2024-09-12T01:26:42.933 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T01:26:42.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:26:43.049 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901560' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMn7zS8qLM0+vyMpucgPlCTiC5B5DPqLV3zUmeeivlponjtmJSrTFRMJ5hIlF3Kuw1Q4KDN4XyT29CspEfHqHBU= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPatDEeGLW1ZoGtzTdOd4rvfGccd1JQ5mc4wPGMoUUoQ1cM74sjrs26PqD52NKx3EtuCvS3VdaKR20lkQyjwrZU= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgHUsr+8+/DdkSXZiMKS3GYxRfRANHnPTdSW6aeiAetnqBX/Ev53TjN5q9p9TkHfdr5DD/kFPYSaFgPipU2OwA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T01:26:43.049 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T01:26:43.050 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T01:26:43.050 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T01:26:43.052 INFO:teuthology.task.internal:Checking packages... 2024-09-12T01:26:43.073 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T01:26:43.073 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T01:26:43.074 INFO:teuthology.packaging:ref: None 2024-09-12T01:26:43.074 INFO:teuthology.packaging:tag: None 2024-09-12T01:26:43.074 INFO:teuthology.packaging:branch: squid 2024-09-12T01:26:43.074 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:26:43.074 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T01:26:43.325 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T01:26:43.327 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T01:26:43.332 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T01:26:43.332 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T01:26:43.338 INFO:teuthology.task.internal:Saving configuration 2024-09-12T01:26:43.357 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T01:26:43.363 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T01:26:43.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:21:41.996589', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMn7zS8qLM0+vyMpucgPlCTiC5B5DPqLV3zUmeeivlponjtmJSrTFRMJ5hIlF3Kuw1Q4KDN4XyT29CspEfHqHBU='} 2024-09-12T01:26:43.426 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:21:41.997721', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPatDEeGLW1ZoGtzTdOd4rvfGccd1JQ5mc4wPGMoUUoQ1cM74sjrs26PqD52NKx3EtuCvS3VdaKR20lkQyjwrZU='} 2024-09-12T01:26:43.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:21:41.994816', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgHUsr+8+/DdkSXZiMKS3GYxRfRANHnPTdSW6aeiAetnqBX/Ev53TjN5q9p9TkHfdr5DD/kFPYSaFgPipU2OwA='} 2024-09-12T01:26:43.458 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T01:26:43.463 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T01:26:43.463 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T01:26:43.463 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T01:26:43.463 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T01:26:43.569 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc9d6f8bbe0>, signals=[15]) 2024-09-12T01:26:43.569 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T01:26:43.579 INFO:teuthology.task.internal:Opening connections... 2024-09-12T01:26:43.579 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T01:26:43.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:26:43.665 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-09-12T01:26:43.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:26:43.749 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-12T01:26:43.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:26:43.833 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T01:26:43.837 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-12T01:26:43.856 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-12T01:26:43.856 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:26:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:26:43.916 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:26:43.917 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-12T01:26:43.944 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-09-12T01:26:43.963 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-09-12T01:26:43.964 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-09-12T01:26:44.021 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:26:44.022 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-09-12T01:26:44.023 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:26:44.023 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:26:44.023 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:26:44.023 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:26:44.023 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:26:44.023 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:26:44.023 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-09-12T01:26:44.078 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-12T01:26:44.098 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-12T01:26:44.098 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:26:44.157 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:26:44.158 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:26:44.158 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:26:44.158 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-12T01:26:44.195 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T01:26:44.205 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T01:26:44.211 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T01:26:44.211 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:26:44.214 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:26:44.216 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:26:44.233 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T01:26:44.241 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T01:26:44.241 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:26:44.272 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:26:44.274 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:26:44.469 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T01:26:44.571 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T01:26:44.585 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T01:26:44.585 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T01:26:44.585 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T01:26:44.586 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:26:44.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:26:44.603 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-12T01:26:44.662 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-503.el9.x86_64 2024-09-12T01:26:44.662 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-503.el9.x86_64 2024-09-12T01:26:44.662 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-09-12T01:26:46.994 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-12T01:26:50.629 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 20 MB 00:02 2024-09-12T01:26:55.080 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-12T01:26:56.697 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-12T01:26:57.643 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-12T01:27:03.074 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T01:27:05.081 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T01:27:05.081 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T01:27:05.172 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T01:27:05.184 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 79 M 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 126 M 2024-09-12T01:27:05.185 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-12T01:27:05.584 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 186 kB/s | 23 kB 00:00 2024-09-12T01:27:06.334 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-12T01:27:06.910 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 24 MB/s | 35 MB 00:01 2024-09-12T01:27:07.177 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 18 MB/s | 29 MB 00:01 2024-09-12T01:27:07.177 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:27:07.178 INFO:teuthology.orchestra.run.smithi122.stdout:Total 40 MB/s | 79 MB 00:01 2024-09-12T01:27:07.862 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-12T01:27:08.193 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-12T01:27:08.193 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-12T01:27:09.161 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-12T01:27:09.162 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-12T01:27:11.604 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-12T01:27:12.264 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:27:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:27:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:27:15.257 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:27:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:27:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:27:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:28:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:28:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:28:03.355 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:28:03.356 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:28:03.356 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:28:03.356 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:28:04.164 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:28:04.164 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:28:04.164 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-12T01:28:04.164 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T01:28:04.164 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T01:28:04.165 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T01:28:04.165 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T01:28:04.165 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:28:04.165 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-12T01:28:04.520 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:28:04.606 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:28:04.606 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:28:04.607 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T01:28:04.607 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T01:28:04.607 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T01:28:04.607 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T01:28:04.607 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T01:28:04.607 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:28:04.666 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:28:04.666 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:28:04.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2024-09-12T01:28:04.698 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2024-09-12T01:28:04.698 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2024-09-12T01:28:04.698 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.06 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 82.el9 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Group : Unspecified 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 8274670 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : builder@centos.org 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2024-09-12T01:28:04.699 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:28:04.700 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T01:28:04.701 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T01:28:04.701 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 5.14.0-505.el9.x86_64 2024-09-12T01:28:04.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T01:28:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-12T01:28:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T01:28:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-12T01:28:06.714 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-12T01:28:06.730 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.6ZtBphZK9W 2024-09-12T01:28:06.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6ZtBphZK9W 2024-09-12T01:28:06.797 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.6ZtBphZK9W 2024-09-12T01:28:06.933 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.6ZtBphZK9W is 6KB 2024-09-12T01:28:06.981 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.6ZtBphZK9W 2024-09-12T01:28:06.996 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T01:28:07.059 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T01:28:07.059 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T01:28:07.059 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T01:28:07.059 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T01:28:07.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T01:28:07.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-12T01:28:37.220 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T01:28:37.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-12T01:28:37.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:28:55.613 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-12T01:29:04.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-12T01:29:04.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:29:07.677 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-12T01:29:19.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-12T01:29:19.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:29:51.548 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-12T01:29:51.917 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-12T01:29:51.918 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T01:29:51.918 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-12T01:29:51.932 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-505.el9.x86_64 2024-09-12T01:29:51.933 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T01:29:51.933 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T01:29:51.933 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T01:29:52.934 DEBUG:teuthology.parallel:result is None 2024-09-12T01:29:52.934 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T01:29:52.943 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T01:29:52.943 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:29:52.946 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:29:52.948 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T01:29:52.968 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T01:29:52.974 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T01:29:52.981 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T01:29:52.981 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:29:53.005 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:29:53.008 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T01:29:53.047 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T01:29:53.054 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T01:29:53.054 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:29:53.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:29:53.095 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:29:53.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:29:53.113 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:29:53.115 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:29:53.137 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T01:29:53.196 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:29:53.214 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:29:53.215 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:29:53.222 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:29:53.235 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:29:53.265 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T01:29:53.267 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T01:29:53.274 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T01:29:53.274 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:29:53.276 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:29:53.278 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T01:29:53.306 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T01:29:53.315 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T01:29:53.315 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:29:53.345 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:29:53.348 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T01:29:53.364 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:29:53.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:29:53.497 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:29:53.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:29:53.624 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:29:53.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:29:53.692 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:29:53.746 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:29:53.784 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:29:53.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:29:53.912 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:29:53.912 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:29:53.979 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:29:54.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T01:29:54.080 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:29:54.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T01:29:54.209 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:29:54.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T01:29:54.276 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-12T01:29:54.279 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-09-12T01:29:54.281 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-12T01:29:54.314 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:29:54.355 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:29:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:29:54.745 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T01:29:54.757 INFO:teuthology.task.internal:Starting timer... 2024-09-12T01:29:54.757 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T01:29:54.768 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T01:29:54.780 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T01:29:54.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-09-12T01:29:54.909 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-09-12T01:29:54.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-09-12T01:29:54.971 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-09-12T01:29:54.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-12T01:29:55.037 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-12T01:29:55.039 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T01:29:55.039 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-09-12T01:29:55.092 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-09-12T01:29:55.092 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-09-12T01:29:55.119 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-09-12T01:29:55.119 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-12T01:29:55.145 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-12T01:29:55.145 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-09-12T01:29:55.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:29:55.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:29:55.191 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:29:55.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:29:55.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:29:55.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:29:55.249 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T01:29:55.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:29:55.325 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:29:55.357 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-12T01:29:55.392 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T01:29:55.402 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T01:29:55.405 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T01:29:55.535 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T01:29:55.545 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T01:29:55.546 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi151.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T01:33:58.242 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-09-12T01:33:58.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-12T01:33:58.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:33:58.334 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-12T01:33:58.421 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-12T01:33:58.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-12T01:33:58.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:33:58.505 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-12T01:33:58.584 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-12T01:33:58.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-12T01:33:58.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:33:58.672 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-12T01:33:58.754 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-12T01:33:58.755 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T01:33:58.766 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T01:33:58.767 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:33:58.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:33:58.770 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:33:58.770 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:33:58.772 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T01:33:58.772 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:33:58.799 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:33:58.801 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:33:58.816 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:33:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:33:58.827 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T01:33:58.841 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-09-12T01:33:58.841 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-09-12T01:33:58.843 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T01:33:58.852 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-09-12T01:33:58.852 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-09-12T01:33:58.866 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:33:58.867 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:33:58.869 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-12T01:33:58.880 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:33:58.881 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-12T01:33:58.881 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:33:58.895 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T01:33:58.910 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T01:33:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:33:58.945 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:33:58.948 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:33:58.948 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-12T01:33:58.948 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.948 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.948 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.948 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:33:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-12T01:33:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.968 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:33:58.971 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:33:58.971 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-12T01:33:58.971 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.972 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.972 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.972 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:33:58.973 INFO:teuthology.run_tasks:Running task install... 2024-09-12T01:33:58.980 DEBUG:teuthology.task.install:project ceph 2024-09-12T01:33:58.981 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T01:33:58.981 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T01:33:58.981 INFO:teuthology.task.install:Using flavor: default 2024-09-12T01:33:58.987 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T01:33:58.987 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T01:33:58.987 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:33:58.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:33:58.990 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:33:58.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:33:58.991 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T01:33:58.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:33:59.137 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:33:59.137 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:33:59.149 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:33:59.149 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:33:59.160 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T01:33:59.160 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T01:33:59.258 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:33:59.259 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:33:59.259 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:33:59.265 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:33:59.265 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:33:59.265 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:33:59.276 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T01:33:59.276 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:33:59.276 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T01:33:59.284 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:33:59.285 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:33:59.292 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:33:59.292 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:33:59.302 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T01:33:59.302 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T01:33:59.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:33:59.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:33:59.368 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T01:33:59.429 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:33:59.435 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:33:59.444 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T01:33:59.493 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-09-12T01:33:59.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-09-12T01:33:59.500 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-09-12T01:33:59.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-09-12T01:33:59.508 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-09-12T01:33:59.510 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-09-12T01:33:59.721 INFO:teuthology.orchestra.run.smithi122.stdout:45 files removed 2024-09-12T01:33:59.729 INFO:teuthology.orchestra.run.smithi151.stdout:45 files removed 2024-09-12T01:33:59.735 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-09-12T01:33:59.748 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:33:59.757 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:33:59.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T01:34:00.295 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-12T01:34:00.303 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-12T01:34:00.310 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-12T01:34:00.571 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-12T01:34:00.587 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T01:34:00.594 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T01:34:00.839 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T01:34:00.845 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 8.8 kB/s | 1.9 kB 00:00 2024-09-12T01:34:00.863 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T01:34:01.354 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-12T01:34:01.444 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-12T01:34:02.539 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-12T01:34:04.366 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-12T01:34:05.196 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 20 MB 00:02 2024-09-12T01:34:05.314 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 20 MB 00:02 2024-09-12T01:34:09.355 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-12T01:34:09.858 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-12T01:34:10.830 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 19 kB 00:00 2024-09-12T01:34:10.881 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-12T01:34:11.293 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 19 kB 00:00 2024-09-12T01:34:11.760 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-12T01:34:12.097 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-12T01:34:12.524 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-12T01:34:13.662 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-12T01:34:17.173 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T01:34:17.512 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T01:34:19.000 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-12T01:34:19.442 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:34:19.442 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:34:19.453 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:34:19.454 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:34:19.455 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:34:19.456 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:34:19.457 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:34:19.460 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:34:19.460 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:34:19.522 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-12T01:34:19.537 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:34:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:34:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:34:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:34:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:34:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:34:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:34:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:34:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:34:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:34:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:34:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:34:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:34:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 8 Packages 2024-09-12T01:34:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:34:19.552 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 226 M 2024-09-12T01:34:19.552 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-12T01:34:19.792 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:34:19.793 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:34:19.805 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:34:19.809 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:34:19.809 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:34:19.809 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:34:19.810 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:34:19.813 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:34:19.814 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:34:19.878 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-09-12T01:34:19.897 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-12T01:34:19.897 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-09-12T01:34:19.897 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-12T01:34:19.897 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:34:19.898 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:34:19.899 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:34:19.900 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:34:19.901 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:34:19.902 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:34:19.903 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:34:19.904 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:34:19.905 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:34:19.906 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:34:19.907 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:34:19.908 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:34:19.909 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-09-12T01:34:19.910 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-12T01:34:19.911 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-09-12T01:34:19.911 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 8 Packages 2024-09-12T01:34:19.911 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:34:19.911 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 226 M 2024-09-12T01:34:19.911 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-09-12T01:34:19.975 INFO:teuthology.orchestra.run.smithi183.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T01:34:20.225 INFO:teuthology.orchestra.run.smithi183.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.5 MB/s | 897 kB 00:00 2024-09-12T01:34:20.309 INFO:teuthology.orchestra.run.smithi183.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-12T01:34:20.334 INFO:teuthology.orchestra.run.smithi183.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.3 MB/s | 145 kB 00:00 2024-09-12T01:34:20.443 INFO:teuthology.orchestra.run.smithi183.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 2.4 MB 00:00 2024-09-12T01:34:20.572 INFO:teuthology.orchestra.run.smithi151.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 29 kB/s | 6.5 kB 00:00 2024-09-12T01:34:20.576 INFO:teuthology.orchestra.run.smithi183.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-12T01:34:20.676 INFO:teuthology.orchestra.run.smithi183.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 20 MB/s | 4.7 MB 00:00 2024-09-12T01:34:21.083 INFO:teuthology.orchestra.run.smithi183.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 21 MB 00:01 2024-09-12T01:34:21.127 INFO:teuthology.orchestra.run.smithi183.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 576 kB/s | 25 kB 00:00 2024-09-12T01:34:21.215 INFO:teuthology.orchestra.run.smithi151.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 897 kB 00:00 2024-09-12T01:34:21.269 INFO:teuthology.orchestra.run.smithi183.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 18 MB/s | 11 MB 00:00 2024-09-12T01:34:21.306 INFO:teuthology.orchestra.run.smithi151.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T01:34:21.319 INFO:teuthology.orchestra.run.smithi183.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T01:34:21.352 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:34:21.353 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T01:34:21.364 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T01:34:21.365 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T01:34:21.366 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T01:34:21.367 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T01:34:21.368 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T01:34:21.371 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T01:34:21.371 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T01:34:21.386 INFO:teuthology.orchestra.run.smithi183.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 406 kB/s | 27 kB 00:00 2024-09-12T01:34:21.434 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-12T01:34:21.445 INFO:teuthology.orchestra.run.smithi183.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 545 kB/s | 32 kB 00:00 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T01:34:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T01:34:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T01:34:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T01:34:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T01:34:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T01:34:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T01:34:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T01:34:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T01:34:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T01:34:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T01:34:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T01:34:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T01:34:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 8 Packages 2024-09-12T01:34:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:34:21.463 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 226 M 2024-09-12T01:34:21.463 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-12T01:34:21.528 INFO:teuthology.orchestra.run.smithi183.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-12T01:34:21.595 INFO:teuthology.orchestra.run.smithi183.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-12T01:34:21.662 INFO:teuthology.orchestra.run.smithi183.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-12T01:34:21.738 INFO:teuthology.orchestra.run.smithi183.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-12T01:34:22.085 INFO:teuthology.orchestra.run.smithi122.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 25 kB/s | 6.5 kB 00:00 2024-09-12T01:34:22.096 INFO:teuthology.orchestra.run.smithi183.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-12T01:34:22.155 INFO:teuthology.orchestra.run.smithi183.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 776 kB/s | 45 kB 00:00 2024-09-12T01:34:22.238 INFO:teuthology.orchestra.run.smithi183.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T01:34:22.313 INFO:teuthology.orchestra.run.smithi183.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-12T01:34:22.389 INFO:teuthology.orchestra.run.smithi183.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-12T01:34:22.472 INFO:teuthology.orchestra.run.smithi183.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.5 MB/s | 301 kB 00:00 2024-09-12T01:34:22.548 INFO:teuthology.orchestra.run.smithi183.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.3 MB/s | 100 kB 00:00 2024-09-12T01:34:22.644 INFO:teuthology.orchestra.run.smithi122.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 897 kB 00:00 2024-09-12T01:34:22.710 INFO:teuthology.orchestra.run.smithi122.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-12T01:34:22.715 INFO:teuthology.orchestra.run.smithi151.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-09-12T01:34:22.731 INFO:teuthology.orchestra.run.smithi183.stdout:(25/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 7.8 MB/s | 17 MB 00:02 2024-09-12T01:34:22.748 INFO:teuthology.orchestra.run.smithi183.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 426 kB/s | 85 kB 00:00 2024-09-12T01:34:22.790 INFO:teuthology.orchestra.run.smithi151.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 1.6 MB/s | 2.4 MB 00:01 2024-09-12T01:34:23.028 INFO:teuthology.orchestra.run.smithi122.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-09-12T01:34:23.066 INFO:teuthology.orchestra.run.smithi151.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 4.3 MB/s | 1.5 MB 00:00 2024-09-12T01:34:23.119 INFO:teuthology.orchestra.run.smithi122.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 5.8 MB/s | 2.4 MB 00:00 2024-09-12T01:34:23.182 INFO:teuthology.orchestra.run.smithi183.stdout:(27/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 24 MB/s | 49 MB 00:02 2024-09-12T01:34:23.195 INFO:teuthology.orchestra.run.smithi122.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-12T01:34:23.207 INFO:teuthology.orchestra.run.smithi183.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 372 kB/s | 171 kB 00:00 2024-09-12T01:34:23.232 INFO:teuthology.orchestra.run.smithi183.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 640 kB/s | 32 kB 00:00 2024-09-12T01:34:23.249 INFO:teuthology.orchestra.run.smithi183.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 3.4 MB/s | 146 kB 00:00 2024-09-12T01:34:23.316 INFO:teuthology.orchestra.run.smithi183.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 5.3 MB/s | 3.1 MB 00:00 2024-09-12T01:34:23.417 INFO:teuthology.orchestra.run.smithi183.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T01:34:23.559 INFO:teuthology.orchestra.run.smithi183.stdout:(33/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 11 MB/s | 3.6 MB 00:00 2024-09-12T01:34:23.584 INFO:teuthology.orchestra.run.smithi183.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 295 kB/s | 49 kB 00:00 2024-09-12T01:34:23.651 INFO:teuthology.orchestra.run.smithi183.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 179 kB/s | 16 kB 00:00 2024-09-12T01:34:23.676 INFO:teuthology.orchestra.run.smithi183.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.0 MB/s | 285 kB 00:00 2024-09-12T01:34:23.743 INFO:teuthology.orchestra.run.smithi183.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-12T01:34:23.768 INFO:teuthology.orchestra.run.smithi183.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 198 kB/s | 18 kB 00:00 2024-09-12T01:34:23.837 INFO:teuthology.orchestra.run.smithi122.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.6 MB/s | 4.7 MB 00:00 2024-09-12T01:34:23.852 INFO:teuthology.orchestra.run.smithi183.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-12T01:34:23.994 INFO:teuthology.orchestra.run.smithi183.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 319 kB/s | 80 kB 00:00 2024-09-12T01:34:24.052 INFO:teuthology.orchestra.run.smithi183.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-12T01:34:24.094 INFO:teuthology.orchestra.run.smithi183.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-12T01:34:24.137 INFO:teuthology.orchestra.run.smithi183.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-12T01:34:24.162 INFO:teuthology.orchestra.run.smithi183.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T01:34:24.195 INFO:teuthology.orchestra.run.smithi183.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-12T01:34:24.221 INFO:teuthology.orchestra.run.smithi183.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-12T01:34:24.256 INFO:teuthology.orchestra.run.smithi183.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.0 MB/s | 1.2 MB 00:00 2024-09-12T01:34:24.281 INFO:teuthology.orchestra.run.smithi183.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T01:34:24.379 INFO:teuthology.orchestra.run.smithi122.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 8.3 MB/s | 21 MB 00:02 2024-09-12T01:34:24.416 INFO:teuthology.orchestra.run.smithi151.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 4.7 MB 00:01 2024-09-12T01:34:24.429 INFO:teuthology.orchestra.run.smithi122.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 505 kB/s | 25 kB 00:00 2024-09-12T01:34:24.679 INFO:teuthology.orchestra.run.smithi122.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 13 MB/s | 11 MB 00:00 2024-09-12T01:34:24.740 INFO:teuthology.orchestra.run.smithi122.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T01:34:24.805 INFO:teuthology.orchestra.run.smithi122.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 407 kB/s | 27 kB 00:00 2024-09-12T01:34:24.872 INFO:teuthology.orchestra.run.smithi122.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 475 kB/s | 32 kB 00:00 2024-09-12T01:34:24.948 INFO:teuthology.orchestra.run.smithi183.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 948 kB/s | 689 kB 00:00 2024-09-12T01:34:24.981 INFO:teuthology.orchestra.run.smithi183.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-12T01:34:24.998 INFO:teuthology.orchestra.run.smithi183.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 149 kB/s | 107 kB 00:00 2024-09-12T01:34:25.015 INFO:teuthology.orchestra.run.smithi183.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 322 kB/s | 22 kB 00:00 2024-09-12T01:34:25.030 INFO:teuthology.orchestra.run.smithi122.stdout:(14/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 9.2 MB/s | 17 MB 00:01 2024-09-12T01:34:25.064 INFO:teuthology.orchestra.run.smithi122.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 3.8 MB/s | 752 kB 00:00 2024-09-12T01:34:25.074 INFO:teuthology.orchestra.run.smithi183.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-12T01:34:25.089 INFO:teuthology.orchestra.run.smithi122.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T01:34:25.091 INFO:teuthology.orchestra.run.smithi183.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-09-12T01:34:25.106 INFO:teuthology.orchestra.run.smithi122.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.9 MB/s | 127 kB 00:00 2024-09-12T01:34:25.141 INFO:teuthology.orchestra.run.smithi183.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-12T01:34:25.166 INFO:teuthology.orchestra.run.smithi183.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-12T01:34:25.204 INFO:teuthology.orchestra.run.smithi122.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.3 MB/s | 506 kB 00:00 2024-09-12T01:34:25.216 INFO:teuthology.orchestra.run.smithi183.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-12T01:34:25.242 INFO:teuthology.orchestra.run.smithi183.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-12T01:34:25.267 INFO:teuthology.orchestra.run.smithi183.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-09-12T01:34:25.290 INFO:teuthology.orchestra.run.smithi122.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 525 kB/s | 45 kB 00:00 2024-09-12T01:34:25.317 INFO:teuthology.orchestra.run.smithi183.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-12T01:34:25.343 INFO:teuthology.orchestra.run.smithi183.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-12T01:34:25.368 INFO:teuthology.orchestra.run.smithi183.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-12T01:34:25.373 INFO:teuthology.orchestra.run.smithi122.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T01:34:25.401 INFO:teuthology.orchestra.run.smithi183.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-12T01:34:25.443 INFO:teuthology.orchestra.run.smithi183.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-12T01:34:25.457 INFO:teuthology.orchestra.run.smithi122.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-12T01:34:25.469 INFO:teuthology.orchestra.run.smithi183.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-12T01:34:25.482 INFO:teuthology.orchestra.run.smithi122.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 162 kB 00:00 2024-09-12T01:34:25.494 INFO:teuthology.orchestra.run.smithi183.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 417 kB/s | 38 kB 00:00 2024-09-12T01:34:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T01:34:25.533 INFO:teuthology.orchestra.run.smithi122.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.8 MB/s | 301 kB 00:00 2024-09-12T01:34:25.558 INFO:teuthology.orchestra.run.smithi122.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T01:34:25.584 INFO:teuthology.orchestra.run.smithi122.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T01:34:25.620 INFO:teuthology.orchestra.run.smithi183.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-09-12T01:34:25.634 INFO:teuthology.orchestra.run.smithi122.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T01:34:25.686 INFO:teuthology.orchestra.run.smithi183.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-12T01:34:25.709 INFO:teuthology.orchestra.run.smithi122.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-12T01:34:25.716 INFO:teuthology.orchestra.run.smithi151.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 3.9 MB/s | 21 MB 00:05 2024-09-12T01:34:25.728 INFO:teuthology.orchestra.run.smithi183.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-12T01:34:25.734 INFO:teuthology.orchestra.run.smithi122.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 318 kB/s | 32 kB 00:00 2024-09-12T01:34:25.760 INFO:teuthology.orchestra.run.smithi122.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-12T01:34:25.761 INFO:teuthology.orchestra.run.smithi183.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-12T01:34:25.787 INFO:teuthology.orchestra.run.smithi183.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-12T01:34:25.798 INFO:teuthology.orchestra.run.smithi151.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 310 kB/s | 25 kB 00:00 2024-09-12T01:34:25.862 INFO:teuthology.orchestra.run.smithi183.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-12T01:34:25.888 INFO:teuthology.orchestra.run.smithi183.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-09-12T01:34:25.981 INFO:teuthology.orchestra.run.smithi183.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-12T01:34:26.008 INFO:teuthology.orchestra.run.smithi183.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-12T01:34:26.055 INFO:teuthology.orchestra.run.smithi183.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-12T01:34:26.130 INFO:teuthology.orchestra.run.smithi183.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-12T01:34:26.156 INFO:teuthology.orchestra.run.smithi183.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-12T01:34:26.181 INFO:teuthology.orchestra.run.smithi183.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-12T01:34:26.206 INFO:teuthology.orchestra.run.smithi183.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-09-12T01:34:26.210 INFO:teuthology.orchestra.run.smithi122.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-12T01:34:26.231 INFO:teuthology.orchestra.run.smithi183.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-12T01:34:26.257 INFO:teuthology.orchestra.run.smithi183.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-09-12T01:34:26.269 INFO:teuthology.orchestra.run.smithi122.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.7 MB/s | 3.6 MB 00:00 2024-09-12T01:34:26.294 INFO:teuthology.orchestra.run.smithi122.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.9 MB/s | 249 kB 00:00 2024-09-12T01:34:26.315 INFO:teuthology.orchestra.run.smithi183.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-12T01:34:26.319 INFO:teuthology.orchestra.run.smithi122.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 986 kB/s | 49 kB 00:00 2024-09-12T01:34:26.340 INFO:teuthology.orchestra.run.smithi183.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-12T01:34:26.344 INFO:teuthology.orchestra.run.smithi122.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-12T01:34:26.369 INFO:teuthology.orchestra.run.smithi122.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T01:34:26.391 INFO:teuthology.orchestra.run.smithi183.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-12T01:34:26.403 INFO:teuthology.orchestra.run.smithi122.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 13 MB/s | 765 kB 00:00 2024-09-12T01:34:26.416 INFO:teuthology.orchestra.run.smithi183.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-09-12T01:34:26.429 INFO:teuthology.orchestra.run.smithi122.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 310 kB/s | 18 kB 00:00 2024-09-12T01:34:26.483 INFO:teuthology.orchestra.run.smithi183.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-09-12T01:34:26.508 INFO:teuthology.orchestra.run.smithi183.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-12T01:34:26.533 INFO:teuthology.orchestra.run.smithi183.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-12T01:34:26.559 INFO:teuthology.orchestra.run.smithi183.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-12T01:34:26.576 INFO:teuthology.orchestra.run.smithi183.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 758 kB/s | 51 kB 00:00 2024-09-12T01:34:26.609 INFO:teuthology.orchestra.run.smithi183.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-09-12T01:34:26.651 INFO:teuthology.orchestra.run.smithi183.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-12T01:34:26.668 INFO:teuthology.orchestra.run.smithi183.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 749 kB/s | 44 kB 00:00 2024-09-12T01:34:26.735 INFO:teuthology.orchestra.run.smithi183.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-09-12T01:34:26.752 INFO:teuthology.orchestra.run.smithi183.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-12T01:34:26.794 INFO:teuthology.orchestra.run.smithi183.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-09-12T01:34:26.828 INFO:teuthology.orchestra.run.smithi183.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.9 MB/s | 304 kB 00:00 2024-09-12T01:34:26.861 INFO:teuthology.orchestra.run.smithi183.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-09-12T01:34:26.871 INFO:teuthology.orchestra.run.smithi122.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 20 MB/s | 49 MB 00:02 2024-09-12T01:34:26.904 INFO:teuthology.orchestra.run.smithi122.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 159 kB/s | 80 kB 00:00 2024-09-12T01:34:26.942 INFO:teuthology.orchestra.run.smithi151.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 4.3 MB/s | 17 MB 00:03 2024-09-12T01:34:26.997 INFO:teuthology.orchestra.run.smithi122.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-12T01:34:27.009 INFO:teuthology.orchestra.run.smithi151.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-12T01:34:27.012 INFO:teuthology.orchestra.run.smithi183.stdout:(100/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-12T01:34:27.037 INFO:teuthology.orchestra.run.smithi183.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 363 kB/s | 64 kB 00:00 2024-09-12T01:34:27.047 INFO:teuthology.orchestra.run.smithi122.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-12T01:34:27.051 INFO:teuthology.orchestra.run.smithi151.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 649 kB/s | 27 kB 00:00 2024-09-12T01:34:27.062 INFO:teuthology.orchestra.run.smithi183.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 409 kB/s | 96 kB 00:00 2024-09-12T01:34:27.088 INFO:teuthology.orchestra.run.smithi183.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-09-12T01:34:27.101 INFO:teuthology.orchestra.run.smithi151.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 632 kB/s | 32 kB 00:00 2024-09-12T01:34:27.105 INFO:teuthology.orchestra.run.smithi122.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-12T01:34:27.131 INFO:teuthology.orchestra.run.smithi122.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-12T01:34:27.138 INFO:teuthology.orchestra.run.smithi183.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T01:34:27.163 INFO:teuthology.orchestra.run.smithi183.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-12T01:34:27.164 INFO:teuthology.orchestra.run.smithi122.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-09-12T01:34:27.189 INFO:teuthology.orchestra.run.smithi122.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T01:34:27.196 INFO:teuthology.orchestra.run.smithi183.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-12T01:34:27.215 INFO:teuthology.orchestra.run.smithi122.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-12T01:34:27.230 INFO:teuthology.orchestra.run.smithi183.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-12T01:34:27.240 INFO:teuthology.orchestra.run.smithi122.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-12T01:34:27.243 INFO:teuthology.orchestra.run.smithi151.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.2 MB/s | 752 kB 00:00 2024-09-12T01:34:27.255 INFO:teuthology.orchestra.run.smithi183.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T01:34:27.281 INFO:teuthology.orchestra.run.smithi183.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T01:34:27.306 INFO:teuthology.orchestra.run.smithi183.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:34:27.339 INFO:teuthology.orchestra.run.smithi183.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-12T01:34:27.352 INFO:teuthology.orchestra.run.smithi151.stdout:(15/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 3.6 MB/s | 11 MB 00:02 2024-09-12T01:34:27.373 INFO:teuthology.orchestra.run.smithi183.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-12T01:34:27.377 INFO:teuthology.orchestra.run.smithi151.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.2 MB/s | 163 kB 00:00 2024-09-12T01:34:27.402 INFO:teuthology.orchestra.run.smithi151.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T01:34:27.423 INFO:teuthology.orchestra.run.smithi183.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-12T01:34:27.449 INFO:teuthology.orchestra.run.smithi183.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:34:27.503 INFO:teuthology.orchestra.run.smithi151.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 3.9 MB/s | 506 kB 00:00 2024-09-12T01:34:27.515 INFO:teuthology.orchestra.run.smithi183.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-12T01:34:27.541 INFO:teuthology.orchestra.run.smithi183.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-09-12T01:34:27.545 INFO:teuthology.orchestra.run.smithi151.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 1.1 MB/s | 45 kB 00:00 2024-09-12T01:34:27.566 INFO:teuthology.orchestra.run.smithi183.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T01:34:27.596 INFO:teuthology.orchestra.run.smithi151.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.6 MB/s | 136 kB 00:00 2024-09-12T01:34:27.600 INFO:teuthology.orchestra.run.smithi183.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-09-12T01:34:27.625 INFO:teuthology.orchestra.run.smithi183.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-12T01:34:27.650 INFO:teuthology.orchestra.run.smithi183.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-12T01:34:27.662 INFO:teuthology.orchestra.run.smithi151.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T01:34:27.675 INFO:teuthology.orchestra.run.smithi183.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-12T01:34:27.701 INFO:teuthology.orchestra.run.smithi183.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-12T01:34:27.707 INFO:teuthology.orchestra.run.smithi122.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 229 kB/s | 107 kB 00:00 2024-09-12T01:34:27.726 INFO:teuthology.orchestra.run.smithi183.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-12T01:34:27.755 INFO:teuthology.orchestra.run.smithi151.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 321 kB 00:00 2024-09-12T01:34:27.760 INFO:teuthology.orchestra.run.smithi183.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-09-12T01:34:27.785 INFO:teuthology.orchestra.run.smithi183.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 300 kB/s | 18 kB 00:00 2024-09-12T01:34:27.810 INFO:teuthology.orchestra.run.smithi183.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-12T01:34:27.815 INFO:teuthology.orchestra.run.smithi122.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-12T01:34:27.822 INFO:teuthology.orchestra.run.smithi151.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-12T01:34:27.836 INFO:teuthology.orchestra.run.smithi183.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-12T01:34:27.853 INFO:teuthology.orchestra.run.smithi183.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 471 kB/s | 20 kB 00:00 2024-09-12T01:34:27.857 INFO:teuthology.orchestra.run.smithi122.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-12T01:34:27.873 INFO:teuthology.orchestra.run.smithi151.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 2.0 MB/s | 100 kB 00:00 2024-09-12T01:34:27.878 INFO:teuthology.orchestra.run.smithi183.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:34:27.903 INFO:teuthology.orchestra.run.smithi183.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:34:27.922 INFO:teuthology.orchestra.run.smithi151.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T01:34:27.929 INFO:teuthology.orchestra.run.smithi183.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T01:34:27.950 INFO:teuthology.orchestra.run.smithi122.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-12T01:34:27.970 INFO:teuthology.orchestra.run.smithi183.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-12T01:34:27.974 INFO:teuthology.orchestra.run.smithi122.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-12T01:34:27.996 INFO:teuthology.orchestra.run.smithi183.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-12T01:34:28.017 INFO:teuthology.orchestra.run.smithi122.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-09-12T01:34:28.021 INFO:teuthology.orchestra.run.smithi183.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-12T01:34:28.047 INFO:teuthology.orchestra.run.smithi183.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T01:34:28.072 INFO:teuthology.orchestra.run.smithi183.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-12T01:34:28.083 INFO:teuthology.orchestra.run.smithi122.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-09-12T01:34:28.097 INFO:teuthology.orchestra.run.smithi183.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-12T01:34:28.122 INFO:teuthology.orchestra.run.smithi183.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T01:34:28.125 INFO:teuthology.orchestra.run.smithi122.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-12T01:34:28.147 INFO:teuthology.orchestra.run.smithi183.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T01:34:28.173 INFO:teuthology.orchestra.run.smithi183.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-12T01:34:28.201 INFO:teuthology.orchestra.run.smithi122.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-12T01:34:28.223 INFO:teuthology.orchestra.run.smithi183.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-12T01:34:28.240 INFO:teuthology.orchestra.run.smithi183.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-12T01:34:28.265 INFO:teuthology.orchestra.run.smithi183.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-12T01:34:28.267 INFO:teuthology.orchestra.run.smithi122.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-09-12T01:34:28.290 INFO:teuthology.orchestra.run.smithi183.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-12T01:34:28.293 INFO:teuthology.orchestra.run.smithi122.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-09-12T01:34:28.315 INFO:teuthology.orchestra.run.smithi183.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-12T01:34:28.318 INFO:teuthology.orchestra.run.smithi122.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-12T01:34:28.341 INFO:teuthology.orchestra.run.smithi183.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:34:28.356 INFO:teuthology.orchestra.run.smithi151.stdout:(26/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.6 MB/s | 5.3 MB 00:00 2024-09-12T01:34:28.366 INFO:teuthology.orchestra.run.smithi183.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T01:34:28.393 INFO:teuthology.orchestra.run.smithi122.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-12T01:34:28.417 INFO:teuthology.orchestra.run.smithi183.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T01:34:28.418 INFO:teuthology.orchestra.run.smithi122.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 195 kB/s | 24 kB 00:00 2024-09-12T01:34:28.431 INFO:teuthology.orchestra.run.smithi151.stdout:(27/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 6.1 MB/s | 3.1 MB 00:00 2024-09-12T01:34:28.465 INFO:teuthology.orchestra.run.smithi151.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.5 MB/s | 171 kB 00:00 2024-09-12T01:34:28.490 INFO:teuthology.orchestra.run.smithi151.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 544 kB/s | 32 kB 00:00 2024-09-12T01:34:28.511 INFO:teuthology.orchestra.run.smithi122.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-12T01:34:28.516 INFO:teuthology.orchestra.run.smithi151.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-12T01:34:28.536 INFO:teuthology.orchestra.run.smithi122.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-12T01:34:28.617 INFO:teuthology.orchestra.run.smithi183.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T01:34:28.628 INFO:teuthology.orchestra.run.smithi122.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-12T01:34:28.828 INFO:teuthology.orchestra.run.smithi183.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.9 MB/s | 3.2 MB 00:00 2024-09-12T01:34:28.867 INFO:teuthology.orchestra.run.smithi183.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 5.2 MB/s | 2.3 MB 00:00 2024-09-12T01:34:28.903 INFO:teuthology.orchestra.run.smithi122.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-12T01:34:28.959 INFO:teuthology.orchestra.run.smithi183.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 305 kB/s | 41 kB 00:00 2024-09-12T01:34:28.966 INFO:teuthology.orchestra.run.smithi151.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 7.5 MB/s | 3.6 MB 00:00 2024-09-12T01:34:29.018 INFO:teuthology.orchestra.run.smithi183.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-12T01:34:29.033 INFO:teuthology.orchestra.run.smithi151.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T01:34:29.045 INFO:teuthology.orchestra.run.smithi122.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 9.7 MB/s | 4.9 MB 00:00 2024-09-12T01:34:29.052 INFO:teuthology.orchestra.run.smithi183.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.2 MB/s | 1.8 MB 00:00 2024-09-12T01:34:29.075 INFO:teuthology.orchestra.run.smithi151.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 1.2 MB/s | 49 kB 00:00 2024-09-12T01:34:29.087 INFO:teuthology.orchestra.run.smithi122.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-12T01:34:29.118 INFO:teuthology.orchestra.run.smithi151.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 393 kB/s | 16 kB 00:00 2024-09-12T01:34:29.185 INFO:teuthology.orchestra.run.smithi151.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-12T01:34:29.187 INFO:teuthology.orchestra.run.smithi122.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-12T01:34:29.213 INFO:teuthology.orchestra.run.smithi122.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T01:34:29.238 INFO:teuthology.orchestra.run.smithi122.stdout:(71/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 52 kB/s | 47 kB 00:00 2024-09-12T01:34:29.293 INFO:teuthology.orchestra.run.smithi183.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 1.4 MB 00:00 2024-09-12T01:34:29.302 INFO:teuthology.orchestra.run.smithi151.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-12T01:34:29.344 INFO:teuthology.orchestra.run.smithi151.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 438 kB/s | 18 kB 00:00 2024-09-12T01:34:29.355 INFO:teuthology.orchestra.run.smithi122.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-09-12T01:34:29.361 INFO:teuthology.orchestra.run.smithi183.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-09-12T01:34:29.366 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:34:29.367 INFO:teuthology.orchestra.run.smithi183.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-12T01:34:29.380 INFO:teuthology.orchestra.run.smithi122.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 617 kB/s | 87 kB 00:00 2024-09-12T01:34:29.455 INFO:teuthology.orchestra.run.smithi122.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T01:34:29.497 INFO:teuthology.orchestra.run.smithi122.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-12T01:34:29.553 INFO:teuthology.orchestra.run.smithi151.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 7.1 MB/s | 7.4 MB 00:01 2024-09-12T01:34:29.573 INFO:teuthology.orchestra.run.smithi122.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-12T01:34:29.665 INFO:teuthology.orchestra.run.smithi122.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-09-12T01:34:29.698 INFO:teuthology.orchestra.run.smithi122.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-09-12T01:34:29.723 INFO:teuthology.orchestra.run.smithi122.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-09-12T01:34:29.765 INFO:teuthology.orchestra.run.smithi122.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-12T01:34:29.807 INFO:teuthology.orchestra.run.smithi122.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-12T01:34:29.824 INFO:teuthology.orchestra.run.smithi122.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-09-12T01:34:29.933 INFO:teuthology.orchestra.run.smithi122.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-12T01:34:29.958 INFO:teuthology.orchestra.run.smithi122.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-09-12T01:34:30.033 INFO:teuthology.orchestra.run.smithi122.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-12T01:34:30.045 INFO:teuthology.orchestra.run.smithi151.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 114 kB/s | 80 kB 00:00 2024-09-12T01:34:30.059 INFO:teuthology.orchestra.run.smithi122.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 615 kB/s | 77 kB 00:00 2024-09-12T01:34:30.084 INFO:teuthology.orchestra.run.smithi122.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-12T01:34:30.151 INFO:teuthology.orchestra.run.smithi122.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-12T01:34:30.176 INFO:teuthology.orchestra.run.smithi122.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-12T01:34:30.201 INFO:teuthology.orchestra.run.smithi122.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-12T01:34:30.260 INFO:teuthology.orchestra.run.smithi122.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-12T01:34:30.285 INFO:teuthology.orchestra.run.smithi122.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-09-12T01:34:30.369 INFO:teuthology.orchestra.run.smithi122.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-12T01:34:30.394 INFO:teuthology.orchestra.run.smithi122.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-12T01:34:30.445 INFO:teuthology.orchestra.run.smithi151.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 898 kB/s | 801 kB 00:00 2024-09-12T01:34:30.486 INFO:teuthology.orchestra.run.smithi122.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-12T01:34:30.511 INFO:teuthology.orchestra.run.smithi122.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-09-12T01:34:30.603 INFO:teuthology.orchestra.run.smithi122.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-12T01:34:30.610 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-12T01:34:30.629 INFO:teuthology.orchestra.run.smithi122.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-12T01:34:30.712 INFO:teuthology.orchestra.run.smithi122.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 532 kB/s | 58 kB 00:00 2024-09-12T01:34:30.746 INFO:teuthology.orchestra.run.smithi122.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 818 kB/s | 96 kB 00:00 2024-09-12T01:34:30.762 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-12T01:34:30.762 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-12T01:34:30.779 INFO:teuthology.orchestra.run.smithi151.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 261 kB/s | 191 kB 00:00 2024-09-12T01:34:30.854 INFO:teuthology.orchestra.run.smithi122.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 447 kB/s | 64 kB 00:00 2024-09-12T01:34:30.888 INFO:teuthology.orchestra.run.smithi122.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 315 kB/s | 45 kB 00:00 2024-09-12T01:34:30.947 INFO:teuthology.orchestra.run.smithi122.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-12T01:34:30.997 INFO:teuthology.orchestra.run.smithi122.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T01:34:31.152 INFO:teuthology.orchestra.run.smithi122.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T01:34:31.152 INFO:teuthology.orchestra.run.smithi122.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-12T01:34:31.152 INFO:teuthology.orchestra.run.smithi122.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-09-12T01:34:31.152 INFO:teuthology.orchestra.run.smithi122.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-12T01:34:31.174 INFO:teuthology.orchestra.run.smithi122.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-12T01:34:31.199 INFO:teuthology.orchestra.run.smithi122.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:34:31.266 INFO:teuthology.orchestra.run.smithi151.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 313 kB/s | 253 kB 00:00 2024-09-12T01:34:31.267 INFO:teuthology.orchestra.run.smithi122.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-09-12T01:34:31.291 INFO:teuthology.orchestra.run.smithi122.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-12T01:34:31.320 INFO:teuthology.orchestra.run.smithi122.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T01:34:31.342 INFO:teuthology.orchestra.run.smithi122.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-12T01:34:31.367 INFO:teuthology.orchestra.run.smithi122.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-09-12T01:34:31.392 INFO:teuthology.orchestra.run.smithi122.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-12T01:34:31.418 INFO:teuthology.orchestra.run.smithi122.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-12T01:34:31.443 INFO:teuthology.orchestra.run.smithi122.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-12T01:34:31.468 INFO:teuthology.orchestra.run.smithi122.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-12T01:34:31.494 INFO:teuthology.orchestra.run.smithi122.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-12T01:34:31.519 INFO:teuthology.orchestra.run.smithi122.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T01:34:31.552 INFO:teuthology.orchestra.run.smithi122.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T01:34:31.569 INFO:teuthology.orchestra.run.smithi122.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 647 kB/s | 11 kB 00:00 2024-09-12T01:34:31.586 INFO:teuthology.orchestra.run.smithi122.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-12T01:34:31.603 INFO:teuthology.orchestra.run.smithi122.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-12T01:34:31.620 INFO:teuthology.orchestra.run.smithi122.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-12T01:34:31.645 INFO:teuthology.orchestra.run.smithi122.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 784 kB/s | 20 kB 00:00 2024-09-12T01:34:31.695 INFO:teuthology.orchestra.run.smithi122.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-09-12T01:34:31.704 INFO:teuthology.orchestra.run.smithi151.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-12T01:34:31.721 INFO:teuthology.orchestra.run.smithi122.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:34:31.746 INFO:teuthology.orchestra.run.smithi122.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:34:31.922 INFO:teuthology.orchestra.run.smithi122.stdout:(131/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.4 MB/s | 16 MB 00:01 2024-09-12T01:34:31.938 INFO:teuthology.orchestra.run.smithi151.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-09-12T01:34:31.947 INFO:teuthology.orchestra.run.smithi122.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 288 kB/s | 58 kB 00:00 2024-09-12T01:34:31.989 INFO:teuthology.orchestra.run.smithi122.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-12T01:34:32.031 INFO:teuthology.orchestra.run.smithi122.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2024-09-12T01:34:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-09-12T01:34:32.081 INFO:teuthology.orchestra.run.smithi122.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T01:34:32.107 INFO:teuthology.orchestra.run.smithi122.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 780 kB/s | 90 kB 00:00 2024-09-12T01:34:32.132 INFO:teuthology.orchestra.run.smithi122.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-12T01:34:32.157 INFO:teuthology.orchestra.run.smithi122.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-12T01:34:32.182 INFO:teuthology.orchestra.run.smithi122.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-12T01:34:32.207 INFO:teuthology.orchestra.run.smithi122.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T01:34:32.233 INFO:teuthology.orchestra.run.smithi122.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-12T01:34:32.258 INFO:teuthology.orchestra.run.smithi122.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T01:34:32.283 INFO:teuthology.orchestra.run.smithi122.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-12T01:34:32.309 INFO:teuthology.orchestra.run.smithi122.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-12T01:34:32.334 INFO:teuthology.orchestra.run.smithi122.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T01:34:32.359 INFO:teuthology.orchestra.run.smithi122.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T01:34:32.410 INFO:teuthology.orchestra.run.smithi122.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T01:34:32.422 INFO:teuthology.orchestra.run.smithi151.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 189 kB/s | 135 kB 00:00 2024-09-12T01:34:32.563 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-12T01:34:32.563 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-12T01:34:32.576 INFO:teuthology.orchestra.run.smithi122.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-12T01:34:32.627 INFO:teuthology.orchestra.run.smithi122.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-12T01:34:32.639 INFO:teuthology.orchestra.run.smithi151.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-12T01:34:32.702 INFO:teuthology.orchestra.run.smithi122.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 14 MB/s | 1.8 MB 00:00 2024-09-12T01:34:32.727 INFO:teuthology.orchestra.run.smithi122.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 406 kB/s | 41 kB 00:00 2024-09-12T01:34:32.794 INFO:teuthology.orchestra.run.smithi122.stdout:(153/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.3 MB/s | 3.2 MB 00:00 2024-09-12T01:34:32.819 INFO:teuthology.orchestra.run.smithi122.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-12T01:34:32.853 INFO:teuthology.orchestra.run.smithi122.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-12T01:34:32.986 INFO:teuthology.orchestra.run.smithi122.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-12T01:34:32.993 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:34:32.993 INFO:teuthology.orchestra.run.smithi122.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-12T01:34:33.364 INFO:teuthology.orchestra.run.smithi151.stdout:(47/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 6.4 MB/s | 49 MB 00:07 2024-09-12T01:34:33.656 INFO:teuthology.orchestra.run.smithi151.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 683 kB/s | 689 kB 00:01 2024-09-12T01:34:33.707 INFO:teuthology.orchestra.run.smithi151.stdout:(49/156): python3-urllib3-1.26.5-6.el9.noarch.r 167 kB/s | 215 kB 00:01 2024-09-12T01:34:33.771 INFO:teuthology.orchestra.run.smithi151.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 937 kB/s | 107 kB 00:00 2024-09-12T01:34:33.888 INFO:teuthology.orchestra.run.smithi151.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T01:34:33.997 INFO:teuthology.orchestra.run.smithi151.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 75 kB/s | 22 kB 00:00 2024-09-12T01:34:34.022 INFO:teuthology.orchestra.run.smithi151.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 244 kB/s | 33 kB 00:00 2024-09-12T01:34:34.139 INFO:teuthology.orchestra.run.smithi151.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-12T01:34:34.189 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-12T01:34:34.214 INFO:teuthology.orchestra.run.smithi151.stdout:(55/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-12T01:34:34.248 INFO:teuthology.orchestra.run.smithi151.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-12T01:34:34.332 INFO:teuthology.orchestra.run.smithi151.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-12T01:34:34.338 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-12T01:34:34.338 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-12T01:34:34.357 INFO:teuthology.orchestra.run.smithi151.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-12T01:34:34.457 INFO:teuthology.orchestra.run.smithi151.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-12T01:34:34.483 INFO:teuthology.orchestra.run.smithi151.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-12T01:34:34.591 INFO:teuthology.orchestra.run.smithi151.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 183 kB/s | 24 kB 00:00 2024-09-12T01:34:34.608 INFO:teuthology.orchestra.run.smithi151.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 378 kB/s | 47 kB 00:00 2024-09-12T01:34:34.709 INFO:teuthology.orchestra.run.smithi151.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-12T01:34:34.734 INFO:teuthology.orchestra.run.smithi151.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-12T01:34:34.809 INFO:teuthology.orchestra.run.smithi151.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-12T01:34:35.618 INFO:teuthology.orchestra.run.smithi151.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.8 MB/s | 3.0 MB 00:01 2024-09-12T01:34:36.444 INFO:teuthology.orchestra.run.smithi151.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-09-12T01:34:36.573 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-12T01:34:36.573 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-12T01:34:36.919 INFO:teuthology.orchestra.run.smithi151.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-12T01:34:37.036 INFO:teuthology.orchestra.run.smithi151.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-12T01:34:37.728 INFO:teuthology.orchestra.run.smithi151.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-09-12T01:34:37.803 INFO:teuthology.orchestra.run.smithi151.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.6 MB/s | 4.9 MB 00:03 2024-09-12T01:34:37.995 INFO:teuthology.orchestra.run.smithi151.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-09-12T01:34:38.892 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-12T01:34:39.005 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:34:39.044 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:34:39.114 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:34:39.121 INFO:teuthology.orchestra.run.smithi151.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 2.8 MB/s | 5.8 MB 00:02 2024-09-12T01:34:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:34:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:34:39.224 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:34:39.237 INFO:teuthology.orchestra.run.smithi151.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 526 kB/s | 61 kB 00:00 2024-09-12T01:34:39.304 INFO:teuthology.orchestra.run.smithi151.stdout:(75/156): protobuf-3.14.0-13.el9.x86_64.rpm 655 kB/s | 1.0 MB 00:01 2024-09-12T01:34:39.355 INFO:teuthology.orchestra.run.smithi151.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-12T01:34:39.472 INFO:teuthology.orchestra.run.smithi151.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-09-12T01:34:39.554 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:34:39.580 INFO:teuthology.orchestra.run.smithi151.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-12T01:34:39.666 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:34:39.697 INFO:teuthology.orchestra.run.smithi151.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 632 kB/s | 248 kB 00:00 2024-09-12T01:34:39.723 INFO:teuthology.orchestra.run.smithi151.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-12T01:34:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:34:39.806 INFO:teuthology.orchestra.run.smithi151.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-09-12T01:34:39.825 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:34:39.917 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:34:39.946 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:34:40.094 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:34:40.143 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:34:40.171 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:34:40.199 INFO:teuthology.orchestra.run.smithi151.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 639 kB/s | 250 kB 00:00 2024-09-12T01:34:40.258 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:34:40.281 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:34:40.343 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:34:40.373 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:34:40.392 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:34:40.582 INFO:teuthology.orchestra.run.smithi151.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 580 kB/s | 222 kB 00:00 2024-09-12T01:34:40.592 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:34:40.745 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:34:40.783 INFO:teuthology.orchestra.run.smithi151.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 385 kB/s | 77 kB 00:00 2024-09-12T01:34:40.888 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:34:40.899 INFO:teuthology.orchestra.run.smithi151.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-12T01:34:40.993 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:41.056 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:41.067 INFO:teuthology.orchestra.run.smithi151.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-09-12T01:34:41.101 INFO:teuthology.orchestra.run.smithi151.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-09-12T01:34:41.193 INFO:teuthology.orchestra.run.smithi151.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-12T01:34:41.235 INFO:teuthology.orchestra.run.smithi151.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-12T01:34:41.265 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:41.310 INFO:teuthology.orchestra.run.smithi151.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-12T01:34:41.344 INFO:teuthology.orchestra.run.smithi151.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-09-12T01:34:41.369 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:34:41.455 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:34:41.461 INFO:teuthology.orchestra.run.smithi151.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-12T01:34:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:34:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:34:41.578 INFO:teuthology.orchestra.run.smithi151.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-12T01:34:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:34:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:34:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:34:41.695 INFO:teuthology.orchestra.run.smithi151.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-12T01:34:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:34:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:34:41.791 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:34:41.812 INFO:teuthology.orchestra.run.smithi151.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-09-12T01:34:41.859 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:34:41.913 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:34:41.921 INFO:teuthology.orchestra.run.smithi151.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-12T01:34:41.944 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:34:41.983 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:34:42.038 INFO:teuthology.orchestra.run.smithi151.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-12T01:34:42.058 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:34:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:34:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:34:42.155 INFO:teuthology.orchestra.run.smithi151.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 496 kB/s | 58 kB 00:00 2024-09-12T01:34:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:34:42.235 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:34:42.263 INFO:teuthology.orchestra.run.smithi151.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 884 kB/s | 96 kB 00:00 2024-09-12T01:34:42.289 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:34:42.362 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:34:42.372 INFO:teuthology.orchestra.run.smithi151.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-09-12T01:34:42.481 INFO:teuthology.orchestra.run.smithi151.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-09-12T01:34:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:34:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:34:42.573 INFO:teuthology.orchestra.run.smithi151.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-12T01:34:42.749 INFO:teuthology.orchestra.run.smithi151.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-09-12T01:34:42.799 INFO:teuthology.orchestra.run.smithi151.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T01:34:42.832 INFO:teuthology.orchestra.run.smithi151.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T01:34:42.850 INFO:teuthology.orchestra.run.smithi151.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-12T01:34:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-12T01:34:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:34:42.959 INFO:teuthology.orchestra.run.smithi151.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T01:34:42.975 INFO:teuthology.orchestra.run.smithi151.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.6 MB/s | 26 kB 00:00 2024-09-12T01:34:42.992 INFO:teuthology.orchestra.run.smithi151.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-12T01:34:43.018 INFO:teuthology.orchestra.run.smithi151.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T01:34:43.043 INFO:teuthology.orchestra.run.smithi151.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.8 MB/s | 151 kB 00:00 2024-09-12T01:34:43.062 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:34:43.077 INFO:teuthology.orchestra.run.smithi151.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 24 MB/s | 839 kB 00:00 2024-09-12T01:34:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:34:43.111 INFO:teuthology.orchestra.run.smithi151.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T01:34:43.136 INFO:teuthology.orchestra.run.smithi151.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T01:34:43.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:34:43.161 INFO:teuthology.orchestra.run.smithi151.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T01:34:43.175 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:34:43.187 INFO:teuthology.orchestra.run.smithi151.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 561 kB/s | 14 kB 00:00 2024-09-12T01:34:43.212 INFO:teuthology.orchestra.run.smithi151.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T01:34:43.217 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:34:43.254 INFO:teuthology.orchestra.run.smithi151.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-12T01:34:43.255 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:34:43.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:34:43.288 INFO:teuthology.orchestra.run.smithi151.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-12T01:34:43.296 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:34:43.313 INFO:teuthology.orchestra.run.smithi151.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T01:34:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:34:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:34:43.389 INFO:teuthology.orchestra.run.smithi151.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-09-12T01:34:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:34:43.414 INFO:teuthology.orchestra.run.smithi151.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-12T01:34:43.439 INFO:teuthology.orchestra.run.smithi151.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-12T01:34:43.465 INFO:teuthology.orchestra.run.smithi151.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-12T01:34:43.490 INFO:teuthology.orchestra.run.smithi151.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-12T01:34:43.516 INFO:teuthology.orchestra.run.smithi151.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-09-12T01:34:43.517 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:34:43.541 INFO:teuthology.orchestra.run.smithi151.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-12T01:34:43.583 INFO:teuthology.orchestra.run.smithi151.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 24 MB/s | 1.0 MB 00:00 2024-09-12T01:34:43.583 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:34:43.608 INFO:teuthology.orchestra.run.smithi151.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T01:34:43.634 INFO:teuthology.orchestra.run.smithi151.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T01:34:43.645 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:34:43.651 INFO:teuthology.orchestra.run.smithi151.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-12T01:34:43.684 INFO:teuthology.orchestra.run.smithi151.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 7.9 MB/s | 272 kB 00:00 2024-09-12T01:34:43.702 INFO:teuthology.orchestra.run.smithi151.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 971 kB/s | 16 kB 00:00 2024-09-12T01:34:43.718 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:34:43.727 INFO:teuthology.orchestra.run.smithi151.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T01:34:43.738 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:34:43.750 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:34:43.752 INFO:teuthology.orchestra.run.smithi151.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T01:34:43.778 INFO:teuthology.orchestra.run.smithi151.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-12T01:34:43.786 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:34:43.803 INFO:teuthology.orchestra.run.smithi151.stdout:(137/156): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-09-12T01:34:43.829 INFO:teuthology.orchestra.run.smithi151.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T01:34:43.833 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:34:43.854 INFO:teuthology.orchestra.run.smithi151.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-12T01:34:43.860 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:34:43.869 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:34:43.879 INFO:teuthology.orchestra.run.smithi151.stdout:(140/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-12T01:34:43.905 INFO:teuthology.orchestra.run.smithi151.stdout:(141/156): python3-webob-1.8.8-2.el9.noarch.rpm 9.1 MB/s | 230 kB 00:00 2024-09-12T01:34:43.912 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:34:43.930 INFO:teuthology.orchestra.run.smithi151.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T01:34:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:34:43.964 INFO:teuthology.orchestra.run.smithi151.stdout:(143/156): python3-werkzeug-2.0.3-3.el9.1.noarc 12 MB/s | 427 kB 00:00 2024-09-12T01:34:43.983 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:34:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-12T01:34:44.012 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:34:44.015 INFO:teuthology.orchestra.run.smithi151.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.4 MB/s | 191 kB 00:00 2024-09-12T01:34:44.073 INFO:teuthology.orchestra.run.smithi151.stdout:(146/156): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-12T01:34:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:34:44.103 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:34:44.133 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:34:44.294 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:34:44.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:34:44.346 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:34:44.374 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:34:44.416 INFO:teuthology.orchestra.run.smithi151.stdout:(147/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-09-12T01:34:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:34:44.478 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:34:44.500 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:34:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:34:44.563 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:34:44.593 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:34:44.593 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:34:44.618 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:34:44.633 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:34:44.659 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:34:44.683 INFO:teuthology.orchestra.run.smithi151.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 5.6 MB/s | 3.4 MB 00:00 2024-09-12T01:34:44.698 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:34:44.777 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:34:44.812 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:34:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:34:44.924 INFO:teuthology.orchestra.run.smithi151.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.2 MB/s | 3.2 MB 00:00 2024-09-12T01:34:44.940 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:34:44.968 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:34:44.986 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:34:45.006 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:34:45.042 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:34:45.074 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:34:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:34:45.117 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:34:45.138 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:34:45.171 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:34:45.213 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:34:45.217 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:45.247 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:34:45.301 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:45.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:34:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:34:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:34:45.521 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:34:45.533 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:45.568 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:34:45.641 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:34:45.736 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:34:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-12T01:34:45.773 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:34:45.807 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:34:45.872 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:34:45.917 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:34:45.957 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:34:46.001 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:34:46.034 INFO:teuthology.orchestra.run.smithi151.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-09-12T01:34:46.034 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:34:46.061 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:34:46.131 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:34:46.184 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:34:46.215 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:34:46.258 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:34:46.285 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:34:46.332 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:34:46.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:34:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:34:46.368 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:34:46.401 INFO:teuthology.orchestra.run.smithi151.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 65 kB/s | 41 kB 00:00 2024-09-12T01:34:46.412 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:34:46.441 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:34:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:34:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:34:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:34:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:34:46.565 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:34:46.629 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:34:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:34:46.719 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:34:46.763 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:34:46.801 INFO:teuthology.orchestra.run.smithi151.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 328 kB/s | 249 kB 00:00 2024-09-12T01:34:46.816 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:34:46.901 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:34:46.933 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:34:47.010 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:34:47.085 INFO:teuthology.orchestra.run.smithi151.stdout:(154/156): python3-babel-2.9.1-2.el9.noarch.rpm 672 kB/s | 6.0 MB 00:09 2024-09-12T01:34:47.348 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:34:47.436 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:34:47.443 INFO:teuthology.orchestra.run.smithi151.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-12T01:34:47.466 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:34:47.508 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:34:47.546 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:34:47.589 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:34:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:34:47.745 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:34:47.791 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:34:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:34:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:34:47.877 INFO:teuthology.orchestra.run.smithi151.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-09-12T01:34:47.882 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:34:47.882 INFO:teuthology.orchestra.run.smithi151.stdout:Total 8.1 MB/s | 226 MB 00:27 2024-09-12T01:34:47.935 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:34:47.967 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:34:48.003 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:34:48.050 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:34:48.086 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:34:48.129 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:34:48.172 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:34:48.308 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:34:48.349 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:34:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:34:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:34:48.533 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:34:48.708 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:34:48.789 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:34:48.814 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:34:48.907 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:34:49.005 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:34:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:34:49.102 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:34:49.103 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:34:49.121 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-09-12T01:34:49.131 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:34:49.164 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:34:49.257 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:34:49.272 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-09-12T01:34:49.272 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-09-12T01:34:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:34:49.473 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:34:49.510 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:34:49.560 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:34:49.599 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:34:49.601 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:34:49.629 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:34:49.632 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:34:49.664 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:34:49.667 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:34:49.697 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:34:49.739 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:34:49.773 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:34:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:34:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:34:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:34:50.036 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:34:50.085 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:34:50.152 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:34:50.188 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:34:50.260 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:34:50.290 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:34:50.804 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:34:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:34:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:34:50.981 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:34:51.029 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:34:51.074 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:34:51.085 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-09-12T01:34:51.085 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-09-12T01:34:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:34:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:34:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:34:51.434 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:34:51.521 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:34:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:34:52.291 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:34:52.897 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:34:52.937 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:34:53.483 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:34:53.525 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:34:53.532 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:34:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:34:53.623 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:34:53.623 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:34:53.623 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:34:53.623 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:34:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:34:53.658 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:34:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:34:53.984 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:34:54.029 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:34:54.104 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:34:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:34:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:34:54.761 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:34:54.965 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:34:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:34:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:34:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:34:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:34:57.477 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-09-12T01:34:57.564 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T01:34:57.603 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T01:34:57.656 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T01:34:57.690 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T01:34:57.724 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T01:34:57.767 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T01:34:57.981 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:34:58.038 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:34:58.128 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:34:58.128 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:34:58.129 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:34:58.129 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:34:58.161 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T01:34:58.162 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:34:58.235 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:34:58.357 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:34:58.386 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:34:58.468 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:34:58.499 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:34:58.648 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T01:34:58.713 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T01:34:58.750 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:34:58.854 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:34:58.885 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:34:58.947 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T01:34:58.978 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:34:59.002 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T01:34:59.195 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:34:59.332 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T01:34:59.461 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T01:34:59.549 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:59.629 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:59.844 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T01:34:59.960 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T01:35:00.054 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T01:35:00.093 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T01:35:00.129 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T01:35:00.202 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T01:35:00.246 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T01:35:00.278 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T01:35:00.331 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T01:35:00.363 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T01:35:00.392 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T01:35:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T01:35:00.521 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T01:35:00.560 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T01:35:00.603 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T01:35:00.680 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T01:35:00.715 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T01:35:00.758 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T01:35:00.789 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T01:35:00.859 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T01:35:00.903 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T01:35:00.985 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T01:35:01.127 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T01:35:01.181 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T01:35:01.705 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T01:35:01.785 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T01:35:01.815 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T01:35:01.858 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T01:35:01.896 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T01:35:01.938 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T01:35:02.115 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T01:35:02.238 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T01:35:02.283 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T01:35:02.370 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T01:35:02.412 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T01:35:02.442 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T01:35:02.528 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T01:35:02.529 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T01:35:02.569 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T01:35:02.608 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T01:35:02.755 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T01:35:02.799 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T01:35:02.849 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T01:35:02.951 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T01:35:02.983 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T01:35:03.023 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T01:35:03.049 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T01:35:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T01:35:03.148 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T01:35:03.276 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T01:35:03.316 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T01:35:03.343 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T01:35:03.379 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T01:35:03.418 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T01:35:03.449 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T01:35:03.480 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T01:35:03.515 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T01:35:03.548 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T01:35:03.598 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T01:35:03.633 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T01:35:03.720 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T01:35:03.763 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T01:35:03.795 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T01:35:03.899 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T01:35:03.955 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T01:35:04.683 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T01:35:04.737 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T01:35:04.771 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T01:35:04.870 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T01:35:04.927 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T01:35:04.971 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T01:35:05.059 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T01:35:05.125 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T01:35:05.312 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T01:35:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-09-12T01:35:05.406 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-09-12T01:35:05.407 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:05.446 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T01:35:05.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:35:05.605 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:35:05.606 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:05.606 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:35:05.606 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:06.094 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T01:35:06.214 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T01:35:07.106 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:35:07.134 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:35:07.134 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:07.134 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:35:07.134 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:35:07.134 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:35:07.134 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:07.357 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:35:07.380 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:35:07.381 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:07.381 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:35:07.381 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:35:07.381 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:35:07.381 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T01:35:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T01:35:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T01:35:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T01:35:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:35:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:35:07.831 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:07.831 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:35:07.831 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:35:07.831 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:35:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T01:35:07.954 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:35:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:35:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:35:08.040 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T01:35:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:35:08.071 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T01:35:08.107 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T01:35:08.249 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:35:08.300 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:35:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:35:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:35:08.585 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T01:35:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:35:08.611 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:35:08.641 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:35:08.673 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:35:08.682 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T01:35:08.701 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:35:08.713 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T01:35:09.185 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:35:09.217 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:35:09.256 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:35:09.318 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:35:09.348 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-09-12T01:35:09.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-09-12T01:35:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:10.016 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:35:10.032 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:35:10.032 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:10.032 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:35:10.032 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:10.387 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:35:10.416 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:35:10.976 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:35:11.228 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:35:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:35:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:35:11.511 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:35:11.519 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:35:11.545 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:35:11.545 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:11.545 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:35:11.546 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:35:11.546 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:35:11.546 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:35:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:35:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:35:11.718 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:11.718 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:35:11.718 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:35:11.718 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:35:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:35:11.774 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T01:35:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:35:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:35:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:35:11.793 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:11.793 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:35:11.793 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:35:11.793 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:35:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:11.824 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:35:11.887 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:35:11.925 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T01:35:11.926 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T01:35:11.926 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T01:35:11.926 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:11.935 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:35:11.959 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:35:12.229 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:35:12.252 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:35:12.252 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:12.252 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:35:12.252 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:35:12.253 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:35:12.253 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:12.360 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:35:12.416 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:35:12.471 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:35:12.657 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:35:12.713 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:35:12.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:35:12.856 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:35:12.888 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:35:12.896 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:35:12.912 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:35:12.913 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:12.913 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:35:12.913 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:35:12.913 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:35:12.913 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:12.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:35:12.978 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:35:12.979 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:12.979 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:35:12.979 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:13.023 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:35:13.037 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:35:13.046 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:35:13.047 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:13.047 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:35:13.047 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:35:13.047 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:35:13.047 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:13.080 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:35:13.112 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:35:13.140 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:35:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:35:13.373 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:35:13.373 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:13.373 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:35:13.373 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:35:13.373 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:35:13.373 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:35:13.555 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:35:13.587 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:35:13.649 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:35:13.678 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:35:14.718 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:35:14.755 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:35:15.728 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:35:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:35:15.874 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:35:16.035 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:35:16.066 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:35:16.089 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:35:16.089 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:16.090 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:35:16.090 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:35:16.090 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:35:16.090 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:16.141 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:35:16.153 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:35:16.250 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:35:16.306 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:35:17.231 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:35:17.261 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:35:17.284 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:35:17.284 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:17.284 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:35:17.285 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:35:17.285 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:35:17.285 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:17.328 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:35:17.351 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:35:17.351 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:17.351 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:35:17.351 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:17.395 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:35:17.418 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:35:17.418 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:17.418 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:35:17.418 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:35:17.418 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:35:17.418 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:17.721 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:35:17.744 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:35:17.744 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:17.744 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:35:17.744 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:35:17.744 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:35:17.745 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:17.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:35:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:35:17.976 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:35:18.008 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:35:18.036 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:35:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:35:18.266 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:35:18.296 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:35:18.326 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:35:18.372 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:35:18.995 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:19.036 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:19.057 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:19.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:35:19.991 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:35:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:35:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:35:20.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:35:20.169 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:35:20.195 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:35:20.195 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:35:20.288 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:35:20.288 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:35:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:35:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:35:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:20.582 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:20.582 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:35:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:35:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:22.325 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:35:22.374 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:35:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:35:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:35:22.483 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:35:22.673 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:35:22.713 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:35:22.743 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:35:22.773 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:35:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:35:23.400 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:23.434 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:23.454 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:23.615 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-09-12T01:35:23.616 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:23.806 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:35:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T01:35:23.823 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:23.823 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T01:35:23.823 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:35:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:35:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:35:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:35:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:35:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:35:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:35:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:35:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:35:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:35:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:35:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:35:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:35:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:35:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:25.302 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:35:25.329 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T01:35:25.329 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:25.329 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T01:35:25.330 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:35:25.330 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T01:35:25.330 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:25.561 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:35:25.585 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T01:35:25.585 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:25.585 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T01:35:25.585 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:35:25.585 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T01:35:25.585 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:26.020 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:35:26.044 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T01:35:26.044 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:26.044 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T01:35:26.045 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:35:26.045 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T01:35:26.045 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:35:26.115 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:35:26.116 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:35:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:35:26.130 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:35:26.131 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:35:26.132 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:35:26.133 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:35:26.134 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:35:26.135 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:35:26.136 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:35:26.137 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:35:26.138 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:35:26.139 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:35:26.140 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:35:26.141 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:35:26.141 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:35:26.150 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T01:35:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T01:35:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T01:35:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T01:35:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T01:35:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T01:35:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T01:35:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T01:35:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T01:35:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T01:35:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T01:35:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:35:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T01:35:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T01:35:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T01:35:27.475 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:35:28.516 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T01:35:28.560 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:35:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:35:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:35:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:35:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:35:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:35:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:35:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:35:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:35:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:35:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:35:28.683 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-12T01:35:29.086 DEBUG:teuthology.parallel:result is None 2024-09-12T01:35:29.525 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T01:35:29.561 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:35:29.664 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T01:35:29.824 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T01:35:29.855 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:35:29.880 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T01:35:29.881 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:29.881 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T01:35:29.881 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:35:29.881 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T01:35:29.881 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:29.931 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:35:29.943 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T01:35:30.040 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T01:35:30.096 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T01:35:31.020 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:31.020 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:35:31.020 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:35:31.020 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:35:31.020 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:35:31.021 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:35:31.022 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:35:31.034 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T01:35:31.035 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:35:31.035 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:35:31.035 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:35:31.036 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:35:31.037 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:35:31.038 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:35:31.039 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:35:31.040 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:35:31.041 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:35:31.042 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:35:31.043 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:35:31.044 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:35:31.045 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:35:31.046 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:35:31.059 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:35:31.084 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T01:35:31.084 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:31.084 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T01:35:31.084 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:35:31.084 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T01:35:31.084 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:35:31.150 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T01:35:31.151 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:31.151 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T01:35:31.151 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:31.203 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:35:31.227 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T01:35:31.227 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:31.227 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T01:35:31.227 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:35:31.227 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T01:35:31.227 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:31.537 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:35:31.561 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T01:35:31.561 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T01:35:31.561 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T01:35:31.561 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:35:31.561 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T01:35:31.561 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:33.791 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:33.791 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:33.791 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-09-12T01:35:33.791 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:33.791 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:35:33.792 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.793 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:35:33.794 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:35:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:35:33.796 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:35:33.797 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.798 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:35:33.799 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:35:33.800 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:35:33.801 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:35:33.801 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:35:33.801 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:35:33.801 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:35:33.801 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:35:33.801 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:35:33.802 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:35:33.803 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:35:33.804 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:35:33.805 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:35:33.805 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:35:33.805 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-12T01:35:34.204 DEBUG:teuthology.parallel:result is None 2024-09-12T01:35:36.117 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T01:35:36.167 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T01:35:36.216 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T01:35:36.249 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T01:35:36.276 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T01:35:36.456 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T01:35:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T01:35:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T01:35:36.568 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T01:35:36.623 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T01:35:37.189 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:37.231 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:37.253 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T01:35:38.084 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T01:35:38.181 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T01:35:38.222 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T01:35:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T01:35:38.337 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T01:35:38.362 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T01:35:38.396 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T01:35:38.396 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:35:38.497 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T01:35:38.497 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:35:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T01:35:38.606 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T01:35:38.611 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:38.612 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:38.791 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T01:35:38.791 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:35:38.883 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T01:35:38.946 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:44.877 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:44.877 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T01:35:44.877 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T01:35:44.877 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T01:35:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T01:35:44.879 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T01:35:44.880 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T01:35:44.892 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T01:35:44.893 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T01:35:44.894 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T01:35:44.895 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T01:35:44.896 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T01:35:44.897 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T01:35:44.898 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T01:35:44.899 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T01:35:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T01:35:44.901 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T01:35:44.902 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T01:35:47.608 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-09-12T01:35:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.610 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.611 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T01:35:47.612 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.613 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T01:35:47.614 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T01:35:47.615 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T01:35:47.616 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T01:35:47.617 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T01:35:47.618 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T01:35:47.619 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T01:35:47.620 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T01:35:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T01:35:47.622 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T01:35:47.622 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:35:47.622 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-09-12T01:35:48.061 DEBUG:teuthology.parallel:result is None 2024-09-12T01:35:48.063 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:35:48.213 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:35:48.240 INFO:teuthology.orchestra.run.smithi122.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:35:48.241 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:35:48.241 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:35:48.242 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:35:48.402 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:35:48.431 INFO:teuthology.orchestra.run.smithi151.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:35:48.431 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:35:48.432 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:35:48.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:35:48.580 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T01:35:48.607 INFO:teuthology.orchestra.run.smithi183.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T01:35:48.608 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T01:35:48.608 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T01:35:48.609 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T01:35:48.610 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:35:48.610 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:35:48.643 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:35:48.643 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:35:48.673 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:35:48.673 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T01:35:48.705 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T01:35:48.705 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:35:48.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:35:48.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:35:48.802 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:35:48.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:35:48.830 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:35:48.896 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:35:48.896 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T01:35:48.926 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T01:35:48.996 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T01:35:48.996 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:35:48.996 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:35:49.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:35:49.087 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:35:49.087 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:35:49.115 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:35:49.183 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:35:49.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T01:35:49.208 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T01:35:49.274 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T01:35:49.274 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:35:49.274 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:35:49.301 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:35:49.367 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:35:49.367 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:35:49.396 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:35:49.464 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:35:49.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T01:35:49.489 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T01:35:49.554 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T01:35:49.650 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-12T01:35:49.651 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:35:49.651 INFO:tasks.cephadm:Cluster fsid is 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:35:49.651 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T01:35:49.652 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.122', 'mon.b': '172.21.15.151', 'mon.c': '172.21.15.183'} 2024-09-12T01:35:49.652 INFO:tasks.cephadm:First mon is mon.a on smithi122 2024-09-12T01:35:49.652 INFO:tasks.cephadm:First mgr is x 2024-09-12T01:35:49.652 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T01:35:49.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-09-12T01:35:49.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-09-12T01:35:49.708 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-12T01:35:49.733 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T01:35:49.733 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:35:49.888 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T01:35:50.040 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:35:50.042 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T01:35:50.042 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T01:35:50.042 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:50.430 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:35 /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:50.431 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:50.805 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:35 /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:50.805 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:51.182 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 01:35 /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:51.182 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:51.202 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:51.222 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T01:35:51.257 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T01:35:51.257 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:35:51.259 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:35:51.264 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T01:35:52.788 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:35:52.868 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:35:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:36:20.879 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:36:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:36:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:36:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-09-12T01:36:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:36:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T01:36:20.880 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:36:40.017 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-12T01:36:40.017 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:36:40.018 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:36:40.018 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-09-12T01:36:40.018 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:36:40.018 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-12T01:36:40.018 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-12T01:36:40.049 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T01:36:40.078 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-09-12T01:36:40.112 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-09-12T01:36:40.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-12T01:36:40.169 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-09-12T01:36:40.197 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-09-12T01:36:40.222 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-12T01:36:40.248 INFO:tasks.cephadm:Writing seed config... 2024-09-12T01:36:40.249 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T01:36:40.250 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T01:36:40.251 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T01:36:40.252 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T01:36:40.253 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T01:36:40.254 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:36:40.254 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T01:36:40.272 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5731f69a-70a7-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T01:36:40.272 DEBUG:teuthology.orchestra.run.smithi122:mon.a> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T01:36:40.315 DEBUG:teuthology.orchestra.run.smithi122:mgr.x> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T01:36:40.358 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T01:36:40.358 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.122 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:36:40.565 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:36:40.565 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '5731f69a-70a7-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.122', '--skip-admin-label'] 2024-09-12T01:36:40.566 INFO:teuthology.orchestra.run.smithi122.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T01:36:40.566 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying podman|docker is present... 2024-09-12T01:36:40.591 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 5.2.2 2024-09-12T01:36:40.591 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying lvm2 is present... 2024-09-12T01:36:40.591 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying time synchronization is in place... 2024-09-12T01:36:40.599 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T01:36:40.599 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T01:36:40.605 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T01:36:40.605 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-12T01:36:40.612 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-09-12T01:36:40.618 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-09-12T01:36:40.618 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-09-12T01:36:40.618 INFO:teuthology.orchestra.run.smithi122.stdout:Repeating the final host check... 2024-09-12T01:36:40.641 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 5.2.2 2024-09-12T01:36:40.641 INFO:teuthology.orchestra.run.smithi122.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T01:36:40.641 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl is present 2024-09-12T01:36:40.641 INFO:teuthology.orchestra.run.smithi122.stdout:lvcreate is present 2024-09-12T01:36:40.647 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T01:36:40.647 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T01:36:40.654 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T01:36:40.654 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-12T01:36:40.660 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-09-12T01:36:40.667 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-09-12T01:36:40.667 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-09-12T01:36:40.667 INFO:teuthology.orchestra.run.smithi122.stdout:Host looks OK 2024-09-12T01:36:40.667 INFO:teuthology.orchestra.run.smithi122.stdout:Cluster fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:36:40.667 INFO:teuthology.orchestra.run.smithi122.stdout:Acquiring lock 140160616588288 on /run/cephadm/5731f69a-70a7-11ef-bcea-c7b262605968.lock 2024-09-12T01:36:40.668 INFO:teuthology.orchestra.run.smithi122.stdout:Lock 140160616588288 acquired on /run/cephadm/5731f69a-70a7-11ef-bcea-c7b262605968.lock 2024-09-12T01:36:40.668 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 3300 ... 2024-09-12T01:36:40.668 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 6789 ... 2024-09-12T01:36:40.668 INFO:teuthology.orchestra.run.smithi122.stdout:Base mon IP(s) is [172.21.15.122:3300, 172.21.15.122:6789], mon addrv is [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-09-12T01:36:40.671 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.122 metric 100 2024-09-12T01:36:40.671 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.122 metric 100 2024-09-12T01:36:40.674 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T01:36:40.674 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-12T01:36:40.674 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d515/64 scope link noprefixroute 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-09-12T01:36:40.677 INFO:teuthology.orchestra.run.smithi122.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T01:36:40.678 INFO:teuthology.orchestra.run.smithi122.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T01:36:40.678 INFO:teuthology.orchestra.run.smithi122.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T01:36:41.188 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T01:36:42.167 INFO:teuthology.orchestra.run.smithi122.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T01:36:42.167 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T01:36:42.167 INFO:teuthology.orchestra.run.smithi122.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T01:36:43.039 INFO:teuthology.orchestra.run.smithi122.stdout:stat: stdout 167 167 2024-09-12T01:36:43.039 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial keys... 2024-09-12T01:36:43.979 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQCrReJmG5G9FRAAvRvM5BvE/ivEX4OV6mlN4Q== 2024-09-12T01:36:45.126 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQCsReJmpc9iEhAATJYTHHoGhyh9+9sj1Syl5Q== 2024-09-12T01:36:46.148 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQCtReJm0adnHBAAYllZ9x5avtsmJugkwgZ4qg== 2024-09-12T01:36:46.149 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial monmap... 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T01:36:47.097 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool for a [v2:172.21.15.122:3300,v1:172.21.15.122:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: set fsid to 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:36:47.098 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mon... 2024-09-12T01:36:48.082 INFO:teuthology.orchestra.run.smithi122.stdout:create mon.a on 2024-09-12T01:36:48.295 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T01:36:48.437 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T01:36:48.608 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5731f69a-70a7-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-5731f69a-70a7-11ef-bcea-c7b262605968.target. 2024-09-12T01:36:48.608 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5731f69a-70a7-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-5731f69a-70a7-11ef-bcea-c7b262605968.target. 2024-09-12T01:36:48.837 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a 2024-09-12T01:36:48.838 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T01:36:49.140 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5731f69a-70a7-11ef-bcea-c7b262605968.target.wants/ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5731f69a-70a7-11ef-bcea-c7b262605968@.service. 2024-09-12T01:36:52.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:52 smithi122 ceph-mon[30559]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:36:53.233 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:36:53.233 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-12T01:36:53.241 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:36:53.241 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-12T01:36:53.241 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-12T01:36:53.241 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T01:36:53.242 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon to start... 2024-09-12T01:36:53.242 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon... 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 bash[30529]: 1797d6b18c0eeb68fb2b35b7317485b06cf88e2d88ec4940745e91bfc4746d84 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 systemd[1]: Started Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: monmap epoch 1 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: last_changed 2024-09-12T01:36:46.446315+0000 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: min_mon_release 19 (squid) 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: election_strategy: 1 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:36:53.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: fsmap 2024-09-12T01:36:53.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T01:36:53.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:53 smithi122 ceph-mon[30559]: mgrmap e1: no daemons active 2024-09-12T01:36:56.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:36:56 smithi122 ceph-mon[30559]: from='client.? 172.21.15.122:0/2989150136' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout id: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout services: 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T01:37:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout data: 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.604+0000 7f18ca96d640 1 Processor -- start 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.604+0000 7f18ca96d640 1 -- start start 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.604+0000 7f18ca96d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c4107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.604+0000 7f18ca96d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4108050 con 0x7f18c4107680 2024-09-12T01:37:01.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.604+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c4107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:01.299 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.605+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c4107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36798/0 (socket says 172.21.15.122:36798) 2024-09-12T01:37:01.299 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.605+0000 7f18c3fff640 1 -- 172.21.15.122:0/1691957459 learned_addr learned my addr 172.21.15.122:0/1691957459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:01.299 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.605+0000 7f18c3fff640 1 -- 172.21.15.122:0/1691957459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c4108830 con 0x7f18c4107680 2024-09-12T01:37:01.299 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.607+0000 7f18c3fff640 1 --2- 172.21.15.122:0/1691957459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c4107a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009820 tx=0x7f18b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac60a4eac856694a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:01.299 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.607+0000 7f18c37fe640 1 -- 172.21.15.122:0/1691957459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4037070 con 0x7f18c4107680 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.607+0000 7f18c37fe640 1 -- 172.21.15.122:0/1691957459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b4032440 con 0x7f18c4107680 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.607+0000 7f18c37fe640 1 -- 172.21.15.122:0/1691957459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4032740 con 0x7f18c4107680 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 -- 172.21.15.122:0/1691957459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 msgr2=0x7f18c4107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 --2- 172.21.15.122:0/1691957459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c4107a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009820 tx=0x7f18b402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 -- 172.21.15.122:0/1691957459 shutdown_connections 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 --2- 172.21.15.122:0/1691957459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c4107a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 -- 172.21.15.122:0/1691957459 >> 172.21.15.122:0/1691957459 conn(0x7f18c4102e50 msgr2=0x7f18c4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 -- 172.21.15.122:0/1691957459 shutdown_connections 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 -- 172.21.15.122:0/1691957459 wait complete. 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 Processor -- start 2024-09-12T01:37:01.300 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.608+0000 7f18ca96d640 1 -- start start 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18ca96d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18ca96d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4108dd0 con 0x7f18c4107680 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36810/0 (socket says 172.21.15.122:36810) 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c3fff640 1 -- 172.21.15.122:0/2989150136 learned_addr learned my addr 172.21.15.122:0/2989150136 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c3fff640 1 -- 172.21.15.122:0/2989150136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c407e0b0 con 0x7f18c4107680 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c3fff640 1 --2- 172.21.15.122:0/2989150136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c407db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f18b40090b0 tx=0x7f18b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4041070 con 0x7f18c4107680 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b403c7a0 con 0x7f18c4107680 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18b4037040 con 0x7f18c4107680 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.609+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c407c2a0 con 0x7f18c4107680 2024-09-12T01:37:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.610+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c407c7e0 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.610+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f18b4044070 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.610+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18b403d3f0 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.610+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c4109420 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.613+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f18b4046450 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.674+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f18c407cf40 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.674+0000 7f18c1ffb640 1 -- 172.21.15.122:0/2989150136 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f18b4046630 con 0x7f18c4107680 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 msgr2=0x7f18c407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 --2- 172.21.15.122:0/2989150136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c407db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f18b40090b0 tx=0x7f18b40046c0 comp rx=0 tx=0).stop 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 shutdown_connections 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 --2- 172.21.15.122:0/2989150136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4107680 0x7f18c407db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 >> 172.21.15.122:0/2989150136 conn(0x7f18c4102e50 msgr2=0x7f18c418f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:01.302 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 shutdown_connections 2024-09-12T01:37:01.303 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:36:56.675+0000 7f18ca96d640 1 -- 172.21.15.122:0/2989150136 wait complete. 2024-09-12T01:37:01.303 INFO:teuthology.orchestra.run.smithi122.stdout:mon is available 2024-09-12T01:37:01.303 INFO:teuthology.orchestra.run.smithi122.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T01:37:02.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:01 smithi122 ceph-mon[30559]: from='client.? 172.21.15.122:0/3747893430' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T01:37:02.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:01 smithi122 ceph-mon[30559]: from='client.? 172.21.15.122:0/3747893430' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T01:37:02.337 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b948a9640 1 Processor -- start 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b948a9640 1 -- start start 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b948a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b90106990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b948a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90106f60 con 0x7f0b90106590 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b8e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b90106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:02.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b8e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b90106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52620/0 (socket says 172.21.15.122:52620) 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.727+0000 7f0b8e59c640 1 -- 172.21.15.122:0/631291945 learned_addr learned my addr 172.21.15.122:0/631291945 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.728+0000 7f0b8e59c640 1 -- 172.21.15.122:0/631291945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b90107790 con 0x7f0b90106590 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.728+0000 7f0b8e59c640 1 --2- 172.21.15.122:0/631291945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b90106990 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009820 tx=0x7f0b8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4249e7f2fb38345 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.728+0000 7f0b8dd9b640 1 -- 172.21.15.122:0/631291945 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80037070 con 0x7f0b90106590 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.728+0000 7f0b8dd9b640 1 -- 172.21.15.122:0/631291945 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80032440 con 0x7f0b90106590 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.728+0000 7f0b8dd9b640 1 -- 172.21.15.122:0/631291945 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80032740 con 0x7f0b90106590 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 -- 172.21.15.122:0/631291945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 msgr2=0x7f0b90106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 --2- 172.21.15.122:0/631291945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b90106990 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009820 tx=0x7f0b8002a270 comp rx=0 tx=0).stop 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 -- 172.21.15.122:0/631291945 shutdown_connections 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 --2- 172.21.15.122:0/631291945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b90106990 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 -- 172.21.15.122:0/631291945 >> 172.21.15.122:0/631291945 conn(0x7f0b90101de0 msgr2=0x7f0b90104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:02.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 -- 172.21.15.122:0/631291945 shutdown_connections 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.729+0000 7f0b948a9640 1 -- 172.21.15.122:0/631291945 wait complete. 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b948a9640 1 Processor -- start 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b948a9640 1 -- start start 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b948a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b9019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b948a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90107cc0 con 0x7f0b90106590 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b8e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b9019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b8e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b9019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52634/0 (socket says 172.21.15.122:52634) 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.730+0000 7f0b8e59c640 1 -- 172.21.15.122:0/3747893430 learned_addr learned my addr 172.21.15.122:0/3747893430 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b8e59c640 1 -- 172.21.15.122:0/3747893430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9019b330 con 0x7f0b90106590 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b8e59c640 1 --2- 172.21.15.122:0/3747893430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b9019adf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b800090b0 tx=0x7f0b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80041070 con 0x7f0b90106590 2024-09-12T01:37:02.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8003c7a0 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b80037040 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9019b5c0 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9019e2b0 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b80044070 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.731+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b8003d3f0 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.732+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90108310 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.734+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0b80046450 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.791+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0b9002d940 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.794+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f0b80046630 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.795+0000 7f0b87fff640 1 -- 172.21.15.122:0/3747893430 <== mon.0 v2:172.21.15.122:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b8003d5f0 con 0x7f0b90106590 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 msgr2=0x7f0b9019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:02.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 --2- 172.21.15.122:0/3747893430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b9019adf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b800090b0 tx=0x7f0b800046c0 comp rx=0 tx=0).stop 2024-09-12T01:37:02.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 shutdown_connections 2024-09-12T01:37:02.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 --2- 172.21.15.122:0/3747893430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b90106590 0x7f0b9019adf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:02.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 >> 172.21.15.122:0/3747893430 conn(0x7f0b90101de0 msgr2=0x7f0b90191af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:02.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 shutdown_connections 2024-09-12T01:37:02.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:01.796+0000 7f0b948a9640 1 -- 172.21.15.122:0/3747893430 wait complete. 2024-09-12T01:37:02.342 INFO:teuthology.orchestra.run.smithi122.stdout:Generating new minimal ceph.conf... 2024-09-12T01:37:03.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:02 smithi122 ceph-mon[30559]: from='client.? 172.21.15.122:0/3308717251' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:37:03.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc081655640 1 Processor -- start 2024-09-12T01:37:03.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc081655640 1 -- start start 2024-09-12T01:37:03.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc081655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:03.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc081655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c1084e0 con 0x7fc07c107b10 2024-09-12T01:37:03.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc07affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc07affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52642/0 (socket says 172.21.15.122:52642) 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.749+0000 7fc07affd640 1 -- 172.21.15.122:0/4122914215 learned_addr learned my addr 172.21.15.122:0/4122914215 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.750+0000 7fc07affd640 1 -- 172.21.15.122:0/4122914215 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c106770 con 0x7fc07c107b10 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.750+0000 7fc07affd640 1 --2- 172.21.15.122:0/4122914215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c107f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009820 tx=0x7fc06c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adeeb03d6b45ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.750+0000 7fc079ffb640 1 -- 172.21.15.122:0/4122914215 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc06c037070 con 0x7fc07c107b10 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.750+0000 7fc079ffb640 1 -- 172.21.15.122:0/4122914215 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc06c032900 con 0x7fc07c107b10 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.750+0000 7fc079ffb640 1 -- 172.21.15.122:0/4122914215 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc06c030850 con 0x7fc07c107b10 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 -- 172.21.15.122:0/4122914215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 msgr2=0x7fc07c107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 --2- 172.21.15.122:0/4122914215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c107f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009820 tx=0x7fc06c02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 -- 172.21.15.122:0/4122914215 shutdown_connections 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 --2- 172.21.15.122:0/4122914215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c107f10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:03.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 -- 172.21.15.122:0/4122914215 >> 172.21.15.122:0/4122914215 conn(0x7fc07c101470 msgr2=0x7fc07c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 -- 172.21.15.122:0/4122914215 shutdown_connections 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.751+0000 7fc081655640 1 -- 172.21.15.122:0/4122914215 wait complete. 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc081655640 1 Processor -- start 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc081655640 1 -- start start 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc081655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc081655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10ab30 con 0x7fc07c107b10 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc07affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc07affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52646/0 (socket says 172.21.15.122:52646) 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc07affd640 1 -- 172.21.15.122:0/3308717251 learned_addr learned my addr 172.21.15.122:0/3308717251 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc07affd640 1 -- 172.21.15.122:0/3308717251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c19aa80 con 0x7fc07c107b10 2024-09-12T01:37:03.360 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.752+0000 7fc07affd640 1 --2- 172.21.15.122:0/3308717251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c19a540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009940 tx=0x7fc06c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc06c042070 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c19ad10 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc06c0308c0 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc06c037040 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c19da00 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc06c045070 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.753+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc06c047bc0 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.754+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c107f90 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.756+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc06c0327b0 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.814+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc07c19df70 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc063fff640 1 -- 172.21.15.122:0/3308717251 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fc06c03c870 con 0x7fc07c107b10 2024-09-12T01:37:03.361 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 msgr2=0x7fc07c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc081655640 1 --2- 172.21.15.122:0/3308717251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c19a540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009940 tx=0x7fc06c032ce0 comp rx=0 tx=0).stop 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 shutdown_connections 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc081655640 1 --2- 172.21.15.122:0/3308717251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc07c107b10 0x7fc07c19a540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 >> 172.21.15.122:0/3308717251 conn(0x7fc07c101470 msgr2=0x7fc07c102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.815+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 shutdown_connections 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:02.816+0000 7fc081655640 1 -- 172.21.15.122:0/3308717251 wait complete. 2024-09-12T01:37:03.362 INFO:teuthology.orchestra.run.smithi122.stdout:Restarting the monitor... 2024-09-12T01:37:03.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:03 smithi122 systemd[1]: Stopping Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:37:03.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30555]: 2024-09-12T01:37:03.630+0000 7f2dc894e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:37:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30555]: 2024-09-12T01:37:03.630+0000 7f2dc894e640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T01:37:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:03 smithi122 podman[30801]: 2024-09-12 01:37:03.825325803 +0000 UTC m=+0.309030201 container died 1797d6b18c0eeb68fb2b35b7317485b06cf88e2d88ec4940745e91bfc4746d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T01:37:04.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 podman[30801]: 2024-09-12 01:37:04.080385176 +0000 UTC m=+0.564089577 container cleanup 1797d6b18c0eeb68fb2b35b7317485b06cf88e2d88ec4940745e91bfc4746d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-12T01:37:04.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 bash[30801]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a 2024-09-12T01:37:04.479 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 podman[30813]: 2024-09-12 01:37:04.248080511 +0000 UTC m=+0.416110521 container remove 1797d6b18c0eeb68fb2b35b7317485b06cf88e2d88ec4940745e91bfc4746d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T01:37:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T01:37:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 systemd[1]: Stopped Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:37:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 systemd[1]: Starting Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:37:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 podman[30906]: 2024-09-12 01:37:04.682145264 +0000 UTC m=+0.091581713 container create 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:37:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 podman[30906]: 2024-09-12 01:37:04.606245466 +0000 UTC m=+0.015681916 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:37:04.962 INFO:teuthology.orchestra.run.smithi122.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T01:37:05.077 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 podman[30906]: 2024-09-12 01:37:04.884640401 +0000 UTC m=+0.294076851 container init 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 podman[30906]: 2024-09-12 01:37:04.888295323 +0000 UTC m=+0.297731774 container start 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: pidfile_write: ignore empty --pid-file 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: load: jerasure load: lrc 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: RocksDB version: 7.9.2 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Git sha 0 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: DB SUMMARY 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: DB Session ID: 1XN0LJDFICUD2AVFCPZ0 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: CURRENT file: CURRENT 2024-09-12T01:37:05.078 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105737 ; 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.error_if_exists: 0 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.create_if_missing: 0 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.paranoid_checks: 1 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.env: 0x558764b3f940 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.info_log: 0x55876682d6e0 2024-09-12T01:37:05.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.statistics: (nil) 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.use_fsync: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_log_file_size: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.allow_fallocate: 1 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.use_direct_reads: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T01:37:05.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.db_log_dir: 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.wal_dir: 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T01:37:05.081 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.write_buffer_manager: 0x55876683cdc0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.unordered_write: 0 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T01:37:05.082 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.row_cache: None 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.wal_filter: None 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.two_write_queues: 0 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.wal_compression: 0 2024-09-12T01:37:05.083 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.atomic_flush: 0 2024-09-12T01:37:05.084 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T01:37:05.084 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T01:37:05.084 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T01:37:05.084 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.log_readahead_size: 0 2024-09-12T01:37:05.084 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T01:37:05.084 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_background_jobs: 2 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_background_compactions: -1 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_subcompactions: 1 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T01:37:05.085 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_open_files: -1 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_background_flushes: -1 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Compression algorithms supported: 2024-09-12T01:37:05.086 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kZSTD supported: 0 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kXpressCompression supported: 0 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kBZip2Compression supported: 0 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kLZ4Compression supported: 1 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kZlibCompression supported: 1 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: kSnappyCompression supported: 1 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.merge_operator: 2024-09-12T01:37:05.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_filter: None 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55876682d320) 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_type: 0 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_index_type: 0 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_shortening: 1 2024-09-12T01:37:05.088 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: checksum: 4 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: no_block_cache: 0 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache: 0x5587668211f0 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_options: 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: capacity : 536870912 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_shard_bits : 4 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: strict_capacity_limit : 0 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-09-12T01:37:05.089 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_compressed: (nil) 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: persistent_cache: (nil) 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size: 4096 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size_deviation: 10 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_restart_interval: 16 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_block_restart_interval: 1 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: metadata_block_size: 4096 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: partition_filters: 0 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: use_delta_encoding: 1 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: filter_policy: bloomfilter 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: whole_key_filtering: 1 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: verify_compression: 0 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-09-12T01:37:05.090 INFO:journalctl@ceph.mon.a.smithi122.stdout: format_version: 5 2024-09-12T01:37:05.091 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_index_compression: 1 2024-09-12T01:37:05.091 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_align: 0 2024-09-12T01:37:05.091 INFO:journalctl@ceph.mon.a.smithi122.stdout: max_auto_readahead_size: 262144 2024-09-12T01:37:05.091 INFO:journalctl@ceph.mon.a.smithi122.stdout: prepopulate_block_cache: 0 2024-09-12T01:37:05.091 INFO:journalctl@ceph.mon.a.smithi122.stdout: initial_auto_readahead_size: 8192 2024-09-12T01:37:05.091 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T01:37:05.092 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T01:37:05.092 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T01:37:05.092 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression: NoCompression 2024-09-12T01:37:05.092 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T01:37:05.092 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T01:37:05.092 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.num_levels: 7 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:37:05.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T01:37:05.094 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T01:37:05.095 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T01:37:05.096 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T01:37:05.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.inplace_update_support: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.bloom_locality: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.max_successive_merges: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.ttl: 2592000 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T01:37:05.098 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.enable_blob_files: false 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.min_blob_size: 0 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T01:37:05.099 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 88c22935-d074-443d-8a3d-45cfac580c07 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105024915144, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105024916168, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100637, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 98531, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14350, "raw_average_key_size": 49, "raw_value_size": 90758, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726105024, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "88c22935-d074-443d-8a3d-45cfac580c07", "db_session_id": "1XN0LJDFICUD2AVFCPZ0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105024916261, "job": 1, "event": "recovery_finished"} 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55876684ee00 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: DB pointer 0x55876694a000 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** DB Stats ** 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:37:05.100 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: L0 2/0 100.14 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Sum 2/0 100.14 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-12T01:37:05.101 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T01:37:05.102 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative compaction: 0.00 GB write, 30.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval compaction: 0.00 GB write, 30.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache BinnedLRUCache@0x5587668211f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: starting mon.a rank 0 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???) e1 preinit fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).mds e1 new map 2024-09-12T01:37:05.103 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).mds e1 print_map 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: e1 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: btime 2024-09-12T01:36:52:283630+0000 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: legacy client fscid: -1 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout: No filesystems configured 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T01:37:05.104 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 bash[30906]: 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 systemd[1]: Started Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: monmap epoch 1 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: last_changed 2024-09-12T01:36:46.446315+0000 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: min_mon_release 19 (squid) 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: election_strategy: 1 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: fsmap 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T01:37:05.105 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:04 smithi122 ceph-mon[30925]: mgrmap e1: no daemons active 2024-09-12T01:37:06.046 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.401+0000 7f44dfa14640 1 Processor -- start 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.401+0000 7f44dfa14640 1 -- start start 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.401+0000 7f44dfa14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d810a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.401+0000 7f44dfa14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44d807be30 con 0x7f44d8108110 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.402+0000 7f44dd7b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d810a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.402+0000 7f44dd7b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d810a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52656/0 (socket says 172.21.15.122:52656) 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.402+0000 7f44dd7b0640 1 -- 172.21.15.122:0/2675113466 learned_addr learned my addr 172.21.15.122:0/2675113466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.402+0000 7f44dd7b0640 1 -- 172.21.15.122:0/2675113466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d810aa40 con 0x7f44d8108110 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.404+0000 7f44dd7b0640 1 --2- 172.21.15.122:0/2675113466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d810a500 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009820 tx=0x7f44d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455e69f75d855426 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.404+0000 7f44cffff640 1 -- 172.21.15.122:0/2675113466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44d4037070 con 0x7f44d8108110 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.404+0000 7f44cffff640 1 -- 172.21.15.122:0/2675113466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44d4032900 con 0x7f44d8108110 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.404+0000 7f44cffff640 1 -- 172.21.15.122:0/2675113466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44d4030850 con 0x7f44d8108110 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.404+0000 7f44dfa14640 1 -- 172.21.15.122:0/2675113466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 msgr2=0x7f44d810a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.404+0000 7f44dfa14640 1 --2- 172.21.15.122:0/2675113466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d810a500 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009820 tx=0x7f44d402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 -- 172.21.15.122:0/2675113466 shutdown_connections 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 --2- 172.21.15.122:0/2675113466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d810a500 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 -- 172.21.15.122:0/2675113466 >> 172.21.15.122:0/2675113466 conn(0x7f44d8101db0 msgr2=0x7f44d81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 -- 172.21.15.122:0/2675113466 shutdown_connections 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 -- 172.21.15.122:0/2675113466 wait complete. 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 Processor -- start 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 -- start start 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dfa14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44d810ba20 con 0x7f44d8108110 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dd7b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.405+0000 7f44dd7b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52662/0 (socket says 172.21.15.122:52662) 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44dd7b0640 1 -- 172.21.15.122:0/2926549442 learned_addr learned my addr 172.21.15.122:0/2926549442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:06.048 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44dd7b0640 1 -- 172.21.15.122:0/2926549442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d819bb60 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44dd7b0640 1 --2- 172.21.15.122:0/2926549442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d819b620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f44d4032ce0 tx=0x7f44d4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44d4042070 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44d40308c0 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d819bdf0 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44d4037040 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d819eae0 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44d4045070 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.406+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f44d4047bb0 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.407+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d8108570 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.409+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f44d403c740 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.466+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f44d819f0c0 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.468+0000 7f44ce7fc640 1 -- 172.21.15.122:0/2926549442 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f44d403c920 con 0x7f44d8108110 2024-09-12T01:37:06.049 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 msgr2=0x7f44d819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 --2- 172.21.15.122:0/2926549442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d819b620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f44d4032ce0 tx=0x7f44d4032d10 comp rx=0 tx=0).stop 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 shutdown_connections 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 --2- 172.21.15.122:0/2926549442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44d8108110 0x7f44d819b620 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 >> 172.21.15.122:0/2926549442 conn(0x7f44d8101db0 msgr2=0x7f44d81028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 shutdown_connections 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:05.469+0000 7f44dfa14640 1 -- 172.21.15.122:0/2926549442 wait complete. 2024-09-12T01:37:06.050 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T01:37:06.056 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:06.057 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mgr... 2024-09-12T01:37:06.057 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T01:37:06.057 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T01:37:06.296 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x 2024-09-12T01:37:06.297 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T01:37:06.525 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5731f69a-70a7-11ef-bcea-c7b262605968.target.wants/ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5731f69a-70a7-11ef-bcea-c7b262605968@.service. 2024-09-12T01:37:06.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:06 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2926549442' entity='client.admin' 2024-09-12T01:37:06.739 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:06 smithi122 systemd[1]: Starting Ceph mgr.x for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:37:07.029 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:06 smithi122 podman[31136]: 2024-09-12 01:37:06.655953179 +0000 UTC m=+0.015724926 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:37:07.029 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:06 smithi122 podman[31136]: 2024-09-12 01:37:06.869373358 +0000 UTC m=+0.229145106 container create bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-12T01:37:07.029 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 podman[31136]: 2024-09-12 01:37:07.029987065 +0000 UTC m=+0.389758816 container init bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:37:07.146 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:37:07.146 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-12T01:37:07.154 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:37:07.154 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-12T01:37:07.154 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-12T01:37:07.154 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T01:37:07.165 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:37:07.165 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-12T01:37:07.173 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:37:07.174 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-12T01:37:07.174 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-12T01:37:07.174 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T01:37:07.174 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr to start... 2024-09-12T01:37:07.174 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr... 2024-09-12T01:37:07.300 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 podman[31136]: 2024-09-12 01:37:07.032666162 +0000 UTC m=+0.392437910 container start bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T01:37:07.300 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 bash[31136]: bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 2024-09-12T01:37:07.300 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 systemd[1]: Started Ceph mgr.x for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:37:07.300 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:07.156+0000 7fbeab3d4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:37:07.300 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:07.222+0000 7fbeab3d4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:37:07.590 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:07.372+0000 7fbeab3d4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:37:07.590 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:07.441+0000 7fbeab3d4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:37:07.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3485924247' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:37:07.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:07.592+0000 7fbeab3d4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:37:07.960 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:07.742+0000 7fbeab3d4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:37:08.238 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:08.061+0000 7fbeab3d4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:37:08.238 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:08.122+0000 7fbeab3d4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:37:08.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:08.249 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:08.249 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:37:08.249 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:37:08.249 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:37:08.249 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:37:08.250 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:37:08.251 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:37:08.258 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:37:08.258 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.259 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:36:52:283630+0000", 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:08.260 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:36:52.284388+0000", 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.604+0000 7f1afc398640 1 Processor -- start 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.604+0000 7f1afc398640 1 -- start start 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.604+0000 7f1afc398640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af41062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.604+0000 7f1afc398640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1af407be50 con 0x7f1af4105ee0 2024-09-12T01:37:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.605+0000 7f1afa134640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af41062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.605+0000 7f1afa134640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af41062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52698/0 (socket says 172.21.15.122:52698) 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.605+0000 7f1afa134640 1 -- 172.21.15.122:0/3055814607 learned_addr learned my addr 172.21.15.122:0/3055814607 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.605+0000 7f1afa134640 1 -- 172.21.15.122:0/3055814607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af4106820 con 0x7f1af4105ee0 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.606+0000 7f1afa134640 1 --2- 172.21.15.122:0/3055814607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af41062e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009820 tx=0x7f1ae402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3982336ebff56960 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.606+0000 7f1af9132640 1 -- 172.21.15.122:0/3055814607 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae4037070 con 0x7f1af4105ee0 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.606+0000 7f1af9132640 1 -- 172.21.15.122:0/3055814607 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1ae4032900 con 0x7f1af4105ee0 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.606+0000 7f1af9132640 1 -- 172.21.15.122:0/3055814607 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae4030850 con 0x7f1af4105ee0 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 -- 172.21.15.122:0/3055814607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 msgr2=0x7f1af41062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 --2- 172.21.15.122:0/3055814607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af41062e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009820 tx=0x7f1ae402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 -- 172.21.15.122:0/3055814607 shutdown_connections 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 --2- 172.21.15.122:0/3055814607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af41062e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 -- 172.21.15.122:0/3055814607 >> 172.21.15.122:0/3055814607 conn(0x7f1af4101db0 msgr2=0x7f1af41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 -- 172.21.15.122:0/3055814607 shutdown_connections 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.607+0000 7f1afc398640 1 -- 172.21.15.122:0/3055814607 wait complete. 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afc398640 1 Processor -- start 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afc398640 1 -- start start 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afc398640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afc398640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4107920 con 0x7f1af4105ee0 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afa134640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afa134640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52710/0 (socket says 172.21.15.122:52710) 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afa134640 1 -- 172.21.15.122:0/3485924247 learned_addr learned my addr 172.21.15.122:0/3485924247 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afa134640 1 -- 172.21.15.122:0/3485924247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af419dcf0 con 0x7f1af4105ee0 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.608+0000 7f1afa134640 1 --2- 172.21.15.122:0/3485924247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af419d7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4032ce0 tx=0x7f1ae4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae4042070 con 0x7f1af4105ee0 2024-09-12T01:37:08.263 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1ae40308c0 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af419df80 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae4037040 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af41a0c70 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ae4045070 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.609+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ae4047c10 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.610+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af41062e0 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.614+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1ae403c850 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.675+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1af41a0e40 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.675+0000 7f1aeb7fe640 1 -- 172.21.15.122:0/3485924247 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1ae403ca30 con 0x7f1af4105ee0 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 msgr2=0x7f1af419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 --2- 172.21.15.122:0/3485924247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af419d7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4032ce0 tx=0x7f1ae4032d10 comp rx=0 tx=0).stop 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 shutdown_connections 2024-09-12T01:37:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 --2- 172.21.15.122:0/3485924247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1af4105ee0 0x7f1af419d7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:08.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 >> 172.21.15.122:0/3485924247 conn(0x7f1af4101db0 msgr2=0x7f1af4191ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:08.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 shutdown_connections 2024-09-12T01:37:08.265 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:07.676+0000 7f1afc398640 1 -- 172.21.15.122:0/3485924247 wait complete. 2024-09-12T01:37:08.265 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (1/15)... 2024-09-12T01:37:08.738 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:08.310+0000 7fbeab3d4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:37:09.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:09.191+0000 7fbeab3d4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:37:09.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:09.257+0000 7fbeab3d4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:37:10.226 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:09.970+0000 7fbeab3d4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:37:10.227 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.086+0000 7fbeab3d4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:37:10.227 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.226+0000 7fbeab3d4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:37:10.488 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.295+0000 7fbeab3d4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:37:10.488 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.444+0000 7fbeab3d4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:37:10.759 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.516+0000 7fbeab3d4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:37:10.759 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.578+0000 7fbeab3d4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:37:10.759 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.742+0000 7fbeab3d4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:37:11.024 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2260997826' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:37:11.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.812+0000 7fbeab3d4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:37:11.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.879+0000 7fbeab3d4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:37:11.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:10.963+0000 7fbeab3d4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:37:11.326 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:11.024+0000 7fbeab3d4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:37:11.327 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:11.084+0000 7fbeab3d4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:37:11.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:37:11.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:37:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:36:52:283630+0000", 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:36:52.284388+0000", 2024-09-12T01:37:11.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4ddba0640 1 Processor -- start 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4ddba0640 1 -- start start 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4ddba0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d810a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4ddba0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d807bdd0 con 0x7fd4d8107f70 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4dcb9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d810a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4dcb9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d810a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38126/0 (socket says 172.21.15.122:38126) 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.691+0000 7fd4dcb9e640 1 -- 172.21.15.122:0/3365703742 learned_addr learned my addr 172.21.15.122:0/3365703742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.692+0000 7fd4dcb9e640 1 -- 172.21.15.122:0/3365703742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d810a8a0 con 0x7fd4d8107f70 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.692+0000 7fd4dcb9e640 1 --2- 172.21.15.122:0/3365703742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d810a360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009820 tx=0x7fd4cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66c0698e29db012 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:11.344 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.692+0000 7fd4d77fe640 1 -- 172.21.15.122:0/3365703742 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc037070 con 0x7fd4d8107f70 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4d77fe640 1 -- 172.21.15.122:0/3365703742 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd4cc032900 con 0x7fd4d8107f70 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4d77fe640 1 -- 172.21.15.122:0/3365703742 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc030850 con 0x7fd4d8107f70 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 -- 172.21.15.122:0/3365703742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 msgr2=0x7fd4d810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 --2- 172.21.15.122:0/3365703742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d810a360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009820 tx=0x7fd4cc02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 -- 172.21.15.122:0/3365703742 shutdown_connections 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 --2- 172.21.15.122:0/3365703742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d810a360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 -- 172.21.15.122:0/3365703742 >> 172.21.15.122:0/3365703742 conn(0x7fd4d8101a70 msgr2=0x7fd4d8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 -- 172.21.15.122:0/3365703742 shutdown_connections 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.693+0000 7fd4ddba0640 1 -- 172.21.15.122:0/3365703742 wait complete. 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4ddba0640 1 Processor -- start 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4ddba0640 1 -- start start 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4ddba0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d81077f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:11.345 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4ddba0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d810b820 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4dcb9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d81077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4dcb9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d81077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38142/0 (socket says 172.21.15.122:38142) 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4dcb9e640 1 -- 172.21.15.122:0/2260997826 learned_addr learned my addr 172.21.15.122:0/2260997826 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.694+0000 7fd4dcb9e640 1 -- 172.21.15.122:0/2260997826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d8107d30 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4dcb9e640 1 --2- 172.21.15.122:0/2260997826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d81077f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0097f0 tx=0x7fd4cc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc042070 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d8105f20 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd4cc032ca0 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc037040 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d8106460 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4cc045070 con 0x7fd4d8107f70 2024-09-12T01:37:11.346 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.695+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4cc047a90 con 0x7fd4d8107f70 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.696+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d81083d0 con 0x7fd4d8107f70 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.698+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd4cc03db10 con 0x7fd4d8107f70 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.759+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd4d81067a0 con 0x7fd4d8107f70 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.759+0000 7fd4d5ffb640 1 -- 172.21.15.122:0/2260997826 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd4cc03d430 con 0x7fd4d8107f70 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.760+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 msgr2=0x7fd4d81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.760+0000 7fd4ddba0640 1 --2- 172.21.15.122:0/2260997826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d81077f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0097f0 tx=0x7fd4cc030ec0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.760+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 shutdown_connections 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.760+0000 7fd4ddba0640 1 --2- 172.21.15.122:0/2260997826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d8107f70 0x7fd4d81077f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.760+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 >> 172.21.15.122:0/2260997826 conn(0x7fd4d8101a70 msgr2=0x7fd4d81026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.761+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 shutdown_connections 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:10.761+0000 7fd4ddba0640 1 -- 172.21.15.122:0/2260997826 wait complete. 2024-09-12T01:37:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (2/15)... 2024-09-12T01:37:12.238 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:11.893+0000 7fbeab3d4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: Activating manager daemon x 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: mgrmap e2: x(active, starting, since 0.00303001s) 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: Manager daemon x is now available 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:37:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' 2024-09-12T01:37:12.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' 2024-09-12T01:37:12.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:11 smithi122 ceph-mon[30925]: from='mgr.14100 172.21.15.122:0/3222850521' entity='mgr.x' 2024-09-12T01:37:13.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:13 smithi122 ceph-mon[30925]: mgrmap e3: x(active, since 1.00749s) 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T01:37:14.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T01:37:14.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:14.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T01:37:14.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T01:37:14.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T01:37:14.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T01:37:14.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T01:37:14.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T01:37:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T01:36:52:283630+0000", 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:37:14.486 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T01:36:52.284388+0000", 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.693+0000 7f68ac3a8640 1 Processor -- start 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.693+0000 7f68ac3a8640 1 -- start start 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.694+0000 7f68ac3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a410a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.694+0000 7f68ac3a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a407be50 con 0x7f68a4108160 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.694+0000 7f68aa144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a410a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:14.487 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.694+0000 7f68aa144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a410a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38226/0 (socket says 172.21.15.122:38226) 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.694+0000 7f68aa144640 1 -- 172.21.15.122:0/2337995299 learned_addr learned my addr 172.21.15.122:0/2337995299 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.694+0000 7f68aa144640 1 -- 172.21.15.122:0/2337995299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a410aa90 con 0x7f68a4108160 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.695+0000 7f68aa144640 1 --2- 172.21.15.122:0/2337995299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a410a550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6894009820 tx=0x7f689402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799eb8c0e06defc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.695+0000 7f68a9142640 1 -- 172.21.15.122:0/2337995299 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894037070 con 0x7f68a4108160 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.695+0000 7f68a9142640 1 -- 172.21.15.122:0/2337995299 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6894032900 con 0x7f68a4108160 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.695+0000 7f68a9142640 1 -- 172.21.15.122:0/2337995299 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894030850 con 0x7f68a4108160 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 -- 172.21.15.122:0/2337995299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 msgr2=0x7f68a410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 --2- 172.21.15.122:0/2337995299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a410a550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6894009820 tx=0x7f689402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 -- 172.21.15.122:0/2337995299 shutdown_connections 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 --2- 172.21.15.122:0/2337995299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a410a550 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 -- 172.21.15.122:0/2337995299 >> 172.21.15.122:0/2337995299 conn(0x7f68a4101de0 msgr2=0x7f68a4104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:14.488 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 -- 172.21.15.122:0/2337995299 shutdown_connections 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 -- 172.21.15.122:0/2337995299 wait complete. 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.696+0000 7f68ac3a8640 1 Processor -- start 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68ac3a8640 1 -- start start 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68ac3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68ac3a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a410ba60 con 0x7f68a4108160 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68aa144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68aa144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38232/0 (socket says 172.21.15.122:38232) 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68aa144640 1 -- 172.21.15.122:0/1841012421 learned_addr learned my addr 172.21.15.122:0/1841012421 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68aa144640 1 -- 172.21.15.122:0/1841012421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a419bb50 con 0x7f68a4108160 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68aa144640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a419b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68940097f0 tx=0x7f6894030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894037040 con 0x7f68a4108160 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6894040070 con 0x7f68a4108160 2024-09-12T01:37:14.489 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a419bde0 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f689403a390 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.697+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a419ead0 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.698+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f689403a5e0 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.698+0000 7f689b7fe640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f6890038e60 0x7f689003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.698+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a41085c0 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.699+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f689406c040 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.699+0000 7f68a9943640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f6890038e60 0x7f689003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.699+0000 7f68a9943640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f6890038e60 0x7f689003b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f68a00098c0 tx=0x7f68a0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.704+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6894032ab0 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.902+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f689402fd60 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.936+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f68a419c260 con 0x7f68a4108160 2024-09-12T01:37:14.490 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.936+0000 7f689b7fe640 1 -- 172.21.15.122:0/1841012421 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f689402ee80 con 0x7f68a4108160 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f6890038e60 msgr2=0x7f689003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f6890038e60 0x7f689003b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f68a00098c0 tx=0x7f68a0006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 msgr2=0x7f68a419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a419b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68940097f0 tx=0x7f6894030d80 comp rx=0 tx=0).stop 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 shutdown_connections 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f6890038e60 0x7f689003b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 --2- 172.21.15.122:0/1841012421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4108160 0x7f68a419b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 >> 172.21.15.122:0/1841012421 conn(0x7f68a4101de0 msgr2=0x7f68a41028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 shutdown_connections 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:13.938+0000 7f68ac3a8640 1 -- 172.21.15.122:0/1841012421 wait complete. 2024-09-12T01:37:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:mgr is available 2024-09-12T01:37:15.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:14 smithi122 ceph-mon[30925]: mgrmap e4: x(active, since 2s) 2024-09-12T01:37:15.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:14 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1841012421' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T01:37:15.609 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.837+0000 7f96117d6640 1 Processor -- start 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.837+0000 7f96117d6640 1 -- start start 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.837+0000 7f96117d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.837+0000 7f96117d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f960c108000 con 0x7f960c107630 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.838+0000 7f960affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.838+0000 7f960affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38246/0 (socket says 172.21.15.122:38246) 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.838+0000 7f960affd640 1 -- 172.21.15.122:0/1876068375 learned_addr learned my addr 172.21.15.122:0/1876068375 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.838+0000 7f960affd640 1 -- 172.21.15.122:0/1876068375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c108830 con 0x7f960c107630 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.839+0000 7f960affd640 1 --2- 172.21.15.122:0/1876068375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c107a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9604009820 tx=0x7f960402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493c2146f06ddc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.839+0000 7f9609ffb640 1 -- 172.21.15.122:0/1876068375 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9604037070 con 0x7f960c107630 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.839+0000 7f9609ffb640 1 -- 172.21.15.122:0/1876068375 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9604032900 con 0x7f960c107630 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.839+0000 7f9609ffb640 1 -- 172.21.15.122:0/1876068375 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9604030850 con 0x7f960c107630 2024-09-12T01:37:15.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 -- 172.21.15.122:0/1876068375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 msgr2=0x7f960c107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 --2- 172.21.15.122:0/1876068375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c107a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9604009820 tx=0x7f960402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 -- 172.21.15.122:0/1876068375 shutdown_connections 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 --2- 172.21.15.122:0/1876068375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c107a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 -- 172.21.15.122:0/1876068375 >> 172.21.15.122:0/1876068375 conn(0x7f960c102de0 msgr2=0x7f960c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 -- 172.21.15.122:0/1876068375 shutdown_connections 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 -- 172.21.15.122:0/1876068375 wait complete. 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.840+0000 7f96117d6640 1 Processor -- start 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f96117d6640 1 -- start start 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f96117d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f96117d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10bbe0 con 0x7f960c107630 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f960affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f960affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38248/0 (socket says 172.21.15.122:38248) 2024-09-12T01:37:15.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f960affd640 1 -- 172.21.15.122:0/946131434 learned_addr learned my addr 172.21.15.122:0/946131434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f960affd640 1 -- 172.21.15.122:0/946131434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c07e0b0 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f960affd640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c07db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9604009940 tx=0x7f96040326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9604042070 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9604032be0 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c07c2a0 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9604037040 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.841+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c07c7e0 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.842+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9604045070 con 0x7f960c107630 2024-09-12T01:37:15.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.842+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c10bd60 con 0x7f960c107630 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.843+0000 7f95f3fff640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95f4038f70 0x7f95f403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.843+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f960406be40 con 0x7f960c107630 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.843+0000 7f960a7fc640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95f4038f70 0x7f95f403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.843+0000 7f960a7fc640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95f4038f70 0x7f95f403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f960000ad30 tx=0x7f96000093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:14.847+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f960404b170 con 0x7f960c107630 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.016+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f960c107ab0 con 0x7f960c107630 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.017+0000 7f95f3fff640 1 -- 172.21.15.122:0/946131434 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f960406b860 con 0x7f960c107630 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95f4038f70 msgr2=0x7f95f403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95f4038f70 0x7f95f403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f960000ad30 tx=0x7f96000093f0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 msgr2=0x7f960c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c07db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9604009940 tx=0x7f96040326e0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 shutdown_connections 2024-09-12T01:37:15.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95f4038f70 0x7f95f403b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.615 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 --2- 172.21.15.122:0/946131434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f960c107630 0x7f960c07db70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:15.615 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.019+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 >> 172.21.15.122:0/946131434 conn(0x7f960c102de0 msgr2=0x7f960c103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:15.615 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.020+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 shutdown_connections 2024-09-12T01:37:15.615 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.020+0000 7f96117d6640 1 -- 172.21.15.122:0/946131434 wait complete. 2024-09-12T01:37:15.615 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling cephadm module... 2024-09-12T01:37:16.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:15 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/946131434' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T01:37:17.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:16 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3508632385' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T01:37:17.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:17.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:17.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.045+0000 7f3f26f07040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:37:17.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.101+0000 7f3f26f07040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:37:17.603 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.247+0000 7f3f26f07040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:37:17.604 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.315+0000 7f3f26f07040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:37:17.604 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.462+0000 7f3f26f07040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:37:17.604 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.604+0000 7f3f26f07040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:37:17.653 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff95715a640 1 Processor -- start 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff95715a640 1 -- start start 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff95715a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff950107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff95715a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff950108110 con 0x7ff950107740 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff954ef6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff950107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff954ef6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff950107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38264/0 (socket says 172.21.15.122:38264) 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff954ef6640 1 -- 172.21.15.122:0/915831960 learned_addr learned my addr 172.21.15.122:0/915831960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.974+0000 7ff954ef6640 1 -- 172.21.15.122:0/915831960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9501088a0 con 0x7ff950107740 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.975+0000 7ff954ef6640 1 --2- 172.21.15.122:0/915831960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff950107b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff944009820 tx=0x7ff94402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46cfdd96d6a73a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.975+0000 7ff94f7fe640 1 -- 172.21.15.122:0/915831960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944037070 con 0x7ff950107740 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.975+0000 7ff94f7fe640 1 -- 172.21.15.122:0/915831960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff944032900 con 0x7ff950107740 2024-09-12T01:37:17.654 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.975+0000 7ff94f7fe640 1 -- 172.21.15.122:0/915831960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944030850 con 0x7ff950107740 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 -- 172.21.15.122:0/915831960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 msgr2=0x7ff950107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 --2- 172.21.15.122:0/915831960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff950107b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff944009820 tx=0x7ff94402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 -- 172.21.15.122:0/915831960 shutdown_connections 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 --2- 172.21.15.122:0/915831960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff950107b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 -- 172.21.15.122:0/915831960 >> 172.21.15.122:0/915831960 conn(0x7ff950102eb0 msgr2=0x7ff950105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 -- 172.21.15.122:0/915831960 shutdown_connections 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.976+0000 7ff95715a640 1 -- 172.21.15.122:0/915831960 wait complete. 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff95715a640 1 Processor -- start 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff95715a640 1 -- start start 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff95715a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff95019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff95715a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff95010bc50 con 0x7ff950107740 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff954ef6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff95019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff954ef6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff95019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38266/0 (socket says 172.21.15.122:38266) 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff954ef6640 1 -- 172.21.15.122:0/3508632385 learned_addr learned my addr 172.21.15.122:0/3508632385 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff954ef6640 1 -- 172.21.15.122:0/3508632385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95019bbc0 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff954ef6640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff95019b680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff9440097f0 tx=0x7ff944030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944042070 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff944032ca0 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944037040 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.977+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95019be50 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.978+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95019eb40 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.978+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff944045070 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.979+0000 7ff94dffb640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7ff940038f70 0x7ff94003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.979+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff94406bfd0 con 0x7ff950107740 2024-09-12T01:37:17.656 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.979+0000 7ff94ffff640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7ff940038f70 0x7ff94003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.979+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950107bc0 con 0x7ff950107740 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.979+0000 7ff94ffff640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7ff940038f70 0x7ff94003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098c0 tx=0x7ff93c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:15.983+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff94403add0 con 0x7ff950107740 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.195+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff95019f160 con 0x7ff950107740 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.905+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff94406b9f0 con 0x7ff950107740 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.907+0000 7ff94dffb640 1 -- 172.21.15.122:0/3508632385 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff94403c6a0 con 0x7ff950107740 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7ff940038f70 msgr2=0x7ff94003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7ff940038f70 0x7ff94003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098c0 tx=0x7ff93c006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 msgr2=0x7ff95019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff95019b680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff9440097f0 tx=0x7ff944030ec0 comp rx=0 tx=0).stop 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 shutdown_connections 2024-09-12T01:37:17.657 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7ff940038f70 0x7ff94003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:17.658 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 --2- 172.21.15.122:0/3508632385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff950107740 0x7ff95019b680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:17.658 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 >> 172.21.15.122:0/3508632385 conn(0x7ff950102eb0 msgr2=0x7ff950103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:17.658 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 shutdown_connections 2024-09-12T01:37:17.658 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:16.908+0000 7ff95715a640 1 -- 172.21.15.122:0/3508632385 wait complete. 2024-09-12T01:37:18.184 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3508632385' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T01:37:18.184 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-mon[30925]: mgrmap e5: x(active, since 5s) 2024-09-12T01:37:18.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.927+0000 7f3f26f07040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:37:18.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:17.990+0000 7f3f26f07040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:37:18.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:18.184+0000 7f3f26f07040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:18.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95d57ac640 1 Processor -- start 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95d57ac640 1 -- start start 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95d57ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95d57ac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95d007be70 con 0x7f95d01080d0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38300/0 (socket says 172.21.15.122:38300) 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.018+0000 7f95ceffd640 1 -- 172.21.15.122:0/3934915810 learned_addr learned my addr 172.21.15.122:0/3934915810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.019+0000 7f95ceffd640 1 -- 172.21.15.122:0/3934915810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d010aa00 con 0x7f95d01080d0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.019+0000 7f95ceffd640 1 --2- 172.21.15.122:0/3934915810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d010a4c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009820 tx=0x7f95c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb1ed24d5cb3d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95ce7fc640 1 -- 172.21.15.122:0/3934915810 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c0037070 con 0x7f95d01080d0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95ce7fc640 1 -- 172.21.15.122:0/3934915810 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95c0032900 con 0x7f95d01080d0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95ce7fc640 1 -- 172.21.15.122:0/3934915810 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c0030850 con 0x7f95d01080d0 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 -- 172.21.15.122:0/3934915810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 msgr2=0x7f95d010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 --2- 172.21.15.122:0/3934915810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d010a4c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009820 tx=0x7f95c002a270 comp rx=0 tx=0).stop 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 -- 172.21.15.122:0/3934915810 shutdown_connections 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 --2- 172.21.15.122:0/3934915810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d010a4c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 -- 172.21.15.122:0/3934915810 >> 172.21.15.122:0/3934915810 conn(0x7f95d0101db0 msgr2=0x7f95d01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 -- 172.21.15.122:0/3934915810 shutdown_connections 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.020+0000 7f95d57ac640 1 -- 172.21.15.122:0/3934915810 wait complete. 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95d57ac640 1 Processor -- start 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95d57ac640 1 -- start start 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95d57ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d0107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95d57ac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95d010ba40 con 0x7f95d01080d0 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d0107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d0107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38308/0 (socket says 172.21.15.122:38308) 2024-09-12T01:37:18.872 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.021+0000 7f95ceffd640 1 -- 172.21.15.122:0/2188054670 learned_addr learned my addr 172.21.15.122:0/2188054670 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95ceffd640 1 -- 172.21.15.122:0/2188054670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d0107e90 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95ceffd640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d0107950 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097f0 tx=0x7f95c0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c0037040 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d0106080 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95c0040070 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95c003a390 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.022+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d0106560 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.023+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f95c003a630 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.023+0000 7f95ccff9640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 0x7f95bc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.023+0000 7f95c7fff640 1 -- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 msgr2=0x7f95bc03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1950823209 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.023+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f95c006c160 con 0x7f95d01080d0 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.023+0000 7f95c7fff640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 0x7f95bc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:37:18.873 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.023+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0108530 con 0x7f95d01080d0 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.028+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95c004a200 con 0x7f95d01080d0 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.224+0000 7f95c7fff640 1 -- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 msgr2=0x7f95bc03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1950823209 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.224+0000 7f95c7fff640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 0x7f95bc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.247+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f95d0106cc0 con 0x7f95d01080d0 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.247+0000 7f95ccff9640 1 -- 172.21.15.122:0/2188054670 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f95c002ee80 con 0x7f95d01080d0 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 msgr2=0x7f95bc03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 0x7f95bc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 msgr2=0x7f95d0107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d0107950 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097f0 tx=0x7f95c0030d80 comp rx=0 tx=0).stop 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 shutdown_connections 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7f95bc038f70 0x7f95bc03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 --2- 172.21.15.122:0/2188054670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d01080d0 0x7f95d0107950 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:18.874 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.249+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 >> 172.21.15.122:0/2188054670 conn(0x7f95d0101db0 msgr2=0x7f95d01028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:18.875 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.250+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 shutdown_connections 2024-09-12T01:37:18.875 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:18.250+0000 7f95d57ac640 1 -- 172.21.15.122:0/2188054670 wait complete. 2024-09-12T01:37:18.875 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-09-12T01:37:18.875 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 5... 2024-09-12T01:37:19.155 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:18 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2188054670' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T01:37:19.155 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:19.085+0000 7f3f26f07040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:37:19.474 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:19.155+0000 7f3f26f07040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:37:20.128 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:19.870+0000 7f3f26f07040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:37:20.128 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:19.986+0000 7f3f26f07040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:37:20.395 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.128+0000 7f3f26f07040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:37:20.396 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.192+0000 7f3f26f07040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:37:20.396 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.348+0000 7f3f26f07040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:37:20.650 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.420+0000 7f3f26f07040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:37:20.650 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.482+0000 7f3f26f07040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:37:20.935 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.649+0000 7f3f26f07040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:37:20.935 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.718+0000 7f3f26f07040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:37:20.936 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.787+0000 7f3f26f07040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:37:20.936 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.873+0000 7f3f26f07040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:37:20.936 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.935+0000 7f3f26f07040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:37:21.238 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:20.996+0000 7f3f26f07040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:37:22.238 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:21.793+0000 7f3f26f07040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: Active manager daemon x restarted 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: Activating manager daemon x 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: mgrmap e6: x(active, starting, since 0.00387299s) 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: Manager daemon x is now available 2024-09-12T01:37:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:22.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:22.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:22.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:22.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:37:22.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:21 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:37:23.050 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:22 smithi122 ceph-mon[30925]: Found migration_current of "None". Setting to last migration. 2024-09-12T01:37:23.050 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:22 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:23.050 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:22 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:23.051 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:22 smithi122 ceph-mon[30925]: mgrmap e7: x(active, since 1.00586s) 2024-09-12T01:37:23.353 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:23.353 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T01:37:23.353 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T01:37:23.353 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:23.353 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.268+0000 7fb680505640 1 Processor -- start 2024-09-12T01:37:23.353 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.268+0000 7fb680505640 1 -- start start 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.268+0000 7fb680505640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb678107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.268+0000 7fb680505640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb678108470 con 0x7fb678107aa0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.269+0000 7fb67e2a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb678107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.269+0000 7fb67e2a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb678107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38310/0 (socket says 172.21.15.122:38310) 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.269+0000 7fb67e2a1640 1 -- 172.21.15.122:0/1978716550 learned_addr learned my addr 172.21.15.122:0/1978716550 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.269+0000 7fb67e2a1640 1 -- 172.21.15.122:0/1978716550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb678106700 con 0x7fb678107aa0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.270+0000 7fb67e2a1640 1 --2- 172.21.15.122:0/1978716550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb678107ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009820 tx=0x7fb66c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7cd3971a5c22a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.270+0000 7fb67d29f640 1 -- 172.21.15.122:0/1978716550 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb66c037070 con 0x7fb678107aa0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.270+0000 7fb67d29f640 1 -- 172.21.15.122:0/1978716550 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb66c032900 con 0x7fb678107aa0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.270+0000 7fb67d29f640 1 -- 172.21.15.122:0/1978716550 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb66c030850 con 0x7fb678107aa0 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.270+0000 7fb680505640 1 -- 172.21.15.122:0/1978716550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 msgr2=0x7fb678107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.270+0000 7fb680505640 1 --2- 172.21.15.122:0/1978716550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb678107ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009820 tx=0x7fb66c02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:23.354 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 -- 172.21.15.122:0/1978716550 shutdown_connections 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 --2- 172.21.15.122:0/1978716550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb678107ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 -- 172.21.15.122:0/1978716550 >> 172.21.15.122:0/1978716550 conn(0x7fb678101420 msgr2=0x7fb678103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 -- 172.21.15.122:0/1978716550 shutdown_connections 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 -- 172.21.15.122:0/1978716550 wait complete. 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 Processor -- start 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 -- start start 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb6781060b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb680505640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb67810aac0 con 0x7fb678107aa0 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb67e2a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb6781060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.271+0000 7fb67e2a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb6781060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38314/0 (socket says 172.21.15.122:38314) 2024-09-12T01:37:23.355 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb67e2a1640 1 -- 172.21.15.122:0/4238552582 learned_addr learned my addr 172.21.15.122:0/4238552582 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb67e2a1640 1 -- 172.21.15.122:0/4238552582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67819e310 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb67e2a1640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb6781060b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0097f0 tx=0x7fb66c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb66c037040 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb66c040070 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67819e4e0 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb66c03a390 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.272+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67819e880 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.273+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb66c03a630 con 0x7fb678107aa0 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.274+0000 7fb66b7fe640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.274+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 --> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb66403bb40 con 0x7fb664038f70 2024-09-12T01:37:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.274+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb66c06c2b0 con 0x7fb678107aa0 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.274+0000 7fb67daa0640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 msgr2=0x7fb66403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1950823209 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.274+0000 7fb67daa0640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.475+0000 7fb67daa0640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 msgr2=0x7fb66403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1950823209 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.475+0000 7fb67daa0640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.876+0000 7fb67daa0640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 msgr2=0x7fb66403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1950823209 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:19.876+0000 7fb67daa0640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:20.677+0000 7fb67daa0640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 msgr2=0x7fb66403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1950823209 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:20.677+0000 7fb67daa0640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:21.798+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fb66c03acf0 con 0x7fb678107aa0 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:21.798+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 msgr2=0x7fb66403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:37:23.357 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:21.798+0000 7fb66b7fe640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.798+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fb66c0457e0 con 0x7fb678107aa0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.798+0000 7fb66b7fe640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fb66403c8a0 0x7fb66403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.798+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb66c00af30 con 0x7fb66403c8a0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.799+0000 7fb67daa0640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fb66403c8a0 0x7fb66403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.799+0000 7fb67daa0640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fb66403c8a0 0x7fb66403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb674003980 tx=0x7fb674006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.800+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb66c00af30 con 0x7fb66403c8a0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb678107f20 con 0x7fb66403c8a0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb66b7fe640 1 -- 172.21.15.122:0/4238552582 <== mgr.14118 v2:172.21.15.122:6800/3688521366 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb678107f20 con 0x7fb66403c8a0 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fb66403c8a0 msgr2=0x7fb66403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fb66403c8a0 0x7fb66403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb674003980 tx=0x7fb674006e10 comp rx=0 tx=0).stop 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 msgr2=0x7fb6781060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb6781060b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0097f0 tx=0x7fb66c030d80 comp rx=0 tx=0).stop 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 shutdown_connections 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fb66403c8a0 0x7fb66403ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:6800/1950823209,v1:172.21.15.122:6801/1950823209] conn(0x7fb664038f70 0x7fb66403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 --2- 172.21.15.122:0/4238552582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb678107aa0 0x7fb6781060b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.807+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 >> 172.21.15.122:0/4238552582 conn(0x7fb678101420 msgr2=0x7fb678101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.808+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 shutdown_connections 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:22.808+0000 7fb680505640 1 -- 172.21.15.122:0/4238552582 wait complete. 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 5 is available 2024-09-12T01:37:23.359 INFO:teuthology.orchestra.run.smithi122.stdout:Setting orchestrator backend to cephadm... 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:23.521+0000 7f3e93a6c640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:01:37:23] ENGINE Error in HTTPServer.serve 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: Traceback (most recent call last): 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self._connections.run(self.expiration_interval) 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self._run(expiration_interval) 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:37:23.647 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: s = self.context.wrap_socket( 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: return self.sslsocket_class._create( 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self.do_handshake() 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self._sslobj.do_handshake() 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:37:23.648 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:23.916 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-mon[30925]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T01:37:23.916 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-mon[30925]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T01:37:23.916 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:23 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:24.391 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.724+0000 7fcd64991640 1 Processor -- start 2024-09-12T01:37:24.391 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd64991640 1 -- start start 2024-09-12T01:37:24.391 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd64991640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd60107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.391 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd64991640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60108110 con 0x7fcd60107740 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd60107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd60107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36066/0 (socket says 172.21.15.122:36066) 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd5e59c640 1 -- 172.21.15.122:0/3478892798 learned_addr learned my addr 172.21.15.122:0/3478892798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.725+0000 7fcd5e59c640 1 -- 172.21.15.122:0/3478892798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd601088a0 con 0x7fcd60107740 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.726+0000 7fcd5e59c640 1 --2- 172.21.15.122:0/3478892798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd60107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009820 tx=0x7fcd5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d33e5a11f837bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.726+0000 7fcd5d59a640 1 -- 172.21.15.122:0/3478892798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd50037070 con 0x7fcd60107740 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.726+0000 7fcd5d59a640 1 -- 172.21.15.122:0/3478892798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd50032900 con 0x7fcd60107740 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.726+0000 7fcd5d59a640 1 -- 172.21.15.122:0/3478892798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd50030850 con 0x7fcd60107740 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 -- 172.21.15.122:0/3478892798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 msgr2=0x7fcd60107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 --2- 172.21.15.122:0/3478892798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd60107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009820 tx=0x7fcd5002a270 comp rx=0 tx=0).stop 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 -- 172.21.15.122:0/3478892798 shutdown_connections 2024-09-12T01:37:24.392 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 --2- 172.21.15.122:0/3478892798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd60107b40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 -- 172.21.15.122:0/3478892798 >> 172.21.15.122:0/3478892798 conn(0x7fcd60102eb0 msgr2=0x7fcd60105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 -- 172.21.15.122:0/3478892798 shutdown_connections 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.727+0000 7fcd64991640 1 -- 172.21.15.122:0/3478892798 wait complete. 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd64991640 1 Processor -- start 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd64991640 1 -- start start 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd64991640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd6019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd64991640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010bc50 con 0x7fcd60107740 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd6019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd5e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd6019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36072/0 (socket says 172.21.15.122:36072) 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd5e59c640 1 -- 172.21.15.122:0/1155140075 learned_addr learned my addr 172.21.15.122:0/1155140075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd5e59c640 1 -- 172.21.15.122:0/1155140075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019bbc0 con 0x7fcd60107740 2024-09-12T01:37:24.393 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd5e59c640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd6019b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097f0 tx=0x7fcd50030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd50042070 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd50032ca0 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019be50 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.728+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd50037040 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.729+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6019eb40 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.729+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fcd50045070 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.729+0000 7fcd577fe640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fcd4c038eb0 0x7fcd4c03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.730+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd5006beb0 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.729+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60107bc0 con 0x7fcd60107740 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.730+0000 7fcd5dd9b640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fcd4c038eb0 0x7fcd4c03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.730+0000 7fcd5dd9b640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fcd4c038eb0 0x7fcd4c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd480098c0 tx=0x7fcd48006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:24.394 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.734+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd50045360 con 0x7fcd60107740 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.852+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcd5003c6a0 con 0x7fcd60107740 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.916+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcd601041c0 con 0x7fcd4c038eb0 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.921+0000 7fcd577fe640 1 -- 172.21.15.122:0/1155140075 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd601041c0 con 0x7fcd4c038eb0 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fcd4c038eb0 msgr2=0x7fcd4c03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fcd4c038eb0 0x7fcd4c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd480098c0 tx=0x7fcd48006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 msgr2=0x7fcd6019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd6019b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097f0 tx=0x7fcd50030ec0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 shutdown_connections 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fcd4c038eb0 0x7fcd4c03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 --2- 172.21.15.122:0/1155140075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd60107740 0x7fcd6019b680 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 >> 172.21.15.122:0/1155140075 conn(0x7fcd60102eb0 msgr2=0x7fcd60103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 shutdown_connections 2024-09-12T01:37:24.396 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:23.923+0000 7fcd64991640 1 -- 172.21.15.122:0/1155140075 wait complete. 2024-09-12T01:37:24.947 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:23] ENGINE Bus STARTING 2024-09-12T01:37:24.947 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:23] ENGINE Serving on https://172.21.15.122:7150 2024-09-12T01:37:24.947 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:23] ENGINE Error in HTTPServer.serve 2024-09-12T01:37:24.947 INFO:journalctl@ceph.mon.a.smithi122.stdout: Traceback (most recent call last): 2024-09-12T01:37:24.947 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:37:24.947 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._run(expiration_interval) 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: s = self.context.wrap_socket( 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: return self.sslsocket_class._create( 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:37:24.948 INFO:journalctl@ceph.mon.a.smithi122.stdout: self.do_handshake() 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._sslobj.do_handshake() 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:23] ENGINE Serving on http://172.21.15.122:8765 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:23] ENGINE Bus STARTED 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: mgrmap e8: x(active, since 2s) 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:24.949 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:24 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:25.506 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T01:37:25.506 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2abad94640 1 Processor -- start 2024-09-12T01:37:25.506 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2abad94640 1 -- start start 2024-09-12T01:37:25.506 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2abad94640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab4107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2abad94640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41080b0 con 0x7f2ab41076e0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2ab8b30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab4107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2ab8b30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab4107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36084/0 (socket says 172.21.15.122:36084) 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2ab8b30640 1 -- 172.21.15.122:0/326388002 learned_addr learned my addr 172.21.15.122:0/326388002 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.766+0000 7f2ab8b30640 1 -- 172.21.15.122:0/326388002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab4108840 con 0x7f2ab41076e0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.767+0000 7f2ab8b30640 1 --2- 172.21.15.122:0/326388002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab4107ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009820 tx=0x7f2aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b145959bdd5b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.767+0000 7f2ab37fe640 1 -- 172.21.15.122:0/326388002 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa8037070 con 0x7f2ab41076e0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.767+0000 7f2ab37fe640 1 -- 172.21.15.122:0/326388002 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2aa8032900 con 0x7f2ab41076e0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.767+0000 7f2ab37fe640 1 -- 172.21.15.122:0/326388002 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa8030850 con 0x7f2ab41076e0 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 -- 172.21.15.122:0/326388002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 msgr2=0x7f2ab4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 --2- 172.21.15.122:0/326388002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab4107ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009820 tx=0x7f2aa802a270 comp rx=0 tx=0).stop 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 -- 172.21.15.122:0/326388002 shutdown_connections 2024-09-12T01:37:25.507 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 --2- 172.21.15.122:0/326388002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab4107ae0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 -- 172.21.15.122:0/326388002 >> 172.21.15.122:0/326388002 conn(0x7f2ab4102eb0 msgr2=0x7f2ab41052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 -- 172.21.15.122:0/326388002 shutdown_connections 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 -- 172.21.15.122:0/326388002 wait complete. 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 Processor -- start 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.768+0000 7f2abad94640 1 -- start start 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2abad94640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2abad94640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab410bbf0 con 0x7f2ab41076e0 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab8b30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab8b30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36086/0 (socket says 172.21.15.122:36086) 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab8b30640 1 -- 172.21.15.122:0/3937043279 learned_addr learned my addr 172.21.15.122:0/3937043279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab8b30640 1 -- 172.21.15.122:0/3937043279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab419bbd0 con 0x7f2ab41076e0 2024-09-12T01:37:25.508 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab8b30640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab419b690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009940 tx=0x7f2aa80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa8042070 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2aa8032be0 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab419be60 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa8037040 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.769+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab419eb50 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.770+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2aa8045070 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.770+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab410bd70 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.771+0000 7f2ab1ffb640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f2aa4038f70 0x7f2aa403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.771+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2aa806bf30 con 0x7f2ab41076e0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.771+0000 7f2ab3fff640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f2aa4038f70 0x7f2aa403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:25.509 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.771+0000 7f2ab3fff640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f2aa4038f70 0x7f2aa403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00098c0 tx=0x7f2aa0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.775+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2aa8045e10 con 0x7f2ab41076e0 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.947+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2ab4104180 con 0x7f2aa4038f70 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.948+0000 7f2ab1ffb640 1 -- 172.21.15.122:0/3937043279 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2ab4104180 con 0x7f2aa4038f70 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.949+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f2aa4038f70 msgr2=0x7f2aa403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.949+0000 7f2abad94640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f2aa4038f70 0x7f2aa403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00098c0 tx=0x7f2aa0006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.949+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 msgr2=0x7f2ab419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.949+0000 7f2abad94640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab419b690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009940 tx=0x7f2aa80326e0 comp rx=0 tx=0).stop 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.950+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 shutdown_connections 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.950+0000 7f2abad94640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f2aa4038f70 0x7f2aa403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.950+0000 7f2abad94640 1 --2- 172.21.15.122:0/3937043279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ab41076e0 0x7f2ab419b690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.950+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 >> 172.21.15.122:0/3937043279 conn(0x7f2ab4102eb0 msgr2=0x7f2ab4103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.950+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 shutdown_connections 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:24.950+0000 7f2abad94640 1 -- 172.21.15.122:0/3937043279 wait complete. 2024-09-12T01:37:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:Generating ssh key... 2024-09-12T01:37:25.880 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:25 smithi122 ceph-mon[30925]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: Generating public/private rsa key pair. 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: Your identification has been saved in /tmp/tmp_cxnshk9/key 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: Your public key has been saved in /tmp/tmp_cxnshk9/key.pub 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: The key fingerprint is: 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: SHA256:vuvguArYY8fmolyMEk0cTuD7Tm6jQSE9zpQRgDkZalQ ceph-5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: The key's randomart image is: 2024-09-12T01:37:26.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: +---[RSA 3072]----+ 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |=B*E | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |O= + | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |o=O | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |o*o. | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |.o+ S | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |oo+. . | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |+o+=+ . . | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |ooBB o . . | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: |.++==...+. | 2024-09-12T01:37:26.490 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: +----[SHA256]-----+ 2024-09-12T01:37:26.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.880+0000 7f60ae55a640 1 Processor -- start 2024-09-12T01:37:26.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.880+0000 7f60ae55a640 1 -- start start 2024-09-12T01:37:26.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.880+0000 7f60ae55a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a810a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.881+0000 7f60ae55a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60a807bdd0 con 0x7f60a8108170 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.881+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.881+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36102/0 (socket says 172.21.15.122:36102) 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.881+0000 7f60a7fff640 1 -- 172.21.15.122:0/1758320709 learned_addr learned my addr 172.21.15.122:0/1758320709 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.881+0000 7f60a7fff640 1 -- 172.21.15.122:0/1758320709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a810aaa0 con 0x7f60a8108170 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.882+0000 7f60a7fff640 1 --2- 172.21.15.122:0/1758320709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a810a560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6098009820 tx=0x7f609802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ea687281381fff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.882+0000 7f60a6ffd640 1 -- 172.21.15.122:0/1758320709 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098037070 con 0x7f60a8108170 2024-09-12T01:37:26.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.882+0000 7f60a6ffd640 1 -- 172.21.15.122:0/1758320709 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6098032900 con 0x7f60a8108170 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.882+0000 7f60a6ffd640 1 -- 172.21.15.122:0/1758320709 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098030850 con 0x7f60a8108170 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 -- 172.21.15.122:0/1758320709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 msgr2=0x7f60a810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 --2- 172.21.15.122:0/1758320709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a810a560 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6098009820 tx=0x7f609802a270 comp rx=0 tx=0).stop 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 -- 172.21.15.122:0/1758320709 shutdown_connections 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 --2- 172.21.15.122:0/1758320709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a810a560 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 -- 172.21.15.122:0/1758320709 >> 172.21.15.122:0/1758320709 conn(0x7f60a8101db0 msgr2=0x7f60a81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 -- 172.21.15.122:0/1758320709 shutdown_connections 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 -- 172.21.15.122:0/1758320709 wait complete. 2024-09-12T01:37:26.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 Processor -- start 2024-09-12T01:37:26.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.883+0000 7f60ae55a640 1 -- start start 2024-09-12T01:37:26.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60ae55a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60ae55a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810ba20 con 0x7f60a8108170 2024-09-12T01:37:26.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a819b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36110/0 (socket says 172.21.15.122:36110) 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a7fff640 1 -- 172.21.15.122:0/2245615836 learned_addr learned my addr 172.21.15.122:0/2245615836 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a7fff640 1 -- 172.21.15.122:0/2245615836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a819bb10 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a7fff640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a819b5d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f60980097f0 tx=0x7f6098030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098037040 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a819bda0 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6098040070 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f609803a390 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.884+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a819ea90 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.885+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f609803a630 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.885+0000 7f60a57fa640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f6090038f70 0x7f609003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.885+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a81085d0 con 0x7f60a8108170 2024-09-12T01:37:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.885+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f609806c6e0 con 0x7f60a8108170 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.885+0000 7f60a77fe640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f6090038f70 0x7f609003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.886+0000 7f60a77fe640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f6090038f70 0x7f609003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098c0 tx=0x7f609c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:25.890+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f609803b430 con 0x7f60a8108170 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.056+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f60a8102ff0 con 0x7f6090038f70 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.230+0000 7f60a57fa640 1 -- 172.21.15.122:0/2245615836 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60a8102ff0 con 0x7f6090038f70 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f6090038f70 msgr2=0x7f609003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f6090038f70 0x7f609003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098c0 tx=0x7f609c006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 msgr2=0x7f60a819b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a819b5d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f60980097f0 tx=0x7f6098030d80 comp rx=0 tx=0).stop 2024-09-12T01:37:26.750 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 shutdown_connections 2024-09-12T01:37:26.751 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f6090038f70 0x7f609003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.751 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 --2- 172.21.15.122:0/2245615836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60a8108170 0x7f60a819b5d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:26.751 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 >> 172.21.15.122:0/2245615836 conn(0x7f60a8101db0 msgr2=0x7f60a81028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:26.751 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 shutdown_connections 2024-09-12T01:37:26.751 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:26.232+0000 7f60ae55a640 1 -- 172.21.15.122:0/2245615836 wait complete. 2024-09-12T01:37:27.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:27 smithi122 ceph-mon[30925]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:27.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:27 smithi122 ceph-mon[30925]: Generating ssh key... 2024-09-12T01:37:27.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:27 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:27.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:27 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd95256e640 1 Processor -- start 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd95256e640 1 -- start start 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd95256e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd95256e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c108030 con 0x7fd94c107660 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:27.744 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36118/0 (socket says 172.21.15.122:36118) 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd94bfff640 1 -- 172.21.15.122:0/428256542 learned_addr learned my addr 172.21.15.122:0/428256542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.100+0000 7fd94bfff640 1 -- 172.21.15.122:0/428256542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c108810 con 0x7fd94c107660 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.101+0000 7fd94bfff640 1 --2- 172.21.15.122:0/428256542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c107a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009820 tx=0x7fd93c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae197cf9de54ce0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.101+0000 7fd94affd640 1 -- 172.21.15.122:0/428256542 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93c037070 con 0x7fd94c107660 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.101+0000 7fd94affd640 1 -- 172.21.15.122:0/428256542 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd93c032900 con 0x7fd94c107660 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.101+0000 7fd94affd640 1 -- 172.21.15.122:0/428256542 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93c030850 con 0x7fd94c107660 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 -- 172.21.15.122:0/428256542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 msgr2=0x7fd94c107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 --2- 172.21.15.122:0/428256542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c107a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009820 tx=0x7fd93c02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 -- 172.21.15.122:0/428256542 shutdown_connections 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 --2- 172.21.15.122:0/428256542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c107a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:27.745 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 -- 172.21.15.122:0/428256542 >> 172.21.15.122:0/428256542 conn(0x7fd94c102e30 msgr2=0x7fd94c105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 -- 172.21.15.122:0/428256542 shutdown_connections 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.102+0000 7fd95256e640 1 -- 172.21.15.122:0/428256542 wait complete. 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd95256e640 1 Processor -- start 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd95256e640 1 -- start start 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd95256e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd95256e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10bbc0 con 0x7fd94c107660 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36134/0 (socket says 172.21.15.122:36134) 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd94bfff640 1 -- 172.21.15.122:0/141650068 learned_addr learned my addr 172.21.15.122:0/141650068 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:27.746 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd94bfff640 1 -- 172.21.15.122:0/141650068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c19bb70 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.103+0000 7fd94bfff640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c19b630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0097f0 tx=0x7fd93c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.104+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93c037040 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.104+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd93c040070 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.104+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c19be00 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.104+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93c03a390 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.104+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c19eaf0 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.104+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd93c03a5e0 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.105+0000 7fd9497fa640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fd934038fc0 0x7fd93403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.105+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd93c06d4f0 con 0x7fd94c107660 2024-09-12T01:37:27.747 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.105+0000 7fd94b7fe640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fd934038fc0 0x7fd93403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.105+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c10bd40 con 0x7fd94c107660 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.105+0000 7fd94b7fe640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fd934038fc0 0x7fd93403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd9400098c0 tx=0x7fd940006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.109+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd93c03a920 con 0x7fd94c107660 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.276+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd94c104010 con 0x7fd934038fc0 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.277+0000 7fd9497fa640 1 -- 172.21.15.122:0/141650068 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd94c104010 con 0x7fd934038fc0 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fd934038fc0 msgr2=0x7fd93403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fd934038fc0 0x7fd93403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd9400098c0 tx=0x7fd940006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 msgr2=0x7fd94c19b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c19b630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0097f0 tx=0x7fd93c030d80 comp rx=0 tx=0).stop 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 shutdown_connections 2024-09-12T01:37:27.748 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fd934038fc0 0x7fd93403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:27.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 --2- 172.21.15.122:0/141650068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd94c107660 0x7fd94c19b630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:27.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 >> 172.21.15.122:0/141650068 conn(0x7fd94c102e30 msgr2=0x7fd94c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:27.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 shutdown_connections 2024-09-12T01:37:27.749 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:27.279+0000 7fd95256e640 1 -- 172.21.15.122:0/141650068 wait complete. 2024-09-12T01:37:27.749 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:37:27.749 INFO:teuthology.orchestra.run.smithi122.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T01:37:27.750 INFO:teuthology.orchestra.run.smithi122.stdout:Adding host smithi122... 2024-09-12T01:37:28.405 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:28 smithi122 ceph-mon[30925]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:29.345 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:29 smithi122 ceph-mon[30925]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:30.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:30 smithi122 ceph-mon[30925]: Deploying cephadm binary to smithi122 2024-09-12T01:37:32.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Added host 'smithi122' with addr '172.21.15.122' 2024-09-12T01:37:32.053 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.112+0000 7f7a7af7f640 1 Processor -- start 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a7af7f640 1 -- start start 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a7af7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a74107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a7af7f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74108110 con 0x7f7a74107740 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a78d1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a74107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a78d1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a74107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36140/0 (socket says 172.21.15.122:36140) 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a78d1b640 1 -- 172.21.15.122:0/51871672 learned_addr learned my addr 172.21.15.122:0/51871672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.113+0000 7f7a78d1b640 1 -- 172.21.15.122:0/51871672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741088a0 con 0x7f7a74107740 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.114+0000 7f7a78d1b640 1 --2- 172.21.15.122:0/51871672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a74107b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009820 tx=0x7f7a6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3591d4711f46e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.114+0000 7f7a737fe640 1 -- 172.21.15.122:0/51871672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68037070 con 0x7f7a74107740 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.114+0000 7f7a737fe640 1 -- 172.21.15.122:0/51871672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a68032900 con 0x7f7a74107740 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.114+0000 7f7a737fe640 1 -- 172.21.15.122:0/51871672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68030850 con 0x7f7a74107740 2024-09-12T01:37:32.054 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 -- 172.21.15.122:0/51871672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 msgr2=0x7f7a74107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 --2- 172.21.15.122:0/51871672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a74107b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009820 tx=0x7f7a6802a270 comp rx=0 tx=0).stop 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 -- 172.21.15.122:0/51871672 shutdown_connections 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 --2- 172.21.15.122:0/51871672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a74107b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 -- 172.21.15.122:0/51871672 >> 172.21.15.122:0/51871672 conn(0x7f7a74102eb0 msgr2=0x7f7a74105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 -- 172.21.15.122:0/51871672 shutdown_connections 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 -- 172.21.15.122:0/51871672 wait complete. 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 Processor -- start 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.115+0000 7f7a7af7f640 1 -- start start 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a7af7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a7419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a7af7f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410bc50 con 0x7f7a74107740 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a78d1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a7419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a78d1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a7419b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36148/0 (socket says 172.21.15.122:36148) 2024-09-12T01:37:32.055 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a78d1b640 1 -- 172.21.15.122:0/2002848154 learned_addr learned my addr 172.21.15.122:0/2002848154 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a78d1b640 1 -- 172.21.15.122:0/2002848154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7419bbf0 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a78d1b640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a7419b6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009940 tx=0x7f7a680326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68042070 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a68032be0 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7419be80 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.116+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68037040 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.117+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7419eb70 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.117+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7a68045070 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.118+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a74107bc0 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.118+0000 7f7a71ffb640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f7a64038f70 0x7f7a6403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.118+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a6806bf20 con 0x7f7a74107740 2024-09-12T01:37:32.056 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.118+0000 7f7a73fff640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f7a64038f70 0x7f7a6403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.118+0000 7f7a73fff640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f7a64038f70 0x7f7a6403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098c0 tx=0x7f7a60006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.122+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a6804b170 con 0x7f7a74107740 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:28.292+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}) -- 0x7f7a741041a0 con 0x7f7a64038f70 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.484+0000 7f7a71ffb640 1 -- 172.21.15.122:0/2002848154 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7a741041a0 con 0x7f7a64038f70 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f7a64038f70 msgr2=0x7f7a6403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f7a64038f70 0x7f7a6403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098c0 tx=0x7f7a60006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 msgr2=0x7f7a7419b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a7419b6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009940 tx=0x7f7a680326e0 comp rx=0 tx=0).stop 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 shutdown_connections 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f7a64038f70 0x7f7a6403b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 --2- 172.21.15.122:0/2002848154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a74107740 0x7f7a7419b6b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.487+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 >> 172.21.15.122:0/2002848154 conn(0x7f7a74102eb0 msgr2=0x7f7a74103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:32.057 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.488+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 shutdown_connections 2024-09-12T01:37:32.058 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:31.488+0000 7f7a7af7f640 1 -- 172.21.15.122:0/2002848154 wait complete. 2024-09-12T01:37:32.058 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mon service... 2024-09-12T01:37:32.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:32 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:32.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:32 smithi122 ceph-mon[30925]: Added host smithi122 2024-09-12T01:37:32.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:32 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc14701a640 1 Processor -- start 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc14701a640 1 -- start start 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc14701a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc140107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc14701a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc140108110 con 0x7fc140107740 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc144db6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc140107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc144db6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc140107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54790/0 (socket says 172.21.15.122:54790) 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc144db6640 1 -- 172.21.15.122:0/2142262203 learned_addr learned my addr 172.21.15.122:0/2142262203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.422+0000 7fc144db6640 1 -- 172.21.15.122:0/2142262203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1401088a0 con 0x7fc140107740 2024-09-12T01:37:33.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.423+0000 7fc144db6640 1 --2- 172.21.15.122:0/2142262203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc140107b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc134009820 tx=0x7fc13402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567a1d5dfc54f679 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.423+0000 7fc13f7fe640 1 -- 172.21.15.122:0/2142262203 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc134037070 con 0x7fc140107740 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.423+0000 7fc13f7fe640 1 -- 172.21.15.122:0/2142262203 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc134032900 con 0x7fc140107740 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.423+0000 7fc13f7fe640 1 -- 172.21.15.122:0/2142262203 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc134030850 con 0x7fc140107740 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 -- 172.21.15.122:0/2142262203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 msgr2=0x7fc140107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 --2- 172.21.15.122:0/2142262203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc140107b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc134009820 tx=0x7fc13402a270 comp rx=0 tx=0).stop 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 -- 172.21.15.122:0/2142262203 shutdown_connections 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 --2- 172.21.15.122:0/2142262203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc140107b40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 -- 172.21.15.122:0/2142262203 >> 172.21.15.122:0/2142262203 conn(0x7fc140102eb0 msgr2=0x7fc140105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 -- 172.21.15.122:0/2142262203 shutdown_connections 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.424+0000 7fc14701a640 1 -- 172.21.15.122:0/2142262203 wait complete. 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc14701a640 1 Processor -- start 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc14701a640 1 -- start start 2024-09-12T01:37:33.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc14701a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc14019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc14701a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc14010bc50 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc144db6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc14019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc144db6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc14019b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54796/0 (socket says 172.21.15.122:54796) 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc144db6640 1 -- 172.21.15.122:0/3367261784 learned_addr learned my addr 172.21.15.122:0/3367261784 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc144db6640 1 -- 172.21.15.122:0/3367261784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14019bb90 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc144db6640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc14019b650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc134009940 tx=0x7fc1340326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc134042070 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc134032be0 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14019be20 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.425+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc134037040 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.426+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14019eb10 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.426+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc134045070 con 0x7fc140107740 2024-09-12T01:37:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.426+0000 7fc13dffb640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc130038f70 0x7fc13003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.426+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc140107bc0 con 0x7fc140107740 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.427+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc13406bf10 con 0x7fc140107740 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.427+0000 7fc13ffff640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc130038f70 0x7fc13003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.427+0000 7fc13ffff640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc130038f70 0x7fc13003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0098c0 tx=0x7fc12c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.431+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc134045e10 con 0x7fc140107740 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.631+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc1401041c0 con 0x7fc130038f70 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.635+0000 7fc13dffb640 1 -- 172.21.15.122:0/3367261784 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc1401041c0 con 0x7fc130038f70 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.637+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc130038f70 msgr2=0x7fc13003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.637+0000 7fc14701a640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc130038f70 0x7fc13003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0098c0 tx=0x7fc12c006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.637+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 msgr2=0x7fc14019b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.637+0000 7fc14701a640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc14019b650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc134009940 tx=0x7fc1340326e0 comp rx=0 tx=0).stop 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.638+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 shutdown_connections 2024-09-12T01:37:33.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.638+0000 7fc14701a640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc130038f70 0x7fc13003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:33.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.638+0000 7fc14701a640 1 --2- 172.21.15.122:0/3367261784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc140107740 0x7fc14019b650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:33.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.638+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 >> 172.21.15.122:0/3367261784 conn(0x7fc140102eb0 msgr2=0x7fc140103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:33.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.638+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 shutdown_connections 2024-09-12T01:37:33.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:32.638+0000 7fc14701a640 1 -- 172.21.15.122:0/3367261784 wait complete. 2024-09-12T01:37:33.292 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mgr service... 2024-09-12T01:37:33.903 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:33 smithi122 ceph-mon[30925]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:33.903 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:33 smithi122 ceph-mon[30925]: Saving service mon spec with placement count:5 2024-09-12T01:37:33.903 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:33 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad529bb640 1 Processor -- start 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad529bb640 1 -- start start 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad529bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad529bb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c108050 con 0x7fad4c107680 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:34.867 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54800/0 (socket says 172.21.15.122:54800) 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad4bfff640 1 -- 172.21.15.122:0/1623204127 learned_addr learned my addr 172.21.15.122:0/1623204127 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.774+0000 7fad4bfff640 1 -- 172.21.15.122:0/1623204127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c108830 con 0x7fad4c107680 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.775+0000 7fad4bfff640 1 --2- 172.21.15.122:0/1623204127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c107a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009820 tx=0x7fad3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b70525ee8a46e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.775+0000 7fad4affd640 1 -- 172.21.15.122:0/1623204127 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c037070 con 0x7fad4c107680 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.775+0000 7fad4affd640 1 -- 172.21.15.122:0/1623204127 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad3c032900 con 0x7fad4c107680 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.775+0000 7fad4affd640 1 -- 172.21.15.122:0/1623204127 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c030850 con 0x7fad4c107680 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 -- 172.21.15.122:0/1623204127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 msgr2=0x7fad4c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 --2- 172.21.15.122:0/1623204127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c107a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009820 tx=0x7fad3c02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 -- 172.21.15.122:0/1623204127 shutdown_connections 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 --2- 172.21.15.122:0/1623204127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c107a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 -- 172.21.15.122:0/1623204127 >> 172.21.15.122:0/1623204127 conn(0x7fad4c102e50 msgr2=0x7fad4c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 -- 172.21.15.122:0/1623204127 shutdown_connections 2024-09-12T01:37:34.868 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 -- 172.21.15.122:0/1623204127 wait complete. 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 Processor -- start 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.776+0000 7fad529bb640 1 -- start start 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad529bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad529bb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10bbe0 con 0x7fad4c107680 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c19b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54812/0 (socket says 172.21.15.122:54812) 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad4bfff640 1 -- 172.21.15.122:0/1796393063 learned_addr learned my addr 172.21.15.122:0/1796393063 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad4bfff640 1 -- 172.21.15.122:0/1796393063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c19bbc0 con 0x7fad4c107680 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad4bfff640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c19b680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009940 tx=0x7fad3c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c042070 con 0x7fad4c107680 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c19be50 con 0x7fad4c107680 2024-09-12T01:37:34.869 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad3c032be0 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.777+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c037040 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.778+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c19eb40 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.778+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fad3c045070 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.778+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c10bd60 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.779+0000 7fad497fa640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fad34038fc0 0x7fad3403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.779+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad3c06c170 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.779+0000 7fad4b7fe640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fad34038fc0 0x7fad3403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.779+0000 7fad4b7fe640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fad34038fc0 0x7fad3403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad400098c0 tx=0x7fad40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.784+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad3c045360 con 0x7fad4c107680 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.968+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fad4c104120 con 0x7fad34038fc0 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.972+0000 7fad497fa640 1 -- 172.21.15.122:0/1796393063 <== mgr.14118 v2:172.21.15.122:6800/3688521366 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fad4c104120 con 0x7fad34038fc0 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fad34038fc0 msgr2=0x7fad3403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fad34038fc0 0x7fad3403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad400098c0 tx=0x7fad40006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 msgr2=0x7fad4c19b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c19b680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009940 tx=0x7fad3c0326e0 comp rx=0 tx=0).stop 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 shutdown_connections 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fad34038fc0 0x7fad3403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 --2- 172.21.15.122:0/1796393063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad4c107680 0x7fad4c19b680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 >> 172.21.15.122:0/1796393063 conn(0x7fad4c102e50 msgr2=0x7fad4c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 shutdown_connections 2024-09-12T01:37:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:33.974+0000 7fad529bb640 1 -- 172.21.15.122:0/1796393063 wait complete. 2024-09-12T01:37:35.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:34 smithi122 ceph-mon[30925]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:34 smithi122 ceph-mon[30925]: Saving service mgr spec with placement count:2 2024-09-12T01:37:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:34 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:36.437 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:36 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:36.437 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:36 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:36.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:36 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2344108522' entity='client.admin' 2024-09-12T01:37:36.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.629+0000 7fc37bb8f640 1 Processor -- start 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.629+0000 7fc37bb8f640 1 -- start start 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.630+0000 7fc37bb8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc37410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.630+0000 7fc37bb8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc37407be50 con 0x7fc3741080f0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.630+0000 7fc37992b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc37410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.630+0000 7fc37992b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc37410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54818/0 (socket says 172.21.15.122:54818) 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.630+0000 7fc37992b640 1 -- 172.21.15.122:0/1600818975 learned_addr learned my addr 172.21.15.122:0/1600818975 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.630+0000 7fc37992b640 1 -- 172.21.15.122:0/1600818975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37410aa20 con 0x7fc3741080f0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc37992b640 1 --2- 172.21.15.122:0/1600818975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc37410a4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc364009e80 tx=0x7fc36402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a56057821e5d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc378929640 1 -- 172.21.15.122:0/1600818975 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc364036070 con 0x7fc3741080f0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc378929640 1 -- 172.21.15.122:0/1600818975 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640318e0 con 0x7fc3741080f0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc378929640 1 -- 172.21.15.122:0/1600818975 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc364007a00 con 0x7fc3741080f0 2024-09-12T01:37:36.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc37bb8f640 1 -- 172.21.15.122:0/1600818975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 msgr2=0x7fc37410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc37bb8f640 1 --2- 172.21.15.122:0/1600818975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc37410a4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc364009e80 tx=0x7fc36402a7a0 comp rx=0 tx=0).stop 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc37bb8f640 1 -- 172.21.15.122:0/1600818975 shutdown_connections 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc37bb8f640 1 --2- 172.21.15.122:0/1600818975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc37410a4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.631+0000 7fc37bb8f640 1 -- 172.21.15.122:0/1600818975 >> 172.21.15.122:0/1600818975 conn(0x7fc374101db0 msgr2=0x7fc3741041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37bb8f640 1 -- 172.21.15.122:0/1600818975 shutdown_connections 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37bb8f640 1 -- 172.21.15.122:0/1600818975 wait complete. 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37bb8f640 1 Processor -- start 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37bb8f640 1 -- start start 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37bb8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc374107970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37bb8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc37410baa0 con 0x7fc3741080f0 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37992b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc374107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37992b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc374107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54830/0 (socket says 172.21.15.122:54830) 2024-09-12T01:37:36.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.632+0000 7fc37992b640 1 -- 172.21.15.122:0/2344108522 learned_addr learned my addr 172.21.15.122:0/2344108522 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc37992b640 1 -- 172.21.15.122:0/2344108522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc374107eb0 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc37992b640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc374107970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc364031cf0 tx=0x7fc364031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc364036050 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc364042070 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3741060a0 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc36403d9b0 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.633+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3741065e0 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.634+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc36403dc60 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.634+0000 7fc372ffd640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc35c038f70 0x7fc35c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.634+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc36406e3f0 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.634+0000 7fc37912a640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc35c038f70 0x7fc35c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.635+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc374108550 con 0x7fc3741080f0 2024-09-12T01:37:36.447 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.635+0000 7fc37912a640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc35c038f70 0x7fc35c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3680098c0 tx=0x7fc368006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.639+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc364044310 con 0x7fc3741080f0 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.815+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc374106860 con 0x7fc3741080f0 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.819+0000 7fc372ffd640 1 -- 172.21.15.122:0/2344108522 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc36404dcd0 con 0x7fc3741080f0 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc35c038f70 msgr2=0x7fc35c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc35c038f70 0x7fc35c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3680098c0 tx=0x7fc368006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 msgr2=0x7fc374107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc374107970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc364031cf0 tx=0x7fc364031d20 comp rx=0 tx=0).stop 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 shutdown_connections 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fc35c038f70 0x7fc35c03b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 --2- 172.21.15.122:0/2344108522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3741080f0 0x7fc374107970 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 >> 172.21.15.122:0/2344108522 conn(0x7fc374101db0 msgr2=0x7fc3741028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 shutdown_connections 2024-09-12T01:37:36.448 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:35.821+0000 7fc37bb8f640 1 -- 172.21.15.122:0/2344108522 wait complete. 2024-09-12T01:37:38.073 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdaecbbe640 1 Processor -- start 2024-09-12T01:37:38.073 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdaecbbe640 1 -- start start 2024-09-12T01:37:38.073 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdaecbbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae810a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:38.073 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdaecbbe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdae807bdd0 con 0x7fdae8108170 2024-09-12T01:37:38.073 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdae659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:38.073 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdae659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54846/0 (socket says 172.21.15.122:54846) 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdae659c640 1 -- 172.21.15.122:0/2909629405 learned_addr learned my addr 172.21.15.122:0/2909629405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.786+0000 7fdae659c640 1 -- 172.21.15.122:0/2909629405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae810aaa0 con 0x7fdae8108170 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.787+0000 7fdae659c640 1 --2- 172.21.15.122:0/2909629405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae810a560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009820 tx=0x7fdad802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc5db1eb052a38a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.787+0000 7fdae5d9b640 1 -- 172.21.15.122:0/2909629405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad8037070 con 0x7fdae8108170 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.787+0000 7fdae5d9b640 1 -- 172.21.15.122:0/2909629405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad8032920 con 0x7fdae8108170 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.787+0000 7fdae5d9b640 1 -- 172.21.15.122:0/2909629405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad8030810 con 0x7fdae8108170 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 -- 172.21.15.122:0/2909629405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 msgr2=0x7fdae810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 --2- 172.21.15.122:0/2909629405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae810a560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009820 tx=0x7fdad802a270 comp rx=0 tx=0).stop 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 -- 172.21.15.122:0/2909629405 shutdown_connections 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 --2- 172.21.15.122:0/2909629405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae810a560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 -- 172.21.15.122:0/2909629405 >> 172.21.15.122:0/2909629405 conn(0x7fdae8101db0 msgr2=0x7fdae81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 -- 172.21.15.122:0/2909629405 shutdown_connections 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 -- 172.21.15.122:0/2909629405 wait complete. 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 Processor -- start 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.788+0000 7fdaecbbe640 1 -- start start 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdaecbbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdaecbbe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810ba60 con 0x7fdae8108170 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdae659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdae659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54862/0 (socket says 172.21.15.122:54862) 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdae659c640 1 -- 172.21.15.122:0/976446462 learned_addr learned my addr 172.21.15.122:0/976446462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdae659c640 1 -- 172.21.15.122:0/976446462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae819bb30 con 0x7fdae8108170 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdae659c640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae819b5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdad80097f0 tx=0x7fdad80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad8037040 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad8030cc0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae819bdc0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad803a6b0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.789+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae819eab0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.790+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdad803baf0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.790+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae81085d0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.791+0000 7fdadf7fe640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fdad4038f70 0x7fdad403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.791+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdad806c4d0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.791+0000 7fdadffff640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fdad4038f70 0x7fdad403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.791+0000 7fdadffff640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fdad4038f70 0x7fdad403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdad00098c0 tx=0x7fdad0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.795+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdad8042dc0 con 0x7fdae8108170 2024-09-12T01:37:38.076 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.978+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdae819ec80 con 0x7fdae8108170 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.981+0000 7fdadf7fe640 1 -- 172.21.15.122:0/976446462 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdad803a8c0 con 0x7fdae8108170 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fdad4038f70 msgr2=0x7fdad403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fdad4038f70 0x7fdad403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdad00098c0 tx=0x7fdad0006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 msgr2=0x7fdae819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae819b5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdad80097f0 tx=0x7fdad80327b0 comp rx=0 tx=0).stop 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 shutdown_connections 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7fdad4038f70 0x7fdad403b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 --2- 172.21.15.122:0/976446462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8108170 0x7fdae819b5f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 >> 172.21.15.122:0/976446462 conn(0x7fdae8101db0 msgr2=0x7fdae81028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 shutdown_connections 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:36.984+0000 7fdaecbbe640 1 -- 172.21.15.122:0/976446462 wait complete. 2024-09-12T01:37:38.077 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling the dashboard module... 2024-09-12T01:37:38.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:37 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/976446462' entity='client.admin' 2024-09-12T01:37:38.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:37 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:39.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-mon[30925]: from='mgr.14118 172.21.15.122:0/2823861750' entity='mgr.x' 2024-09-12T01:37:39.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2496223883' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T01:37:39.853 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: ignoring --setuser ceph since I am not root 2024-09-12T01:37:39.853 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: ignoring --setgroup ceph since I am not root 2024-09-12T01:37:39.853 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:39.648+0000 7fdae4df8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:37:39.854 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:39.706+0000 7fdae4df8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:37:39.854 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:39.853+0000 7fdae4df8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:37:40.213 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:39.920+0000 7fdae4df8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:37:40.213 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:40.070+0000 7fdae4df8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:37:40.213 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:40.213+0000 7fdae4df8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:37:40.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.461+0000 7efede3ea640 1 Processor -- start 2024-09-12T01:37:40.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.461+0000 7efede3ea640 1 -- start start 2024-09-12T01:37:40.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.461+0000 7efede3ea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed8107a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:40.338 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.461+0000 7efede3ea640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efed8108010 con 0x7efed8107640 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.462+0000 7efed7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed8107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.462+0000 7efed7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed8107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54874/0 (socket says 172.21.15.122:54874) 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.462+0000 7efed7fff640 1 -- 172.21.15.122:0/3506388302 learned_addr learned my addr 172.21.15.122:0/3506388302 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.462+0000 7efed7fff640 1 -- 172.21.15.122:0/3506388302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed8108840 con 0x7efed8107640 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.463+0000 7efed7fff640 1 --2- 172.21.15.122:0/3506388302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed8107a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efec8009820 tx=0x7efec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9302871eca17639 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.463+0000 7efed6ffd640 1 -- 172.21.15.122:0/3506388302 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec8037070 con 0x7efed8107640 2024-09-12T01:37:40.339 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.463+0000 7efed6ffd640 1 -- 172.21.15.122:0/3506388302 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8032920 con 0x7efed8107640 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.463+0000 7efed6ffd640 1 -- 172.21.15.122:0/3506388302 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec8030810 con 0x7efed8107640 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 -- 172.21.15.122:0/3506388302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 msgr2=0x7efed8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 --2- 172.21.15.122:0/3506388302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed8107a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efec8009820 tx=0x7efec802a270 comp rx=0 tx=0).stop 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 -- 172.21.15.122:0/3506388302 shutdown_connections 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 --2- 172.21.15.122:0/3506388302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed8107a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 -- 172.21.15.122:0/3506388302 >> 172.21.15.122:0/3506388302 conn(0x7efed8102e10 msgr2=0x7efed8105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 -- 172.21.15.122:0/3506388302 shutdown_connections 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.464+0000 7efede3ea640 1 -- 172.21.15.122:0/3506388302 wait complete. 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.465+0000 7efede3ea640 1 Processor -- start 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.465+0000 7efede3ea640 1 -- start start 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.465+0000 7efede3ea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.465+0000 7efede3ea640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efed810bbf0 con 0x7efed8107640 2024-09-12T01:37:40.340 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54880/0 (socket says 172.21.15.122:54880) 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed7fff640 1 -- 172.21.15.122:0/2496223883 learned_addr learned my addr 172.21.15.122:0/2496223883 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed7fff640 1 -- 172.21.15.122:0/2496223883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed819bc00 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed7fff640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed819b6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efec80097f0 tx=0x7efec80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec8037040 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec8030c40 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed819be90 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed819eb80 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.466+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efec803a5b0 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.467+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efec803a7c0 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.467+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed810bd70 con 0x7efed8107640 2024-09-12T01:37:40.341 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.467+0000 7efed57fa640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7efec0038fc0 0x7efec003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.468+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efec806c4a0 con 0x7efed8107640 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.468+0000 7efed77fe640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7efec0038fc0 0x7efec003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.468+0000 7efed77fe640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7efec0038fc0 0x7efec003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efecc0098c0 tx=0x7efecc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.473+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efec80371e0 con 0x7efed8107640 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:38.707+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7efed819ed50 con 0x7efed8107640 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.501+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7efec8030df0 con 0x7efed8107640 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.502+0000 7efed57fa640 1 -- 172.21.15.122:0/2496223883 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7efec8044400 con 0x7efed8107640 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7efec0038fc0 msgr2=0x7efec003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7efec0038fc0 0x7efec003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efecc0098c0 tx=0x7efecc006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 msgr2=0x7efed819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed819b6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efec80097f0 tx=0x7efec80327b0 comp rx=0 tx=0).stop 2024-09-12T01:37:40.342 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 shutdown_connections 2024-09-12T01:37:40.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7efec0038fc0 0x7efec003b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:40.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 --2- 172.21.15.122:0/2496223883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efed8107640 0x7efed819b6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:40.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 >> 172.21.15.122:0/2496223883 conn(0x7efed8102e10 msgr2=0x7efed81039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:40.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 shutdown_connections 2024-09-12T01:37:40.343 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:39.504+0000 7efede3ea640 1 -- 172.21.15.122:0/2496223883 wait complete. 2024-09-12T01:37:40.790 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2496223883' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T01:37:40.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-mon[30925]: mgrmap e9: x(active, since 17s) 2024-09-12T01:37:40.791 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:40.535+0000 7fdae4df8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:37:40.791 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:40.598+0000 7fdae4df8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:37:40.791 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:40.790+0000 7fdae4df8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:37:41.474 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.720+0000 7f1deaece640 1 Processor -- start 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.720+0000 7f1deaece640 1 -- start start 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.720+0000 7f1deaece640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.720+0000 7f1deaece640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1de407be50 con 0x7f1de41080f0 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.721+0000 7f1de8c6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.721+0000 7f1de8c6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47940/0 (socket says 172.21.15.122:47940) 2024-09-12T01:37:41.475 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.721+0000 7f1de8c6a640 1 -- 172.21.15.122:0/459546978 learned_addr learned my addr 172.21.15.122:0/459546978 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:41.476 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.721+0000 7f1de8c6a640 1 -- 172.21.15.122:0/459546978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de410aa20 con 0x7f1de41080f0 2024-09-12T01:37:41.476 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1de8c6a640 1 --2- 172.21.15.122:0/459546978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de410a4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009820 tx=0x7f1dd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ecfde5aa6e27da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:41.476 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1de37fe640 1 -- 172.21.15.122:0/459546978 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1dd8037070 con 0x7f1de41080f0 2024-09-12T01:37:41.476 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1de37fe640 1 -- 172.21.15.122:0/459546978 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd8032920 con 0x7f1de41080f0 2024-09-12T01:37:41.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1de37fe640 1 -- 172.21.15.122:0/459546978 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1dd8030810 con 0x7f1de41080f0 2024-09-12T01:37:41.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1deaece640 1 -- 172.21.15.122:0/459546978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 msgr2=0x7f1de410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:41.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1deaece640 1 --2- 172.21.15.122:0/459546978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de410a4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009820 tx=0x7f1dd802a270 comp rx=0 tx=0).stop 2024-09-12T01:37:41.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1deaece640 1 -- 172.21.15.122:0/459546978 shutdown_connections 2024-09-12T01:37:41.481 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1deaece640 1 --2- 172.21.15.122:0/459546978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de410a4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.722+0000 7f1deaece640 1 -- 172.21.15.122:0/459546978 >> 172.21.15.122:0/459546978 conn(0x7f1de4101db0 msgr2=0x7f1de41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1deaece640 1 -- 172.21.15.122:0/459546978 shutdown_connections 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1deaece640 1 -- 172.21.15.122:0/459546978 wait complete. 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1deaece640 1 Processor -- start 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1deaece640 1 -- start start 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1deaece640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1deaece640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410baa0 con 0x7f1de41080f0 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.723+0000 7f1de8c6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de8c6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de419b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47956/0 (socket says 172.21.15.122:47956) 2024-09-12T01:37:41.482 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de8c6a640 1 -- 172.21.15.122:0/1711478622 learned_addr learned my addr 172.21.15.122:0/1711478622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de8c6a640 1 -- 172.21.15.122:0/1711478622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de419bb80 con 0x7f1de41080f0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de8c6a640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de419b640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8030380 tx=0x7f1dd8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1dd8037040 con 0x7f1de41080f0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de419be10 con 0x7f1de41080f0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd8040070 con 0x7f1de41080f0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1dd803a7b0 con 0x7f1de41080f0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.724+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de419eb00 con 0x7f1de41080f0 2024-09-12T01:37:41.483 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.725+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1dd803aa30 con 0x7f1de41080f0 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.725+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de4108550 con 0x7f1de41080f0 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.725+0000 7f1de1ffb640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 0x7f1dd403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.725+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1dd803b900 con 0x7f1de41080f0 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.725+0000 7f1de3fff640 1 -- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 msgr2=0x7f1dd403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3688521366 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.725+0000 7f1de3fff640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 0x7f1dd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.730+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1dd8041270 con 0x7f1de41080f0 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.927+0000 7f1de3fff640 1 -- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 msgr2=0x7f1dd403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3688521366 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.927+0000 7f1de3fff640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 0x7f1dd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.949+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1de419ecd0 con 0x7f1de41080f0 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.950+0000 7f1de1ffb640 1 -- 172.21.15.122:0/1711478622 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1de419ecd0 con 0x7f1de41080f0 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 msgr2=0x7f1dd403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:37:41.484 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 0x7f1dd403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 msgr2=0x7f1de419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de419b640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8030380 tx=0x7f1dd8032670 comp rx=0 tx=0).stop 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 shutdown_connections 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7f1dd4038fc0 0x7f1dd403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 --2- 172.21.15.122:0/1711478622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1de41080f0 0x7f1de419b640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 >> 172.21.15.122:0/1711478622 conn(0x7f1de4101db0 msgr2=0x7f1de4102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 shutdown_connections 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:40.952+0000 7f1deaece640 1 -- 172.21.15.122:0/1711478622 wait complete. 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-09-12T01:37:41.485 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 9... 2024-09-12T01:37:41.769 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:41 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1711478622' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T01:37:41.769 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:41 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:41.700+0000 7fdae4df8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:37:42.082 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:41 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:41.769+0000 7fdae4df8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:37:42.738 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:42.479+0000 7fdae4df8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:37:42.738 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:42.596+0000 7fdae4df8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:37:43.017 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:42.739+0000 7fdae4df8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:37:43.017 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:42.800+0000 7fdae4df8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:37:43.018 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:42.950+0000 7fdae4df8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:37:43.018 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.017+0000 7fdae4df8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:37:43.284 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.077+0000 7fdae4df8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:37:43.285 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.237+0000 7fdae4df8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:37:43.571 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.303+0000 7fdae4df8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:37:43.571 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.369+0000 7fdae4df8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:37:43.571 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.452+0000 7fdae4df8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:37:43.571 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.512+0000 7fdae4df8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:37:43.572 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:43.571+0000 7fdae4df8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:37:44.738 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:44.365+0000 7fdae4df8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: Active manager daemon x restarted 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: Activating manager daemon x 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: mgrmap e10: x(active, starting, since 0.00318508s) 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T01:37:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T01:37:44.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: Manager daemon x is now available 2024-09-12T01:37:44.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:44.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T01:37:44.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T01:37:45.932 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:45.494+0000 7fda5010c640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:01:37:45] ENGINE Error in HTTPServer.serve 2024-09-12T01:37:45.932 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: Traceback (most recent call last): 2024-09-12T01:37:45.932 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:37:45.932 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self._connections.run(self.expiration_interval) 2024-09-12T01:37:45.932 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:37:45.932 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self._run(expiration_interval) 2024-09-12T01:37:45.933 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:37:45.933 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:37:45.933 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:37:45.933 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:37:45.933 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: s = self.context.wrap_socket( 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: return self.sslsocket_class._create( 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self.do_handshake() 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: self._sslobj.do_handshake() 2024-09-12T01:37:45.934 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:37:45.935 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:37:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:37:46.194 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-09-12T01:37:46.194 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T01:37:46.194 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafb656d640 1 Processor -- start 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafb656d640 1 -- start start 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafb656d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb010a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafb656d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafb007be20 con 0x7fafb01080c0 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafaffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb010a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafaffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb010a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47966/0 (socket says 172.21.15.122:47966) 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.877+0000 7fafaffff640 1 -- 172.21.15.122:0/713065150 learned_addr learned my addr 172.21.15.122:0/713065150 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.878+0000 7fafaffff640 1 -- 172.21.15.122:0/713065150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb010a9f0 con 0x7fafb01080c0 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.878+0000 7fafaffff640 1 --2- 172.21.15.122:0/713065150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb010a4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009820 tx=0x7fafa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208468a627d3d6a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafaf7fe640 1 -- 172.21.15.122:0/713065150 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa0037070 con 0x7fafb01080c0 2024-09-12T01:37:46.195 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafaf7fe640 1 -- 172.21.15.122:0/713065150 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa0032920 con 0x7fafb01080c0 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafaf7fe640 1 -- 172.21.15.122:0/713065150 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa0030810 con 0x7fafb01080c0 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 -- 172.21.15.122:0/713065150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 msgr2=0x7fafb010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 --2- 172.21.15.122:0/713065150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb010a4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009820 tx=0x7fafa002a270 comp rx=0 tx=0).stop 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 -- 172.21.15.122:0/713065150 shutdown_connections 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 --2- 172.21.15.122:0/713065150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb010a4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 -- 172.21.15.122:0/713065150 >> 172.21.15.122:0/713065150 conn(0x7fafb0101ca0 msgr2=0x7fafb01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 -- 172.21.15.122:0/713065150 shutdown_connections 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.879+0000 7fafb656d640 1 -- 172.21.15.122:0/713065150 wait complete. 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafb656d640 1 Processor -- start 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafb656d640 1 -- start start 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafb656d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafb656d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010ba10 con 0x7fafb01080c0 2024-09-12T01:37:46.196 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafaffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafaffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47974/0 (socket says 172.21.15.122:47974) 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafaffff640 1 -- 172.21.15.122:0/851601552 learned_addr learned my addr 172.21.15.122:0/851601552 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.880+0000 7fafaffff640 1 -- 172.21.15.122:0/851601552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb019bb20 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.881+0000 7fafaffff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb019b5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafa0030380 tx=0x7fafa0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.881+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa0037040 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.881+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb019bdb0 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.881+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa0040070 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.881+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa003a7b0 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.881+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb019eaa0 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.882+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fafa003aa30 con 0x7fafb01080c0 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.882+0000 7fafadffb640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:46.197 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.882+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 --> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fafb0108520 con 0x7faf9c038fc0 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.882+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fafa006c740 con 0x7fafb01080c0 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.882+0000 7fafa7fff640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 msgr2=0x7faf9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3688521366 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:41.882+0000 7fafa7fff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:42.083+0000 7fafa7fff640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 msgr2=0x7faf9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3688521366 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:42.083+0000 7fafa7fff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:42.484+0000 7fafa7fff640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 msgr2=0x7faf9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3688521366 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:42.484+0000 7fafa7fff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:43.285+0000 7fafa7fff640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 msgr2=0x7faf9c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3688521366 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:43.285+0000 7fafa7fff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:44.368+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fafa006b1d0 con 0x7fafb01080c0 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:44.368+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 msgr2=0x7faf9c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:44.368+0000 7fafadffb640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:46.198 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.370+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fafa0045980 con 0x7fafb01080c0 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.370+0000 7fafadffb640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf9c03c890 0x7faf9c03ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.370+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fafb0108520 con 0x7faf9c03c890 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.371+0000 7fafa7fff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf9c03c890 0x7faf9c03ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.371+0000 7fafa7fff640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf9c03c890 0x7faf9c03ec80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf98003980 tx=0x7faf98006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.372+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fafb0108520 con 0x7faf9c03c890 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.378+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fafb0102fe0 con 0x7faf9c03c890 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafadffb640 1 -- 172.21.15.122:0/851601552 <== mgr.14150 v2:172.21.15.122:6800/3245830997 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fafb0102fe0 con 0x7faf9c03c890 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf9c03c890 msgr2=0x7faf9c03ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf9c03c890 0x7faf9c03ec80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf98003980 tx=0x7faf98006e10 comp rx=0 tx=0).stop 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 msgr2=0x7fafb019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:46.199 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb019b5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafa0030380 tx=0x7fafa0032670 comp rx=0 tx=0).stop 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 shutdown_connections 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf9c03c890 0x7faf9c03ec80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:6800/3688521366,v1:172.21.15.122:6801/3688521366] conn(0x7faf9c038fc0 0x7faf9c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 --2- 172.21.15.122:0/851601552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb01080c0 0x7fafb019b5e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 >> 172.21.15.122:0/851601552 conn(0x7fafb0101ca0 msgr2=0x7fafb01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 shutdown_connections 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:45.379+0000 7fafb656d640 1 -- 172.21.15.122:0/851601552 wait complete. 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 9 is available 2024-09-12T01:37:46.200 INFO:teuthology.orchestra.run.smithi122.stdout:Generating a dashboard self-signed certificate... 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:45] ENGINE Bus STARTING 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: mgrmap e11: x(active, since 1.0062s) 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:45] ENGINE Serving on https://172.21.15.122:7150 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:45] ENGINE Error in HTTPServer.serve 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout: Traceback (most recent call last): 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T01:37:46.434 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._connections.run(self.expiration_interval) 2024-09-12T01:37:46.435 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T01:37:46.435 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._run(expiration_interval) 2024-09-12T01:37:46.435 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T01:37:46.435 INFO:journalctl@ceph.mon.a.smithi122.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T01:37:46.435 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T01:37:46.435 INFO:journalctl@ceph.mon.a.smithi122.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T01:37:46.436 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T01:37:46.436 INFO:journalctl@ceph.mon.a.smithi122.stdout: s = self.context.wrap_socket( 2024-09-12T01:37:46.436 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T01:37:46.436 INFO:journalctl@ceph.mon.a.smithi122.stdout: return self.sslsocket_class._create( 2024-09-12T01:37:46.436 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T01:37:46.436 INFO:journalctl@ceph.mon.a.smithi122.stdout: self.do_handshake() 2024-09-12T01:37:46.437 INFO:journalctl@ceph.mon.a.smithi122.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T01:37:46.437 INFO:journalctl@ceph.mon.a.smithi122.stdout: self._sslobj.do_handshake() 2024-09-12T01:37:46.437 INFO:journalctl@ceph.mon.a.smithi122.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T01:37:46.437 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:47.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T01:37:47.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.598+0000 7f4b3847a640 1 Processor -- start 2024-09-12T01:37:47.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.598+0000 7f4b3847a640 1 -- start start 2024-09-12T01:37:47.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.599+0000 7f4b3847a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b30106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:47.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.599+0000 7f4b3847a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30106ef0 con 0x7f4b30106520 2024-09-12T01:37:47.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.599+0000 7f4b36216640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b30106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:47.325 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.599+0000 7f4b36216640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b30106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48042/0 (socket says 172.21.15.122:48042) 2024-09-12T01:37:47.325 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.599+0000 7f4b36216640 1 -- 172.21.15.122:0/3866479353 learned_addr learned my addr 172.21.15.122:0/3866479353 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:47.325 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.599+0000 7f4b36216640 1 -- 172.21.15.122:0/3866479353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b30107720 con 0x7f4b30106520 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.600+0000 7f4b36216640 1 --2- 172.21.15.122:0/3866479353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b30106920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009e80 tx=0x7f4b2002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93058a63d56d8f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.600+0000 7f4b35214640 1 -- 172.21.15.122:0/3866479353 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b20036070 con 0x7f4b30106520 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.600+0000 7f4b35214640 1 -- 172.21.15.122:0/3866479353 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b200318e0 con 0x7f4b30106520 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.600+0000 7f4b35214640 1 -- 172.21.15.122:0/3866479353 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b20007a00 con 0x7f4b30106520 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 -- 172.21.15.122:0/3866479353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 msgr2=0x7f4b30106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 --2- 172.21.15.122:0/3866479353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b30106920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009e80 tx=0x7f4b2002a7a0 comp rx=0 tx=0).stop 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 -- 172.21.15.122:0/3866479353 shutdown_connections 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 --2- 172.21.15.122:0/3866479353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b30106920 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 -- 172.21.15.122:0/3866479353 >> 172.21.15.122:0/3866479353 conn(0x7f4b30101db0 msgr2=0x7f4b301041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 -- 172.21.15.122:0/3866479353 shutdown_connections 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 -- 172.21.15.122:0/3866479353 wait complete. 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 Processor -- start 2024-09-12T01:37:47.326 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.601+0000 7f4b3847a640 1 -- start start 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b3847a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b3019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b3847a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010aad0 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b36216640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b3019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b36216640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b3019b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48052/0 (socket says 172.21.15.122:48052) 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b36216640 1 -- 172.21.15.122:0/596855493 learned_addr learned my addr 172.21.15.122:0/596855493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b36216640 1 -- 172.21.15.122:0/596855493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3019bb90 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b36216640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b3019b650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4b20031cc0 tx=0x7f4b20031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b20036050 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b20042070 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3019be20 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b2003d8e0 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.602+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3019eb10 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.603+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f4b2003bc80 con 0x7f4b30106520 2024-09-12T01:37:47.327 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.603+0000 7f4b2f7fe640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b18038f00 0x7f4b1803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.603+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3010ac50 con 0x7f4b30106520 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.603+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b2006d300 con 0x7f4b30106520 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.603+0000 7f4b35a15640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b18038f00 0x7f4b1803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.604+0000 7f4b35a15640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b18038f00 0x7f4b1803b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b240098c0 tx=0x7f4b24006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.608+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b2004c200 con 0x7f4b30106520 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.792+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4b2006be30 con 0x7f4b30106520 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.798+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f4b30103000 con 0x7f4b18038f00 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.844+0000 7f4b2f7fe640 1 -- 172.21.15.122:0/596855493 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f4b30103000 con 0x7f4b18038f00 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b18038f00 msgr2=0x7f4b1803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b18038f00 0x7f4b1803b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b240098c0 tx=0x7f4b24006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 msgr2=0x7f4b3019b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:47.328 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b3019b650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4b20031cc0 tx=0x7f4b20031cf0 comp rx=0 tx=0).stop 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 shutdown_connections 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b18038f00 0x7f4b1803b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 --2- 172.21.15.122:0/596855493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b30106520 0x7f4b3019b650 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.846+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 >> 172.21.15.122:0/596855493 conn(0x7f4b30101db0 msgr2=0x7f4b301028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.847+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 shutdown_connections 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:46.847+0000 7f4b3847a640 1 -- 172.21.15.122:0/596855493 wait complete. 2024-09-12T01:37:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial admin user... 2024-09-12T01:37:47.643 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:47 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:45] ENGINE Serving on http://172.21.15.122:8765 2024-09-12T01:37:47.643 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:47 smithi122 ceph-mon[30925]: [12/Sep/2024:01:37:45] ENGINE Bus STARTED 2024-09-12T01:37:47.643 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:47.643 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:47 smithi122 ceph-mon[30925]: mgrmap e12: x(active, since 2s) 2024-09-12T01:37:47.643 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:47.643 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:48.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:48 smithi122 ceph-mon[30925]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:48.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:49.372 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$g0FiW30uow7OiHNBPjjZzeQZ8SD4IXfXqu.t5nCHS33fxeY3NDepe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726105068, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T01:37:49.372 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.724+0000 7f51e0f59640 1 Processor -- start 2024-09-12T01:37:49.372 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.724+0000 7f51e0f59640 1 -- start start 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.725+0000 7f51e0f59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc105cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.725+0000 7f51e0f59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1062a0 con 0x7f51dc1058d0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.725+0000 7f51da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc105cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.725+0000 7f51da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc105cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48064/0 (socket says 172.21.15.122:48064) 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.725+0000 7f51da59c640 1 -- 172.21.15.122:0/952146096 learned_addr learned my addr 172.21.15.122:0/952146096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.725+0000 7f51da59c640 1 -- 172.21.15.122:0/952146096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc1088b0 con 0x7f51dc1058d0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.726+0000 7f51da59c640 1 --2- 172.21.15.122:0/952146096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc105cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009e80 tx=0x7f51cc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7920e124d6e5256 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.726+0000 7f51d959a640 1 -- 172.21.15.122:0/952146096 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51cc036070 con 0x7f51dc1058d0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.726+0000 7f51d959a640 1 -- 172.21.15.122:0/952146096 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51cc0318e0 con 0x7f51dc1058d0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.726+0000 7f51d959a640 1 -- 172.21.15.122:0/952146096 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51cc007a00 con 0x7f51dc1058d0 2024-09-12T01:37:49.373 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.726+0000 7f51e0f59640 1 -- 172.21.15.122:0/952146096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 msgr2=0x7f51dc105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.726+0000 7f51e0f59640 1 --2- 172.21.15.122:0/952146096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc105cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009e80 tx=0x7f51cc02a7a0 comp rx=0 tx=0).stop 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 -- 172.21.15.122:0/952146096 shutdown_connections 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 --2- 172.21.15.122:0/952146096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc105cd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 -- 172.21.15.122:0/952146096 >> 172.21.15.122:0/952146096 conn(0x7f51dc101480 msgr2=0x7f51dc1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 -- 172.21.15.122:0/952146096 shutdown_connections 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 -- 172.21.15.122:0/952146096 wait complete. 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 Processor -- start 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 -- start start 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.727+0000 7f51e0f59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc10bae0 con 0x7f51dc1058d0 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48066/0 (socket says 172.21.15.122:48066) 2024-09-12T01:37:49.374 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51da59c640 1 -- 172.21.15.122:0/286930431 learned_addr learned my addr 172.21.15.122:0/286930431 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51da59c640 1 -- 172.21.15.122:0/286930431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc19ddd0 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51da59c640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc19d890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51cc031cc0 tx=0x7f51cc031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51cc036050 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51cc042070 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc19e060 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51cc03d8e0 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.728+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc1a0d50 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.729+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f51cc03dbe0 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.729+0000 7f51cb7fe640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51c4038fc0 0x7f51c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.729+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f51cc06d740 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.729+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51dc105d50 con 0x7f51dc1058d0 2024-09-12T01:37:49.375 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.729+0000 7f51d9d9b640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51c4038fc0 0x7f51c403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.730+0000 7f51d9d9b640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51c4038fc0 0x7f51c403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098c0 tx=0x7f51d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.734+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51cc044310 con 0x7f51dc1058d0 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:47.919+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f51dc1a0f20 con 0x7f51c4038fc0 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.144+0000 7f51cb7fe640 1 -- 172.21.15.122:0/286930431 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f51dc1a0f20 con 0x7f51c4038fc0 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51c4038fc0 msgr2=0x7f51c403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51c4038fc0 0x7f51c403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098c0 tx=0x7f51d0006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 msgr2=0x7f51dc19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc19d890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51cc031cc0 tx=0x7f51cc031cf0 comp rx=0 tx=0).stop 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 shutdown_connections 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51c4038fc0 0x7f51c403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 --2- 172.21.15.122:0/286930431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51dc1058d0 0x7f51dc19d890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:49.376 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 >> 172.21.15.122:0/286930431 conn(0x7f51dc101480 msgr2=0x7f51dc101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:49.377 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 shutdown_connections 2024-09-12T01:37:49.377 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:48.147+0000 7f51e0f59640 1 -- 172.21.15.122:0/286930431 wait complete. 2024-09-12T01:37:49.377 INFO:teuthology.orchestra.run.smithi122.stdout:Fetching dashboard port number... 2024-09-12T01:37:49.690 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:49 smithi122 ceph-mon[30925]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:50.508 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:50 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2134630860' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d74093640 1 Processor -- start 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d74093640 1 -- start start 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d74093640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d74093640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c108520 con 0x7f3d6c107b50 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d71e2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:50.554 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d71e2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48074/0 (socket says 172.21.15.122:48074) 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.769+0000 7f3d71e2f640 1 -- 172.21.15.122:0/1226606978 learned_addr learned my addr 172.21.15.122:0/1226606978 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.770+0000 7f3d71e2f640 1 -- 172.21.15.122:0/1226606978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c1067b0 con 0x7f3d6c107b50 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.770+0000 7f3d71e2f640 1 --2- 172.21.15.122:0/1226606978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c107f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009820 tx=0x7f3d5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229982448e095c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.770+0000 7f3d70e2d640 1 -- 172.21.15.122:0/1226606978 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d5c037070 con 0x7f3d6c107b50 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.770+0000 7f3d70e2d640 1 -- 172.21.15.122:0/1226606978 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5c032920 con 0x7f3d6c107b50 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.770+0000 7f3d70e2d640 1 -- 172.21.15.122:0/1226606978 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d5c030810 con 0x7f3d6c107b50 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 -- 172.21.15.122:0/1226606978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 msgr2=0x7f3d6c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 --2- 172.21.15.122:0/1226606978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c107f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009820 tx=0x7f3d5c02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 -- 172.21.15.122:0/1226606978 shutdown_connections 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 --2- 172.21.15.122:0/1226606978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c107f50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 -- 172.21.15.122:0/1226606978 >> 172.21.15.122:0/1226606978 conn(0x7f3d6c101550 msgr2=0x7f3d6c103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:50.555 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 -- 172.21.15.122:0/1226606978 shutdown_connections 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.771+0000 7f3d74093640 1 -- 172.21.15.122:0/1226606978 wait complete. 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d74093640 1 Processor -- start 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d74093640 1 -- start start 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d74093640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d74093640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1056a0 con 0x7f3d6c107b50 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d71e2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d71e2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48086/0 (socket says 172.21.15.122:48086) 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d71e2f640 1 -- 172.21.15.122:0/2134630860 learned_addr learned my addr 172.21.15.122:0/2134630860 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d71e2f640 1 -- 172.21.15.122:0/2134630860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c199960 con 0x7f3d6c107b50 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d71e2f640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c199420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c030560 tx=0x7f3d5c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d5c037040 con 0x7f3d6c107b50 2024-09-12T01:37:50.556 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5c040070 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c199bf0 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.772+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d5c03a740 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.773+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c19c8e0 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.773+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3d5c03a970 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.773+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c107fd0 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.773+0000 7f3d62ffd640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d5803b220 0x7f3d5803d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.773+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d5c06c830 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.774+0000 7f3d7162e640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d5803b220 0x7f3d5803d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.774+0000 7f3d7162e640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d5803b220 0x7f3d5803d6e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d680098c0 tx=0x7f3d68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.778+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d5c044c70 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.945+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3d6c192890 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.946+0000 7f3d62ffd640 1 -- 172.21.15.122:0/2134630860 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3d5c0328f0 con 0x7f3d6c107b50 2024-09-12T01:37:50.557 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d5803b220 msgr2=0x7f3d5803d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d5803b220 0x7f3d5803d6e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d680098c0 tx=0x7f3d68006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 msgr2=0x7f3d6c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c199420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c030560 tx=0x7f3d5c032670 comp rx=0 tx=0).stop 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 shutdown_connections 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d5803b220 0x7f3d5803d6e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 --2- 172.21.15.122:0/2134630860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d6c107b50 0x7f3d6c199420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 >> 172.21.15.122:0/2134630860 conn(0x7f3d6c101550 msgr2=0x7f3d6c18f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 shutdown_connections 2024-09-12T01:37:50.558 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:49.948+0000 7f3d74093640 1 -- 172.21.15.122:0/2134630860 wait complete. 2024-09-12T01:37:50.560 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T01:37:50.560 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-09-12T01:37:50.567 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T01:37:50.567 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-09-12T01:37:50.567 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-09-12T01:37:50.567 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph Dashboard is now available at: 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: URL: https://smithi122.front.sepia.ceph.com:8443/ 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: User: admin 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: Password: trdze6it07 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:Saving cluster configuration to /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config directory 2024-09-12T01:37:51.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:51 smithi122 ceph-mon[30925]: mgrmap e13: x(active, since 6s) 2024-09-12T01:37:51.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:51 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2567986607' entity='client.admin' 2024-09-12T01:37:51.930 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.967+0000 7fb1023c4640 1 Processor -- start 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.967+0000 7fb1023c4640 1 -- start start 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.968+0000 7fb1023c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc10a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.968+0000 7fb1023c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc07beb0 con 0x7fb0fc108090 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.968+0000 7fb0fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc10a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.968+0000 7fb0fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc10a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50540/0 (socket says 172.21.15.122:50540) 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.968+0000 7fb0fbfff640 1 -- 172.21.15.122:0/4148268329 learned_addr learned my addr 172.21.15.122:0/4148268329 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.968+0000 7fb0fbfff640 1 -- 172.21.15.122:0/4148268329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc10a9c0 con 0x7fb0fc108090 2024-09-12T01:37:51.938 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.969+0000 7fb0fbfff640 1 --2- 172.21.15.122:0/4148268329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc10a480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009820 tx=0x7fb0ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaa3a02cee7d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:51.939 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.969+0000 7fb0fb7fe640 1 -- 172.21.15.122:0/4148268329 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0ec037070 con 0x7fb0fc108090 2024-09-12T01:37:51.944 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.969+0000 7fb0fb7fe640 1 -- 172.21.15.122:0/4148268329 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ec032920 con 0x7fb0fc108090 2024-09-12T01:37:51.944 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.969+0000 7fb0fb7fe640 1 -- 172.21.15.122:0/4148268329 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0ec030810 con 0x7fb0fc108090 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 -- 172.21.15.122:0/4148268329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 msgr2=0x7fb0fc10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 --2- 172.21.15.122:0/4148268329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc10a480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009820 tx=0x7fb0ec02a270 comp rx=0 tx=0).stop 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 -- 172.21.15.122:0/4148268329 shutdown_connections 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 --2- 172.21.15.122:0/4148268329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc10a480 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 -- 172.21.15.122:0/4148268329 >> 172.21.15.122:0/4148268329 conn(0x7fb0fc101db0 msgr2=0x7fb0fc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 -- 172.21.15.122:0/4148268329 shutdown_connections 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 -- 172.21.15.122:0/4148268329 wait complete. 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 Processor -- start 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.970+0000 7fb1023c4640 1 -- start start 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb1023c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:51.945 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb1023c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10baa0 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50556/0 (socket says 172.21.15.122:50556) 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0fbfff640 1 -- 172.21.15.122:0/2567986607 learned_addr learned my addr 172.21.15.122:0/2567986607 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0fbfff640 1 -- 172.21.15.122:0/2567986607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc19bb10 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0fbfff640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc19b5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097f0 tx=0x7fb0ec0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0ec037040 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ec030cc0 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc19bda0 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0ec03a6b0 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.971+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc19ea90 con 0x7fb0fc108090 2024-09-12T01:37:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.972+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb0ec03b3f0 con 0x7fb0fc108090 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.972+0000 7fb0f9ffb640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0e8039060 0x7fb0e803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.972+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc1084f0 con 0x7fb0fc108090 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.972+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb0ec06c6c0 con 0x7fb0fc108090 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.972+0000 7fb0f3fff640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0e8039060 0x7fb0e803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.973+0000 7fb0f3fff640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0e8039060 0x7fb0e803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098c0 tx=0x7fb0e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:50.977+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0ec03b6e0 con 0x7fb0fc108090 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.230+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb0fc19efe0 con 0x7fb0fc108090 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.233+0000 7fb0f9ffb640 1 -- 172.21.15.122:0/2567986607 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb0ec049c80 con 0x7fb0fc108090 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0e8039060 msgr2=0x7fb0e803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0e8039060 0x7fb0e803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098c0 tx=0x7fb0e4006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 msgr2=0x7fb0fc19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc19b5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0097f0 tx=0x7fb0ec0327b0 comp rx=0 tx=0).stop 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 shutdown_connections 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0e8039060 0x7fb0e803b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 --2- 172.21.15.122:0/2567986607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0fc108090 0x7fb0fc19b5d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 >> 172.21.15.122:0/2567986607 conn(0x7fb0fc101db0 msgr2=0x7fb0fc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.235+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 shutdown_connections 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-09-12T01:37:51.236+0000 7fb1023c4640 1 -- 172.21.15.122:0/2567986607 wait complete. 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.948 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: ceph telemetry on 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout:For more information see: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:37:51.949 INFO:teuthology.orchestra.run.smithi122.stdout:Bootstrap complete. 2024-09-12T01:37:51.977 INFO:tasks.cephadm:Fetching config... 2024-09-12T01:37:51.977 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:37:51.977 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T01:37:52.017 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T01:37:52.017 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:37:52.018 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T01:37:52.083 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T01:37:52.083 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:37:52.083 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T01:37:52.153 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T01:37:52.153 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:37:52.153 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T01:37:52.223 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T01:37:52.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:37:52.305 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:52.316 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:37:52.360 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:52.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T01:37:52.418 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1pqJCEmXqInmiQOEUpNKDqSX14lcCHLsOAOUUYFOdfMuujE6v1icHMXB3og4xgHPm28voRTK16+kpduHzhTy15IMHrtgjG8dZxB2E2Yt+eTGrbCSlUJCBQ1fZq+kAIU/Ov0GoDq9smvvzAwtyAckfSwdoXX+5x817PTkixsqlJql+lpIRzspVcfEm9Mcw+xViQDpanSpMYosd+/5BX2YEjBEOaFjuIpdy/4N6PD/5+Fj1Meu13JSxsDIDkBe/5y1igzRP33ZpOYsoJZUeOMnuxSXFZHB0N+pXDRy/Q2kq97wTs3EWwKhaOUQY9CAlVWzeday6T89w8T4WeKH9awXHgZWFrZa9+8UJvrD++tXgrtNJCEdzxRavp8yQMucfw2kRCgWSZpK4Cy1MLV7bvNGATedIOQk6XIe2m3kOtNb0qxggU/xje36U159hBhGztjCaRj4W83hciJQ9S1mVoOw9ND5mLyBq7XY6uAPcVYJgxMs+w0z/qCvjLi3qu/NdUc= ceph-5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:37:52.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T01:37:52.678 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:53.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 -- 172.21.15.122:0/126688575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 msgr2=0x7f18b410da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:53.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 --2- 172.21.15.122:0/126688575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b410da00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009880 tx=0x7f18a402a5d0 comp rx=0 tx=0).stop 2024-09-12T01:37:53.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 -- 172.21.15.122:0/126688575 shutdown_connections 2024-09-12T01:37:53.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 --2- 172.21.15.122:0/126688575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b410da00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:53.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 -- 172.21.15.122:0/126688575 >> 172.21.15.122:0/126688575 conn(0x7f18b406b7e0 msgr2=0x7f18b406bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:53.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 -- 172.21.15.122:0/126688575 shutdown_connections 2024-09-12T01:37:53.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.181+0000 7f18b8ce1640 1 -- 172.21.15.122:0/126688575 wait complete. 2024-09-12T01:37:53.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.182+0000 7f18b8ce1640 1 Processor -- start 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.182+0000 7f18b8ce1640 1 -- start start 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b8ce1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b41a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b8ce1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4112130 con 0x7f18b410d600 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b41a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b41a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50564/0 (socket says 172.21.15.122:50564) 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b259c640 1 -- 172.21.15.122:0/43151522 learned_addr learned my addr 172.21.15.122:0/43151522 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b259c640 1 -- 172.21.15.122:0/43151522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b41a1d20 con 0x7f18b410d600 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b259c640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b41a17e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18a4004f60 tx=0x7f18a40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a403f070 con 0x7f18b410d600 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a4004d40 con 0x7f18b410d600 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b41a1fb0 con 0x7f18b410d600 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a4035030 con 0x7f18b410d600 2024-09-12T01:37:53.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.183+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b41a4ca0 con 0x7f18b410d600 2024-09-12T01:37:53.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.184+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f18a4042070 con 0x7f18b410d600 2024-09-12T01:37:53.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.184+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b41128b0 con 0x7f18b410d600 2024-09-12T01:37:53.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.185+0000 7f18a37fe640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f189c0390c0 0x7f189c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:53.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.185+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f18a406ccc0 con 0x7f18b410d600 2024-09-12T01:37:53.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.185+0000 7f18b1d9b640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f189c0390c0 0x7f189c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:53.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.185+0000 7f18b1d9b640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f189c0390c0 0x7f189c03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098c0 tx=0x7f18a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.189+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18a4042360 con 0x7f18b410d600 2024-09-12T01:37:53.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.361+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f18b402d830 con 0x7f18b410d600 2024-09-12T01:37:53.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.363+0000 7f18a37fe640 1 -- 172.21.15.122:0/43151522 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f18a402f300 con 0x7f18b410d600 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f189c0390c0 msgr2=0x7f189c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f189c0390c0 0x7f189c03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098c0 tx=0x7f18a8006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 msgr2=0x7f18b41a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b41a17e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18a4004f60 tx=0x7f18a40048f0 comp rx=0 tx=0).stop 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 shutdown_connections 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f189c0390c0 0x7f189c03b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 --2- 172.21.15.122:0/43151522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18b410d600 0x7f18b41a17e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 >> 172.21.15.122:0/43151522 conn(0x7f18b406b7e0 msgr2=0x7f18b410a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 shutdown_connections 2024-09-12T01:37:53.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:53.366+0000 7f18b8ce1640 1 -- 172.21.15.122:0/43151522 wait complete. 2024-09-12T01:37:54.275 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T01:37:54.276 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T01:37:54.496 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:54 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/43151522' entity='client.admin' 2024-09-12T01:37:54.736 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:55.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 -- 172.21.15.122:0/1062743964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 msgr2=0x7fc1ec100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:55.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 --2- 172.21.15.122:0/1062743964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec100930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009880 tx=0x7fc1dc02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:37:55.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 -- 172.21.15.122:0/1062743964 shutdown_connections 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 --2- 172.21.15.122:0/1062743964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec100930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 -- 172.21.15.122:0/1062743964 >> 172.21.15.122:0/1062743964 conn(0x7fc1ec0fbca0 msgr2=0x7fc1ec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 -- 172.21.15.122:0/1062743964 shutdown_connections 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.179+0000 7fc1f2456640 1 -- 172.21.15.122:0/1062743964 wait complete. 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1f2456640 1 Processor -- start 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1f2456640 1 -- start start 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1f2456640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1f2456640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec106640 con 0x7fc1ec100530 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50586/0 (socket says 172.21.15.122:50586) 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.180+0000 7fc1ebfff640 1 -- 172.21.15.122:0/3518381300 learned_addr learned my addr 172.21.15.122:0/3518381300 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1ebfff640 1 -- 172.21.15.122:0/3518381300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec197350 con 0x7fc1ec100530 2024-09-12T01:37:55.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1ebfff640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec196e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc004f60 tx=0x7fc1dc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1dc03f070 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1dc004d40 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec1975e0 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1dc035030 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.181+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec19a2d0 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.182+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc1dc042070 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.182+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec106dc0 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.182+0000 7fc1e97fa640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc1d4039060 0x7fc1d403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.182+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc1dc06cd20 con 0x7fc1ec100530 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.182+0000 7fc1eb7fe640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc1d4039060 0x7fc1d403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:55.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.183+0000 7fc1eb7fe640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc1d4039060 0x7fc1d403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00098c0 tx=0x7fc1e0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:55.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.188+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1dc042360 con 0x7fc1ec100530 2024-09-12T01:37:55.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.378+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc1ec02d830 con 0x7fc1d4039060 2024-09-12T01:37:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.381+0000 7fc1e97fa640 1 -- 172.21.15.122:0/3518381300 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1ec02d830 con 0x7fc1d4039060 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc1d4039060 msgr2=0x7fc1d403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc1d4039060 0x7fc1d403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00098c0 tx=0x7fc1e0006db0 comp rx=0 tx=0).stop 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 msgr2=0x7fc1ec196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec196e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc004f60 tx=0x7fc1dc0048f0 comp rx=0 tx=0).stop 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 shutdown_connections 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc1d4039060 0x7fc1d403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 --2- 172.21.15.122:0/3518381300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1ec100530 0x7fc1ec196e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 >> 172.21.15.122:0/3518381300 conn(0x7fc1ec0fbca0 msgr2=0x7fc1ec069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 shutdown_connections 2024-09-12T01:37:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:55.383+0000 7fc1f2456640 1 -- 172.21.15.122:0/3518381300 wait complete. 2024-09-12T01:37:55.902 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-09-12T01:37:55.902 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:37:55.902 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-09-12T01:37:55.922 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:37:55.922 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:55.981 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-09-12T01:37:55.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch host add smithi151 2024-09-12T01:37:56.406 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:37:56.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:37:57.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 -- 172.21.15.122:0/3943604691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 msgr2=0x7f0c6c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:37:57.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 --2- 172.21.15.122:0/3943604691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c101a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009880 tx=0x7f0c6802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:37:57.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 -- 172.21.15.122:0/3943604691 shutdown_connections 2024-09-12T01:37:57.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 --2- 172.21.15.122:0/3943604691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c101a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:37:57.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 -- 172.21.15.122:0/3943604691 >> 172.21.15.122:0/3943604691 conn(0x7f0c6c0fcdf0 msgr2=0x7f0c6c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 -- 172.21.15.122:0/3943604691 shutdown_connections 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.983+0000 7f0c7349f640 1 -- 172.21.15.122:0/3943604691 wait complete. 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7349f640 1 Processor -- start 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7349f640 1 -- start start 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7349f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7349f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c107750 con 0x7f0c6c101640 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7123b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7123b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50606/0 (socket says 172.21.15.122:50606) 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7123b640 1 -- 172.21.15.122:0/2379534028 learned_addr learned my addr 172.21.15.122:0/2379534028 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.984+0000 7f0c7123b640 1 -- 172.21.15.122:0/2379534028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c197390 con 0x7f0c6c101640 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.985+0000 7f0c7123b640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c196e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009230 tx=0x7f0c680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:57.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.985+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c6803f070 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.985+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c68003ad0 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.985+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c197620 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.985+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c68035030 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.985+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c19a310 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.986+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0c68042070 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.986+0000 7f0c627fc640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c580390c0 0x7f0c5803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.986+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c107ed0 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.986+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0c6806cd40 con 0x7f0c6c101640 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.986+0000 7f0c70a3a640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c580390c0 0x7f0c5803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.987+0000 7f0c70a3a640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c580390c0 0x7f0c5803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0098c0 tx=0x7f0c5c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:37:57.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:56.991+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c68039350 con 0x7f0c6c101640 2024-09-12T01:37:57.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:37:57.174+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7f0c6c02d830 con 0x7f0c580390c0 2024-09-12T01:37:57.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:37:58.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:58 smithi122 ceph-mon[30925]: Updating smithi122:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:37:58.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:58 smithi122 ceph-mon[30925]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:37:58.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:58 smithi122 ceph-mon[30925]: Updating smithi122:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:37:58.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:58 smithi122 ceph-mon[30925]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:37:59.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:37:59 smithi122 ceph-mon[30925]: Deploying cephadm binary to smithi151 2024-09-12T01:38:00.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.406+0000 7f0c627fc640 1 -- 172.21.15.122:0/2379534028 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0c6c02d830 con 0x7f0c580390c0 2024-09-12T01:38:00.406 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-09-12T01:38:00.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c580390c0 msgr2=0x7f0c5803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:00.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c580390c0 0x7f0c5803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0098c0 tx=0x7f0c5c006db0 comp rx=0 tx=0).stop 2024-09-12T01:38:00.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 msgr2=0x7f0c6c196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c196e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009230 tx=0x7f0c680048f0 comp rx=0 tx=0).stop 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 shutdown_connections 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c580390c0 0x7f0c5803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 --2- 172.21.15.122:0/2379534028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c101640 0x7f0c6c196e50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 >> 172.21.15.122:0/2379534028 conn(0x7f0c6c0fcdf0 msgr2=0x7f0c6c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 shutdown_connections 2024-09-12T01:38:00.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:00.408+0000 7f0c7349f640 1 -- 172.21.15.122:0/2379534028 wait complete. 2024-09-12T01:38:00.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T01:38:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:01.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:01.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:01 smithi122 ceph-mon[30925]: Added host smithi151 2024-09-12T01:38:01.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:01.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:01.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 -- 172.21.15.122:0/15684600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 msgr2=0x7fbadc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 --2- 172.21.15.122:0/15684600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009880 tx=0x7fbacc02a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:01.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 -- 172.21.15.122:0/15684600 shutdown_connections 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 --2- 172.21.15.122:0/15684600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 -- 172.21.15.122:0/15684600 >> 172.21.15.122:0/15684600 conn(0x7fbadc0fd2b0 msgr2=0x7fbadc0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 -- 172.21.15.122:0/15684600 shutdown_connections 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.621+0000 7fbae0d04640 1 -- 172.21.15.122:0/15684600 wait complete. 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbae0d04640 1 Processor -- start 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbae0d04640 1 -- start start 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbae0d04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbae0d04640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc10bc70 con 0x7fbadc107660 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbada59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbada59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35952/0 (socket says 172.21.15.122:35952) 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbada59c640 1 -- 172.21.15.122:0/3656881799 learned_addr learned my addr 172.21.15.122:0/3656881799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.622+0000 7fbada59c640 1 -- 172.21.15.122:0/3656881799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc1975e0 con 0x7fbadc107660 2024-09-12T01:38:01.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.623+0000 7fbada59c640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc1970a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009230 tx=0x7fbacc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.623+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbacc03f070 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.623+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc1978d0 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.623+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc03bc80 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.623+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbacc035030 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.623+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc19b970 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.624+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbacc042070 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.624+0000 7fbacb7fe640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbac4039060 0x7fbac403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.624+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbadc102d10 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.624+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbacc06ce40 con 0x7fbadc107660 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.624+0000 7fbad9d9b640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbac4039060 0x7fbac403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.624+0000 7fbad9d9b640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbac4039060 0x7fbac403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098c0 tx=0x7fbad0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:01.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.629+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbacc04c350 con 0x7fbadc107660 2024-09-12T01:38:01.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.801+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbadc198400 con 0x7fbac4039060 2024-09-12T01:38:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.802+0000 7fbacb7fe640 1 -- 172.21.15.122:0/3656881799 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fbadc198400 con 0x7fbac4039060 2024-09-12T01:38:01.802 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:38:01.802 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbac4039060 msgr2=0x7fbac403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbac4039060 0x7fbac403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098c0 tx=0x7fbad0006db0 comp rx=0 tx=0).stop 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 msgr2=0x7fbadc1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc1970a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009230 tx=0x7fbacc003d40 comp rx=0 tx=0).stop 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 shutdown_connections 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbac4039060 0x7fbac403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 --2- 172.21.15.122:0/3656881799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc107660 0x7fbadc1970a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 >> 172.21.15.122:0/3656881799 conn(0x7fbadc0fd2b0 msgr2=0x7fbadc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 shutdown_connections 2024-09-12T01:38:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:01.804+0000 7fbae0d04640 1 -- 172.21.15.122:0/3656881799 wait complete. 2024-09-12T01:38:02.363 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-12T01:38:02.364 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:38:02.364 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-12T01:38:02.384 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:38:02.384 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:02.442 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-12T01:38:02.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch host add smithi183 2024-09-12T01:38:02.685 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:02.925 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:02.925 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:02 smithi122 ceph-mon[30925]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:38:03.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 -- 172.21.15.122:0/3995590501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 msgr2=0x7f62d8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:03.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 --2- 172.21.15.122:0/3995590501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8106970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009c20 tx=0x7f62c802aa90 comp rx=0 tx=0).stop 2024-09-12T01:38:03.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 -- 172.21.15.122:0/3995590501 shutdown_connections 2024-09-12T01:38:03.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 --2- 172.21.15.122:0/3995590501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8106970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:03.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 -- 172.21.15.122:0/3995590501 >> 172.21.15.122:0/3995590501 conn(0x7f62d80fc1a0 msgr2=0x7f62d80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:03.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 -- 172.21.15.122:0/3995590501 shutdown_connections 2024-09-12T01:38:03.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.121+0000 7f62dd1b7640 1 -- 172.21.15.122:0/3995590501 wait complete. 2024-09-12T01:38:03.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62dd1b7640 1 Processor -- start 2024-09-12T01:38:03.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62dd1b7640 1 -- start start 2024-09-12T01:38:03.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62dd1b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:03.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62dd1b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62d810abc0 con 0x7f62d8106590 2024-09-12T01:38:03.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62d6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:03.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62d6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35962/0 (socket says 172.21.15.122:35962) 2024-09-12T01:38:03.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.122+0000 7f62d6d9d640 1 -- 172.21.15.122:0/4116739491 learned_addr learned my addr 172.21.15.122:0/4116739491 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62d6d9d640 1 -- 172.21.15.122:0/4116739491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d8198460 con 0x7f62d8106590 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62d6d9d640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8197f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005e30 tx=0x7f62c802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c8047070 con 0x7f62d8106590 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c8003df0 con 0x7f62d8106590 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c8033050 con 0x7f62d8106590 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d81986f0 con 0x7f62d8106590 2024-09-12T01:38:03.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.123+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d8198b50 con 0x7f62d8106590 2024-09-12T01:38:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.124+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f62c804a070 con 0x7f62d8106590 2024-09-12T01:38:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.124+0000 7f62cffff640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f62c4039060 0x7f62c403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.124+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f62c803e030 con 0x7f62d8106590 2024-09-12T01:38:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.124+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d8101c00 con 0x7f62d8106590 2024-09-12T01:38:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.124+0000 7f62d659c640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f62c4039060 0x7f62c403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.125+0000 7f62d659c640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f62c4039060 0x7f62c403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098c0 tx=0x7f62c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:03.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.129+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62c8051080 con 0x7f62d8106590 2024-09-12T01:38:03.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:03.304+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f62d81069f0 con 0x7f62c4039060 2024-09-12T01:38:04.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:03 smithi122 ceph-mon[30925]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:05.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: Deploying cephadm binary to smithi183 2024-09-12T01:38:05.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:06.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.486+0000 7f62cffff640 1 -- 172.21.15.122:0/4116739491 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f62d81069f0 con 0x7f62c4039060 2024-09-12T01:38:06.486 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-12T01:38:06.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f62c4039060 msgr2=0x7f62c403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:06.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f62c4039060 0x7f62c403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098c0 tx=0x7f62c0006db0 comp rx=0 tx=0).stop 2024-09-12T01:38:06.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 msgr2=0x7f62d8197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:06.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8197f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005e30 tx=0x7f62c802fc10 comp rx=0 tx=0).stop 2024-09-12T01:38:06.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 shutdown_connections 2024-09-12T01:38:06.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f62c4039060 0x7f62c403b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:06.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 --2- 172.21.15.122:0/4116739491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8106590 0x7f62d8197f20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:06.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.488+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 >> 172.21.15.122:0/4116739491 conn(0x7f62d80fc1a0 msgr2=0x7f62d80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:06.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.489+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 shutdown_connections 2024-09-12T01:38:06.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:06.489+0000 7f62dd1b7640 1 -- 172.21.15.122:0/4116739491 wait complete. 2024-09-12T01:38:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:06 smithi122 ceph-mon[30925]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-12T01:38:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:06 smithi122 ceph-mon[30925]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:06 smithi122 ceph-mon[30925]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:07.099 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T01:38:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: Added host smithi183 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:07.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 -- 172.21.15.122:0/1107270530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 msgr2=0x7f2918106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 --2- 172.21.15.122:0/1107270530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918106a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2914009880 tx=0x7f291402a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 -- 172.21.15.122:0/1107270530 shutdown_connections 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 --2- 172.21.15.122:0/1107270530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918106a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 -- 172.21.15.122:0/1107270530 >> 172.21.15.122:0/1107270530 conn(0x7f29180fb920 msgr2=0x7f29180fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 -- 172.21.15.122:0/1107270530 shutdown_connections 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.737+0000 7f291fbf2640 1 -- 172.21.15.122:0/1107270530 wait complete. 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291fbf2640 1 Processor -- start 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291fbf2640 1 -- start start 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291fbf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291fbf2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f291810ac30 con 0x7f2918106620 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291d98e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291d98e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35988/0 (socket says 172.21.15.122:35988) 2024-09-12T01:38:07.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291d98e640 1 -- 172.21.15.122:0/2145853065 learned_addr learned my addr 172.21.15.122:0/2145853065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.738+0000 7f291d98e640 1 -- 172.21.15.122:0/2145853065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2918197650 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.739+0000 7f291d98e640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918197110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2914009230 tx=0x7f2914003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.739+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291403f070 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.739+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f291403bc80 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.739+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2914035030 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.739+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29181978e0 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.739+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291819b9e0 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.740+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2914042070 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.740+0000 7f290effd640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29040390c0 0x7f290403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.740+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f291406cd70 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.740+0000 7f291d18d640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29040390c0 0x7f290403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.741+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2918101c70 con 0x7f2918106620 2024-09-12T01:38:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.741+0000 7f291d18d640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29040390c0 0x7f290403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29080098c0 tx=0x7f2908006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:07.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.746+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f291404c350 con 0x7f2918106620 2024-09-12T01:38:07.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.919+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2918198470 con 0x7f29040390c0 2024-09-12T01:38:07.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.920+0000 7f290effd640 1 -- 172.21.15.122:0/2145853065 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f2918198470 con 0x7f29040390c0 2024-09-12T01:38:07.920 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:38:07.920 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29040390c0 msgr2=0x7f290403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29040390c0 0x7f290403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29080098c0 tx=0x7f2908006db0 comp rx=0 tx=0).stop 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 msgr2=0x7f2918197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918197110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2914009230 tx=0x7f2914003d40 comp rx=0 tx=0).stop 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 shutdown_connections 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29040390c0 0x7f290403b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 --2- 172.21.15.122:0/2145853065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918106620 0x7f2918197110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 >> 172.21.15.122:0/2145853065 conn(0x7f29180fb920 msgr2=0x7f29180ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:07.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 shutdown_connections 2024-09-12T01:38:07.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:07.922+0000 7f291fbf2640 1 -- 172.21.15.122:0/2145853065 wait complete. 2024-09-12T01:38:08.414 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T01:38:08.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T01:38:08.652 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:09.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:09.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:08 smithi122 ceph-mon[30925]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:38:09.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:08 smithi122 ceph-mon[30925]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 -- 172.21.15.122:0/21889616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 msgr2=0x7faee8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 --2- 172.21.15.122:0/21889616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faee0009880 tx=0x7faee002a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 -- 172.21.15.122:0/21889616 shutdown_connections 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 --2- 172.21.15.122:0/21889616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 -- 172.21.15.122:0/21889616 >> 172.21.15.122:0/21889616 conn(0x7faee80fc1a0 msgr2=0x7faee80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 -- 172.21.15.122:0/21889616 shutdown_connections 2024-09-12T01:38:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.044+0000 7faeefb3b640 1 -- 172.21.15.122:0/21889616 wait complete. 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeefb3b640 1 Processor -- start 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeefb3b640 1 -- start start 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeefb3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeefb3b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faee810abc0 con 0x7faee8106590 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeed8d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeed8d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36004/0 (socket says 172.21.15.122:36004) 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeed8d7640 1 -- 172.21.15.122:0/3280359373 learned_addr learned my addr 172.21.15.122:0/3280359373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeed8d7640 1 -- 172.21.15.122:0/3280359373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee8197640 con 0x7faee8106590 2024-09-12T01:38:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeed8d7640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faee0009230 tx=0x7faee0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:09.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee003f070 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee003bc80 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0035030 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.045+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee81978d0 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.046+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee819b9d0 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.046+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7faee0042070 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.046+0000 7faee6ffd640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faed00390c0 0x7faed003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.046+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee8101c00 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.047+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faee006ccb0 con 0x7faee8106590 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.047+0000 7faeed0d6640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faed00390c0 0x7faed003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:09.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.047+0000 7faeed0d6640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faed00390c0 0x7faed003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faedc00ad30 tx=0x7faedc0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:09.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.053+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faee00314c0 con 0x7faee8106590 2024-09-12T01:38:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.224+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7faee8198460 con 0x7faee8106590 2024-09-12T01:38:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.837+0000 7faee6ffd640 1 -- 172.21.15.122:0/3280359373 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7faee003a890 con 0x7faee8106590 2024-09-12T01:38:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faed00390c0 msgr2=0x7faed003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faed00390c0 0x7faed003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faedc00ad30 tx=0x7faedc0093f0 comp rx=0 tx=0).stop 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 msgr2=0x7faee8197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faee0009230 tx=0x7faee0003d40 comp rx=0 tx=0).stop 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 shutdown_connections 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faed00390c0 0x7faed003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 --2- 172.21.15.122:0/3280359373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faee8106590 0x7faee8197100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 >> 172.21.15.122:0/3280359373 conn(0x7faee80fc1a0 msgr2=0x7faee8069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 shutdown_connections 2024-09-12T01:38:09.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:09.839+0000 7faeefb3b640 1 -- 172.21.15.122:0/3280359373 wait complete. 2024-09-12T01:38:10.142 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:09 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3280359373' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:38:10.405 INFO:tasks.cephadm:Adding mon.a on smithi122 2024-09-12T01:38:10.406 INFO:tasks.cephadm:Adding mon.b on smithi151 2024-09-12T01:38:10.406 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-09-12T01:38:10.406 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c' 2024-09-12T01:38:10.650 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T01:38:10.694 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T01:38:11.165 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 -- 172.21.15.183:0/877872792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 msgr2=0x7f1978101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:11.165 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 --2- 172.21.15.183:0/877872792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978101a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1970009c20 tx=0x7f197002aa90 comp rx=0 tx=0).stop 2024-09-12T01:38:11.165 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 -- 172.21.15.183:0/877872792 shutdown_connections 2024-09-12T01:38:11.165 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 --2- 172.21.15.183:0/877872792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978101a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:11.165 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 -- 172.21.15.183:0/877872792 >> 172.21.15.183:0/877872792 conn(0x7f19780fcdb0 msgr2=0x7f19780ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 -- 172.21.15.183:0/877872792 shutdown_connections 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.139+0000 7f197f59c640 1 -- 172.21.15.183:0/877872792 wait complete. 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197f59c640 1 Processor -- start 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197f59c640 1 -- start start 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197f59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197f59c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1978107730 con 0x7f1978101620 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197d338640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197d338640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.183:41886/0 (socket says 172.21.15.183:41886) 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.140+0000 7f197d338640 1 -- 172.21.15.183:0/192861361 learned_addr learned my addr 172.21.15.183:0/192861361 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f197d338640 1 -- 172.21.15.183:0/192861361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1978197270 con 0x7f1978101620 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f197d338640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1970005cc0 tx=0x7f197002fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1970047070 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1970003e50 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1978197500 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1970033050 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.141+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197819a1f0 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.142+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f197004a070 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.142+0000 7f19767fc640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1960039060 0x7f196003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.142+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1978107eb0 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.142+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f197003e030 con 0x7f1978101620 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.142+0000 7f197cb37640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1960039060 0x7f196003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.143+0000 7f197cb37640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1960039060 0x7f196003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098c0 tx=0x7f196c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.147+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19700373b0 con 0x7f1978101620 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3280359373' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:11.337 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.336+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) -- 0x7f197802d8f0 con 0x7f1960039060 2024-09-12T01:38:11.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.341+0000 7f19767fc640 1 -- 172.21.15.183:0/192861361 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f197802d8f0 con 0x7f1960039060 2024-09-12T01:38:11.342 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-09-12T01:38:11.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1960039060 msgr2=0x7f196003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:11.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1960039060 0x7f196003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098c0 tx=0x7f196c006db0 comp rx=0 tx=0).stop 2024-09-12T01:38:11.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 msgr2=0x7f1978196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1970005cc0 tx=0x7f197002fc10 comp rx=0 tx=0).stop 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 shutdown_connections 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1960039060 0x7f196003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 --2- 172.21.15.183:0/192861361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1978101620 0x7f1978196d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.343+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 >> 172.21.15.183:0/192861361 conn(0x7f19780fcdb0 msgr2=0x7f19780fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.344+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 shutdown_connections 2024-09-12T01:38:11.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:11.344+0000 7f197f59c640 1 -- 172.21.15.183:0/192861361 wait complete. 2024-09-12T01:38:11.989 DEBUG:teuthology.orchestra.run.smithi151:mon.b> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T01:38:11.992 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T01:38:11.994 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:38:11.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:12 smithi122 ceph-mon[30925]: Deploying daemon mon.c on smithi183 2024-09-12T01:38:13.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:13 smithi122 ceph-mon[30925]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:14.952 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:14.953 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 -- 172.21.15.183:0/2354885654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 msgr2=0x7f1e14107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:14.953 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 --2- 172.21.15.183:0/2354885654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009880 tx=0x7f1e0402a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:14.953 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 -- 172.21.15.183:0/2354885654 shutdown_connections 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 --2- 172.21.15.183:0/2354885654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 -- 172.21.15.183:0/2354885654 >> 172.21.15.183:0/2354885654 conn(0x7f1e14076250 msgr2=0x7f1e14076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 -- 172.21.15.183:0/2354885654 shutdown_connections 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 -- 172.21.15.183:0/2354885654 wait complete. 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.565+0000 7f1e1a868640 1 Processor -- start 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.566+0000 7f1e1a868640 1 -- start start 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.566+0000 7f1e1a868640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.566+0000 7f1e1a868640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1410bce0 con 0x7f1e14107690 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.566+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.566+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.183:41900/0 (socket says 172.21.15.183:41900) 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.566+0000 7f1e13fff640 1 -- 172.21.15.183:0/2793253993 learned_addr learned my addr 172.21.15.183:0/2793253993 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:38:14.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e13fff640 1 -- 172.21.15.183:0/2793253993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e141026f0 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e13fff640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14100800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009230 tx=0x7f1e04003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e0403f070 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e14100d40 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e14101200 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0403bc80 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.567+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e04035030 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.568+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1e04042070 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.568+0000 7f1e117fa640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dfc0390c0 0x7f1dfc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.568+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e0406ccf0 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.568+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e00005290 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.569+0000 7f1e137fe640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dfc0390c0 0x7f1dfc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.569+0000 7f1e137fe640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dfc0390c0 0x7f1dfc03b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098c0 tx=0x7f1e08006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.573+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e0403a890 con 0x7f1e14107690 2024-09-12T01:38:14.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.828+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1e00005580 con 0x7f1e14107690 2024-09-12T01:38:14.956 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.831+0000 7f1e117fa640 1 -- 172.21.15.183:0/2793253993 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f1e0403a3c0 con 0x7f1e14107690 2024-09-12T01:38:14.956 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:38:14.956 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","modified":"2024-09-12T01:36:46.446315Z","created":"2024-09-12T01:36:46.446315Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dfc0390c0 msgr2=0x7f1dfc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dfc0390c0 0x7f1dfc03b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098c0 tx=0x7f1e08006db0 comp rx=0 tx=0).stop 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 msgr2=0x7f1e14100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14100800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009230 tx=0x7f1e04003d40 comp rx=0 tx=0).stop 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 shutdown_connections 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dfc0390c0 0x7f1dfc03b580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 --2- 172.21.15.183:0/2793253993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e14107690 0x7f1e14100800 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 >> 172.21.15.183:0/2793253993 conn(0x7f1e14076250 msgr2=0x7f1e140fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:14.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 shutdown_connections 2024-09-12T01:38:14.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:14.835+0000 7f1e1a868640 1 -- 172.21.15.183:0/2793253993 wait complete. 2024-09-12T01:38:15.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:15 smithi122 ceph-mon[30925]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:15 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/2793253993' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:15 smithi183 podman[35108]: 2024-09-12 01:38:15.847609052 +0000 UTC m=+0.126119975 container create 9a051b6e9f2993edd11110bcdc955a8c5505bd91d5a4d2974da56d9ec81dbc45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 podman[35108]: 2024-09-12 01:38:16.057943051 +0000 UTC m=+0.336453929 container init 9a051b6e9f2993edd11110bcdc955a8c5505bd91d5a4d2974da56d9ec81dbc45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 podman[35108]: 2024-09-12 01:38:16.060935503 +0000 UTC m=+0.339446383 container start 9a051b6e9f2993edd11110bcdc955a8c5505bd91d5a4d2974da56d9ec81dbc45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: pidfile_write: ignore empty --pid-file 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: load: jerasure load: lrc 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: RocksDB version: 7.9.2 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Git sha 0 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T01:38:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: DB SUMMARY 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: DB Session ID: UMO6UF386SBVMEFFD9FX 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: CURRENT file: CURRENT 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.error_if_exists: 0 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.create_if_missing: 0 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.paranoid_checks: 1 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T01:38:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T01:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T01:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.env: 0x56166f247940 2024-09-12T01:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T01:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.info_log: 0x561670d776e0 2024-09-12T01:38:16.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T01:38:16.456 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.statistics: (nil) 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.use_fsync: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_log_file_size: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.allow_fallocate: 1 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.use_direct_reads: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T01:38:16.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.db_log_dir: 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.wal_dir: 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.write_buffer_manager: 0x561670d86dc0 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T01:38:16.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.unordered_write: 0 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.row_cache: None 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.wal_filter: None 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T01:38:16.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.two_write_queues: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.wal_compression: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.atomic_flush: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.log_readahead_size: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T01:38:16.460 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_background_jobs: 2 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_background_compactions: -1 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_subcompactions: 1 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T01:38:16.461 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_open_files: -1 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_background_flushes: -1 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Compression algorithms supported: 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kZSTD supported: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kXpressCompression supported: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kBZip2Compression supported: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kLZ4Compression supported: 1 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kZlibCompression supported: 1 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T01:38:16.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: kSnappyCompression supported: 1 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.merge_operator: 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_filter: None 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561670d77320) 2024-09-12T01:38:16.463 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_type: 0 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_index_type: 0 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_shortening: 1 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: checksum: 4 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: no_block_cache: 0 2024-09-12T01:38:16.464 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache: 0x561670d6b1f0 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_options: 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: capacity : 536870912 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_shard_bits : 4 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: strict_capacity_limit : 0 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_compressed: (nil) 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: persistent_cache: (nil) 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size: 4096 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size_deviation: 10 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_restart_interval: 16 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_block_restart_interval: 1 2024-09-12T01:38:16.465 INFO:journalctl@ceph.mon.c.smithi183.stdout: metadata_block_size: 4096 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: partition_filters: 0 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: use_delta_encoding: 1 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: filter_policy: bloomfilter 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: whole_key_filtering: 1 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: verify_compression: 0 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: format_version: 5 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_index_compression: 1 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_align: 0 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: max_auto_readahead_size: 262144 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: prepopulate_block_cache: 0 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: initial_auto_readahead_size: 8192 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T01:38:16.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression: NoCompression 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.num_levels: 7 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T01:38:16.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T01:38:16.468 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T01:38:16.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T01:38:16.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.inplace_update_support: 0 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.bloom_locality: 0 2024-09-12T01:38:16.471 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.max_successive_merges: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.ttl: 2592000 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.enable_blob_files: false 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.min_blob_size: 0 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T01:38:16.472 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f840fe7b-9ba6-454c-834f-af496112b393 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105096087606, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105096088014, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726105096, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f840fe7b-9ba6-454c-834f-af496112b393", "db_session_id": "UMO6UF386SBVMEFFD9FX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105096088106, "job": 1, "event": "recovery_finished"} 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T01:38:16.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561670d98e00 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: DB pointer 0x561670e94000 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** DB Stats ** 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:38:16.474 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T01:38:16.475 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache BinnedLRUCache@0x561670d6b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T01:38:16.476 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: starting mon.c rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(???) e0 preinit fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: btime 2024-09-12T01:36:52:283630+0000 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2024-09-12T01:38:16.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Added host smithi183 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3280359373' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3280359373' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.479 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:16.480 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: Deploying daemon mon.c on smithi183 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/2793253993' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 ceph-mon[35127]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 bash[35108]: 9a051b6e9f2993edd11110bcdc955a8c5505bd91d5a4d2974da56d9ec81dbc45 2024-09-12T01:38:16.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:16 smithi183 systemd[1]: Started Ceph mon.c for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:38:16.673 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:38:16.673 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:38:16.925 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: mon.a calling monitor election 2024-09-12T01:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: mon.c calling monitor election 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: monmap epoch 2 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: last_changed 2024-09-12T01:38:16.111939+0000 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:38:21.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: min_mon_release 19 (squid) 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: election_strategy: 1 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: fsmap 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: mgrmap e13: x(active, since 36s) 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: overall HEALTH_OK 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:21.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:21.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:38:21.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: mon.a calling monitor election 2024-09-12T01:38:21.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:21.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: mon.c calling monitor election 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: monmap epoch 2 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: last_changed 2024-09-12T01:38:16.111939+0000 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:38:21.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: min_mon_release 19 (squid) 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: election_strategy: 1 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: fsmap 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: mgrmap e13: x(active, since 36s) 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: overall HEALTH_OK 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:21.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 -- 172.21.15.183:0/2232313764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b4003530 msgr2=0x7f69b40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2232313764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b4003530 0x7f69b40059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f69c806aa60 tx=0x7f69c402c490 comp rx=0 tx=0).stop 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 -- 172.21.15.183:0/2232313764 shutdown_connections 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2232313764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b4003530 0x7f69b40059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 -- 172.21.15.183:0/2232313764 >> 172.21.15.183:0/2232313764 conn(0x7f69c80fc220 msgr2=0x7f69c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 -- 172.21.15.183:0/2232313764 shutdown_connections 2024-09-12T01:38:21.720 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.719+0000 7f69cf97b640 1 -- 172.21.15.183:0/2232313764 wait complete. 2024-09-12T01:38:21.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cf97b640 1 Processor -- start 2024-09-12T01:38:21.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cf97b640 1 -- start start 2024-09-12T01:38:21.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cf97b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:21.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cf97b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c81a3b70 0x7f69c819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:21.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cf97b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69c810fd90 con 0x7f69c81a3b70 2024-09-12T01:38:21.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cf97b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69c810fc10 con 0x7f69b4003530 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.720+0000 7f69cd717640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cd717640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36250/0 (socket says 172.21.15.183:36250) 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cd717640 1 -- 172.21.15.183:0/2626375687 learned_addr learned my addr 172.21.15.183:0/2626375687 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ccf16640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c81a3b70 0x7f69c819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cd717640 1 -- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 msgr2=0x7f69c81a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cd717640 1 -- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 msgr2=0x7f69c81a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cd717640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cd717640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ccf16640 1 -- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 msgr2=0x7f69c81a3630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ccf16640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ccf16640 1 -- 172.21.15.183:0/2626375687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c81a4110 con 0x7f69c81a3b70 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ccf16640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c81a3b70 0x7f69c819d700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b3d0 tx=0x7f69bc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:21.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f69bc011630 con 0x7f69c81a3b70 2024-09-12T01:38:21.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc011cb0 con 0x7f69c81a3b70 2024-09-12T01:38:21.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c819dca0 con 0x7f69c81a3b70 2024-09-12T01:38:21.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f69bc010df0 con 0x7f69c81a3b70 2024-09-12T01:38:21.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.721+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c819e230 con 0x7f69c81a3b70 2024-09-12T01:38:21.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.722+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f69bc018020 con 0x7f69c81a3b70 2024-09-12T01:38:21.725 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.723+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c806ad10 con 0x7f69c81a3b70 2024-09-12T01:38:21.725 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.723+0000 7f69ba7fc640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69ac038e00 0x7f69ac03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:21.725 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.723+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f69bc04bd70 con 0x7f69c81a3b70 2024-09-12T01:38:21.725 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.723+0000 7f69cd717640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69ac038e00 0x7f69ac03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:21.725 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.723+0000 7f69cd717640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69ac038e00 0x7f69ac03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f69c40129e0 tx=0x7f69c402c9d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:21.733 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.733+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69bc051020 con 0x7f69c81a3b70 2024-09-12T01:38:21.966 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.965+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f69c819e5a0 con 0x7f69c81a3b70 2024-09-12T01:38:21.966 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.965+0000 7f69ba7fc640 1 -- 172.21.15.183:0/2626375687 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f69bc01f020 con 0x7f69c81a3b70 2024-09-12T01:38:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:38:21.967 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","modified":"2024-09-12T01:38:16.111939Z","created":"2024-09-12T01:36:46.446315Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:38:21.969 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-12T01:38:21.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69ac038e00 msgr2=0x7f69ac03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:21.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69ac038e00 0x7f69ac03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f69c40129e0 tx=0x7f69c402c9d0 comp rx=0 tx=0).stop 2024-09-12T01:38:21.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c81a3b70 msgr2=0x7f69c819d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:21.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c81a3b70 0x7f69c819d700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b3d0 tx=0x7f69bc00b890 comp rx=0 tx=0).stop 2024-09-12T01:38:21.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 shutdown_connections 2024-09-12T01:38:21.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69ac038e00 0x7f69ac03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:21.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c81a3b70 0x7f69c819d700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:21.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 --2- 172.21.15.183:0/2626375687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b4003530 0x7f69c81a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:21.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 >> 172.21.15.183:0/2626375687 conn(0x7f69c80fc220 msgr2=0x7f69c80fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:21.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.967+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 shutdown_connections 2024-09-12T01:38:21.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:21.968+0000 7f69cf97b640 1 -- 172.21.15.183:0/2626375687 wait complete. 2024-09-12T01:38:22.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:22 smithi183 ceph-mon[35127]: Deploying daemon mon.b on smithi151 2024-09-12T01:38:22.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:22 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/2626375687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:22.262 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:22 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:22.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:22 smithi122 ceph-mon[30925]: Deploying daemon mon.b on smithi151 2024-09-12T01:38:22.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:22 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/2626375687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:22.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:22 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:23 smithi183 ceph-mon[35127]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:23.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:23 smithi122 ceph-mon[30925]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:23.489 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:38:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:38:23.112+0000 7fda83b01640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-12T01:38:23.541 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:38:23.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:38:23.794 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:38:24.241 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 -- 172.21.15.183:0/3906422192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 msgr2=0x7feb3c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 --2- 172.21.15.183:0/3906422192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb38009880 tx=0x7feb3802a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 -- 172.21.15.183:0/3906422192 shutdown_connections 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 --2- 172.21.15.183:0/3906422192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 -- 172.21.15.183:0/3906422192 >> 172.21.15.183:0/3906422192 conn(0x7feb3c0fc220 msgr2=0x7feb3c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 -- 172.21.15.183:0/3906422192 shutdown_connections 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.237+0000 7feb441c0640 1 -- 172.21.15.183:0/3906422192 wait complete. 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb441c0640 1 Processor -- start 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb441c0640 1 -- start start 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb441c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c0fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:24.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb441c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb441c0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c10ac60 con 0x7feb3c106630 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb441c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c10aae0 con 0x7feb3c0fcc10 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36260/0 (socket says 172.21.15.183:36260) 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 -- 172.21.15.183:0/3611870191 learned_addr learned my addr 172.21.15.183:0/3611870191 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 -- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 msgr2=0x7feb3c0fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 -- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 msgr2=0x7feb3c0fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb4175b640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.238+0000 7feb41f5c640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c0fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb41f5c640 1 -- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 msgr2=0x7feb3c0fd070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb41f5c640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:24.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb41f5c640 1 -- 172.21.15.183:0/3611870191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c0fe5c0 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb41f5c640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c0fc6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb38000c00 tx=0x7feb38032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7feb38035070 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb38032e40 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c1a3650 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7feb3803bc30 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.239+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1a3b80 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.240+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3c101ca0 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.240+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7feb38042030 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.240+0000 7feb32ffd640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feb28038e00 0x7feb2803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.240+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feb38040070 con 0x7feb3c106630 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.241+0000 7feb4175b640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feb28038e00 0x7feb2803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:24.244 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.242+0000 7feb4175b640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feb28038e00 0x7feb2803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009100 tx=0x7feb2c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:24.245 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.245+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb3804a260 con 0x7feb3c106630 2024-09-12T01:38:24.477 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.474+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feb3c106a90 con 0x7feb3c106630 2024-09-12T01:38:24.477 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.475+0000 7feb32ffd640 1 -- 172.21.15.183:0/3611870191 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7feb38003d20 con 0x7feb3c106630 2024-09-12T01:38:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:38:24.478 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","modified":"2024-09-12T01:38:16.111939Z","created":"2024-09-12T01:36:46.446315Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T01:38:24.478 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feb28038e00 msgr2=0x7feb2803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feb28038e00 0x7feb2803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009100 tx=0x7feb2c008040 comp rx=0 tx=0).stop 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 msgr2=0x7feb3c0fc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c0fc6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb38000c00 tx=0x7feb38032c50 comp rx=0 tx=0).stop 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 shutdown_connections 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feb28038e00 0x7feb2803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c0fcc10 0x7feb3c0fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:24.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 --2- 172.21.15.183:0/3611870191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb3c106630 0x7feb3c0fc6d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:24.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 >> 172.21.15.183:0/3611870191 conn(0x7feb3c0fc220 msgr2=0x7feb3c1900f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:24.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 shutdown_connections 2024-09-12T01:38:24.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:24.477+0000 7feb441c0640 1 -- 172.21.15.183:0/3611870191 wait complete. 2024-09-12T01:38:24.986 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: load: jerasure load: lrc 2024-09-12T01:38:25.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: RocksDB version: 7.9.2 2024-09-12T01:38:25.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Git sha 0 2024-09-12T01:38:25.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T01:38:25.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: DB SUMMARY 2024-09-12T01:38:25.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: DB Session ID: XRMSO06L53T0Y7VZ7R20 2024-09-12T01:38:25.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: CURRENT file: CURRENT 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.error_if_exists: 0 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.create_if_missing: 0 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.paranoid_checks: 1 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.env: 0x564a2d1bc940 2024-09-12T01:38:25.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.info_log: 0x564a2f0dd6e0 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.statistics: (nil) 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.use_fsync: 0 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_log_file_size: 0 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.allow_fallocate: 1 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T01:38:25.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.use_direct_reads: 0 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.db_log_dir: 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.wal_dir: 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T01:38:25.369 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.write_buffer_manager: 0x564a2f0ecdc0 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T01:38:25.371 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.unordered_write: 0 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.row_cache: None 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.wal_filter: None 2024-09-12T01:38:25.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.two_write_queues: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.wal_compression: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.atomic_flush: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.log_readahead_size: 0 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T01:38:25.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_background_jobs: 2 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_background_compactions: -1 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_subcompactions: 1 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T01:38:25.374 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_open_files: -1 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_background_flushes: -1 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Compression algorithms supported: 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kZSTD supported: 0 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kXpressCompression supported: 0 2024-09-12T01:38:25.375 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kBZip2Compression supported: 0 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kLZ4Compression supported: 1 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kZlibCompression supported: 1 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: kSnappyCompression supported: 1 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.merge_operator: 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_filter: None 2024-09-12T01:38:25.376 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564a2f0dd320) 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: index_type: 0 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: data_block_index_type: 0 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: index_shortening: 1 2024-09-12T01:38:25.377 INFO:journalctl@ceph.mon.b.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: checksum: 4 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: no_block_cache: 0 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache: 0x564a2f0d11f0 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache_options: 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: capacity : 536870912 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: num_shard_bits : 4 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: strict_capacity_limit : 0 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache_compressed: (nil) 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: persistent_cache: (nil) 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_size: 4096 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_size_deviation: 10 2024-09-12T01:38:25.378 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_restart_interval: 16 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: index_block_restart_interval: 1 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: metadata_block_size: 4096 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: partition_filters: 0 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: use_delta_encoding: 1 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: filter_policy: bloomfilter 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: whole_key_filtering: 1 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: verify_compression: 0 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: format_version: 5 2024-09-12T01:38:25.379 INFO:journalctl@ceph.mon.b.smithi151.stdout: enable_index_compression: 1 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_align: 0 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout: max_auto_readahead_size: 262144 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout: prepopulate_block_cache: 0 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout: initial_auto_readahead_size: 8192 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression: NoCompression 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.num_levels: 7 2024-09-12T01:38:25.380 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T01:38:25.381 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T01:38:25.382 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T01:38:25.383 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T01:38:25.384 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.inplace_update_support: 0 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.bloom_locality: 0 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.max_successive_merges: 0 2024-09-12T01:38:25.385 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.ttl: 2592000 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.enable_blob_files: false 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.min_blob_size: 0 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T01:38:25.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 62fd92f9-1dc7-4d40-8df9-f92e32d1e1d9 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105104988443, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105104988852, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726105104, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "62fd92f9-1dc7-4d40-8df9-f92e32d1e1d9", "db_session_id": "XRMSO06L53T0Y7VZ7R20", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726105104988944, "job": 1, "event": "recovery_finished"} 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T01:38:25.387 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564a2f0fee00 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: DB pointer 0x564a2f1fa000 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** DB Stats ** 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T01:38:25.388 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** Compaction Stats [default] ** 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** Compaction Stats [default] ** 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-09-12T01:38:25.389 INFO:journalctl@ceph.mon.b.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Block cache BinnedLRUCache@0x564a2f0d11f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-09-12T01:38:25.390 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: using public_addr v2:172.21.15.151:0/0 -> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: starting mon.b rank -1 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:24 smithi151 ceph-mon[35229]: mon.b@-1(???) e0 preinit fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: e1 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: btime 2024-09-12T01:36:52:283630+0000 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: legacy client fscid: -1 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-09-12T01:38:25.391 INFO:journalctl@ceph.mon.b.smithi151.stdout: No filesystems configured 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.392 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Added host smithi183 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3280359373' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3280359373' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.393 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.394 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Deploying daemon mon.c on smithi183 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/2793253993' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.a calling monitor election 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.395 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.c calling monitor election 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: monmap epoch 2 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: last_changed 2024-09-12T01:38:16.111939+0000 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: min_mon_release 19 (squid) 2024-09-12T01:38:25.396 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: election_strategy: 1 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: fsmap 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mgrmap e13: x(active, since 36s) 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: overall HEALTH_OK 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: Deploying daemon mon.b on smithi151 2024-09-12T01:38:25.397 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/2626375687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:25.398 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:25.398 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:25.398 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 ceph-mon[35229]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T01:38:25.398 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 bash[35211]: 088859bd539afd174ae540f68806c6924b3b620e82a6e132dc0bc0c109569e67 2024-09-12T01:38:25.398 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:25 smithi151 systemd[1]: Started Ceph mon.b for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:38:26.065 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T01:38:26.065 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T01:38:26.315 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.346+0000 7f4643e5f640 1 -- 172.21.15.183:0/2426295770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c1076b0 msgr2=0x7f4628005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.346+0000 7f4643e5f640 1 --2- 172.21.15.183:0/2426295770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c1076b0 0x7f4628005580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4638014d10 tx=0x7f463802c570 comp rx=0 tx=0).stop 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.346+0000 7f4643e5f640 1 -- 172.21.15.183:0/2426295770 shutdown_connections 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.346+0000 7f4643e5f640 1 --2- 172.21.15.183:0/2426295770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c1076b0 0x7f4628005580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.346+0000 7f4643e5f640 1 -- 172.21.15.183:0/2426295770 >> 172.21.15.183:0/2426295770 conn(0x7f463c0fd390 msgr2=0x7f463c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.347+0000 7f4643e5f640 1 -- 172.21.15.183:0/2426295770 shutdown_connections 2024-09-12T01:38:30.347 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.347+0000 7f4643e5f640 1 -- 172.21.15.183:0/2426295770 wait complete. 2024-09-12T01:38:30.348 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 Processor -- start 2024-09-12T01:38:30.348 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 -- start start 2024-09-12T01:38:30.348 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:38:30.348 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: mon.a calling monitor election 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: mon.c calling monitor election 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: mon.b calling monitor election 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:38:30.349 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: monmap epoch 3 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: last_changed 2024-09-12T01:38:25.014350+0000 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: min_mon_release 19 (squid) 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: election_strategy: 1 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: fsmap 2024-09-12T01:38:30.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:30.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: mgrmap e13: x(active, since 45s) 2024-09-12T01:38:30.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: overall HEALTH_OK 2024-09-12T01:38:30.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 0x7f463c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:30.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c197810 0x7f463c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:30.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:30.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f463c109770 con 0x7f463c197810 2024-09-12T01:38:30.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1095f0 con 0x7f463c1076b0 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4643e5f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1098f0 con 0x7f463c19c0e0 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:46612/0 (socket says 172.21.15.183:46612) 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 -- 172.21.15.183:0/1641708879 learned_addr learned my addr 172.21.15.183:0/1641708879 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46413fa640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c197810 0x7f463c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4641bfb640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 0x7f463c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 msgr2=0x7f463c19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 msgr2=0x7f463c19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f46423fc640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4641bfb640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 msgr2=0x7f463c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T01:38:30.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4641bfb640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 msgr2=0x7f463c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4641bfb640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 0x7f463c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.348+0000 7f4641bfb640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 0x7f463c1972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f46413fa640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 msgr2=0x7f463c19c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f46413fa640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f46413fa640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 msgr2=0x7f463c1972d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f46413fa640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 0x7f463c1972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f46413fa640 1 -- 172.21.15.183:0/1641708879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a3e50 con 0x7f463c197810 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f46413fa640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c197810 0x7f463c19bba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f462c0089e0 tx=0x7f462c008ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c004230 con 0x7f463c197810 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f462c0043d0 con 0x7f463c197810 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c074480 con 0x7f463c197810 2024-09-12T01:38:30.355 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c003b80 con 0x7f463c197810 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.349+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c074a10 con 0x7f463c197810 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.350+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f462c01e3f0 con 0x7f463c197810 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.350+0000 7f4632ffd640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46200394f0 0x7f462003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.351+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f462c04b250 con 0x7f463c197810 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.351+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c076430 con 0x7f463c197810 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.351+0000 7f4641bfb640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46200394f0 0x7f462003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.351+0000 7f4641bfb640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46200394f0 0x7f462003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4638004f80 tx=0x7f4638005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:30.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.356+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f462c012500 con 0x7f463c197810 2024-09-12T01:38:30.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:38:30.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: mon.a calling monitor election 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: mon.c calling monitor election 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: mon.b calling monitor election 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: monmap epoch 3 2024-09-12T01:38:30.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: last_changed 2024-09-12T01:38:25.014350+0000 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: min_mon_release 19 (squid) 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: election_strategy: 1 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: fsmap 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: mgrmap e13: x(active, since 45s) 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: overall HEALTH_OK 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.368 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: mon.a calling monitor election 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: mon.c calling monitor election 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: mon.b calling monitor election 2024-09-12T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: monmap epoch 3 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: last_changed 2024-09-12T01:38:25.014350+0000 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: created 2024-09-12T01:36:46.446315+0000 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: min_mon_release 19 (squid) 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: election_strategy: 1 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-09-12T01:38:30.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: fsmap 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: mgrmap e13: x(active, since 45s) 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: overall HEALTH_OK 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.491 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:30.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.622+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f463c075ea0 con 0x7f463c197810 2024-09-12T01:38:30.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.622+0000 7f4632ffd640 1 -- 172.21.15.183:0/1641708879 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f462c0122a0 con 0x7f463c197810 2024-09-12T01:38:30.625 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:38:30.625 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","modified":"2024-09-12T01:38:25.014350Z","created":"2024-09-12T01:36:46.446315Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-12T01:38:30.626 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-09-12T01:38:30.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46200394f0 msgr2=0x7f462003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:30.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46200394f0 0x7f462003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4638004f80 tx=0x7f4638005da0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c197810 msgr2=0x7f463c19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c197810 0x7f463c19bba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f462c0089e0 tx=0x7f462c008ea0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 shutdown_connections 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46200394f0 0x7f462003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f463c19c0e0 0x7f463c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f463c197810 0x7f463c19bba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 --2- 172.21.15.183:0/1641708879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f463c1076b0 0x7f463c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 >> 172.21.15.183:0/1641708879 conn(0x7f463c0fd390 msgr2=0x7f463c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 shutdown_connections 2024-09-12T01:38:30.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:30.625+0000 7f4643e5f640 1 -- 172.21.15.183:0/1641708879 wait complete. 2024-09-12T01:38:31.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:31.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:31 smithi183 ceph-mon[35127]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:31.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/1641708879' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:31.352 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:31.352 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:31.352 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:31.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:31.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:31 smithi151 ceph-mon[35229]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:31.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/1641708879' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:31.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:31.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:31.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:31.379 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T01:38:31.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T01:38:31.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:31.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:31 smithi122 ceph-mon[30925]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:31.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/1641708879' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T01:38:31.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:38:31.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:31.409 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:32.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.116+0000 7f4ef058a640 1 -- 172.21.15.122:0/2830490647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 msgr2=0x7f4ee8100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.116+0000 7f4ef058a640 1 --2- 172.21.15.122:0/2830490647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8100d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009880 tx=0x7f4ee402a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- 172.21.15.122:0/2830490647 shutdown_connections 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 --2- 172.21.15.122:0/2830490647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8100d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- 172.21.15.122:0/2830490647 >> 172.21.15.122:0/2830490647 conn(0x7f4ee8076250 msgr2=0x7f4ee8076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- 172.21.15.122:0/2830490647 shutdown_connections 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- 172.21.15.122:0/2830490647 wait complete. 2024-09-12T01:38:32.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 Processor -- start 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- start start 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ee8199ab0 0x7f4ee819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee819e380 0x7f4ee819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee810ab40 con 0x7f4ee8100990 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee810a9c0 con 0x7f4ee8199ab0 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4ef058a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee810acc0 con 0x7f4ee819e380 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4eee326640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.117+0000 7f4eee326640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34722/0 (socket says 172.21.15.122:34722) 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 -- 172.21.15.122:0/3737501676 learned_addr learned my addr 172.21.15.122:0/3737501676 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eeeb27640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee819e380 0x7f4ee819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:32.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eedb25640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ee8199ab0 0x7f4ee819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 -- 172.21.15.122:0/3737501676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee819e380 msgr2=0x7f4ee819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee819e380 0x7f4ee819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 -- 172.21.15.122:0/3737501676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ee8199ab0 msgr2=0x7f4ee819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ee8199ab0 0x7f4ee819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 -- 172.21.15.122:0/3737501676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee81a60f0 con 0x7f4ee8100990 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eedb25640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ee8199ab0 0x7f4ee819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eeeb27640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee819e380 0x7f4ee819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4eee326640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8199570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4031550 tx=0x7f4ee40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee403f030 con 0x7f4ee8100990 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee4003ce0 con 0x7f4ee8100990 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81a62c0 con 0x7f4ee8100990 2024-09-12T01:38:32.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4035070 con 0x7f4ee8100990 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.118+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a68a0 con 0x7f4ee8100990 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.119+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4ee4042030 con 0x7f4ee8100990 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.119+0000 7f4edf7fe640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ed40394f0 0x7f4ed403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.119+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4ee406cce0 con 0x7f4ee8100990 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.119+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee8100df0 con 0x7f4ee8100990 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.119+0000 7f4eedb25640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ed40394f0 0x7f4ed403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.120+0000 7f4eedb25640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ed40394f0 0x7f4ed403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007920 tx=0x7f4ed8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:32.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.124+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ee4036260 con 0x7f4ee8100990 2024-09-12T01:38:32.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.302+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4ee81a6ad0 con 0x7f4ee8100990 2024-09-12T01:38:32.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.303+0000 7f4edf7fe640 1 -- 172.21.15.122:0/3737501676 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f4ee404dc30 con 0x7f4ee8100990 2024-09-12T01:38:32.303 INFO:teuthology.orchestra.run.smithi122.stdout:# minimal ceph.conf for 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:32.303 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-12T01:38:32.303 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:38:32.303 INFO:teuthology.orchestra.run.smithi122.stdout: mon_host = [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ed40394f0 msgr2=0x7f4ed403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ed40394f0 0x7f4ed403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007920 tx=0x7f4ed8008040 comp rx=0 tx=0).stop 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 msgr2=0x7f4ee8199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8199570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4031550 tx=0x7f4ee40042f0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 shutdown_connections 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ed40394f0 0x7f4ed403b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee819e380 0x7f4ee819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ee8199ab0 0x7f4ee819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 --2- 172.21.15.122:0/3737501676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ee8100990 0x7f4ee8199570 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:32.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 >> 172.21.15.122:0/3737501676 conn(0x7f4ee8076250 msgr2=0x7f4ee80696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:32.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 shutdown_connections 2024-09-12T01:38:32.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:32.305+0000 7f4ef058a640 1 -- 172.21.15.122:0/3737501676 wait complete. 2024-09-12T01:38:32.878 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T01:38:32.879 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:38:32.879 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:38:32.941 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Updating smithi122:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: Reconfiguring daemon mon.a on smithi122 2024-09-12T01:38:32.943 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3737501676' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:32.944 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:32 smithi122 ceph-mon[30925]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:32.945 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:38:32.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:32.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Updating smithi122:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: Reconfiguring daemon mon.a on smithi122 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3737501676' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:32.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:32 smithi183 ceph-mon[35127]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:33.012 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:38:33.012 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Updating smithi122:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Updating smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Updating smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T01:38:33.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T01:38:33.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:33.041 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:33.041 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:33.041 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: Reconfiguring daemon mon.a on smithi122 2024-09-12T01:38:33.041 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3737501676' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:33.041 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:32 smithi151 ceph-mon[35229]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:33.043 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:38:33.043 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:33.111 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:38:33.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T01:38:33.141 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:38:33.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T01:38:33.209 INFO:tasks.cephadm:Adding mgr.x on smithi122 2024-09-12T01:38:33.209 INFO:tasks.cephadm:Adding mgr.y on smithi151 2024-09-12T01:38:33.210 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-09-12T01:38:33.210 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi122=x;smithi151=y;smithi183=z' 2024-09-12T01:38:33.486 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:38:33.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 -- 172.21.15.183:0/3159227542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e41016e0 msgr2=0x7f30e4101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:33.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 --2- 172.21.15.183:0/3159227542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e41016e0 0x7f30e4101ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009880 tx=0x7f30e002a5d0 comp rx=0 tx=0).stop 2024-09-12T01:38:33.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 -- 172.21.15.183:0/3159227542 shutdown_connections 2024-09-12T01:38:33.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 --2- 172.21.15.183:0/3159227542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e41016e0 0x7f30e4101ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:33.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 -- 172.21.15.183:0/3159227542 >> 172.21.15.183:0/3159227542 conn(0x7f30e40fce90 msgr2=0x7f30e40ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 -- 172.21.15.183:0/3159227542 shutdown_connections 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.912+0000 7f30ec15b640 1 -- 172.21.15.183:0/3159227542 wait complete. 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 Processor -- start 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 -- start start 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30e41016e0 0x7f30e4196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 0x7f30e419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e4197990 0x7f30e419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30e41077d0 con 0x7f30e4197990 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4107650 con 0x7f30e41016e0 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30ec15b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4107950 con 0x7f30e4197510 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30e96f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 0x7f30e419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30e96f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 0x7f30e419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:46616/0 (socket says 172.21.15.183:46616) 2024-09-12T01:38:33.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.913+0000 7f30e96f6640 1 -- 172.21.15.183:0/1584653062 learned_addr learned my addr 172.21.15.183:0/1584653062 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30ea6f8640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e4197990 0x7f30e419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e9ef7640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30e41016e0 0x7f30e4196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e96f6640 1 -- 172.21.15.183:0/1584653062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30e41016e0 msgr2=0x7f30e4196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e96f6640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30e41016e0 0x7f30e4196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e96f6640 1 -- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e4197990 msgr2=0x7f30e419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e96f6640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e4197990 0x7f30e419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e96f6640 1 -- 172.21.15.183:0/1584653062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e419f3d0 con 0x7f30e4197510 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30ea6f8640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e4197990 0x7f30e419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30e96f6640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 0x7f30e419c580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b520 tx=0x7f30d400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4004230 con 0x7f30e4197510 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d40043d0 con 0x7f30e4197510 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e419f6c0 con 0x7f30e4197510 2024-09-12T01:38:33.942 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4003d60 con 0x7f30e4197510 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.914+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e419fc80 con 0x7f30e4197510 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.915+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f30d400e070 con 0x7f30e4197510 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.915+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e4190590 con 0x7f30e4197510 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.915+0000 7f30daffd640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30d00394f0 0x7f30d003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.915+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f30d404ba60 con 0x7f30e4197510 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.915+0000 7f30e9ef7640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30d00394f0 0x7f30d003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.916+0000 7f30e9ef7640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30d00394f0 0x7f30d003b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009230 tx=0x7f30e00315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:33.943 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:33.920+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30d4014020 con 0x7f30e4197510 2024-09-12T01:38:34.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.101+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}) -- 0x7f30e40626c0 con 0x7f30d00394f0 2024-09-12T01:38:34.105 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.106+0000 7f30daffd640 1 -- 172.21.15.183:0/1584653062 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f30e40626c0 con 0x7f30d00394f0 2024-09-12T01:38:34.106 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-09-12T01:38:34.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30d00394f0 msgr2=0x7f30d003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:34.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30d00394f0 0x7f30d003b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009230 tx=0x7f30e00315d0 comp rx=0 tx=0).stop 2024-09-12T01:38:34.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 msgr2=0x7f30e419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:34.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 0x7f30e419c580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f30d400b520 tx=0x7f30d400b9e0 comp rx=0 tx=0).stop 2024-09-12T01:38:34.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 shutdown_connections 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30d00394f0 0x7f30d003b9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30e4197990 0x7f30e419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30e4197510 0x7f30e419c580 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 --2- 172.21.15.183:0/1584653062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30e41016e0 0x7f30e4196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 >> 172.21.15.183:0/1584653062 conn(0x7f30e40fce90 msgr2=0x7f30e40fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 shutdown_connections 2024-09-12T01:38:34.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:38:34.108+0000 7f30ec15b640 1 -- 172.21.15.183:0/1584653062 wait complete. 2024-09-12T01:38:34.592 DEBUG:teuthology.orchestra.run.smithi151:mgr.y> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T01:38:34.594 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T01:38:34.597 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T01:38:34.597 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:38:34.597 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:38:34.617 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:38:34.617 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-12T01:38:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:38:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:34.675 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T01:38:34.675 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:34.675 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:34.675 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-12 01:37:55.067483498 +0000 2024-09-12T01:38:34.676 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-12 01:33:20.169730177 +0000 2024-09-12T01:38:34.676 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-12 01:33:20.169730177 +0000 2024-09-12T01:38:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-12 01:33:20.169730177 +0000 2024-09-12T01:38:34.676 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:38:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-12T01:38:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-12T01:38:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000304672 s, 1.7 MB/s 2024-09-12T01:38:34.746 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:38:34.806 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-12 01:37:55.098482743 +0000 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-12 01:33:20.507721874 +0000 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-12 01:33:20.507721874 +0000 2024-09-12T01:38:34.865 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-12 01:33:20.507721874 +0000 2024-09-12T01:38:34.866 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:38:34.934 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-12T01:38:34.934 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-12T01:38:34.934 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000225505 s, 2.3 MB/s 2024-09-12T01:38:34.935 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:38:34.995 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-12T01:38:35.054 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:38:35.054 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:35.054 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T01:38:35.054 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:35.055 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:35.055 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-12 01:37:55.128482011 +0000 2024-09-12T01:38:35.055 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-12 01:33:20.821714161 +0000 2024-09-12T01:38:35.055 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-12 01:33:20.821714161 +0000 2024-09-12T01:38:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-12 01:33:20.821714161 +0000 2024-09-12T01:38:35.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:38:35.120 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: Reconfiguring daemon mon.b on smithi151 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: Saving service mgr spec with placement smithi122=x;smithi151=y;smithi183=z;count:3 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:35.121 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:34 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.123 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-12T01:38:35.123 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-12T01:38:35.123 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000199843 s, 2.6 MB/s 2024-09-12T01:38:35.125 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:38:35.184 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-12T01:38:35.243 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:38:35.243 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:35.243 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-12T01:38:35.243 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:35.244 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:35.244 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-12 01:37:55.159481256 +0000 2024-09-12T01:38:35.244 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-12 01:33:21.162705785 +0000 2024-09-12T01:38:35.244 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-12 01:33:21.162705785 +0000 2024-09-12T01:38:35.244 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-12 01:33:21.162705785 +0000 2024-09-12T01:38:35.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: Reconfiguring daemon mon.b on smithi151 2024-09-12T01:38:35.299 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:35.300 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: Saving service mgr spec with placement smithi122=x;smithi151=y;smithi183=z;count:3 2024-09-12T01:38:35.300 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.300 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:35.300 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:34 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-12T01:38:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-12T01:38:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000202643 s, 2.5 MB/s 2024-09-12T01:38:35.314 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:38:35.374 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-12T01:38:35.374 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:38:35.390 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:38:35.390 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-09-12T01:38:35.445 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-12 01:38:24.424435490 +0000 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-12 01:33:37.546320071 +0000 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-12 01:33:37.546320071 +0000 2024-09-12T01:38:35.446 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-12 01:33:37.546320071 +0000 2024-09-12T01:38:35.446 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:38:35.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: Reconfiguring mon.b (monmap changed)... 2024-09-12T01:38:35.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:35.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: Reconfiguring daemon mon.b on smithi151 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: Saving service mgr spec with placement smithi122=x;smithi151=y;smithi183=z;count:3 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:34 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:35.510 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-12T01:38:35.510 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-12T01:38:35.510 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000218951 s, 2.3 MB/s 2024-09-12T01:38:35.511 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:38:35.568 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-12 01:38:24.424435490 +0000 2024-09-12T01:38:35.625 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-12 01:33:37.892315218 +0000 2024-09-12T01:38:35.626 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-12 01:33:37.892315218 +0000 2024-09-12T01:38:35.626 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-12 01:33:37.892315218 +0000 2024-09-12T01:38:35.626 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:38:35.688 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-12T01:38:35.689 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-12T01:38:35.689 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000258345 s, 2.0 MB/s 2024-09-12T01:38:35.690 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:38:35.746 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-09-12T01:38:35.801 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:38:35.801 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:35.801 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-12T01:38:35.802 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:35.802 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:35.802 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-12 01:38:24.425435476 +0000 2024-09-12T01:38:35.802 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-12 01:33:38.223310576 +0000 2024-09-12T01:38:35.802 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-12 01:33:38.223310576 +0000 2024-09-12T01:38:35.802 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-12 01:33:38.223310576 +0000 2024-09-12T01:38:35.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:38:35.864 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-12T01:38:35.864 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-12T01:38:35.865 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000254567 s, 2.0 MB/s 2024-09-12T01:38:35.866 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:38:35.922 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-09-12T01:38:35.978 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-12 01:38:24.425435476 +0000 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-12 01:33:38.555305920 +0000 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-12 01:33:38.555305920 +0000 2024-09-12T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-12 01:33:38.555305920 +0000 2024-09-12T01:38:35.979 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:38:36.042 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-12T01:38:36.043 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-12T01:38:36.043 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.00024004 s, 2.1 MB/s 2024-09-12T01:38:36.044 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:38:36.100 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T01:38:36.100 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T01:38:36.116 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T01:38:36.116 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-12T01:38:36.172 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 01:38:15.405837407 +0000 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 01:33:33.680270269 +0000 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 01:33:33.680270269 +0000 2024-09-12T01:38:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 01:33:33.680270269 +0000 2024-09-12T01:38:36.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T01:38:36.236 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T01:38:36.236 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T01:38:36.236 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000272001 s, 1.9 MB/s 2024-09-12T01:38:36.237 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T01:38:36.294 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-12T01:38:36.350 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T01:38:36.350 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:36.350 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T01:38:36.351 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:36.351 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:36.351 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 01:38:15.405837407 +0000 2024-09-12T01:38:36.351 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 01:33:34.013267010 +0000 2024-09-12T01:38:36.351 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 01:33:34.013267010 +0000 2024-09-12T01:38:36.351 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 01:33:34.013267010 +0000 2024-09-12T01:38:36.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T01:38:36.414 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T01:38:36.414 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T01:38:36.414 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000269179 s, 1.9 MB/s 2024-09-12T01:38:36.415 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T01:38:36.471 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 01:38:15.405837407 +0000 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 01:33:34.339263820 +0000 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 01:33:34.339263820 +0000 2024-09-12T01:38:36.528 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 01:33:34.339263820 +0000 2024-09-12T01:38:36.529 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T01:38:36.591 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T01:38:36.592 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T01:38:36.592 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000245836 s, 2.1 MB/s 2024-09-12T01:38:36.593 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:36 smithi151 ceph-mon[35229]: Reconfiguring daemon mon.c on smithi183 2024-09-12T01:38:36.649 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-12T01:38:36.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:36.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:36.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:38:36.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:36.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:36.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:36.683 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:36 smithi183 ceph-mon[35127]: Reconfiguring daemon mon.c on smithi183 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 01:38:15.406837397 +0000 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 01:33:34.669260591 +0000 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 01:33:34.669260591 +0000 2024-09-12T01:38:36.698 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 01:33:34.669260591 +0000 2024-09-12T01:38:36.699 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T01:38:36.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: Reconfiguring mon.c (monmap changed)... 2024-09-12T01:38:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T01:38:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T01:38:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:36 smithi122 ceph-mon[30925]: Reconfiguring daemon mon.c on smithi183 2024-09-12T01:38:36.763 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T01:38:36.763 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T01:38:36.763 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000183867 s, 2.8 MB/s 2024-09-12T01:38:36.764 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T01:38:36.820 INFO:tasks.cephadm:Deploying osd.0 on smithi122 with /dev/vg_nvme/lv_4... 2024-09-12T01:38:36.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:38:37.068 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:37.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:37.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:37.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:37.472 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:37.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:37.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:37.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:37.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:37.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:37.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:37.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:38:37.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:37.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:38.372 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:38 smithi122 ceph-mon[30925]: Deploying daemon mgr.z on smithi183 2024-09-12T01:38:38.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:38 smithi151 ceph-mon[35229]: Deploying daemon mgr.z on smithi183 2024-09-12T01:38:38.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:38 smithi183 ceph-mon[35127]: Deploying daemon mgr.z on smithi183 2024-09-12T01:38:38.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 systemd[1]: Starting Ceph mgr.z for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 podman[36912]: 2024-09-12 01:38:38.609716098 +0000 UTC m=+0.015758057 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 podman[36912]: 2024-09-12 01:38:38.715931582 +0000 UTC m=+0.121973570 container create 673b42d8bd2ecb33eeb78c4f6a5a0d2a17a1dc6f78b0e774e7b1afd41bf97c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 podman[36912]: 2024-09-12 01:38:38.868096655 +0000 UTC m=+0.274138615 container init 673b42d8bd2ecb33eeb78c4f6a5a0d2a17a1dc6f78b0e774e7b1afd41bf97c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 podman[36912]: 2024-09-12 01:38:38.870802279 +0000 UTC m=+0.276844226 container start 673b42d8bd2ecb33eeb78c4f6a5a0d2a17a1dc6f78b0e774e7b1afd41bf97c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 ceph-mgr[36931]: -- 172.21.15.183:0/1064192208 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6989dd0e0 con 0x55e6989bb000 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 bash[36912]: 673b42d8bd2ecb33eeb78c4f6a5a0d2a17a1dc6f78b0e774e7b1afd41bf97c3d 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 systemd[1]: Started Ceph mgr.z for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:38:38.999 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:38.998+0000 7f32e1804040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T01:38:39.276 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.065+0000 7f32e1804040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T01:38:39.276 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.211+0000 7f32e1804040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:38:39.276 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.276+0000 7f32e1804040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:39.459 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:39.461 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:38:39.487 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-09-12T01:38:39.560 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.423+0000 7f32e1804040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:38:39.560 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.560+0000 7f32e1804040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:38:39.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:39.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:39.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:38:39.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:39.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:39.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:38:40.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 -- 172.21.15.122:0/4215097985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c1037c0 msgr2=0x7f6e2c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:40.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 --2- 172.21.15.122:0/4215097985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c1037c0 0x7f6e2c10a050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ca0 tx=0x7f6e2402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:38:40.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 -- 172.21.15.122:0/4215097985 shutdown_connections 2024-09-12T01:38:40.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 --2- 172.21.15.122:0/4215097985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c1037c0 0x7f6e2c10a050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:40.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 --2- 172.21.15.122:0/4215097985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c102e00 0x7f6e2c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:40.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 --2- 172.21.15.122:0/4215097985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c101c00 0x7f6e2c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:40.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 -- 172.21.15.122:0/4215097985 >> 172.21.15.122:0/4215097985 conn(0x7f6e2c0fd390 msgr2=0x7f6e2c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:38:40.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 -- 172.21.15.122:0/4215097985 shutdown_connections 2024-09-12T01:38:40.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.120+0000 7f6e31895640 1 -- 172.21.15.122:0/4215097985 wait complete. 2024-09-12T01:38:40.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 Processor -- start 2024-09-12T01:38:40.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 -- start start 2024-09-12T01:38:40.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 0x7f6e2c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:40.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c102e00 0x7f6e2c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:40.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c1037c0 0x7f6e2c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:40.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10d860 con 0x7f6e2c101c00 2024-09-12T01:38:40.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10d6e0 con 0x7f6e2c102e00 2024-09-12T01:38:40.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e31895640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10d9e0 con 0x7f6e2c1037c0 2024-09-12T01:38:40.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 0x7f6e2c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:40.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e2a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c102e00 0x7f6e2c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:40.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 0x7f6e2c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34742/0 (socket says 172.21.15.122:34742) 2024-09-12T01:38:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e2a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c102e00 0x7f6e2c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:35006/0 (socket says 172.21.15.122:35006) 2024-09-12T01:38:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e2affd640 1 -- 172.21.15.122:0/2184289466 learned_addr learned my addr 172.21.15.122:0/2184289466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:38:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.121+0000 7f6e2b7fe640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c1037c0 0x7f6e2c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e2affd640 1 -- 172.21.15.122:0/2184289466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c1037c0 msgr2=0x7f6e2c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e2affd640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c1037c0 0x7f6e2c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e2affd640 1 -- 172.21.15.122:0/2184289466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c102e00 msgr2=0x7f6e2c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e2affd640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c102e00 0x7f6e2c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e2affd640 1 -- 172.21.15.122:0/2184289466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c0753e0 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e2affd640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 0x7f6e2c075f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00d770 tx=0x7f6e1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c017070 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1c009c30 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c003da0 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1a1fb0 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.122+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1a2460 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.123+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6e1c003f40 con 0x7f6e2c101c00 2024-09-12T01:38:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.123+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c109000 con 0x7f6e2c101c00 2024-09-12T01:38:40.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.123+0000 7f6e30893640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e18039290 0x7f6e1803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:38:40.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.124+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6e1c04bd70 con 0x7f6e2c101c00 2024-09-12T01:38:40.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.124+0000 7f6e2a7fc640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e18039290 0x7f6e1803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:38:40.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.124+0000 7f6e2a7fc640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e18039290 0x7f6e1803b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c075d10 tx=0x7f6e1400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:38:40.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.128+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e1c050070 con 0x7f6e2c101c00 2024-09-12T01:38:40.202 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.872+0000 7f32e1804040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:38:40.202 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:39.933+0000 7f32e1804040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:38:40.202 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:40.115+0000 7f32e1804040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:38:40.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:40.303+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6e2c062630 con 0x7f6e18039290 2024-09-12T01:38:40.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:40 smithi122 ceph-mon[30925]: Deploying daemon mgr.y on smithi151 2024-09-12T01:38:40.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:38:40.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:38:40.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:40 smithi151 ceph-mon[35229]: Deploying daemon mgr.y on smithi151 2024-09-12T01:38:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:38:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:38:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:40.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:40 smithi183 ceph-mon[35127]: Deploying daemon mgr.y on smithi151 2024-09-12T01:38:40.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:38:40.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:38:40.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:41.365 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:40.991+0000 7f32e1804040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:38:41.366 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:41.056+0000 7f32e1804040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:38:41.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:41 smithi122 ceph-mon[30925]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:41.603 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:41 smithi122 ceph-mon[30925]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:41 smithi183 ceph-mon[35127]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:41 smithi183 ceph-mon[35127]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:41.753 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:41 smithi151 ceph-mon[35229]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:38:41.753 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:41 smithi151 ceph-mon[35229]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:41.753 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:41 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:41.753+0000 7f089d526040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T01:38:42.057 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:41.778+0000 7f32e1804040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:38:42.057 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:41.898+0000 7f32e1804040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:38:42.058 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.057+0000 7f32e1804040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:38:42.115 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:41 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:41.823+0000 7f089d526040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T01:38:42.115 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:41 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:41.981+0000 7f089d526040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:38:42.343 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.121+0000 7f32e1804040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:38:42.343 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.272+0000 7f32e1804040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:38:42.344 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.343+0000 7f32e1804040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:38:42.411 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:42.139+0000 7f089d526040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T01:38:42.635 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.635 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.635 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3735225776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]: dispatch 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]: dispatch 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]': finished 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:38:42.636 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:38:42.637 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.409+0000 7f32e1804040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:38:42.637 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.570+0000 7f32e1804040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:38:42.637 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.635+0000 7f32e1804040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:38:42.720 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:42.464+0000 7f089d526040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T01:38:42.720 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:42.526+0000 7f089d526040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T01:38:42.720 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3735225776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]: dispatch 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]: dispatch 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]': finished 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:38:42.721 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:38:42.874 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.874 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.874 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.874 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:42.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:42.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3735225776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]: dispatch 2024-09-12T01:38:42.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]: dispatch 2024-09-12T01:38:42.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5f05d0c-f7d1-4a68-8a67-6daf86c09856"}]': finished 2024-09-12T01:38:42.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T01:38:42.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:38:42.912 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.701+0000 7f32e1804040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:38:42.912 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.787+0000 7f32e1804040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:38:42.912 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.852+0000 7f32e1804040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:38:42.912 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:42.912+0000 7f32e1804040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:38:43.115 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:42 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:42.720+0000 7f089d526040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T01:38:43.865 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:43.608+0000 7f089d526040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T01:38:43.865 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:43.674+0000 7f089d526040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:43.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:43 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2570138980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:38:43.952 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z[36927]: 2024-09-12T01:38:43.696+0000 7f32e1804040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:43 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2570138980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:38:43.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:43.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:43 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2570138980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:38:44.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:44.576+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f6e1c010460 con 0x7f6e2c101c00 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: Reconfiguring daemon mgr.x on smithi122 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: Standby manager daemon z started 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:38:44.874 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:38:44.875 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:44.875 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.391+0000 7f089d526040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T01:38:44.875 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.507+0000 7f089d526040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T01:38:44.875 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.650+0000 7f089d526040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T01:38:44.927 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.711+0000 7f089d526040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T01:38:44.927 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.860+0000 7f089d526040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T01:38:44.927 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.927+0000 7f089d526040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: Reconfiguring daemon mgr.x on smithi122 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: Standby manager daemon z started 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:38:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:38:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: Reconfiguring daemon mgr.x on smithi122 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: Standby manager daemon z started 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.183:0/3489251420' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:38:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:45.209 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:44.987+0000 7f089d526040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T01:38:45.210 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:45.145+0000 7f089d526040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T01:38:45.210 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:45.210+0000 7f089d526040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T01:38:45.474 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:45.276+0000 7f089d526040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T01:38:45.474 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:45.357+0000 7f089d526040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T01:38:45.474 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:45.415+0000 7f089d526040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T01:38:45.474 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:45.474+0000 7f089d526040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T01:38:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-mon[35229]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-mon[35229]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-12T01:38:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:38:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:45 smithi183 ceph-mon[35127]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:45 smithi183 ceph-mon[35127]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-12T01:38:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:38:45.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:45 smithi122 ceph-mon[30925]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:45.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:45 smithi122 ceph-mon[30925]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-12T01:38:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T01:38:46.573 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y[36595]: 2024-09-12T01:38:46.260+0000 7f089d526040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T01:38:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:38:46.585+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f6e1c019b10 con 0x7f6e2c101c00 2024-09-12T01:38:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-mon[35229]: Standby manager daemon y started 2024-09-12T01:38:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:38:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:38:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:38:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-mon[35229]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:38:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:46 smithi151 ceph-mon[35229]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:46 smithi183 ceph-mon[35127]: Standby manager daemon y started 2024-09-12T01:38:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:46 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:38:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:46 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:38:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:46 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:38:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:46 smithi183 ceph-mon[35127]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:38:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:46 smithi183 ceph-mon[35127]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:46 smithi122 ceph-mon[30925]: Standby manager daemon y started 2024-09-12T01:38:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:46 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T01:38:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:46 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T01:38:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:46 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T01:38:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:46 smithi122 ceph-mon[30925]: from='mgr.? 172.21.15.151:0/3662967498' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T01:38:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:46 smithi122 ceph-mon[30925]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:47.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:47 smithi122 ceph-mon[30925]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-12T01:38:47.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:38:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:47 smithi151 ceph-mon[35229]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-12T01:38:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:38:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:47 smithi183 ceph-mon[35127]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-12T01:38:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T01:38:48.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:48 smithi122 ceph-mon[30925]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:48.868 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:48 smithi151 ceph-mon[35229]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:48 smithi183 ceph-mon[35127]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:50.219 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.219 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.220 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:50.220 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:50.220 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:50.220 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:50.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:50.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:50.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:38:50.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:51.385 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:51 smithi122 ceph-mon[30925]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:51.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:51 smithi183 ceph-mon[35127]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:51.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:51 smithi151 ceph-mon[35229]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:38:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:52 smithi183 ceph-mon[35127]: Deploying daemon osd.0 on smithi122 2024-09-12T01:38:52.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:38:52.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:52.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:52 smithi122 ceph-mon[30925]: Deploying daemon osd.0 on smithi122 2024-09-12T01:38:52.516 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T01:38:52.516 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:38:52.517 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:52 smithi151 ceph-mon[35229]: Deploying daemon osd.0 on smithi122 2024-09-12T01:38:53.334 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:53 smithi122 ceph-mon[30925]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:53.533 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:53 smithi151 ceph-mon[35229]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:53.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:53 smithi183 ceph-mon[35127]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:55.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:55 smithi183 ceph-mon[35127]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:55.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:55 smithi122 ceph-mon[30925]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:55.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:55 smithi151 ceph-mon[35229]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:57.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:57 smithi183 ceph-mon[35127]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:57.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:57 smithi122 ceph-mon[30925]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:57 smithi151 ceph-mon[35229]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:58.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:58.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:58.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:58.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:58.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:38:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:38:59.675 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:38:59 smithi122 ceph-mon[30925]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:59.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:38:59 smithi183 ceph-mon[35127]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:38:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:38:59 smithi151 ceph-mon[35229]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:01.451 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.451 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:01 smithi122 ceph-mon[30925]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:01.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:01.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:01 smithi183 ceph-mon[35127]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.493+0000 7f6e30893640 1 -- 172.21.15.122:0/2184289466 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6e2c062630 con 0x7f6e18039290 2024-09-12T01:39:01.493 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 0 on host 'smithi122' 2024-09-12T01:39:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.495+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e18039290 msgr2=0x7f6e1803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.495+0000 7f6e31895640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e18039290 0x7f6e1803b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c075d10 tx=0x7f6e1400b410 comp rx=0 tx=0).stop 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.495+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 msgr2=0x7f6e2c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.495+0000 7f6e31895640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 0x7f6e2c075f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00d770 tx=0x7f6e1c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 shutdown_connections 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e18039290 0x7f6e1803b750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c1037c0 0x7f6e2c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2c102e00 0x7f6e2c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 --2- 172.21.15.122:0/2184289466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c101c00 0x7f6e2c075f90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 >> 172.21.15.122:0/2184289466 conn(0x7f6e2c0fd390 msgr2=0x7f6e2c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 shutdown_connections 2024-09-12T01:39:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:01.496+0000 7f6e31895640 1 -- 172.21.15.122:0/2184289466 wait complete. 2024-09-12T01:39:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:01 smithi151 ceph-mon[35229]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:02.124 DEBUG:teuthology.orchestra.run.smithi122:osd.0> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T01:39:02.126 INFO:tasks.cephadm:Deploying osd.1 on smithi151 with /dev/vg_nvme/lv_4... 2024-09-12T01:39:02.126 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:39:02.382 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:39:02.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:02.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:02.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:02.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:02.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:02.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:02.496 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:02.497 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:02.497 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:02.702 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:39:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0[45064]: 2024-09-12T01:39:02.536+0000 7f3b97dce640 -1 osd.0 0 log_to_monitors true 2024-09-12T01:39:03.366 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:03 smithi122 ceph-mon[30925]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:03.366 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:03 smithi122 ceph-mon[30925]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:39:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:03 smithi183 ceph-mon[35127]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:03 smithi183 ceph-mon[35127]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:39:03.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:03 smithi151 ceph-mon[35229]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:03.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:03 smithi151 ceph-mon[35229]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T01:39:04.351 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:04 smithi122 ceph-mon[30925]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:39:04.351 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:04 smithi122 ceph-mon[30925]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:39:04.351 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:04 smithi122 ceph-mon[30925]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:39:04.351 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:04.351 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:39:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0[45064]: 2024-09-12T01:39:04.211+0000 7f3b93d51640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T01:39:04.351 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:39:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0[45064]: 2024-09-12T01:39:04.214+0000 7f3b8fb79640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:39:04.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:04 smithi151 ceph-mon[35229]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:39:04.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:04 smithi151 ceph-mon[35229]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:39:04.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:04 smithi151 ceph-mon[35229]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:39:04.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:04 smithi183 ceph-mon[35127]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T01:39:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:04 smithi183 ceph-mon[35127]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T01:39:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:04 smithi183 ceph-mon[35127]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:39:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:04.745 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:39:04.772 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-09-12T01:39:05.015 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:05.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.511+0000 7f9e8213b640 1 -- 172.21.15.151:0/646802852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 msgr2=0x7f9e7c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.511+0000 7f9e8213b640 1 --2- 172.21.15.151:0/646802852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c109d10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009ca0 tx=0x7f9e7402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:39:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 -- 172.21.15.151:0/646802852 shutdown_connections 2024-09-12T01:39:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 --2- 172.21.15.151:0/646802852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c109d10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 --2- 172.21.15.151:0/646802852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c104ff0 0x7f9e7c1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 --2- 172.21.15.151:0/646802852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e7c1026c0 0x7f9e7c104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 -- 172.21.15.151:0/646802852 >> 172.21.15.151:0/646802852 conn(0x7f9e7c0fc280 msgr2=0x7f9e7c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 -- 172.21.15.151:0/646802852 shutdown_connections 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 -- 172.21.15.151:0/646802852 wait complete. 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 Processor -- start 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 -- start start 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c1026c0 0x7f9e7c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e7c104ff0 0x7f9e7c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.512+0000 7f9e8213b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c10d880 con 0x7f9e7c1026c0 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e8213b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c10d700 con 0x7f9e7c107920 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e8213b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c10da00 con 0x7f9e7c104ff0 2024-09-12T01:39:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c1026c0 0x7f9e7c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c19f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50770/0 (socket says 172.21.15.151:50770) 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7bfff640 1 -- 172.21.15.151:0/1249145064 learned_addr learned my addr 172.21.15.151:0/1249145064 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e72ffd640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e7c104ff0 0x7f9e7c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 -- 172.21.15.151:0/1249145064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e7c104ff0 msgr2=0x7f9e7c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e7c104ff0 0x7f9e7c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 -- 172.21.15.151:0/1249145064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 msgr2=0x7f9e7c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 -- 172.21.15.151:0/1249145064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c19fb10 con 0x7f9e7c1026c0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e7b7fe640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c1026c0 0x7f9e7c197ee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00d720 tx=0x7f9e6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c017070 con 0x7f9e7c1026c0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c19fe00 con 0x7f9e7c1026c0 2024-09-12T01:39:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c00de30 con 0x7f9e7c1026c0 2024-09-12T01:39:05.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0047a0 con 0x7f9e7c1026c0 2024-09-12T01:39:05.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.513+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c1a0250 con 0x7f9e7c1026c0 2024-09-12T01:39:05.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.515+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c0fdd70 con 0x7f9e7c1026c0 2024-09-12T01:39:05.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.515+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f9e6c0099d0 con 0x7f9e7c1026c0 2024-09-12T01:39:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.516+0000 7f9e797fa640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9e680a2b20 0x7f9e680a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.516+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f9e6c0cd250 con 0x7f9e7c1026c0 2024-09-12T01:39:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.516+0000 7f9e72ffd640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9e680a2b20 0x7f9e680a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.517+0000 7f9e72ffd640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9e680a2b20 0x7f9e680a4fe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c199400 tx=0x7f9e64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:39:05.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.520+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e6c020020 con 0x7f9e7c1026c0 2024-09-12T01:39:05.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:05.692+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9e7c062630 con 0x7f9e680a2b20 2024-09-12T01:39:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:39:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T01:39:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T01:39:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:06.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:39:06.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi122 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi122 to 17087M 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] boot 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:06.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi122 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi122 to 17087M 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] boot 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:06.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:06.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:39:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:39:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi122 2024-09-12T01:39:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi122 to 17087M 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: osd.0 [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] boot 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:07.364 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:07 smithi151 ceph-mon[35229]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:07.364 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:07 smithi151 ceph-mon[35229]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:39:07.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:07 smithi151 ceph-mon[35229]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:07.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:07 smithi122 ceph-mon[30925]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:07.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:07 smithi122 ceph-mon[30925]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:39:07.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:07 smithi122 ceph-mon[30925]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:07 smithi183 ceph-mon[35127]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:07 smithi183 ceph-mon[35127]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T01:39:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:07 smithi183 ceph-mon[35127]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:08.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:08 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/3423758519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]: dispatch 2024-09-12T01:39:08.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:08 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]: dispatch 2024-09-12T01:39:08.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:08 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]': finished 2024-09-12T01:39:08.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:08 smithi122 ceph-mon[30925]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:39:08.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:08 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/3423758519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]: dispatch 2024-09-12T01:39:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:08 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]: dispatch 2024-09-12T01:39:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:08 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]': finished 2024-09-12T01:39:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:08 smithi151 ceph-mon[35229]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:39:08.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:08 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:08.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:08 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/3423758519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]: dispatch 2024-09-12T01:39:08.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:08 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]: dispatch 2024-09-12T01:39:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:08 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474588fe-b8aa-4c63-a179-9fb1a91609c6"}]': finished 2024-09-12T01:39:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:08 smithi183 ceph-mon[35127]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T01:39:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:08 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:09.488 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:09 smithi122 ceph-mon[30925]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:09 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/1336276868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:09.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:09 smithi151 ceph-mon[35229]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:09.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:09 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/1336276868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:09.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:09 smithi183 ceph-mon[35127]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:09.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:09 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/1336276868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:11 smithi183 ceph-mon[35127]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:11.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:11 smithi122 ceph-mon[30925]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:11 smithi151 ceph-mon[35229]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:13.609 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:13 smithi151 ceph-mon[35229]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:13.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:13 smithi183 ceph-mon[35127]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:13.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:13 smithi122 ceph-mon[30925]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:15.545 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:15 smithi151 ceph-mon[35229]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:15.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:15 smithi183 ceph-mon[35127]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:15.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:15 smithi122 ceph-mon[30925]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:17.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:17 smithi151 ceph-mon[35229]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:17.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:17 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:39:17.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:17 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:17.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:17 smithi151 ceph-mon[35229]: Deploying daemon osd.1 on smithi151 2024-09-12T01:39:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:17 smithi183 ceph-mon[35127]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:17 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:39:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:17 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:17 smithi183 ceph-mon[35127]: Deploying daemon osd.1 on smithi151 2024-09-12T01:39:17.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:17 smithi122 ceph-mon[30925]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:17 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T01:39:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:17 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:17 smithi122 ceph-mon[30925]: Deploying daemon osd.1 on smithi151 2024-09-12T01:39:19.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:19 smithi183 ceph-mon[35127]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:19.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:19 smithi122 ceph-mon[30925]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:19.819 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:19 smithi151 ceph-mon[35229]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:21.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:21 smithi183 ceph-mon[35127]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:21.736 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:21 smithi151 ceph-mon[35229]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:21.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:21 smithi122 ceph-mon[30925]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:23.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:23 smithi151 ceph-mon[35229]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:23.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:23.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:23.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:23.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:23 smithi122 ceph-mon[30925]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:23.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:23 smithi183 ceph-mon[35127]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:23.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:23.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:23.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:25.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:25 smithi122 ceph-mon[30925]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:25 smithi151 ceph-mon[35229]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:25 smithi183 ceph-mon[35127]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:26.696 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:26 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.696 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:26 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.696 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:26 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:26.696 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:26 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:26.697 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:26 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.734+0000 7f9e797fa640 1 -- 172.21.15.151:0/1249145064 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e7c062630 con 0x7f9e680a2b20 2024-09-12T01:39:26.734 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 1 on host 'smithi151' 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9e680a2b20 msgr2=0x7f9e680a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9e680a2b20 0x7f9e680a4fe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c199400 tx=0x7f9e64005f90 comp rx=0 tx=0).stop 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c1026c0 msgr2=0x7f9e7c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c1026c0 0x7f9e7c197ee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00d720 tx=0x7f9e6c00dbe0 comp rx=0 tx=0).stop 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 shutdown_connections 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9e680a2b20 0x7f9e680a4fe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e7c107920 0x7f9e7c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e7c104ff0 0x7f9e7c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 --2- 172.21.15.151:0/1249145064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e7c1026c0 0x7f9e7c197ee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 >> 172.21.15.151:0/1249145064 conn(0x7f9e7c0fc280 msgr2=0x7f9e7c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:39:26.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 shutdown_connections 2024-09-12T01:39:26.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:39:26.736+0000 7f9e8213b640 1 -- 172.21.15.151:0/1249145064 wait complete. 2024-09-12T01:39:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:26 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:26 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:26 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:26 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:26 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:26 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:26 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:26 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:26 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:26 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:27.439 DEBUG:teuthology.orchestra.run.smithi151:osd.1> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T01:39:27.442 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-09-12T01:39:27.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T01:39:27.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:27 smithi151 ceph-mon[35229]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:27.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:27.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:27.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:27.697 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:39:27.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:27 smithi183 ceph-mon[35127]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:27.815 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:27.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:27.816 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:27.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:27 smithi122 ceph-mon[30925]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:28.984 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:39:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1[39583]: 2024-09-12T01:39:28.730+0000 7f3266d45640 -1 osd.1 0 log_to_monitors true 2024-09-12T01:39:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:29 smithi183 ceph-mon[35127]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:29 smithi183 ceph-mon[35127]: from='osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:39:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:29 smithi183 ceph-mon[35127]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:39:29.803 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:29 smithi151 ceph-mon[35229]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:29.804 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:29 smithi151 ceph-mon[35229]: from='osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:39:29.804 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:29 smithi151 ceph-mon[35229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:39:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:29 smithi122 ceph-mon[30925]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:29 smithi122 ceph-mon[30925]: from='osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:39:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:29 smithi122 ceph-mon[30925]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T01:39:30.066 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:39:30.093 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-09-12T01:39:30.336 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi151 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:39:30.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 -- 172.21.15.183:0/3432283675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10101c00 msgr2=0x7ffa10102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 --2- 172.21.15.183:0/3432283675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10101c00 0x7ffa10102000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009900 tx=0x7ffa0002a410 comp rx=0 tx=0).stop 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 -- 172.21.15.183:0/3432283675 shutdown_connections 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 --2- 172.21.15.183:0/3432283675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa101037c0 0x7ffa1010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 --2- 172.21.15.183:0/3432283675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa10102e00 0x7ffa10103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 --2- 172.21.15.183:0/3432283675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10101c00 0x7ffa10102000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 -- 172.21.15.183:0/3432283675 >> 172.21.15.183:0/3432283675 conn(0x7ffa100fd390 msgr2=0x7ffa100ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 -- 172.21.15.183:0/3432283675 shutdown_connections 2024-09-12T01:39:30.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.782+0000 7ffa14928640 1 -- 172.21.15.183:0/3432283675 wait complete. 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.783+0000 7ffa14928640 1 Processor -- start 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.783+0000 7ffa14928640 1 -- start start 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.783+0000 7ffa14928640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa10101c00 0x7ffa10197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.783+0000 7ffa14928640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 0x7ffa10198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.783+0000 7ffa14928640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa101037c0 0x7ffa1019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.783+0000 7ffa14928640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010d890 con 0x7ffa101037c0 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa14928640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010d710 con 0x7ffa10101c00 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa14928640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010da10 con 0x7ffa10102e00 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 0x7ffa10198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa10101c00 0x7ffa10197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 0x7ffa10198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:47424/0 (socket says 172.21.15.183:47424) 2024-09-12T01:39:30.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 -- 172.21.15.183:0/3916774937 learned_addr learned my addr 172.21.15.183:0/3916774937 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0ed9d640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa101037c0 0x7ffa1019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 -- 172.21.15.183:0/3916774937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa10101c00 msgr2=0x7ffa10197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa10101c00 0x7ffa10197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 -- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa101037c0 msgr2=0x7ffa1019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa101037c0 0x7ffa1019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 -- 172.21.15.183:0/3916774937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019fb10 con 0x7ffa10102e00 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0ed9d640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa101037c0 0x7ffa1019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa0dd9b640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 0x7ffa10198420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9f0 tx=0x7ffa0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400cd80 con 0x7ffa10102e00 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04010910 con 0x7ffa10102e00 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018990 con 0x7ffa10102e00 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019fe00 con 0x7ffa10102e00 2024-09-12T01:39:30.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.784+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a0300 con 0x7ffa10102e00 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.785+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa10108ff0 con 0x7ffa10102e00 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.787+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7ffa04010ac0 con 0x7ffa10102e00 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.787+0000 7ff9ff7fe640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9f80a2ba0 0x7ff9f80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.787+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7ffa040cdd80 con 0x7ffa10102e00 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.788+0000 7ffa0e59c640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9f80a2ba0 0x7ff9f80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.788+0000 7ffa0e59c640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9f80a2ba0 0x7ff9f80a5060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005ab0 tx=0x7ffa00000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:39:30.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.790+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa0409c650 con 0x7ffa10102e00 2024-09-12T01:39:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:39:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:39:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:39:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi151 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:30.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:39:30.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:39:30.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:30.965 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:30.963+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ffa10062630 con 0x7ff9f80a2ba0 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi151 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-12T01:39:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T01:39:30.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:31.365 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1[39583]: 2024-09-12T01:39:30.966+0000 7f3262cc8640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T01:39:31.365 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:39:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1[39583]: 2024-09-12T01:39:30.972+0000 7f325aae8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:39:31.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:31.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:31 smithi183 ceph-mon[35127]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:31.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:31.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:31 smithi183 ceph-mon[35127]: from='osd.1 ' entity='osd.1' 2024-09-12T01:39:31.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:31.686 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:31 smithi151 ceph-mon[35229]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:31 smithi151 ceph-mon[35229]: from='osd.1 ' entity='osd.1' 2024-09-12T01:39:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:31 smithi122 ceph-mon[30925]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:31 smithi122 ceph-mon[30925]: from='osd.1 ' entity='osd.1' 2024-09-12T01:39:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:31.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:39:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:39:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:32.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] boot 2024-09-12T01:39:32.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:39:32.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:32.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:32 smithi122 ceph-mon[30925]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:32.836 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:39:32.836 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:39:32.836 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:32.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] boot 2024-09-12T01:39:32.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:39:32.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:32.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:32 smithi183 ceph-mon[35127]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: osd.1 [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] boot 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T01:39:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:32 smithi151 ceph-mon[35229]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T01:39:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:33 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/1749092359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]: dispatch 2024-09-12T01:39:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:33 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]: dispatch 2024-09-12T01:39:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:33 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]': finished 2024-09-12T01:39:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:33 smithi151 ceph-mon[35229]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T01:39:34.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:34.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:33 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/1357362610' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:34.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:33 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/1749092359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]: dispatch 2024-09-12T01:39:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:33 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]: dispatch 2024-09-12T01:39:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:33 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]': finished 2024-09-12T01:39:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:33 smithi122 ceph-mon[30925]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T01:39:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:33 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/1357362610' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:34.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:33 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/1749092359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]: dispatch 2024-09-12T01:39:34.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:33 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]: dispatch 2024-09-12T01:39:34.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:33 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1bc9ea7-5717-4f77-a9a1-dee0231402da"}]': finished 2024-09-12T01:39:34.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:33 smithi183 ceph-mon[35127]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T01:39:34.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:33 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/1357362610' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:34 smithi183 ceph-mon[35127]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:35.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:34 smithi122 ceph-mon[30925]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:35.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:34 smithi151 ceph-mon[35229]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:37 smithi183 ceph-mon[35127]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:37.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:37 smithi122 ceph-mon[30925]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:37.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:37 smithi151 ceph-mon[35229]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:39.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:39 smithi183 ceph-mon[35127]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:39.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:39 smithi122 ceph-mon[30925]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:39 smithi151 ceph-mon[35229]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:41 smithi183 ceph-mon[35127]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:39:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:41.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:41 smithi122 ceph-mon[30925]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:39:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:41 smithi151 ceph-mon[35229]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T01:39:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:42.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:42 smithi183 ceph-mon[35127]: Deploying daemon osd.2 on smithi183 2024-09-12T01:39:42.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:42 smithi122 ceph-mon[30925]: Deploying daemon osd.2 on smithi183 2024-09-12T01:39:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:42 smithi151 ceph-mon[35229]: Deploying daemon osd.2 on smithi183 2024-09-12T01:39:43.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:43 smithi183 ceph-mon[35127]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:43.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:43 smithi122 ceph-mon[30925]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:43 smithi151 ceph-mon[35229]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:45.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:45 smithi122 ceph-mon[30925]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:45 smithi151 ceph-mon[35229]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:45 smithi183 ceph-mon[35127]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:47.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:47 smithi183 ceph-mon[35127]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:47.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:47 smithi122 ceph-mon[30925]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:47 smithi151 ceph-mon[35229]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:48.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:48.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:48.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:49.664 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:49 smithi183 ceph-mon[35127]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:49.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:49 smithi122 ceph-mon[30925]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:49.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:49 smithi151 ceph-mon[35229]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:51.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:51.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:51.457 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.458 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:51 smithi183 ceph-mon[35127]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:51.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:51.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:51.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:51 smithi122 ceph-mon[30925]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:51.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:51.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:51.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:51.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:51 smithi151 ceph-mon[35229]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:51.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.875+0000 7ff9ff7fe640 1 -- 172.21.15.183:0/3916774937 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffa10062630 con 0x7ff9f80a2ba0 2024-09-12T01:39:51.877 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-09-12T01:39:51.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9f80a2ba0 msgr2=0x7ff9f80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9f80a2ba0 0x7ff9f80a5060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005ab0 tx=0x7ffa00000bc0 comp rx=0 tx=0).stop 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 msgr2=0x7ffa10198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 0x7ffa10198420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9f0 tx=0x7ffa0400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 shutdown_connections 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9f80a2ba0 0x7ff9f80a5060 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa101037c0 0x7ffa1019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa10102e00 0x7ffa10198420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 --2- 172.21.15.183:0/3916774937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa10101c00 0x7ffa10197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 >> 172.21.15.183:0/3916774937 conn(0x7ffa100fd390 msgr2=0x7ffa101061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 shutdown_connections 2024-09-12T01:39:51.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:39:51.878+0000 7ffa14928640 1 -- 172.21.15.183:0/3916774937 wait complete. 2024-09-12T01:39:52.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:52.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:52.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:52.503 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T01:39:52.506 INFO:tasks.cephadm:Deploying osd.3 on smithi122 with /dev/vg_nvme/lv_3... 2024-09-12T01:39:52.506 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:39:52.532 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:52.532 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:52.532 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:52.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:39:52.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:52.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:52.746 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:39:53.411 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:53 smithi122 ceph-mon[30925]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:53.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:53 smithi151 ceph-mon[35229]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:53.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:53 smithi183 ceph-mon[35127]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:53.939 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:39:53 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2[40084]: 2024-09-12T01:39:53.705+0000 7f84478a8640 -1 osd.2 0 log_to_monitors true 2024-09-12T01:39:54.494 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:54 smithi122 ceph-mon[30925]: from='osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:39:54.494 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:54 smithi122 ceph-mon[30925]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:39:54.595 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:54 smithi183 ceph-mon[35127]: from='osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:39:54.595 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:54 smithi183 ceph-mon[35127]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:39:54.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:54 smithi151 ceph-mon[35229]: from='osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:39:54.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:54 smithi151 ceph-mon[35229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T01:39:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:39:55.094 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-09-12T01:39:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:39:55.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-mon[35127]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:39:55.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-mon[35127]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:39:55.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:55.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-mon[35127]: from='osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:39:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-mon[35127]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:39:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-mon[35127]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:55.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:55 smithi122 ceph-mon[30925]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:39:55.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:55 smithi122 ceph-mon[30925]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:39:55.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:55 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:55.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:55 smithi122 ceph-mon[30925]: from='osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:39:55.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:55 smithi122 ceph-mon[30925]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:39:55.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:55 smithi122 ceph-mon[30925]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:55 smithi151 ceph-mon[35229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T01:39:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:55 smithi151 ceph-mon[35229]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T01:39:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:55 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:55 smithi151 ceph-mon[35229]: from='osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:39:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:55 smithi151 ceph-mon[35229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:39:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:55 smithi151 ceph-mon[35229]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 -- 172.21.15.122:0/2408569617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4128107920 msgr2=0x7f4128109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 --2- 172.21.15.122:0/2408569617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4128107920 0x7f4128109d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4120009ca0 tx=0x7f412002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:39:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 -- 172.21.15.122:0/2408569617 shutdown_connections 2024-09-12T01:39:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 --2- 172.21.15.122:0/2408569617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4128107920 0x7f4128109d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 --2- 172.21.15.122:0/2408569617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4128104ff0 0x7f41281073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 --2- 172.21.15.122:0/2408569617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41281026c0 0x7f4128104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.802+0000 7f412cb45640 1 -- 172.21.15.122:0/2408569617 >> 172.21.15.122:0/2408569617 conn(0x7f41280fc280 msgr2=0x7f41280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.803+0000 7f412cb45640 1 -- 172.21.15.122:0/2408569617 shutdown_connections 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.803+0000 7f412cb45640 1 -- 172.21.15.122:0/2408569617 wait complete. 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.803+0000 7f412cb45640 1 Processor -- start 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.803+0000 7f412cb45640 1 -- start start 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 0x7f4128197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4128104ff0 0x7f4128198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4128107920 0x7f412819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f412810d880 con 0x7f41281026c0 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f412810d700 con 0x7f4128104ff0 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f412810da00 con 0x7f4128107920 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 0x7f4128197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 0x7f4128197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35398/0 (socket says 172.21.15.122:35398) 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 -- 172.21.15.122:0/2651295831 learned_addr learned my addr 172.21.15.122:0/2651295831 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f4125d9b640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4128104ff0 0x7f4128198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f4126d9d640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4128107920 0x7f412819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 -- 172.21.15.122:0/2651295831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4128107920 msgr2=0x7f412819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4128107920 0x7f412819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 -- 172.21.15.122:0/2651295831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4128104ff0 msgr2=0x7f4128198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4128104ff0 0x7f4128198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 -- 172.21.15.122:0/2651295831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412819faf0 con 0x7f41281026c0 2024-09-12T01:39:55.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412659c640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 0x7f4128197ec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f411c00d950 tx=0x7f411c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:39:55.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c017070 con 0x7f41281026c0 2024-09-12T01:39:55.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412819fde0 con 0x7f41281026c0 2024-09-12T01:39:55.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411c009c30 con 0x7f41281026c0 2024-09-12T01:39:55.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.804+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c00f6b0 con 0x7f41281026c0 2024-09-12T01:39:55.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.805+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a0240 con 0x7f41281026c0 2024-09-12T01:39:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.805+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41280fdd70 con 0x7f41281026c0 2024-09-12T01:39:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.807+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f411c0109d0 con 0x7f41281026c0 2024-09-12T01:39:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.807+0000 7f41177fe640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41100a2b20 0x7f41100a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:39:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.807+0000 7f4125d9b640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41100a2b20 0x7f41100a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:39:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.807+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7f411c0cd790 con 0x7f41281026c0 2024-09-12T01:39:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.808+0000 7f4125d9b640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41100a2b20 0x7f41100a4fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41281993e0 tx=0x7f411800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:39:55.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.812+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f411c09b7f0 con 0x7f41281026c0 2024-09-12T01:39:55.952 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2[40084]: 2024-09-12T01:39:55.699+0000 7f844382b640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T01:39:55.952 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:39:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2[40084]: 2024-09-12T01:39:55.703+0000 7f843b64b640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:39:55.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:39:55.989+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4128062630 con 0x7f41100a2b20 2024-09-12T01:39:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:39:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:39:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi183 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='osd.2 ' entity='osd.2' 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi183 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:56.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='osd.2 ' entity='osd.2' 2024-09-12T01:39:56.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:56.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:56.621 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi183 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:39:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:39:56.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='osd.2 ' entity='osd.2' 2024-09-12T01:39:56.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:39:56.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:39:56.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:39:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] boot 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T01:39:57.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] boot 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T01:39:57.618 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:57.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:39:57.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: osd.2 [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] boot 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T01:39:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/209919396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a02adc8a-32be-42b2-9b14-d6f23ac36887"}]: dispatch 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/209919396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a02adc8a-32be-42b2-9b14-d6f23ac36887"}]': finished 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: pgmap v76: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:39:58.909 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:39:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:39:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/209919396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a02adc8a-32be-42b2-9b14-d6f23ac36887"}]: dispatch 2024-09-12T01:39:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/209919396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a02adc8a-32be-42b2-9b14-d6f23ac36887"}]': finished 2024-09-12T01:39:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T01:39:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:39:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: pgmap v76: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:39:59.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:39:59.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T01:39:59.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/209919396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a02adc8a-32be-42b2-9b14-d6f23ac36887"}]: dispatch 2024-09-12T01:39:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/209919396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a02adc8a-32be-42b2-9b14-d6f23ac36887"}]': finished 2024-09-12T01:39:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T01:39:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:39:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: pgmap v76: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:39:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:39:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:59 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2997296541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:39:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:39:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:59 smithi122 ceph-mon[30925]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T01:39:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:39:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:39:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:40:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:59 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2997296541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:40:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:59 smithi151 ceph-mon[35229]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T01:40:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:39:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:40:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:59 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2997296541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T01:40:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:59 smithi183 ceph-mon[35127]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T01:40:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:00.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:39:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:40:00.937 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50326]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:40:00.937 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:00.937 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50326]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:00.937 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50326]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:00.938 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:00 smithi122 ceph-mon[30925]: overall HEALTH_OK 2024-09-12T01:40:00.938 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:00 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:40:00.938 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:00 smithi122 ceph-mon[30925]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T01:40:00.938 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:00 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:00.938 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:00 smithi122 ceph-mon[30925]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:00 smithi151 ceph-mon[35229]: overall HEALTH_OK 2024-09-12T01:40:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:00 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:40:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:00 smithi151 ceph-mon[35229]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T01:40:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:00 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:00 smithi151 ceph-mon[35229]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:00 smithi183 ceph-mon[35127]: overall HEALTH_OK 2024-09-12T01:40:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:00 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:40:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:00 smithi183 ceph-mon[35127]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T01:40:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:00 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:01.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:00 smithi183 ceph-mon[35127]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:01.238 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50331]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:40:01.239 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50331]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:01.239 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50331]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:01.239 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 01:40:00 smithi122 sudo[50331]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:01.462 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43691]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:40:01.462 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:01.462 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43691]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:01.462 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43691]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:01.865 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43694]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:40:01.865 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:01.865 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43694]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:01.865 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 01:40:01 smithi151 sudo[43694]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:01.992 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:01.993 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:01 smithi183 ceph-mon[35127]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:40:01.993 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:01.993 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:01.993 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43723]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:40:01.993 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:01.993 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43723]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:01.993 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43723]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:02.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:01 smithi122 ceph-mon[30925]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:40:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:02.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:02.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:01 smithi151 ceph-mon[35229]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T01:40:02.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:02.367 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:02.452 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43726]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:40:02.452 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43726]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:02.452 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43726]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:02.453 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 01:40:01 smithi183 sudo[43726]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:02.519 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[50892]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:40:02.519 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[50892]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:02.519 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[50892]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:02.519 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[50892]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:02.894 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[51082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:40:02.894 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[51082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:02.894 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[51082]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:02.894 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:02 smithi122 sudo[51082]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:03.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:02 smithi183 sudo[43729]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:40:03.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:02 smithi183 sudo[43729]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:03.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:02 smithi183 sudo[43729]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:03.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:02 smithi183 sudo[43729]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:03.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 sudo[43732]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:40:03.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 sudo[43732]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 sudo[43732]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 sudo[43732]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:03.515 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43698]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T01:40:03.515 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43698]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:03.515 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43698]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:03.515 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43698]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:03.515 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T01:40:03.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:03.528+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f411c097e60 con 0x7f41281026c0 2024-09-12T01:40:03.809 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43701]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 sudo[43701]: pam_unix(sudo:session): session closed for user root 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:03.810 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:03.811 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:03.885 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:03.885 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:03.885 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:03.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:03.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:04.614 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:04.614 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:04.614 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:04.614 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:04.615 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:04.615 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:04.615 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:04.615 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:04 smithi122 ceph-mon[30925]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:04.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:04 smithi151 ceph-mon[35229]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T01:40:04.880 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:04 smithi183 ceph-mon[35127]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T01:40:05.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:05 smithi122 ceph-mon[30925]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:05 smithi151 ceph-mon[35229]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:05 smithi183 ceph-mon[35127]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:06.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:40:06.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:06 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:40:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:06 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T01:40:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:06 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:07.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:07 smithi122 ceph-mon[30925]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:07.793 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:07 smithi122 ceph-mon[30925]: Deploying daemon osd.3 on smithi122 2024-09-12T01:40:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:07 smithi151 ceph-mon[35229]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:07 smithi151 ceph-mon[35229]: Deploying daemon osd.3 on smithi122 2024-09-12T01:40:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:07 smithi183 ceph-mon[35127]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:07 smithi183 ceph-mon[35127]: Deploying daemon osd.3 on smithi122 2024-09-12T01:40:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:09 smithi151 ceph-mon[35229]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:40:09.898 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:09 smithi122 ceph-mon[30925]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:40:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:09 smithi183 ceph-mon[35127]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T01:40:11.684 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:11 smithi122 ceph-mon[30925]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:40:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:11 smithi151 ceph-mon[35229]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:40:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:11 smithi183 ceph-mon[35127]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T01:40:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:12.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:12.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:12.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:12.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:13.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:13 smithi122 ceph-mon[30925]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:40:13.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:13 smithi151 ceph-mon[35229]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:40:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:13 smithi183 ceph-mon[35127]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T01:40:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:15 smithi151 ceph-mon[35229]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:40:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:15 smithi122 ceph-mon[30925]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:40:15.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:15.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:15.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:15 smithi183 ceph-mon[35127]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T01:40:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:16.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.598+0000 7f41177fe640 1 -- 172.21.15.122:0/2651295831 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4128062630 con 0x7f41100a2b20 2024-09-12T01:40:16.598 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 3 on host 'smithi122' 2024-09-12T01:40:16.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.600+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41100a2b20 msgr2=0x7f41100a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:16.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.600+0000 7f412cb45640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41100a2b20 0x7f41100a4fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41281993e0 tx=0x7f411800b390 comp rx=0 tx=0).stop 2024-09-12T01:40:16.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.600+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 msgr2=0x7f4128197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.600+0000 7f412cb45640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 0x7f4128197ec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f411c00d950 tx=0x7f411c00de10 comp rx=0 tx=0).stop 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.600+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 shutdown_connections 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.600+0000 7f412cb45640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41100a2b20 0x7f41100a4fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.601+0000 7f412cb45640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4128107920 0x7f412819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.601+0000 7f412cb45640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4128104ff0 0x7f4128198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.601+0000 7f412cb45640 1 --2- 172.21.15.122:0/2651295831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41281026c0 0x7f4128197ec0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.601+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 >> 172.21.15.122:0/2651295831 conn(0x7f41280fc280 msgr2=0x7f4128102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.601+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 shutdown_connections 2024-09-12T01:40:16.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:40:16.601+0000 7f412cb45640 1 -- 172.21.15.122:0/2651295831 wait complete. 2024-09-12T01:40:17.309 DEBUG:teuthology.orchestra.run.smithi122:osd.3> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T01:40:17.312 INFO:tasks.cephadm:Deploying osd.4 on smithi151 with /dev/vg_nvme/lv_3... 2024-09-12T01:40:17.312 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:40:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:40:17.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:17 smithi151 ceph-mon[35229]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:40:17.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:17 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:17.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:17 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:17.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:17 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:17 smithi183 ceph-mon[35127]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:40:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:17 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:17 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:17 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:17 smithi122 ceph-mon[30925]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:40:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:17 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:17 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:17 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:18.989 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 01:40:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3[51916]: 2024-09-12T01:40:18.618+0000 7efcd0a42640 -1 osd.3 0 log_to_monitors true 2024-09-12T01:40:19.632 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:19 smithi151 ceph-mon[35229]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:40:19.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:19 smithi151 ceph-mon[35229]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:40:19.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:19 smithi122 ceph-mon[30925]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:40:19.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:19 smithi122 ceph-mon[30925]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:40:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:19 smithi183 ceph-mon[35127]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T01:40:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:19 smithi183 ceph-mon[35127]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T01:40:20.170 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:40:20.195 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-09-12T01:40:20.448 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.853 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:20.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:20.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:20.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:20.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:20 smithi151 ceph-mon[35229]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:40:20.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:40:20.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:40:20.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:40:20.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:20.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.934 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-mon[30925]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 -- 172.21.15.151:0/3779950592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 msgr2=0x7ff57c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 --2- 172.21.15.151:0/3779950592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c100ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff564009900 tx=0x7ff56402a410 comp rx=0 tx=0).stop 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 -- 172.21.15.151:0/3779950592 shutdown_connections 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 --2- 172.21.15.151:0/3779950592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff57c102690 0x7ff57c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 --2- 172.21.15.151:0/3779950592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 0x7ff57c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 --2- 172.21.15.151:0/3779950592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c100ed0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:20.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 -- 172.21.15.151:0/3779950592 >> 172.21.15.151:0/3779950592 conn(0x7ff57c0fc280 msgr2=0x7ff57c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 -- 172.21.15.151:0/3779950592 shutdown_connections 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 -- 172.21.15.151:0/3779950592 wait complete. 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.939+0000 7ff58080a640 1 Processor -- start 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 -- start start 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 0x7ff57c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff57c102690 0x7ff57c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10c7b0 con 0x7ff57c101cd0 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10c630 con 0x7ff57c100ad0 2024-09-12T01:40:20.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff58080a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10c930 con 0x7ff57c102690 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43228/0 (socket says 172.21.15.151:43228) 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 -- 172.21.15.151:0/1369682354 learned_addr learned my addr 172.21.15.151:0/1369682354 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff579d9b640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 0x7ff57c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57ad9d640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff57c102690 0x7ff57c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 -- 172.21.15.151:0/1369682354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff57c102690 msgr2=0x7ff57c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:20.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff57c102690 0x7ff57c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 -- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 msgr2=0x7ff57c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 0x7ff57c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 -- 172.21.15.151:0/1369682354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c19fbf0 con 0x7ff57c100ad0 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff579d9b640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 0x7ff57c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.940+0000 7ff57a59c640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c197fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff564031440 tx=0x7ff564032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.941+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56403f030 con 0x7ff57c100ad0 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.941+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff564003ce0 con 0x7ff57c100ad0 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.941+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564035070 con 0x7ff57c100ad0 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.941+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c19fe80 con 0x7ff57c100ad0 2024-09-12T01:40:20.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.941+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c1a0280 con 0x7ff57c100ad0 2024-09-12T01:40:20.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.942+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff57c1a04b0 con 0x7ff57c100ad0 2024-09-12T01:40:20.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.943+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff564042030 con 0x7ff57c100ad0 2024-09-12T01:40:20.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.944+0000 7ff5737fe640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5680a2c30 0x7ff5680a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:20.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.944+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7ff5640ee020 con 0x7ff57c100ad0 2024-09-12T01:40:20.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.944+0000 7ff579d9b640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5680a2c30 0x7ff5680a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:20.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.944+0000 7ff579d9b640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5680a2c30 0x7ff5680a50f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff57c1994e0 tx=0x7ff56c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:40:20.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:20.948+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5640bbc70 con 0x7ff57c100ad0 2024-09-12T01:40:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T01:40:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T01:40:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:40:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:20 smithi183 ceph-mon[35127]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:40:21.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:21.122+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff57c06a710 con 0x7ff5680a2c30 2024-09-12T01:40:21.238 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3[51916]: 2024-09-12T01:40:20.932+0000 7efccc9c5640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T01:40:21.239 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 01:40:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3[51916]: 2024-09-12T01:40:20.937+0000 7efcc47e5640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:40:21.789 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi122 2024-09-12T01:40:21.789 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi122 to 8543M 2024-09-12T01:40:21.789 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:21.789 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T01:40:21.789 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:21.789 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:21.790 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' 2024-09-12T01:40:21.790 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:40:21.790 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:40:21.790 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi122 2024-09-12T01:40:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi122 to 8543M 2024-09-12T01:40:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:40:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:21.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi122 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi122 to 8543M 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: from='osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685]' entity='osd.3' 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:40:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:22 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:40:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:22 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:40:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:22 smithi151 ceph-mon[35229]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:40:22.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:22 smithi151 ceph-mon[35229]: osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] boot 2024-09-12T01:40:22.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:22 smithi151 ceph-mon[35229]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:40:22.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:22 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:22 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:40:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:22 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:40:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:22 smithi183 ceph-mon[35127]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:40:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:22 smithi183 ceph-mon[35127]: osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] boot 2024-09-12T01:40:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:22 smithi183 ceph-mon[35127]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:40:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:22 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:22.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:22 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:40:22.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:22 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:40:22.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:22 smithi122 ceph-mon[30925]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:40:22.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:22 smithi122 ceph-mon[30925]: osd.3 [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] boot 2024-09-12T01:40:22.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:22 smithi122 ceph-mon[30925]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T01:40:22.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:22 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T01:40:23.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:23 smithi122 ceph-mon[30925]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:23 smithi122 ceph-mon[30925]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T01:40:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:23 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/1110713766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]: dispatch 2024-09-12T01:40:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:23 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]: dispatch 2024-09-12T01:40:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:23 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]': finished 2024-09-12T01:40:23.831 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:23 smithi151 ceph-mon[35229]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:23.832 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:23 smithi151 ceph-mon[35229]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T01:40:23.832 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:23 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/1110713766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]: dispatch 2024-09-12T01:40:23.832 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:23 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]: dispatch 2024-09-12T01:40:23.832 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:23 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]': finished 2024-09-12T01:40:23.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:23 smithi183 ceph-mon[35127]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T01:40:23.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:23 smithi183 ceph-mon[35127]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T01:40:23.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:23 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/1110713766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]: dispatch 2024-09-12T01:40:23.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:23 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]: dispatch 2024-09-12T01:40:23.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:23 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ee3483-5327-4ea0-ab19-e2614d5aec71"}]': finished 2024-09-12T01:40:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:24 smithi151 ceph-mon[35229]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T01:40:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:24 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/1223818533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:24.869 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:24 smithi183 ceph-mon[35127]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T01:40:24.869 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:24.869 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:24 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/1223818533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:24.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:24 smithi122 ceph-mon[30925]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T01:40:24.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:24.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:24 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/1223818533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:25 smithi151 ceph-mon[35229]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:25 smithi151 ceph-mon[35229]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T01:40:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:25 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:25 smithi183 ceph-mon[35127]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:25 smithi183 ceph-mon[35127]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T01:40:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:25 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:25.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:25 smithi122 ceph-mon[30925]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:25 smithi122 ceph-mon[30925]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T01:40:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:25 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:26 smithi151 ceph-mon[35229]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T01:40:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:26 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:26 smithi151 ceph-mon[35229]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:26 smithi183 ceph-mon[35127]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T01:40:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:26 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:26 smithi183 ceph-mon[35127]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:26.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:26 smithi122 ceph-mon[30925]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T01:40:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:26 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:26 smithi122 ceph-mon[30925]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:29.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:29 smithi183 ceph-mon[35127]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-12T01:40:29.737 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:29 smithi151 ceph-mon[35229]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-12T01:40:29.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:29 smithi122 ceph-mon[30925]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-12T01:40:31.573 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:31 smithi151 ceph-mon[35229]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T01:40:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:31 smithi183 ceph-mon[35127]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T01:40:31.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:31 smithi122 ceph-mon[30925]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T01:40:32.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:40:32.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:32.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:40:32.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:32.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T01:40:32.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:33.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:33 smithi122 ceph-mon[30925]: Deploying daemon osd.4 on smithi151 2024-09-12T01:40:33.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:33 smithi122 ceph-mon[30925]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:40:33.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:33 smithi151 ceph-mon[35229]: Deploying daemon osd.4 on smithi151 2024-09-12T01:40:33.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:33 smithi151 ceph-mon[35229]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:40:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:33 smithi183 ceph-mon[35127]: Deploying daemon osd.4 on smithi151 2024-09-12T01:40:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:33 smithi183 ceph-mon[35127]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T01:40:35.718 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:35 smithi151 ceph-mon[35229]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:40:35.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:35 smithi122 ceph-mon[30925]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:40:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:35 smithi183 ceph-mon[35127]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T01:40:37.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:37 smithi122 ceph-mon[30925]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-12T01:40:37.780 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:37 smithi151 ceph-mon[35229]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-12T01:40:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:37 smithi183 ceph-mon[35127]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-12T01:40:38.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:38 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:38.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:38 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:38.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:38 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:38.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:38 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:38 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:38 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:38 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:38 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:38 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:39.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:39 smithi122 ceph-mon[30925]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:40:39.798 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:39 smithi151 ceph-mon[35229]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:40:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:39 smithi183 ceph-mon[35127]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:40:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:41 smithi122 ceph-mon[30925]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:41 smithi151 ceph-mon[35229]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:41.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:41.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:41.749 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:41 smithi183 ceph-mon[35127]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:42.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.442+0000 7ff5737fe640 1 -- 172.21.15.151:0/1369682354 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff57c06a710 con 0x7ff5680a2c30 2024-09-12T01:40:42.442 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 4 on host 'smithi151' 2024-09-12T01:40:42.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5680a2c30 msgr2=0x7ff5680a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:42.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5680a2c30 0x7ff5680a50f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff57c1994e0 tx=0x7ff56c005f90 comp rx=0 tx=0).stop 2024-09-12T01:40:42.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 msgr2=0x7ff57c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:42.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c197fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff564031440 tx=0x7ff564032ce0 comp rx=0 tx=0).stop 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 shutdown_connections 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5680a2c30 0x7ff5680a50f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff57c102690 0x7ff57c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57c101cd0 0x7ff57c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 --2- 172.21.15.151:0/1369682354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c100ad0 0x7ff57c197fc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 >> 172.21.15.151:0/1369682354 conn(0x7ff57c0fc280 msgr2=0x7ff57c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 shutdown_connections 2024-09-12T01:40:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:40:42.445+0000 7ff58080a640 1 -- 172.21.15.151:0/1369682354 wait complete. 2024-09-12T01:40:42.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:42.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:42.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:40:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:43.082 DEBUG:teuthology.orchestra.run.smithi151:osd.4> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T01:40:43.084 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-09-12T01:40:43.085 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T01:40:43.339 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:40:43.541 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:43 smithi151 ceph-mon[35229]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:43.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:43 smithi183 ceph-mon[35127]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:43.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:43 smithi122 ceph-mon[30925]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:44.304 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 01:40:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T01:40:44.150+0000 7f4705dea640 -1 osd.4 0 log_to_monitors true 2024-09-12T01:40:44.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:44 smithi151 ceph-mon[35229]: from='osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:40:44.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:44 smithi151 ceph-mon[35229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:40:44.753 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:44 smithi183 ceph-mon[35127]: from='osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:40:44.753 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:44 smithi183 ceph-mon[35127]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:40:44.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:44 smithi122 ceph-mon[30925]: from='osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:40:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:44 smithi122 ceph-mon[30925]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T01:40:45.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-mon[35229]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:45.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-mon[35229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:40:45.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-mon[35229]: from='osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:40:45.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-mon[35229]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:40:45.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:45.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-mon[35229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:40:45.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:45 smithi183 ceph-mon[35127]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:45.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:45 smithi183 ceph-mon[35127]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:40:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:45 smithi183 ceph-mon[35127]: from='osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:40:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:45 smithi183 ceph-mon[35127]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:40:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:45 smithi183 ceph-mon[35127]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:40:45.945 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:40:45.971 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-09-12T01:40:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:45 smithi122 ceph-mon[30925]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:45 smithi122 ceph-mon[30925]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T01:40:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:45 smithi122 ceph-mon[30925]: from='osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:40:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:45 smithi122 ceph-mon[30925]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T01:40:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:45 smithi122 ceph-mon[30925]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:40:46.062 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T01:40:45.884+0000 7f4702581640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T01:40:46.062 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 01:40:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T01:40:45.890+0000 7f46f9b8d640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:40:46.226 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='osd.4 ' entity='osd.4' 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:46.627 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:46.628 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.628 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:46.628 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:46.628 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 -- 172.21.15.183:0/400605116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c101d10 msgr2=0x7f4a8c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 --2- 172.21.15.183:0/400605116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c101d10 0x7f4a8c102190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009900 tx=0x7f4a8002a410 comp rx=0 tx=0).stop 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 -- 172.21.15.183:0/400605116 shutdown_connections 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 --2- 172.21.15.183:0/400605116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 --2- 172.21.15.183:0/400605116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c101d10 0x7f4a8c102190 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 --2- 172.21.15.183:0/400605116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 0x7f4a8c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 -- 172.21.15.183:0/400605116 >> 172.21.15.183:0/400605116 conn(0x7f4a8c0fc2a0 msgr2=0x7f4a8c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 -- 172.21.15.183:0/400605116 shutdown_connections 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.710+0000 7f4a92874640 1 -- 172.21.15.183:0/400605116 wait complete. 2024-09-12T01:40:46.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 Processor -- start 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 -- start start 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 0x7f4a8c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c101d10 0x7f4a8c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c10c770 con 0x7f4a8c101d10 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c10c5f0 con 0x7f4a8c100b10 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a92874640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c10c8f0 con 0x7f4a8c1026d0 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:50940/0 (socket says 172.21.15.183:50940) 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 -- 172.21.15.183:0/4230826375 learned_addr learned my addr 172.21.15.183:0/4230826375 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:40:46.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a8bfff640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 0x7f4a8c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a8b7fe640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c101d10 0x7f4a8c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 -- 172.21.15.183:0/4230826375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 msgr2=0x7f4a8c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 0x7f4a8c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 -- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c101d10 msgr2=0x7f4a8c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c101d10 0x7f4a8c1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a90e11640 1 -- 172.21.15.183:0/4230826375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c103ef0 con 0x7f4a8c1026d0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a8bfff640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 0x7f4a8c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.711+0000 7f4a8b7fe640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c101d10 0x7f4a8c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.712+0000 7f4a90e11640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c103630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009fd0 tx=0x7f4a8400b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.712+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84017070 con 0x7f4a8c1026d0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.712+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c1a2180 con 0x7f4a8c1026d0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.712+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a84003e50 con 0x7f4a8c1026d0 2024-09-12T01:40:46.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.712+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840106c0 con 0x7f4a8c1026d0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.712+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c1a2640 con 0x7f4a8c1026d0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.713+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8c1a2870 con 0x7f4a8c1026d0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.714+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4a84003970 con 0x7f4a8c1026d0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.715+0000 7f4a897fa640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a740a2b80 0x7f4a740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.715+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f4a840cdcd0 con 0x7f4a8c1026d0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.715+0000 7f4a8bfff640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a740a2b80 0x7f4a740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.716+0000 7f4a8bfff640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a740a2b80 0x7f4a740a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0058c0 tx=0x7f4a7c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:40:46.730 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.719+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a8409bdc0 con 0x7f4a8c1026d0 2024-09-12T01:40:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:40:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T01:40:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='osd.4 ' entity='osd.4' 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:46.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.897 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:40:46.897+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4a8c06af30 con 0x7f4a740a2b80 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='osd.4 ' entity='osd.4' 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:40:46.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:46.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:46.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:40:46.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi151 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] boot 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:47.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:40:47.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:40:47.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:40:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:40:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi151 2024-09-12T01:40:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-12T01:40:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] boot 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:40:47.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:47.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi151 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: osd.4 [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] boot 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:40:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:48 smithi183 ceph-mon[35127]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:40:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:48 smithi183 ceph-mon[35127]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T01:40:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:48 smithi151 ceph-mon[35229]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:40:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:48 smithi151 ceph-mon[35229]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T01:40:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:48 smithi122 ceph-mon[30925]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:40:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:48 smithi122 ceph-mon[30925]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T01:40:49.826 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:49 smithi183 ceph-mon[35127]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:49.826 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:49 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/2116176246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]: dispatch 2024-09-12T01:40:49.826 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:49 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]: dispatch 2024-09-12T01:40:49.826 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:49 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]': finished 2024-09-12T01:40:49.826 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:49 smithi183 ceph-mon[35127]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T01:40:49.826 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:40:49.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:49 smithi151 ceph-mon[35229]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:49.952 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:49 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/2116176246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]: dispatch 2024-09-12T01:40:49.952 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:49 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]: dispatch 2024-09-12T01:40:49.953 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:49 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]': finished 2024-09-12T01:40:49.953 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:49 smithi151 ceph-mon[35229]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T01:40:49.953 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:40:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:49 smithi122 ceph-mon[30925]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:49 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/2116176246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]: dispatch 2024-09-12T01:40:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:49 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]: dispatch 2024-09-12T01:40:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:49 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f02377d1-5979-48eb-945c-65d7675d8283"}]': finished 2024-09-12T01:40:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:49 smithi122 ceph-mon[30925]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T01:40:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:40:50.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:50 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/2370086013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:50 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/2370086013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:50.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:50 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/2370086013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:40:51.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:51 smithi151 ceph-mon[35229]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:51.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:51 smithi122 ceph-mon[30925]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:51 smithi183 ceph-mon[35127]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:53.777 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:53 smithi183 ceph-mon[35127]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:53.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:53 smithi151 ceph-mon[35229]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:53.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:53 smithi122 ceph-mon[30925]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:55.861 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:55 smithi183 ceph-mon[35127]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:55.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:55 smithi151 ceph-mon[35229]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:55.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:55 smithi122 ceph-mon[30925]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:57 smithi151 ceph-mon[35229]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:57.892 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:57 smithi183 ceph-mon[35127]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:57.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:57 smithi122 ceph-mon[30925]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:58.632 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:40:58.632 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:40:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:58.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T01:40:58.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:40:59.805 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:59 smithi183 ceph-mon[35127]: Deploying daemon osd.5 on smithi183 2024-09-12T01:40:59.805 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:40:59 smithi183 ceph-mon[35127]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:59 smithi151 ceph-mon[35229]: Deploying daemon osd.5 on smithi183 2024-09-12T01:40:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:40:59 smithi151 ceph-mon[35229]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:40:59.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:59 smithi122 ceph-mon[30925]: Deploying daemon osd.5 on smithi183 2024-09-12T01:40:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:40:59 smithi122 ceph-mon[30925]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:01 smithi151 ceph-mon[35229]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:01 smithi183 ceph-mon[35127]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:01.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:01 smithi122 ceph-mon[30925]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:03.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:03 smithi183 ceph-mon[35127]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:03 smithi151 ceph-mon[35229]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:03 smithi122 ceph-mon[30925]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:04.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:04.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:04.806 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:04.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:05.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:05 smithi183 ceph-mon[35127]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:05 smithi151 ceph-mon[35229]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:05 smithi122 ceph-mon[30925]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:07.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:07 smithi183 ceph-mon[35127]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:07.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:07.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:07.795 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:07 smithi151 ceph-mon[35229]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:07 smithi122 ceph-mon[30925]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:08.136 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-09-12T01:41:08.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.135+0000 7f4a897fa640 1 -- 172.21.15.183:0/4230826375 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a8c06af30 con 0x7f4a740a2b80 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a740a2b80 msgr2=0x7f4a740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a740a2b80 0x7f4a740a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0058c0 tx=0x7f4a7c005850 comp rx=0 tx=0).stop 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 msgr2=0x7f4a8c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c103630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009fd0 tx=0x7f4a8400b260 comp rx=0 tx=0).stop 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 shutdown_connections 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a740a2b80 0x7f4a740a5040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a8c1026d0 0x7f4a8c103630 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a8c101d10 0x7f4a8c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:08.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 --2- 172.21.15.183:0/4230826375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a8c100b10 0x7f4a8c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:08.139 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 >> 172.21.15.183:0/4230826375 conn(0x7f4a8c0fc2a0 msgr2=0x7f4a8c06a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:41:08.139 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 shutdown_connections 2024-09-12T01:41:08.139 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:41:08.138+0000 7f4a92874640 1 -- 172.21.15.183:0/4230826375 wait complete. 2024-09-12T01:41:08.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:08 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:08.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:08 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:08.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:08 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:08.827 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T01:41:08.829 INFO:tasks.cephadm:Deploying osd.6 on smithi122 with /dev/vg_nvme/lv_2... 2024-09-12T01:41:08.829 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:41:08.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:08.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:08.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:08 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:08 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:08 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:41:09.839 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:09 smithi122 ceph-mon[30925]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:09 smithi151 ceph-mon[35229]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:09 smithi183 ceph-mon[35127]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:10.342 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 01:41:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T01:41:10.151+0000 7f25c3aad640 -1 osd.5 0 log_to_monitors true 2024-09-12T01:41:10.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:10 smithi183 ceph-mon[35127]: from='osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:41:10.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:10 smithi183 ceph-mon[35127]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:41:10.781 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:10 smithi122 ceph-mon[30925]: from='osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:41:10.781 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:10 smithi122 ceph-mon[30925]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:41:10.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:10 smithi151 ceph-mon[35229]: from='osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:41:10.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:10 smithi151 ceph-mon[35229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T01:41:11.459 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:41:11.487 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-09-12T01:41:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-mon[35127]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-mon[35127]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:41:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-mon[35127]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:41:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-mon[35127]: from='osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:41:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-mon[35127]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:41:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:11 smithi122 ceph-mon[30925]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:11 smithi122 ceph-mon[30925]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:41:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:11 smithi122 ceph-mon[30925]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:41:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:11 smithi122 ceph-mon[30925]: from='osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:41:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:11 smithi122 ceph-mon[30925]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:41:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:41:11.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:11 smithi151 ceph-mon[35229]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:11.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:11 smithi151 ceph-mon[35229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T01:41:11.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:11 smithi151 ceph-mon[35229]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T01:41:11.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:11.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:11 smithi151 ceph-mon[35229]: from='osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:41:11.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:11 smithi151 ceph-mon[35229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:41:12.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 -- 172.21.15.122:0/3623584882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 msgr2=0x7f5678102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:12.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 --2- 172.21.15.122:0/3623584882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678102150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f566c009900 tx=0x7f566c02a410 comp rx=0 tx=0).stop 2024-09-12T01:41:12.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 -- 172.21.15.122:0/3623584882 shutdown_connections 2024-09-12T01:41:12.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 --2- 172.21.15.122:0/3623584882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5678102690 0x7f5678108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 --2- 172.21.15.122:0/3623584882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678102150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 --2- 172.21.15.122:0/3623584882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678100ad0 0x7f5678100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 -- 172.21.15.122:0/3623584882 >> 172.21.15.122:0/3623584882 conn(0x7f56780fc280 msgr2=0x7f56780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.246+0000 7f567d65a640 1 -- 172.21.15.122:0/3623584882 shutdown_connections 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 -- 172.21.15.122:0/3623584882 wait complete. 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 Processor -- start 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 -- start start 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5678100ad0 0x7f5678196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102690 0x7f567819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f567810c7a0 con 0x7f5678101cd0 2024-09-12T01:41:12.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f567810c620 con 0x7f5678100ad0 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.247+0000 7f567d65a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f567810c920 con 0x7f5678102690 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56777fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102690 0x7f567819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59986/0 (socket says 172.21.15.122:59986) 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 -- 172.21.15.122:0/3386180602 learned_addr learned my addr 172.21.15.122:0/3386180602 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f5676ffd640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5678100ad0 0x7f5678196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 -- 172.21.15.122:0/3386180602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102690 msgr2=0x7f567819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102690 0x7f567819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:12.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 -- 172.21.15.122:0/3386180602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5678100ad0 msgr2=0x7f5678196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5678100ad0 0x7f5678196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 -- 172.21.15.122:0/3386180602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567819e9f0 con 0x7f5678101cd0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56777fe640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102690 0x7f567819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f56767fc640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678197300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f566c031440 tx=0x7f566c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c03f030 con 0x7f5678101cd0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566c003ce0 con 0x7f5678101cd0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c035070 con 0x7f5678101cd0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567819ec80 con 0x7f5678101cd0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.248+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567819f160 con 0x7f5678101cd0 2024-09-12T01:41:12.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.250+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678107f20 con 0x7f5678101cd0 2024-09-12T01:41:12.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.251+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f566c042030 con 0x7f5678101cd0 2024-09-12T01:41:12.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.251+0000 7f565ffff640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56600a2bb0 0x7f56600a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:12.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.252+0000 7f5676ffd640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56600a2bb0 0x7f56600a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:12.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.252+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f566c0ee540 con 0x7f5678101cd0 2024-09-12T01:41:12.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.252+0000 7f5676ffd640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56600a2bb0 0x7f56600a5070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5678101b30 tx=0x7f5668005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:41:12.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.256+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f566c0bbe30 con 0x7f5678101cd0 2024-09-12T01:41:12.452 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T01:41:11.973+0000 7f25bfa30640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T01:41:12.452 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 01:41:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T01:41:11.979+0000 7f25b7850640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:41:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:12.457+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5678062630 con 0x7f56600a2bb0 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='osd.5 ' entity='osd.5' 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:41:12.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:41:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T01:41:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='osd.5 ' entity='osd.5' 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:41:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:12.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:41:12.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='osd.5 ' entity='osd.5' 2024-09-12T01:41:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:41:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:41:12.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi183 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] boot 2024-09-12T01:41:13.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T01:41:13.862 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:41:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:41:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi183 2024-09-12T01:41:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-12T01:41:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:13.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:41:13.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:13.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] boot 2024-09-12T01:41:13.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T01:41:13.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi183 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: osd.5 [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] boot 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T01:41:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T01:41:14.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:14 smithi122 ceph-mon[30925]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T01:41:14.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:14 smithi151 ceph-mon[35229]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T01:41:14.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:14 smithi183 ceph-mon[35127]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1291540037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4782f3-b6a1-45f7-83b4-e109d523dffc"}]: dispatch 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1291540037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c4782f3-b6a1-45f7-83b4-e109d523dffc"}]': finished 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:15 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2963832091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:41:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T01:41:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1291540037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4782f3-b6a1-45f7-83b4-e109d523dffc"}]: dispatch 2024-09-12T01:41:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1291540037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c4782f3-b6a1-45f7-83b4-e109d523dffc"}]': finished 2024-09-12T01:41:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T01:41:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:15 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2963832091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:41:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T01:41:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1291540037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c4782f3-b6a1-45f7-83b4-e109d523dffc"}]: dispatch 2024-09-12T01:41:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1291540037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c4782f3-b6a1-45f7-83b4-e109d523dffc"}]': finished 2024-09-12T01:41:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T01:41:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:15.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:15 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2963832091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:41:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:16 smithi151 ceph-mon[35229]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T01:41:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:16 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:16 smithi151 ceph-mon[35229]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:16 smithi183 ceph-mon[35127]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T01:41:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:16 smithi183 ceph-mon[35127]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:17.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:16 smithi122 ceph-mon[30925]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T01:41:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:16 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:16 smithi122 ceph-mon[30925]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:19.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:19 smithi122 ceph-mon[30925]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-12T01:41:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:19 smithi151 ceph-mon[35229]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-12T01:41:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:19 smithi183 ceph-mon[35127]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-12T01:41:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:21 smithi122 ceph-mon[30925]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:41:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:21 smithi151 ceph-mon[35229]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:41:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:21 smithi183 ceph-mon[35127]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:41:23.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:23 smithi122 ceph-mon[30925]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T01:41:23.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:41:23.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:23.755 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:23 smithi183 ceph-mon[35127]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T01:41:23.755 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:41:23.755 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:23 smithi151 ceph-mon[35229]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T01:41:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T01:41:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:24.585 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:24 smithi122 ceph-mon[30925]: Deploying daemon osd.6 on smithi122 2024-09-12T01:41:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:24 smithi151 ceph-mon[35229]: Deploying daemon osd.6 on smithi122 2024-09-12T01:41:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:24 smithi183 ceph-mon[35127]: Deploying daemon osd.6 on smithi122 2024-09-12T01:41:25.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:25 smithi122 ceph-mon[30925]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T01:41:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:25 smithi151 ceph-mon[35229]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T01:41:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:25 smithi183 ceph-mon[35127]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T01:41:27.731 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:27 smithi122 ceph-mon[30925]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:41:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:27 smithi151 ceph-mon[35229]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:41:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:27 smithi183 ceph-mon[35127]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:41:29.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:29 smithi122 ceph-mon[30925]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:41:29.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:29 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:29.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:29 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:29.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:29 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:29 smithi151 ceph-mon[35229]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:41:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:29 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:29 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:29 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:29 smithi183 ceph-mon[35127]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:41:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:29 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:29 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:29.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:29 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:31.493 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:31 smithi122 ceph-mon[30925]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:31 smithi151 ceph-mon[35229]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:31 smithi183 ceph-mon[35127]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:32.459 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 6 on host 'smithi122' 2024-09-12T01:41:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.459+0000 7f565ffff640 1 -- 172.21.15.122:0/3386180602 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5678062630 con 0x7f56600a2bb0 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56600a2bb0 msgr2=0x7f56600a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56600a2bb0 0x7f56600a5070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5678101b30 tx=0x7f5668005f90 comp rx=0 tx=0).stop 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 msgr2=0x7f5678197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678197300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f566c031440 tx=0x7f566c032ce0 comp rx=0 tx=0).stop 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 shutdown_connections 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56600a2bb0 0x7f56600a5070 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102690 0x7f567819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678101cd0 0x7f5678197300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 --2- 172.21.15.122:0/3386180602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5678100ad0 0x7f5678196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 >> 172.21.15.122:0/3386180602 conn(0x7f56780fc280 msgr2=0x7f56781050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 shutdown_connections 2024-09-12T01:41:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:41:32.462+0000 7f567d65a640 1 -- 172.21.15.122:0/3386180602 wait complete. 2024-09-12T01:41:33.132 DEBUG:teuthology.orchestra.run.smithi122:osd.6> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T01:41:33.135 INFO:tasks.cephadm:Deploying osd.7 on smithi151 with /dev/vg_nvme/lv_2... 2024-09-12T01:41:33.135 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:41:33.395 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:41:33.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:33 smithi122 ceph-mon[30925]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:33.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.658 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:33 smithi151 ceph-mon[35229]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:33.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:33 smithi183 ceph-mon[35127]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:35.588 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:35 smithi151 ceph-mon[35229]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:35.588 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:35 smithi151 ceph-mon[35229]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:41:35.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:35 smithi122 ceph-mon[30925]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:35.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:35 smithi122 ceph-mon[30925]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:41:35.720 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 01:41:35 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T01:41:35.325+0000 7f3532dda640 -1 osd.6 0 log_to_monitors true 2024-09-12T01:41:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:35 smithi183 ceph-mon[35127]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:35 smithi183 ceph-mon[35127]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T01:41:36.197 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:41:36.223 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-09-12T01:41:36.475 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:41:36.739 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T01:41:36.465+0000 7f352ed5d640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T01:41:36.739 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T01:41:36.477+0000 7f352ab85640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:41:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:36.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:36.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:36.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:36.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:36.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:36 smithi122 ceph-mon[30925]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:41:36.758 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:41:36.758 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T01:41:36.758 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:41:36.758 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:36.758 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.758 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.759 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:36 smithi151 ceph-mon[35229]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:41:36.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:41:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:36 smithi183 ceph-mon[35127]: from='osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:41:36.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 -- 172.21.15.151:0/3287304040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4101720 msgr2=0x7f06a4108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:36.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/3287304040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4101720 0x7f06a4108fc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f069c009ca0 tx=0x7f069c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:41:36.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 -- 172.21.15.151:0/3287304040 shutdown_connections 2024-09-12T01:41:36.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/3287304040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4101720 0x7f06a4108fc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/3287304040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06a4100d60 0x7f06a41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/3287304040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4102190 0x7f06a4102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 -- 172.21.15.151:0/3287304040 >> 172.21.15.151:0/3287304040 conn(0x7f06a40fba40 msgr2=0x7f06a40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 -- 172.21.15.151:0/3287304040 shutdown_connections 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.963+0000 7f06aa2e3640 1 -- 172.21.15.151:0/3287304040 wait complete. 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 Processor -- start 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 -- start start 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06a4100d60 0x7f06a410b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4101720 0x7f06a4109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 0x7f06a410a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:36.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06a410d830 con 0x7f06a4100d60 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06a410d6b0 con 0x7f06a4102190 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06aa2e3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06a410d9b0 con 0x7f06a4101720 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06a8880640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 0x7f06a410a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06a8880640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 0x7f06a410a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57328/0 (socket says 172.21.15.151:57328) 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06a8880640 1 -- 172.21.15.151:0/493857952 learned_addr learned my addr 172.21.15.151:0/493857952 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.964+0000 7f06a3fff640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06a4100d60 0x7f06a410b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a37fe640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4101720 0x7f06a4109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a8880640 1 -- 172.21.15.151:0/493857952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4101720 msgr2=0x7f06a4109bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a8880640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4101720 0x7f06a4109bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a8880640 1 -- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06a4100d60 msgr2=0x7f06a410b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a8880640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06a4100d60 0x7f06a410b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a8880640 1 -- 172.21.15.151:0/493857952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a410a9f0 con 0x7f06a4102190 2024-09-12T01:41:36.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a37fe640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4101720 0x7f06a4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a8880640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 0x7f06a410a130 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f069c006010 tx=0x7f069c031ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c02fb20 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a41a1f90 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069c02fcc0 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c046870 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.965+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41a2440 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.966+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a4107f60 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.967+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f069c030bc0 con 0x7f06a4102190 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.968+0000 7f06a17fa640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f068c0a2b80 0x7f068c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:41:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.968+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7f069c0efa20 con 0x7f06a4102190 2024-09-12T01:41:36.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.968+0000 7f06a3fff640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f068c0a2b80 0x7f068c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:41:36.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.968+0000 7f06a3fff640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f068c0a2b80 0x7f068c0a5040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0694005d00 tx=0x7f069400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:41:36.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:36.971+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f069c04b060 con 0x7f06a4102190 2024-09-12T01:41:37.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:37.146+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f06a4062630 con 0x7f068c0a2b80 2024-09-12T01:41:37.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi122 2024-09-12T01:41:37.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi122 to 5695M 2024-09-12T01:41:37.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:37.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T01:41:37.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:37.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:37.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:41:37.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:41:37.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:37 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:37.800 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi122 2024-09-12T01:41:37.800 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi122 to 5695M 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:41:37.801 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:37 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi122 2024-09-12T01:41:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi122 to 5695M 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:41:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:37 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:38.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:38 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:41:38.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:38 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:41:38.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:38 smithi151 ceph-mon[35229]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:41:38.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:38 smithi151 ceph-mon[35229]: osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] boot 2024-09-12T01:41:38.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:38 smithi151 ceph-mon[35229]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T01:41:38.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:38 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:38 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:41:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:38 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:41:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:38 smithi122 ceph-mon[30925]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:41:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:38 smithi122 ceph-mon[30925]: osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] boot 2024-09-12T01:41:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:38 smithi122 ceph-mon[30925]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T01:41:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:38 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:38 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:41:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:38 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:41:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:38 smithi183 ceph-mon[35127]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:41:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:38 smithi183 ceph-mon[35127]: osd.6 [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] boot 2024-09-12T01:41:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:38 smithi183 ceph-mon[35127]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T01:41:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:38 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T01:41:39.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:39 smithi122 ceph-mon[30925]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:39.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:39 smithi122 ceph-mon[30925]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T01:41:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:39 smithi151 ceph-mon[35229]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:39 smithi151 ceph-mon[35229]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T01:41:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:39 smithi183 ceph-mon[35127]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T01:41:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:39 smithi183 ceph-mon[35127]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T01:41:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:40 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/899656273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]: dispatch 2024-09-12T01:41:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:40 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]: dispatch 2024-09-12T01:41:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:40 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]': finished 2024-09-12T01:41:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:40 smithi151 ceph-mon[35229]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T01:41:40.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:41:40.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:40 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/2543423971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:41:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:40 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/899656273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]: dispatch 2024-09-12T01:41:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:40 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]: dispatch 2024-09-12T01:41:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:40 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]': finished 2024-09-12T01:41:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:40 smithi122 ceph-mon[30925]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T01:41:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:41:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:40 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/2543423971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:41:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:40 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/899656273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]: dispatch 2024-09-12T01:41:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:40 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]: dispatch 2024-09-12T01:41:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:40 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "142f0be6-d6a8-49d4-a4a1-462d5aece8d1"}]': finished 2024-09-12T01:41:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:40 smithi183 ceph-mon[35127]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T01:41:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:41:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:40 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/2543423971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:41:41.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:41 smithi122 ceph-mon[30925]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:41 smithi151 ceph-mon[35229]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:41 smithi183 ceph-mon[35127]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:43.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:43 smithi122 ceph-mon[30925]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:43.751 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:43 smithi151 ceph-mon[35229]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:43 smithi183 ceph-mon[35127]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:45.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:45 smithi122 ceph-mon[30925]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:45.750 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:45 smithi151 ceph-mon[35229]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:45 smithi183 ceph-mon[35127]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:47.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:47 smithi122 ceph-mon[30925]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:47.796 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:47 smithi151 ceph-mon[35229]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:47 smithi183 ceph-mon[35127]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:48.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:41:48.499 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:48.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:41:48.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T01:41:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:49.595 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:49 smithi151 ceph-mon[35229]: Deploying daemon osd.7 on smithi151 2024-09-12T01:41:49.595 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:49 smithi151 ceph-mon[35229]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:49 smithi183 ceph-mon[35127]: Deploying daemon osd.7 on smithi151 2024-09-12T01:41:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:49 smithi183 ceph-mon[35127]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:49.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:49 smithi122 ceph-mon[30925]: Deploying daemon osd.7 on smithi151 2024-09-12T01:41:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:49 smithi122 ceph-mon[30925]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:51.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:51 smithi151 ceph-mon[35229]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:51.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:51 smithi122 ceph-mon[30925]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:51 smithi183 ceph-mon[35127]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:53.754 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:53 smithi183 ceph-mon[35127]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:53.814 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:53 smithi151 ceph-mon[35229]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:53 smithi122 ceph-mon[30925]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:54.608 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:54.608 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:54.608 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:54.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:54.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:54.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:55.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:55 smithi151 ceph-mon[35229]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:55 smithi183 ceph-mon[35127]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:55 smithi122 ceph-mon[30925]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:57 smithi151 ceph-mon[35229]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:57.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:57 smithi183 ceph-mon[35127]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:57 smithi122 ceph-mon[30925]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:41:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:41:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:58.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.054+0000 7f06a17fa640 1 -- 172.21.15.151:0/493857952 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f06a4062630 con 0x7f068c0a2b80 2024-09-12T01:41:58.054 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 7 on host 'smithi151' 2024-09-12T01:41:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f068c0a2b80 msgr2=0x7f068c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f068c0a2b80 0x7f068c0a5040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0694005d00 tx=0x7f069400b410 comp rx=0 tx=0).stop 2024-09-12T01:41:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 msgr2=0x7f06a410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:41:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 0x7f06a410a130 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f069c006010 tx=0x7f069c031ec0 comp rx=0 tx=0).stop 2024-09-12T01:41:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 shutdown_connections 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f068c0a2b80 0x7f068c0a5040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a4102190 0x7f06a410a130 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06a4101720 0x7f06a4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 --2- 172.21.15.151:0/493857952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06a4100d60 0x7f06a410b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 >> 172.21.15.151:0/493857952 conn(0x7f06a40fba40 msgr2=0x7f06a4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 shutdown_connections 2024-09-12T01:41:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:41:58.057+0000 7f06aa2e3640 1 -- 172.21.15.151:0/493857952 wait complete. 2024-09-12T01:41:58.700 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:58.700 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:58.700 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:58.729 DEBUG:teuthology.orchestra.run.smithi151:osd.7> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T01:41:58.732 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-09-12T01:41:58.732 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T01:41:58.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:58.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:58.762 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:58.984 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:41:58.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:41:58.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:58.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:41:59.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:41:59 smithi183 ceph-mon[35127]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:41:59 smithi151 ceph-mon[35229]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:41:59.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:41:59 smithi122 ceph-mon[30925]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:00.267 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 01:41:59 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T01:41:59.917+0000 7fe6f02d3640 -1 osd.7 0 log_to_monitors true 2024-09-12T01:42:00.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:00 smithi151 ceph-mon[35229]: from='osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:42:00.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:00 smithi151 ceph-mon[35229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:42:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:00 smithi183 ceph-mon[35127]: from='osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:42:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:00 smithi183 ceph-mon[35127]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:42:00.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:00 smithi122 ceph-mon[30925]: from='osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:42:00.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:00 smithi122 ceph-mon[30925]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T01:42:01.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T01:42:01.530+0000 7fe6ec256640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T01:42:01.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T01:42:01.540+0000 7fe6e807e640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:42:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-mon[35229]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:01.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-mon[35229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:42:01.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-mon[35229]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T01:42:01.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:01.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-mon[35229]: from='osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:42:01.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:01 smithi151 ceph-mon[35229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:42:01.649 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:01 smithi183 ceph-mon[35127]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:01.650 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:01 smithi183 ceph-mon[35127]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:42:01.650 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:01 smithi183 ceph-mon[35127]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T01:42:01.650 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:01.650 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:01 smithi183 ceph-mon[35127]: from='osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:42:01.650 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:01 smithi183 ceph-mon[35127]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:42:01.653 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:42:01.682 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-09-12T01:42:01.973 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:42:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:01 smithi122 ceph-mon[30925]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:01 smithi122 ceph-mon[30925]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T01:42:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:01 smithi122 ceph-mon[30925]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T01:42:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:01 smithi122 ceph-mon[30925]: from='osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:42:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:01 smithi122 ceph-mon[30925]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:42:02.478 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/3853888495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48107920 msgr2=0x7f1d48109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:02.478 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/3853888495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48107920 0x7f1d48109d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009ca0 tx=0x7f1d4002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/3853888495 shutdown_connections 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/3853888495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48107920 0x7f1d48109d10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/3853888495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d48104ff0 0x7f1d481073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/3853888495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d481026c0 0x7f1d48104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/3853888495 >> 172.21.15.183:0/3853888495 conn(0x7f1d480fc280 msgr2=0x7f1d480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/3853888495 shutdown_connections 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/3853888495 wait complete. 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 Processor -- start 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.460+0000 7f1d4ec0f640 1 -- start start 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d481026c0 0x7f1d48197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 0x7f1d48198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:02.479 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d48107920 0x7f1d4819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810d860 con 0x7f1d48107920 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810d6e0 con 0x7f1d481026c0 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4810d9e0 con 0x7f1d48104ff0 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 0x7f1d48198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 0x7f1d48198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:35138/0 (socket says 172.21.15.183:35138) 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 -- 172.21.15.183:0/1982909356 learned_addr learned my addr 172.21.15.183:0/1982909356 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4c9ab640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d481026c0 0x7f1d48197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4d1ac640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d48107920 0x7f1d4819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 -- 172.21.15.183:0/1982909356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d481026c0 msgr2=0x7f1d48197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d481026c0 0x7f1d48197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:02.480 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 -- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d48107920 msgr2=0x7f1d4819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d48107920 0x7f1d4819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 -- 172.21.15.183:0/1982909356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4819faf0 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4d1ac640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d48107920 0x7f1d4819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3ffff640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 0x7f1d48198400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e9c0 tx=0x7f1d3800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800cd80 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d38005500 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4819fde0 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.461+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38010690 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.462+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d481a0290 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.463+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d480fdd70 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.464+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1d380046c0 con 0x7f1d48104ff0 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.464+0000 7f1d3dffb640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d300a2bb0 0x7f1d300a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.464+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f1d380ce7a0 con 0x7f1d48104ff0 2024-09-12T01:42:02.482 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.465+0000 7f1d4c9ab640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d300a2bb0 0x7f1d300a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:02.482 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.465+0000 7f1d4c9ab640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d300a2bb0 0x7f1d300a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400bbe0 tx=0x7f1d4400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:42:02.482 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.469+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d3809c380 con 0x7f1d48104ff0 2024-09-12T01:42:02.647 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:02.647+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1d48062630 con 0x7f1d300a2bb0 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.648 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:02.649 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:02.649 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:42:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:02.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:02.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:02.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:03.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:42:03.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:42:03.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi151 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] boot 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:03 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:42:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:42:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi151 2024-09-12T01:42:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-12T01:42:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:03.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] boot 2024-09-12T01:42:03.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T01:42:03.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:03.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:03.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:03.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:03 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi151 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: osd.7 [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] boot 2024-09-12T01:42:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T01:42:03.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T01:42:03.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:03.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:03.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:03 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:04.655 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:04 smithi183 ceph-mon[35127]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:04.655 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:04 smithi183 ceph-mon[35127]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T01:42:04.655 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:04 smithi183 ceph-mon[35127]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T01:42:04.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:04 smithi151 ceph-mon[35229]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:04.854 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:04 smithi151 ceph-mon[35229]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T01:42:04.855 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:04 smithi151 ceph-mon[35229]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T01:42:04.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:04 smithi122 ceph-mon[30925]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:04 smithi122 ceph-mon[30925]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T01:42:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:04 smithi122 ceph-mon[30925]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T01:42:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T01:42:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/94452243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]: dispatch 2024-09-12T01:42:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]: dispatch 2024-09-12T01:42:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]': finished 2024-09-12T01:42:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T01:42:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T01:42:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/94452243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]: dispatch 2024-09-12T01:42:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]: dispatch 2024-09-12T01:42:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]': finished 2024-09-12T01:42:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T01:42:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/94452243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]: dispatch 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]: dispatch 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0da6dee8-5cbe-43a4-8be6-b4c58619d9a5"}]': finished 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T01:42:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:06 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/1516658441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:06 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/1516658441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:06.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:06 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/1516658441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:07 smithi151 ceph-mon[35229]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:07 smithi183 ceph-mon[35127]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:07.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:07 smithi122 ceph-mon[30925]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:09 smithi151 ceph-mon[35229]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:09.938 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:09 smithi183 ceph-mon[35127]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:09.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:09 smithi122 ceph-mon[30925]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:11 smithi183 ceph-mon[35127]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:11 smithi151 ceph-mon[35229]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:11.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:11 smithi122 ceph-mon[30925]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:13.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:13 smithi183 ceph-mon[35127]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:13 smithi151 ceph-mon[35229]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:13.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:13 smithi122 ceph-mon[30925]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:14.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:42:14.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:14.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:42:14.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:14.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T01:42:14.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:15.663 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:15 smithi183 ceph-mon[35127]: Deploying daemon osd.8 on smithi183 2024-09-12T01:42:15.664 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:15 smithi183 ceph-mon[35127]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:15 smithi151 ceph-mon[35229]: Deploying daemon osd.8 on smithi183 2024-09-12T01:42:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:15 smithi151 ceph-mon[35229]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:15.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:15 smithi122 ceph-mon[30925]: Deploying daemon osd.8 on smithi183 2024-09-12T01:42:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:15 smithi122 ceph-mon[30925]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:17.621 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:17 smithi183 ceph-mon[35127]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:17 smithi151 ceph-mon[35229]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:17.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:17 smithi122 ceph-mon[30925]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:19.843 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:19 smithi183 ceph-mon[35127]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:19 smithi151 ceph-mon[35229]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:19 smithi122 ceph-mon[30925]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:20.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:20.849 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:20.849 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:20.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:20.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:20.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:21.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:21 smithi183 ceph-mon[35127]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:21 smithi151 ceph-mon[35229]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:21.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:21 smithi122 ceph-mon[30925]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:23.846 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:23 smithi183 ceph-mon[35127]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:23.846 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.846 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.846 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:23.847 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:23.847 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:23 smithi151 ceph-mon[35229]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.884 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.884+0000 7f1d3dffb640 1 -- 172.21.15.183:0/1982909356 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d48062630 con 0x7f1d300a2bb0 2024-09-12T01:42:23.884 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d300a2bb0 msgr2=0x7f1d300a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d300a2bb0 0x7f1d300a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400bbe0 tx=0x7f1d4400b390 comp rx=0 tx=0).stop 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 msgr2=0x7f1d48198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 0x7f1d48198400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e9c0 tx=0x7f1d3800ee80 comp rx=0 tx=0).stop 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 shutdown_connections 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d300a2bb0 0x7f1d300a5070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d48107920 0x7f1d4819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48104ff0 0x7f1d48198400 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:23.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 --2- 172.21.15.183:0/1982909356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d481026c0 0x7f1d48197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:23.888 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 >> 172.21.15.183:0/1982909356 conn(0x7f1d480fc280 msgr2=0x7f1d48102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:42:23.888 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 shutdown_connections 2024-09-12T01:42:23.888 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:42:23.887+0000 7f1d4ec0f640 1 -- 172.21.15.183:0/1982909356 wait complete. 2024-09-12T01:42:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:23 smithi122 ceph-mon[30925]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:24.586 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T01:42:24.588 INFO:tasks.cephadm:Deploying osd.9 on smithi122 with /dev/vg_nvme/lv_1... 2024-09-12T01:42:24.588 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:42:24.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:24.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:24.837 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:24.850 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:42:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:24.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:24.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:24.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:25.757 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:25 smithi183 ceph-mon[35127]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:25 smithi151 ceph-mon[35229]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:25.982 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:25 smithi122 ceph-mon[30925]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:26.453 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 01:42:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T01:42:26.164+0000 7f0b75aa0640 -1 osd.8 0 log_to_monitors true 2024-09-12T01:42:26.820 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:26 smithi183 ceph-mon[35127]: from='osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:42:26.821 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:26 smithi183 ceph-mon[35127]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:42:26.852 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:26 smithi122 ceph-mon[30925]: from='osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:42:26.852 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:26 smithi122 ceph-mon[30925]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:42:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:26 smithi151 ceph-mon[35229]: from='osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:42:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:26 smithi151 ceph-mon[35229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T01:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:42:27.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-09-12T01:42:27.849 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:42:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:27 smithi151 ceph-mon[35229]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:27 smithi151 ceph-mon[35229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:42:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:27 smithi151 ceph-mon[35229]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:42:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:27 smithi151 ceph-mon[35229]: from='osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:42:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:27 smithi151 ceph-mon[35229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:42:27.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T01:42:27.607+0000 7f0b71a23640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T01:42:27.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T01:42:27.620+0000 7f0b6d84b640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:42:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-mon[35127]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-mon[35127]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:42:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-mon[35127]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:42:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-mon[35127]: from='osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:42:27.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:27 smithi183 ceph-mon[35127]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:42:27.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:27 smithi122 ceph-mon[30925]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:27.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:27 smithi122 ceph-mon[30925]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T01:42:27.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:27 smithi122 ceph-mon[30925]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T01:42:27.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:27.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:27 smithi122 ceph-mon[30925]: from='osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:42:27.969 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:27 smithi122 ceph-mon[30925]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:42:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.341+0000 7fc51e6bd640 1 -- 172.21.15.122:0/725810086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518107900 msgr2=0x7fc518109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.341+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/725810086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518107900 0x7fc518109d50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc510009ca0 tx=0x7fc51002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.341+0000 7fc51e6bd640 1 -- 172.21.15.122:0/725810086 shutdown_connections 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.341+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/725810086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518107900 0x7fc518109d50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.341+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/725810086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc518104fd0 0x7fc5181073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.341+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/725810086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5181026a0 0x7fc518104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- 172.21.15.122:0/725810086 >> 172.21.15.122:0/725810086 conn(0x7fc5180fc2a0 msgr2=0x7fc5180fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- 172.21.15.122:0/725810086 shutdown_connections 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- 172.21.15.122:0/725810086 wait complete. 2024-09-12T01:42:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 Processor -- start 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- start start 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5181026a0 0x7fc518197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 0x7fc518198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc518107900 0x7fc51819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc51810d8c0 con 0x7fc518104fd0 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc51810d740 con 0x7fc518107900 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.342+0000 7fc51e6bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc51810da40 con 0x7fc5181026a0 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 0x7fc518198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 0x7fc518198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52528/0 (socket says 172.21.15.122:52528) 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 -- 172.21.15.122:0/3455738910 learned_addr learned my addr 172.21.15.122:0/3455738910 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc517fff640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5181026a0 0x7fc518197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc51cc5a640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc518107900 0x7fc51819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 -- 172.21.15.122:0/3455738910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5181026a0 msgr2=0x7fc518197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5181026a0 0x7fc518197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 -- 172.21.15.122:0/3455738910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc518107900 msgr2=0x7fc51819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc518107900 0x7fc51819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 -- 172.21.15.122:0/3455738910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51819fb20 con 0x7fc518104fd0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc517fff640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5181026a0 0x7fc518197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5177fe640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 0x7fc518198430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c970 tx=0x7fc50c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c007bb0 con 0x7fc518104fd0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50c007d50 con 0x7fc518104fd0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51819fe10 con 0x7fc518104fd0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0187a0 con 0x7fc518104fd0 2024-09-12T01:42:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.343+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5181a02c0 con 0x7fc518104fd0 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.344+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5180fdd90 con 0x7fc518104fd0 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.345+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc50c00f450 con 0x7fc518104fd0 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.346+0000 7fc5157fa640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc5000a2c30 0x7fc5000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.346+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7fc50c0ce7b0 con 0x7fc518104fd0 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.346+0000 7fc517fff640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc5000a2c30 0x7fc5000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.347+0000 7fc517fff640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc5000a2c30 0x7fc5000a50f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc508005d00 tx=0x7fc508005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:42:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.349+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc50c014030 con 0x7fc518104fd0 2024-09-12T01:42:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:28.525+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc518062630 con 0x7fc5000a2c30 2024-09-12T01:42:28.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi183 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-12T01:42:28.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:28.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:28.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi183 2024-09-12T01:42:28.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:42:28.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T01:42:28.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:28.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:28.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:28.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:28.932 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi183 2024-09-12T01:42:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:42:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T01:42:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T01:42:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:28.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:29.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:29 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:42:29.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:29 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:42:29.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:29 smithi122 ceph-mon[30925]: osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] boot 2024-09-12T01:42:29.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:29 smithi122 ceph-mon[30925]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T01:42:29.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:29 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:29 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:42:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:29 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:42:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:29 smithi151 ceph-mon[35229]: osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] boot 2024-09-12T01:42:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:29 smithi151 ceph-mon[35229]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T01:42:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:29 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:29 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:42:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:29 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:42:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:29 smithi183 ceph-mon[35127]: osd.8 [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] boot 2024-09-12T01:42:29.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:29 smithi183 ceph-mon[35127]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T01:42:29.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:29 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T01:42:30.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:30 smithi122 ceph-mon[30925]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T01:42:30.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:30 smithi122 ceph-mon[30925]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:30.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:30 smithi183 ceph-mon[35127]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T01:42:30.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:30 smithi183 ceph-mon[35127]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:30 smithi151 ceph-mon[35229]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T01:42:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:30 smithi151 ceph-mon[35229]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:31 smithi183 ceph-mon[35127]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T01:42:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2088852971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc2fd685-4e31-49ab-b2ed-8244461230a6"}]: dispatch 2024-09-12T01:42:31.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2088852971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc2fd685-4e31-49ab-b2ed-8244461230a6"}]': finished 2024-09-12T01:42:31.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:31 smithi183 ceph-mon[35127]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T01:42:31.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:31.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2070781548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:31 smithi122 ceph-mon[30925]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T01:42:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2088852971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc2fd685-4e31-49ab-b2ed-8244461230a6"}]: dispatch 2024-09-12T01:42:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2088852971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc2fd685-4e31-49ab-b2ed-8244461230a6"}]': finished 2024-09-12T01:42:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:31 smithi122 ceph-mon[30925]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T01:42:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2070781548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:31 smithi151 ceph-mon[35229]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T01:42:32.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2088852971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc2fd685-4e31-49ab-b2ed-8244461230a6"}]: dispatch 2024-09-12T01:42:32.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2088852971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc2fd685-4e31-49ab-b2ed-8244461230a6"}]': finished 2024-09-12T01:42:32.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:31 smithi151 ceph-mon[35229]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T01:42:32.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:32.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2070781548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:32 smithi151 ceph-mon[35229]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T01:42:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:32 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:32 smithi151 ceph-mon[35229]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:32 smithi183 ceph-mon[35127]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T01:42:33.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:32 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:33.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:32 smithi183 ceph-mon[35127]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:32 smithi122 ceph-mon[30925]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T01:42:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:32 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:32 smithi122 ceph-mon[30925]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:35.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:35 smithi122 ceph-mon[30925]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-12T01:42:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:35 smithi151 ceph-mon[35229]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-12T01:42:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:35 smithi183 ceph-mon[35127]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-12T01:42:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:37 smithi183 ceph-mon[35127]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:42:37.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:37 smithi122 ceph-mon[30925]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:42:37.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:37 smithi151 ceph-mon[35229]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:42:39.586 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:39 smithi122 ceph-mon[30925]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T01:42:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:39 smithi151 ceph-mon[35229]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T01:42:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:39 smithi183 ceph-mon[35127]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T01:42:40.717 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:42:40.718 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:40.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:42:40.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T01:42:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:41.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:41 smithi122 ceph-mon[30925]: Deploying daemon osd.9 on smithi122 2024-09-12T01:42:41.706 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:41 smithi122 ceph-mon[30925]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T01:42:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:41 smithi151 ceph-mon[35229]: Deploying daemon osd.9 on smithi122 2024-09-12T01:42:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:41 smithi151 ceph-mon[35229]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T01:42:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:41 smithi183 ceph-mon[35127]: Deploying daemon osd.9 on smithi122 2024-09-12T01:42:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:41 smithi183 ceph-mon[35127]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T01:42:43.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:43 smithi122 ceph-mon[30925]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:42:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:43 smithi151 ceph-mon[35229]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:42:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:43 smithi183 ceph-mon[35127]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T01:42:45.783 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:45 smithi122 ceph-mon[30925]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:42:45.783 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:45.783 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:45.783 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:45 smithi151 ceph-mon[35229]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:42:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:45 smithi183 ceph-mon[35127]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T01:42:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:47.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:47 smithi122 ceph-mon[30925]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:47 smithi151 ceph-mon[35229]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:47 smithi183 ceph-mon[35127]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:49.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.010+0000 7fc5157fa640 1 -- 172.21.15.122:0/3455738910 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc518062630 con 0x7fc5000a2c30 2024-09-12T01:42:49.011 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 9 on host 'smithi122' 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc5000a2c30 msgr2=0x7fc5000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc5000a2c30 0x7fc5000a50f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc508005d00 tx=0x7fc508005c70 comp rx=0 tx=0).stop 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 msgr2=0x7fc518198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 0x7fc518198430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c970 tx=0x7fc50c00ce30 comp rx=0 tx=0).stop 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 shutdown_connections 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc5000a2c30 0x7fc5000a50f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc518107900 0x7fc51819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc518104fd0 0x7fc518198430 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 --2- 172.21.15.122:0/3455738910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5181026a0 0x7fc518197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:49.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 >> 172.21.15.122:0/3455738910 conn(0x7fc5180fc2a0 msgr2=0x7fc518102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:42:49.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 shutdown_connections 2024-09-12T01:42:49.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:42:49.013+0000 7fc51e6bd640 1 -- 172.21.15.122:0/3455738910 wait complete. 2024-09-12T01:42:49.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:49.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:49.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:49.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:49.617 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.617 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:49.793 DEBUG:teuthology.orchestra.run.smithi122:osd.9> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T01:42:49.796 INFO:tasks.cephadm:Deploying osd.10 on smithi151 with /dev/vg_nvme/lv_1... 2024-09-12T01:42:49.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:42:50.057 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:42:51.485 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 01:42:51 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T01:42:51.405+0000 7f1fc6c5a640 -1 osd.9 0 log_to_monitors true 2024-09-12T01:42:51.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:51 smithi122 ceph-mon[30925]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:51.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:51 smithi122 ceph-mon[30925]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:42:51.782 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:51 smithi151 ceph-mon[35229]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:51.782 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:51 smithi151 ceph-mon[35229]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:42:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:51 smithi183 ceph-mon[35127]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:51 smithi183 ceph-mon[35127]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T01:42:52.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:42:52.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T01:42:52.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:42:52.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:52.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:42:52.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T01:42:52.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:52.677 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T01:42:52.486+0000 7f1fc2bdd640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T01:42:52.677 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T01:42:52.501+0000 7f1fbea05640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T01:42:52.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:52.717 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-12T01:42:52.751 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-09-12T01:42:52.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T01:42:52.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T01:42:52.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-12T01:42:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: from='osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-12T01:42:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T01:42:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:53.026 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 -- 172.21.15.151:0/3940906607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 msgr2=0x7fec3c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 --2- 172.21.15.151:0/3940906607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c100ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009900 tx=0x7fec2c02a410 comp rx=0 tx=0).stop 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 -- 172.21.15.151:0/3940906607 shutdown_connections 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 --2- 172.21.15.151:0/3940906607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec3c102690 0x7fec3c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 --2- 172.21.15.151:0/3940906607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c101cd0 0x7fec3c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 --2- 172.21.15.151:0/3940906607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c100ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 -- 172.21.15.151:0/3940906607 >> 172.21.15.151:0/3940906607 conn(0x7fec3c0fc280 msgr2=0x7fec3c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 -- 172.21.15.151:0/3940906607 shutdown_connections 2024-09-12T01:42:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 -- 172.21.15.151:0/3940906607 wait complete. 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 Processor -- start 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.501+0000 7fec422f5640 1 -- start start 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec422f5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec422f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec3c101cd0 0x7fec3c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec422f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c102690 0x7fec3c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec422f5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c10c780 con 0x7fec3c102690 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec422f5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c10c600 con 0x7fec3c100ad0 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec422f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c10c900 con 0x7fec3c101cd0 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c06ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44144/0 (socket says 172.21.15.151:44144) 2024-09-12T01:42:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 -- 172.21.15.151:0/1074491089 learned_addr learned my addr 172.21.15.151:0/1074491089 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec40892640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c102690 0x7fec3c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3b7fe640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec3c101cd0 0x7fec3c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 -- 172.21.15.151:0/1074491089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec3c101cd0 msgr2=0x7fec3c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec3c101cd0 0x7fec3c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 -- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c102690 msgr2=0x7fec3c19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c102690 0x7fec3c19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 -- 172.21.15.151:0/1074491089 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c19fb50 con 0x7fec3c100ad0 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec40892640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c102690 0x7fec3c19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.502+0000 7fec3bfff640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c06ab10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fec2c031440 tx=0x7fec2c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.503+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c03f030 con 0x7fec3c100ad0 2024-09-12T01:42:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.503+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec2c003ce0 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.503+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c19fde0 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.503+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c035070 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.503+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c1a0240 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.504+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c107ed0 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.505+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fec2c042030 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.505+0000 7fec397fa640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec240a2c00 0x7fec240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.505+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7fec2c0eec10 con 0x7fec3c100ad0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.505+0000 7fec3b7fe640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec240a2c00 0x7fec240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.506+0000 7fec3b7fe640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec240a2c00 0x7fec240a50c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec3c06a890 tx=0x7fec30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:42:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.509+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec2c0bbd20 con 0x7fec3c100ad0 2024-09-12T01:42:53.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:42:53.686+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fec3c062630 con 0x7fec240a2c00 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.687 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.688 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.688 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:53.688 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:53.688 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.688 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:53.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:53.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:53.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:53.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:53.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:53.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:42:53.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:54.816 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi122 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi122 to 4271M 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] boot 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:54.817 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi122 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi122 to 4271M 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] boot 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:54.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi122 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi122 to 4271M 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: osd.9 [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] boot 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:42:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:42:55.765 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:55 smithi151 ceph-mon[35229]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:55.765 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:55 smithi151 ceph-mon[35229]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:55.765 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:55 smithi151 ceph-mon[35229]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T01:42:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:55 smithi183 ceph-mon[35127]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:55 smithi183 ceph-mon[35127]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:55 smithi183 ceph-mon[35127]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T01:42:55.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:55 smithi122 ceph-mon[30925]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:42:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:55 smithi122 ceph-mon[30925]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T01:42:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:55 smithi122 ceph-mon[30925]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T01:42:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:56 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/2311413788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]: dispatch 2024-09-12T01:42:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:56 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]: dispatch 2024-09-12T01:42:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:56 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]': finished 2024-09-12T01:42:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:56 smithi151 ceph-mon[35229]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T01:42:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:42:56.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:56 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/2311413788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]: dispatch 2024-09-12T01:42:56.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:56 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]: dispatch 2024-09-12T01:42:56.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:56 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]': finished 2024-09-12T01:42:56.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:56 smithi183 ceph-mon[35127]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T01:42:56.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:42:56.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:56 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/2311413788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]: dispatch 2024-09-12T01:42:56.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:56 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]: dispatch 2024-09-12T01:42:56.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:56 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d577aa12-49ce-464c-b1b7-18e5b0765883"}]': finished 2024-09-12T01:42:56.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:56 smithi122 ceph-mon[30925]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T01:42:56.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:42:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:57 smithi151 ceph-mon[35229]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:42:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:57 smithi151 ceph-mon[35229]: from='client.? 172.21.15.151:0/71121216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:57 smithi183 ceph-mon[35127]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:42:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:57 smithi183 ceph-mon[35127]: from='client.? 172.21.15.151:0/71121216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:57.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:57 smithi122 ceph-mon[30925]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:42:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:57 smithi122 ceph-mon[30925]: from='client.? 172.21.15.151:0/71121216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:42:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:42:59 smithi151 ceph-mon[35229]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:42:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:42:59 smithi183 ceph-mon[35127]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:42:59.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:42:59 smithi122 ceph-mon[30925]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:01.744 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:01 smithi151 ceph-mon[35229]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:01 smithi183 ceph-mon[35127]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:01 smithi122 ceph-mon[30925]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:03.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:03 smithi151 ceph-mon[35229]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:03.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:03 smithi122 ceph-mon[30925]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:03 smithi183 ceph-mon[35127]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:05.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:05 smithi151 ceph-mon[35229]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:05.606 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:43:05.606 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:05 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:05 smithi183 ceph-mon[35127]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:43:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:05 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:05 smithi122 ceph-mon[30925]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T01:43:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:05 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:06.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:06 smithi151 ceph-mon[35229]: Deploying daemon osd.10 on smithi151 2024-09-12T01:43:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:06 smithi183 ceph-mon[35127]: Deploying daemon osd.10 on smithi151 2024-09-12T01:43:06.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:06 smithi122 ceph-mon[30925]: Deploying daemon osd.10 on smithi151 2024-09-12T01:43:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:07 smithi151 ceph-mon[35229]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:07 smithi183 ceph-mon[35127]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:07.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:07 smithi122 ceph-mon[30925]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:09.581 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:09 smithi151 ceph-mon[35229]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:09 smithi183 ceph-mon[35127]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:09.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:09 smithi122 ceph-mon[30925]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:11 smithi151 ceph-mon[35229]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:11 smithi183 ceph-mon[35127]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:11 smithi122 ceph-mon[30925]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:13.805 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:13 smithi151 ceph-mon[35229]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:13 smithi183 ceph-mon[35127]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:13 smithi122 ceph-mon[30925]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:14.499 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 10 on host 'smithi151' 2024-09-12T01:43:14.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.499+0000 7fec397fa640 1 -- 172.21.15.151:0/1074491089 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fec3c062630 con 0x7fec240a2c00 2024-09-12T01:43:14.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec240a2c00 msgr2=0x7fec240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:14.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec240a2c00 0x7fec240a50c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec3c06a890 tx=0x7fec30005f90 comp rx=0 tx=0).stop 2024-09-12T01:43:14.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 msgr2=0x7fec3c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c06ab10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fec2c031440 tx=0x7fec2c032ce0 comp rx=0 tx=0).stop 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 shutdown_connections 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec240a2c00 0x7fec240a50c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec3c102690 0x7fec3c19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec3c101cd0 0x7fec3c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 --2- 172.21.15.151:0/1074491089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec3c100ad0 0x7fec3c06ab10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 >> 172.21.15.151:0/1074491089 conn(0x7fec3c0fc280 msgr2=0x7fec3c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 shutdown_connections 2024-09-12T01:43:14.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-12T01:43:14.502+0000 7fec422f5640 1 -- 172.21.15.151:0/1074491089 wait complete. 2024-09-12T01:43:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:15.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:15.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:14 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.202 DEBUG:teuthology.orchestra.run.smithi151:osd.10> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T01:43:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:14 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.205 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-09-12T01:43:15.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:14 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:15.460 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:43:17.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 01:43:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T01:43:16.977+0000 7f476d252640 -1 osd.10 0 log_to_monitors true 2024-09-12T01:43:17.722 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:17 smithi151 ceph-mon[35229]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:17.722 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:17 smithi151 ceph-mon[35229]: from='osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:43:17.723 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:17 smithi151 ceph-mon[35229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:43:17.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:17 smithi122 ceph-mon[30925]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:17 smithi122 ceph-mon[30925]: from='osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:43:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:17 smithi122 ceph-mon[30925]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:43:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:17 smithi183 ceph-mon[35127]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:17 smithi183 ceph-mon[35127]: from='osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:43:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:17 smithi183 ceph-mon[35127]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T01:43:18.299 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T01:43:18.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-09-12T01:43:18.575 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.696 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:18 smithi183 ceph-mon[35127]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:43:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:43:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:43:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T01:43:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:43:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-mon[35229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:43:18.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T01:43:18.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:43:18.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T01:43:18.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:18.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-12T01:43:18.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:18.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:18.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:18.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:18.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:18 smithi122 ceph-mon[30925]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-12T01:43:19.058 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 -- 172.21.15.183:0/3617674732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc100ad0 msgr2=0x7f88dc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:19.058 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/3617674732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc100ad0 0x7f88dc100ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009900 tx=0x7f88d802a410 comp rx=0 tx=0).stop 2024-09-12T01:43:19.058 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 -- 172.21.15.183:0/3617674732 shutdown_connections 2024-09-12T01:43:19.058 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/3617674732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc102690 0x7f88dc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:19.058 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/3617674732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88dc101cd0 0x7f88dc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:19.059 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/3617674732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc100ad0 0x7f88dc100ed0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:19.059 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 -- 172.21.15.183:0/3617674732 >> 172.21.15.183:0/3617674732 conn(0x7f88dc0fc280 msgr2=0x7f88dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:19.059 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 -- 172.21.15.183:0/3617674732 shutdown_connections 2024-09-12T01:43:19.059 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 -- 172.21.15.183:0/3617674732 wait complete. 2024-09-12T01:43:19.059 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.039+0000 7f88e3ebf640 1 Processor -- start 2024-09-12T01:43:19.060 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 -- start start 2024-09-12T01:43:19.060 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88dc100ad0 0x7f88dc196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:19.060 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc101cd0 0x7f88dc197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:19.060 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 0x7f88dc19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:19.061 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10c830 con 0x7f88dc100ad0 2024-09-12T01:43:19.061 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10c6b0 con 0x7f88dc101cd0 2024-09-12T01:43:19.061 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e3ebf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10c9b0 con 0x7f88dc102690 2024-09-12T01:43:19.061 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 0x7f88dc19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:19.061 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 0x7f88dc19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:39184/0 (socket says 172.21.15.183:39184) 2024-09-12T01:43:19.062 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 -- 172.21.15.183:0/2438705301 learned_addr learned my addr 172.21.15.183:0/2438705301 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:43:19.062 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e1c5b640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88dc100ad0 0x7f88dc196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:19.062 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e145a640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc101cd0 0x7f88dc197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:19.062 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 -- 172.21.15.183:0/2438705301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc101cd0 msgr2=0x7f88dc197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:19.062 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc101cd0 0x7f88dc197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:19.063 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 -- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88dc100ad0 msgr2=0x7f88dc196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:19.063 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88dc100ad0 0x7f88dc196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:19.063 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 -- 172.21.15.183:0/2438705301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc19eb70 con 0x7f88dc102690 2024-09-12T01:43:19.063 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e145a640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc101cd0 0x7f88dc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:43:19.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.040+0000 7f88e245c640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 0x7f88dc19e500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88d400db90 tx=0x7f88d400dc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:19.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.041+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d400e980 con 0x7f88dc102690 2024-09-12T01:43:19.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.041+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc069f70 con 0x7f88dc102690 2024-09-12T01:43:19.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.041+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d4009580 con 0x7f88dc102690 2024-09-12T01:43:19.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.041+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4016650 con 0x7f88dc102690 2024-09-12T01:43:19.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.041+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc06a530 con 0x7f88dc102690 2024-09-12T01:43:19.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.042+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc100f50 con 0x7f88dc102690 2024-09-12T01:43:19.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.043+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f88d4009730 con 0x7f88dc102690 2024-09-12T01:43:19.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.043+0000 7f88d2ffd640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88c80a2bb0 0x7f88c80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:19.065 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.043+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7f88d40cff80 con 0x7f88dc102690 2024-09-12T01:43:19.066 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.044+0000 7f88e1c5b640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88c80a2bb0 0x7f88c80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:19.066 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.045+0000 7f88e1c5b640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88c80a2bb0 0x7f88c80a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88d8005ab0 tx=0x7f88d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:19.066 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.048+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88d409d650 con 0x7f88dc102690 2024-09-12T01:43:19.227 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:19.227+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f88dc06a790 con 0x7f88c80a2bb0 2024-09-12T01:43:19.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T01:43:18.881+0000 7f47691d5640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T01:43:19.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 01:43:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T01:43:18.892+0000 7f4760ff5640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:43:19.788 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi151 2024-09-12T01:43:19.788 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-12T01:43:19.788 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:19.788 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T01:43:19.788 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:19.788 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:19.789 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: from='osd.10 ' entity='osd.10' 2024-09-12T01:43:19.789 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:43:19.789 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:43:19.789 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:19 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi151 2024-09-12T01:43:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: from='osd.10 ' entity='osd.10' 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:43:19.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:19 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi151 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: from='osd.10 ' entity='osd.10' 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T01:43:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:19 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:20.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] boot 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:20 smithi183 ceph-mon[35127]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T01:43:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:43:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:43:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:43:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] boot 2024-09-12T01:43:20.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T01:43:20.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:20.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:20 smithi151 ceph-mon[35229]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T01:43:20.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: osd.10 [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] boot 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T01:43:20.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:20 smithi122 ceph-mon[30925]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T01:43:21.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:21 smithi183 ceph-mon[35127]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:21.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:21 smithi183 ceph-mon[35127]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T01:43:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:21 smithi151 ceph-mon[35229]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:21 smithi151 ceph-mon[35229]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T01:43:21.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:21 smithi122 ceph-mon[30925]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-12T01:43:21.890 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:21 smithi122 ceph-mon[30925]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T01:43:22.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:22 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/3166860151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]: dispatch 2024-09-12T01:43:22.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:22 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]: dispatch 2024-09-12T01:43:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:22 smithi183 ceph-mon[35127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]': finished 2024-09-12T01:43:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:22 smithi183 ceph-mon[35127]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T01:43:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:22 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:22 smithi183 ceph-mon[35127]: from='client.? 172.21.15.183:0/463056216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:43:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:22 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/3166860151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]: dispatch 2024-09-12T01:43:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:22 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]: dispatch 2024-09-12T01:43:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:22 smithi151 ceph-mon[35229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]': finished 2024-09-12T01:43:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:22 smithi151 ceph-mon[35229]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T01:43:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:22 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:22.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:22 smithi151 ceph-mon[35229]: from='client.? 172.21.15.183:0/463056216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:43:22.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:22 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/3166860151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]: dispatch 2024-09-12T01:43:22.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:22 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]: dispatch 2024-09-12T01:43:22.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:22 smithi122 ceph-mon[30925]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07eb6a15-ca7e-4169-8e16-6f47e7046f9e"}]': finished 2024-09-12T01:43:22.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:22 smithi122 ceph-mon[30925]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T01:43:22.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:22 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:22.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:22 smithi122 ceph-mon[30925]: from='client.? 172.21.15.183:0/463056216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T01:43:23.767 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:23 smithi151 ceph-mon[35229]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:23.767 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:23 smithi151 ceph-mon[35229]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T01:43:23.767 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:23 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:23 smithi183 ceph-mon[35127]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:23 smithi183 ceph-mon[35127]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T01:43:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:23 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:23.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:23 smithi122 ceph-mon[30925]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:23.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:23 smithi122 ceph-mon[30925]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T01:43:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:23 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:25 smithi151 ceph-mon[35229]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-12T01:43:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:25 smithi183 ceph-mon[35127]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-12T01:43:25.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:25 smithi122 ceph-mon[30925]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-12T01:43:27.845 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:27 smithi183 ceph-mon[35127]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:43:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:27 smithi151 ceph-mon[35229]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:43:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:27 smithi122 ceph-mon[30925]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T01:43:29.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:29 smithi183 ceph-mon[35127]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T01:43:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:29 smithi151 ceph-mon[35229]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T01:43:29.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:29 smithi122 ceph-mon[30925]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T01:43:30.751 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:43:30.752 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:43:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T01:43:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:31.790 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:31 smithi183 ceph-mon[35127]: Deploying daemon osd.11 on smithi183 2024-09-12T01:43:31.791 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:31 smithi183 ceph-mon[35127]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-12T01:43:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:31 smithi151 ceph-mon[35229]: Deploying daemon osd.11 on smithi183 2024-09-12T01:43:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:31 smithi151 ceph-mon[35229]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-12T01:43:31.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:31 smithi122 ceph-mon[30925]: Deploying daemon osd.11 on smithi183 2024-09-12T01:43:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:31 smithi122 ceph-mon[30925]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-12T01:43:33.772 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:33 smithi183 ceph-mon[35127]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:43:33.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:33 smithi151 ceph-mon[35229]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:43:33.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:33 smithi122 ceph-mon[30925]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T01:43:35.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:35 smithi183 ceph-mon[35127]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:35 smithi151 ceph-mon[35229]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:35.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:35 smithi122 ceph-mon[30925]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:36.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:36.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:36.814 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:36.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:36.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:36.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:36.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:36.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:36.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:37 smithi183 ceph-mon[35127]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:37.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:37 smithi151 ceph-mon[35229]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:37 smithi122 ceph-mon[30925]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:39.404 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-09-12T01:43:39.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.404+0000 7f88d2ffd640 1 -- 172.21.15.183:0/2438705301 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f88dc06a790 con 0x7f88c80a2bb0 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88c80a2bb0 msgr2=0x7f88c80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88c80a2bb0 0x7f88c80a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88d8005ab0 tx=0x7f88d8000bc0 comp rx=0 tx=0).stop 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 msgr2=0x7f88dc19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 0x7f88dc19e500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88d400db90 tx=0x7f88d400dc80 comp rx=0 tx=0).stop 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 shutdown_connections 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88c80a2bb0 0x7f88c80a5070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88dc102690 0x7f88dc19e500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88dc101cd0 0x7f88dc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 --2- 172.21.15.183:0/2438705301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88dc100ad0 0x7f88dc196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 >> 172.21.15.183:0/2438705301 conn(0x7f88dc0fc280 msgr2=0x7f88dc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:39.408 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 shutdown_connections 2024-09-12T01:43:39.408 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:39.407+0000 7f88e3ebf640 1 -- 172.21.15.183:0/2438705301 wait complete. 2024-09-12T01:43:39.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:39 smithi183 ceph-mon[35127]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:39.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:39 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:39 smithi151 ceph-mon[35229]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:39 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:39.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:39 smithi122 ceph-mon[30925]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:39 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:40.449 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T01:43:40.451 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T01:43:40.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:43:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:41.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 -- 172.21.15.122:0/3928317207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c102100 msgr2=0x7f1c3c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3928317207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c102100 0x7f1c3c102500 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009900 tx=0x7f1c2c02a410 comp rx=0 tx=0).stop 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 -- 172.21.15.122:0/3928317207 shutdown_connections 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3928317207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c3c101700 0x7f1c3c108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3928317207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 0x7f1c3c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3928317207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c102100 0x7f1c3c102500 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 -- 172.21.15.122:0/3928317207 >> 172.21.15.122:0/3928317207 conn(0x7f1c3c0fba80 msgr2=0x7f1c3c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 -- 172.21.15.122:0/3928317207 shutdown_connections 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.196+0000 7f1c41d28640 1 -- 172.21.15.122:0/3928317207 wait complete. 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 Processor -- start 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 -- start start 2024-09-12T01:43:41.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 0x7f1c3c196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 0x7f1c3c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c3c102100 0x7f1c3c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c10c7e0 con 0x7f1c3c101700 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c10c660 con 0x7f1c3c100d60 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c41d28640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c10c960 con 0x7f1c3c102100 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 0x7f1c3c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 0x7f1c3c1973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37754/0 (socket says 172.21.15.122:37754) 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 -- 172.21.15.122:0/3988106378 learned_addr learned my addr 172.21.15.122:0/3988106378 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 -- 172.21.15.122:0/3988106378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c3c102100 msgr2=0x7f1c3c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c3c102100 0x7f1c3c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3b7fe640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 0x7f1c3c196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 -- 172.21.15.122:0/3988106378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 msgr2=0x7f1c3c196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 0x7f1c3c196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3affd640 1 -- 172.21.15.122:0/3988106378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c19eaa0 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.197+0000 7f1c3b7fe640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 0x7f1c3c196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.198+0000 7f1c3affd640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 0x7f1c3c1973b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b4a0 tx=0x7f1c3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.198+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30014070 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.198+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c19ed90 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.198+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c30010af0 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.198+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30017d10 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.198+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c19f2f0 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.199+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c3c108010 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.200+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1c30017070 con 0x7f1c3c101700 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.201+0000 7f1c38ff9640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c240a2bb0 0x7f1c240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.201+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7f1c300ce580 con 0x7f1c3c101700 2024-09-12T01:43:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.201+0000 7f1c3b7fe640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c240a2bb0 0x7f1c240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.202+0000 7f1c3b7fe640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c240a2bb0 0x7f1c240a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000c00 tx=0x7f1c2c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.205+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c3009bac0 con 0x7f1c3c101700 2024-09-12T01:43:41.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.379+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1c3c02da10 con 0x7f1c3c101700 2024-09-12T01:43:41.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.380+0000 7f1c38ff9640 1 -- 172.21.15.122:0/3988106378 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f1c3009b3c0 con 0x7f1c3c101700 2024-09-12T01:43:41.380 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:43:41.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c240a2bb0 msgr2=0x7f1c240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c240a2bb0 0x7f1c240a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000c00 tx=0x7f1c2c031400 comp rx=0 tx=0).stop 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 msgr2=0x7f1c3c1973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 0x7f1c3c1973b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b4a0 tx=0x7f1c3000b960 comp rx=0 tx=0).stop 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 shutdown_connections 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c240a2bb0 0x7f1c240a5070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c3c102100 0x7f1c3c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c3c101700 0x7f1c3c1973b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 --2- 172.21.15.122:0/3988106378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c3c100d60 0x7f1c3c196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.382+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 >> 172.21.15.122:0/3988106378 conn(0x7f1c3c0fba80 msgr2=0x7f1c3c1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.383+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 shutdown_connections 2024-09-12T01:43:41.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:41.383+0000 7f1c41d28640 1 -- 172.21.15.122:0/3988106378 wait complete. 2024-09-12T01:43:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:41 smithi183 ceph-mon[35127]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:41 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3988106378' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:41 smithi122 ceph-mon[30925]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:41.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:41 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3988106378' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:41 smithi151 ceph-mon[35229]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:41 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3988106378' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:42.146 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726105399,"num_in_osds":12,"osd_in_since":1726105401,"num_remapped_pgs":0} 2024-09-12T01:43:42.324 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 01:43:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T01:43:42.171+0000 7f4be9ffe640 -1 osd.11 0 log_to_monitors true 2024-09-12T01:43:42.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:42 smithi183 ceph-mon[35127]: from='osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:43:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:42 smithi183 ceph-mon[35127]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:43:42.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:42 smithi122 ceph-mon[30925]: from='osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:43:42.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:42 smithi122 ceph-mon[30925]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:43:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:42 smithi151 ceph-mon[35229]: from='osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:43:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:42 smithi151 ceph-mon[35229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T01:43:43.147 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:43:43.395 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:43.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T01:43:43.551+0000 7f4be5f81640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T01:43:43.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T01:43:43.561+0000 7f4be1da9640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T01:43:43.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-mon[35127]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-mon[35127]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:43:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-mon[35127]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T01:43:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-mon[35127]: from='osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:43:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:43 smithi183 ceph-mon[35127]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:43:43.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:43 smithi122 ceph-mon[30925]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:43.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:43 smithi122 ceph-mon[30925]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:43:43.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:43 smithi122 ceph-mon[30925]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T01:43:43.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:43.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:43 smithi122 ceph-mon[30925]: from='osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:43:43.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:43 smithi122 ceph-mon[30925]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:43:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:43 smithi151 ceph-mon[35229]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:43 smithi151 ceph-mon[35229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T01:43:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:43 smithi151 ceph-mon[35229]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T01:43:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:43 smithi151 ceph-mon[35229]: from='osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:43:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:43 smithi151 ceph-mon[35229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T01:43:43.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 -- 172.21.15.122:0/1448186504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00102690 msgr2=0x7f8f00108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:43.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 --2- 172.21.15.122:0/1448186504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00102690 0x7f8f00108f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009ca0 tx=0x7f8ef802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:43:43.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 -- 172.21.15.122:0/1448186504 shutdown_connections 2024-09-12T01:43:43.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 --2- 172.21.15.122:0/1448186504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00102690 0x7f8f00108f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:43.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 --2- 172.21.15.122:0/1448186504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f00101cd0 0x7f8f00102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:43.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 --2- 172.21.15.122:0/1448186504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f00100ad0 0x7f8f00100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 -- 172.21.15.122:0/1448186504 >> 172.21.15.122:0/1448186504 conn(0x7f8f000fc280 msgr2=0x7f8f000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 -- 172.21.15.122:0/1448186504 shutdown_connections 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 -- 172.21.15.122:0/1448186504 wait complete. 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.888+0000 7f8f0607a640 1 Processor -- start 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 -- start start 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f00100ad0 0x7f8f0006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 0x7f8f0019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f00102690 0x7f8f001a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010c7b0 con 0x7f8f00101cd0 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010c630 con 0x7f8f00100ad0 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8f0607a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010c930 con 0x7f8f00102690 2024-09-12T01:43:43.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 0x7f8f0019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 0x7f8f0019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37778/0 (socket says 172.21.15.122:37778) 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 -- 172.21.15.122:0/2952670330 learned_addr learned my addr 172.21.15.122:0/2952670330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efffff640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f00102690 0x7f8f001a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8eff7fe640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f00100ad0 0x7f8f0006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 -- 172.21.15.122:0/2952670330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f00102690 msgr2=0x7f8f001a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f00102690 0x7f8f001a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 -- 172.21.15.122:0/2952670330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f00100ad0 msgr2=0x7f8f0006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f00100ad0 0x7f8f0006aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.889+0000 7f8efeffd640 1 -- 172.21.15.122:0/2952670330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f001a1da0 con 0x7f8f00101cd0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.890+0000 7f8efeffd640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 0x7f8f0019f060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400b4a0 tx=0x7f8ef400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.890+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4014070 con 0x7f8f00101cd0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.890+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef4010af0 con 0x7f8f00101cd0 2024-09-12T01:43:43.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.890+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4017d10 con 0x7f8f00101cd0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.890+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f001a1fd0 con 0x7f8f00101cd0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.890+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f001a2510 con 0x7f8f00101cd0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.891+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f00107ef0 con 0x7f8f00101cd0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.892+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8ef4017070 con 0x7f8f00101cd0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.893+0000 7f8efcff9640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ee80a2c30 0x7f8ee80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.893+0000 7f8eff7fe640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ee80a2c30 0x7f8ee80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.893+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7f8ef40cdcd0 con 0x7f8f00101cd0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.894+0000 7f8eff7fe640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ee80a2c30 0x7f8ee80a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f00101b30 tx=0x7f8ef000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:43.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:43.897+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ef409aba0 con 0x7f8f00101cd0 2024-09-12T01:43:44.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.073+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8f00062df0 con 0x7f8f00101cd0 2024-09-12T01:43:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.074+0000 7f8efcff9640 1 -- 172.21.15.122:0/2952670330 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f8ef40cd0a0 con 0x7f8f00101cd0 2024-09-12T01:43:44.075 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:43:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ee80a2c30 msgr2=0x7f8ee80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ee80a2c30 0x7f8ee80a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f00101b30 tx=0x7f8ef000b390 comp rx=0 tx=0).stop 2024-09-12T01:43:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 msgr2=0x7f8f0019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 0x7f8f0019f060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400b4a0 tx=0x7f8ef400b960 comp rx=0 tx=0).stop 2024-09-12T01:43:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 shutdown_connections 2024-09-12T01:43:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ee80a2c30 0x7f8ee80a50f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f00102690 0x7f8f001a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f00101cd0 0x7f8f0019f060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 --2- 172.21.15.122:0/2952670330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f00100ad0 0x7f8f0006aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 >> 172.21.15.122:0/2952670330 conn(0x7f8f000fc280 msgr2=0x7f8f001050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 shutdown_connections 2024-09-12T01:43:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:44.076+0000 7f8f0607a640 1 -- 172.21.15.122:0/2952670330 wait complete. 2024-09-12T01:43:44.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:43:44.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T01:43:44.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:44.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:44.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:44.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:44 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2952670330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:44.799 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726105399,"num_in_osds":12,"osd_in_since":1726105401,"num_remapped_pgs":0} 2024-09-12T01:43:44.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:43:44.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T01:43:44.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:44.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:44.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:44.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:44 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2952670330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T01:43:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:44 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2952670330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:45.800 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T01:43:45.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: purged_snaps scrub starts 2024-09-12T01:43:45.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: purged_snaps scrub ok 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: Detected new or changed devices on smithi183 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] boot 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T01:43:45.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: purged_snaps scrub starts 2024-09-12T01:43:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: purged_snaps scrub ok 2024-09-12T01:43:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: Detected new or changed devices on smithi183 2024-09-12T01:43:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-12T01:43:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:45.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:45.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] boot 2024-09-12T01:43:45.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T01:43:45.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: purged_snaps scrub starts 2024-09-12T01:43:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: purged_snaps scrub ok 2024-09-12T01:43:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: Detected new or changed devices on smithi183 2024-09-12T01:43:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-12T01:43:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T01:43:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: osd.11 [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] boot 2024-09-12T01:43:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T01:43:45.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T01:43:46.053 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:46.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.643+0000 7f232378c640 1 -- 172.21.15.122:0/2395653634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c102df0 msgr2=0x7f231c103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:46.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.643+0000 7f232378c640 1 --2- 172.21.15.122:0/2395653634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c102df0 0x7f231c103270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f230c009900 tx=0x7f230c02a410 comp rx=0 tx=0).stop 2024-09-12T01:43:46.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- 172.21.15.122:0/2395653634 shutdown_connections 2024-09-12T01:43:46.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 --2- 172.21.15.122:0/2395653634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f231c1037b0 0x7f231c10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 --2- 172.21.15.122:0/2395653634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c102df0 0x7f231c103270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 --2- 172.21.15.122:0/2395653634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c101bf0 0x7f231c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- 172.21.15.122:0/2395653634 >> 172.21.15.122:0/2395653634 conn(0x7f231c0fd3c0 msgr2=0x7f231c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:46.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- 172.21.15.122:0/2395653634 shutdown_connections 2024-09-12T01:43:46.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- 172.21.15.122:0/2395653634 wait complete. 2024-09-12T01:43:46.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 Processor -- start 2024-09-12T01:43:46.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- start start 2024-09-12T01:43:46.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f231c101bf0 0x7f231c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:46.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c102df0 0x7f231c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:46.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 0x7f231c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:46.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f231c10d970 con 0x7f231c1037b0 2024-09-12T01:43:46.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f231c10d7f0 con 0x7f231c101bf0 2024-09-12T01:43:46.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.644+0000 7f232378c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f231c10daf0 con 0x7f231c102df0 2024-09-12T01:43:46.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 0x7f231c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 0x7f231c19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37794/0 (socket says 172.21.15.122:37794) 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 -- 172.21.15.122:0/3023222540 learned_addr learned my addr 172.21.15.122:0/3023222540 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321528640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f231c101bf0 0x7f231c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2320d27640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c102df0 0x7f231c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 -- 172.21.15.122:0/3023222540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c102df0 msgr2=0x7f231c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c102df0 0x7f231c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 -- 172.21.15.122:0/3023222540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f231c101bf0 msgr2=0x7f231c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f231c101bf0 0x7f231c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 -- 172.21.15.122:0/3023222540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c19fc50 con 0x7f231c1037b0 2024-09-12T01:43:46.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2320d27640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c102df0 0x7f231c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f2321d29640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 0x7f231c19f5e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f231400b970 tx=0x7f231400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314019070 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2314014440 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c19ff40 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23140075d0 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.645+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1a0420 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.646+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231c109040 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.647+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f231401c070 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.648+0000 7f23127fc640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23080a2bb0 0x7f23080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.648+0000 7f2321528640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23080a2bb0 0x7f23080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.648+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f23140ce190 con 0x7f231c1037b0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.648+0000 7f2321528640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23080a2bb0 0x7f23080a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f231c102c50 tx=0x7f2318009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:46.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.652+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f231409b590 con 0x7f231c1037b0 2024-09-12T01:43:46.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.820+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f231c02d9c0 con 0x7f231c1037b0 2024-09-12T01:43:46.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.821+0000 7f23127fc640 1 -- 172.21.15.122:0/3023222540 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f231409ae90 con 0x7f231c1037b0 2024-09-12T01:43:46.821 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:43:46.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23080a2bb0 msgr2=0x7f23080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:46.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23080a2bb0 0x7f23080a5070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f231c102c50 tx=0x7f2318009110 comp rx=0 tx=0).stop 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 msgr2=0x7f231c19f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 0x7f231c19f5e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f231400b970 tx=0x7f231400bd70 comp rx=0 tx=0).stop 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 shutdown_connections 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23080a2bb0 0x7f23080a5070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1037b0 0x7f231c19f5e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f231c102df0 0x7f231c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 --2- 172.21.15.122:0/3023222540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f231c101bf0 0x7f231c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 >> 172.21.15.122:0/3023222540 conn(0x7f231c0fd3c0 msgr2=0x7f231c106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 shutdown_connections 2024-09-12T01:43:46.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:46.823+0000 7f232378c640 1 -- 172.21.15.122:0/3023222540 wait complete. 2024-09-12T01:43:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:47 smithi122 ceph-mon[30925]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T01:43:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:47 smithi122 ceph-mon[30925]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:47 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3023222540' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:47.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:47 smithi151 ceph-mon[35229]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T01:43:47.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:47 smithi151 ceph-mon[35229]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:47.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:47 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3023222540' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:47.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:47 smithi183 ceph-mon[35127]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T01:43:47.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:47 smithi183 ceph-mon[35127]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:47.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:47 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3023222540' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T01:43:47.540 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726105424,"num_in_osds":12,"osd_in_since":1726105401,"num_remapped_pgs":0} 2024-09-12T01:43:47.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:43:47.823 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 -- 172.21.15.122:0/3413411805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54100a80 msgr2=0x7f0c54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 --2- 172.21.15.122:0/3413411805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54100a80 0x7f0c54100ee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009930 tx=0x7f0c4802a420 comp rx=0 tx=0).stop 2024-09-12T01:43:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 -- 172.21.15.122:0/3413411805 shutdown_connections 2024-09-12T01:43:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 --2- 172.21.15.122:0/3413411805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c54101420 0x7f0c5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 --2- 172.21.15.122:0/3413411805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54100a80 0x7f0c54100ee0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 --2- 172.21.15.122:0/3413411805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c54106a80 0x7f0c54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 -- 172.21.15.122:0/3413411805 >> 172.21.15.122:0/3413411805 conn(0x7f0c540fc7a0 msgr2=0x7f0c540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 -- 172.21.15.122:0/3413411805 shutdown_connections 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.304+0000 7f0c5c505640 1 -- 172.21.15.122:0/3413411805 wait complete. 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 Processor -- start 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 -- start start 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c54100a80 0x7f0c54199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c54101420 0x7f0c54199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 0x7f0c541a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c54110f20 con 0x7f0c54106a80 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c54110da0 con 0x7f0c54101420 2024-09-12T01:43:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5c505640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541110a0 con 0x7f0c54100a80 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 0x7f0c541a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 0x7f0c541a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37810/0 (socket says 172.21.15.122:37810) 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 -- 172.21.15.122:0/312033047 learned_addr learned my addr 172.21.15.122:0/312033047 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5a2a1640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c54100a80 0x7f0c54199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c59aa0640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c54101420 0x7f0c54199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 -- 172.21.15.122:0/312033047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c54100a80 msgr2=0x7f0c54199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c54100a80 0x7f0c54199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 -- 172.21.15.122:0/312033047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c54101420 msgr2=0x7f0c54199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c54101420 0x7f0c54199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.305+0000 7f0c5aaa2640 1 -- 172.21.15.122:0/312033047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c541a49f0 con 0x7f0c54106a80 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.306+0000 7f0c5aaa2640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 0x7f0c541a44b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009fd0 tx=0x7f0c4c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.306+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c011830 con 0x7f0c54106a80 2024-09-12T01:43:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.306+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c541a4bc0 con 0x7f0c54106a80 2024-09-12T01:43:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.306+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c016070 con 0x7f0c54106a80 2024-09-12T01:43:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.306+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00f2e0 con 0x7f0c54106a80 2024-09-12T01:43:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.306+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c541a5080 con 0x7f0c54106a80 2024-09-12T01:43:48.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.307+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c54100f60 con 0x7f0c54106a80 2024-09-12T01:43:48.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.308+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0c4c0119d0 con 0x7f0c54106a80 2024-09-12T01:43:48.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.308+0000 7f0c537fe640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c3c0a2b80 0x7f0c3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:48.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.309+0000 7f0c5a2a1640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c3c0a2b80 0x7f0c3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:48.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.309+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f0c4c0ce1d0 con 0x7f0c54106a80 2024-09-12T01:43:48.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.309+0000 7f0c5a2a1640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c3c0a2b80 0x7f0c3c0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400a400 tx=0x7f0c4400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.313+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c4c09b5d0 con 0x7f0c54106a80 2024-09-12T01:43:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.480+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c54109010 con 0x7f0c54106a80 2024-09-12T01:43:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.481+0000 7f0c537fe640 1 -- 172.21.15.122:0/312033047 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18491 (secure 0 0 0) 0x7f0c4c09aed0 con 0x7f0c54106a80 2024-09-12T01:43:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:43:48.481 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:43:46.008957+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:43:48.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c3c0a2b80 msgr2=0x7f0c3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:48.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c3c0a2b80 0x7f0c3c0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400a400 tx=0x7f0c4400d040 comp rx=0 tx=0).stop 2024-09-12T01:43:48.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 msgr2=0x7f0c541a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:48.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 0x7f0c541a44b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009fd0 tx=0x7f0c4c00b8e0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 shutdown_connections 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c3c0a2b80 0x7f0c3c0a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c54106a80 0x7f0c541a44b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c54101420 0x7f0c54199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 --2- 172.21.15.122:0/312033047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c54100a80 0x7f0c54199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 >> 172.21.15.122:0/312033047 conn(0x7f0c540fc7a0 msgr2=0x7f0c540fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 shutdown_connections 2024-09-12T01:43:48.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:48.483+0000 7f0c5c505640 1 -- 172.21.15.122:0/312033047 wait complete. 2024-09-12T01:43:48.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:48 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/312033047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:43:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:48 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/312033047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:43:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:48 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/312033047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:43:49.277 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:39:58.671405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T01:43:49.277 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T01:43:49.521 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:49.793 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:49 smithi122 ceph-mon[30925]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:49.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:49 smithi151 ceph-mon[35229]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:49 smithi183 ceph-mon[35127]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 -- 172.21.15.122:0/4181133707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 msgr2=0x7f1e00101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 --2- 172.21.15.122:0/4181133707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00101a80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002a410 comp rx=0 tx=0).stop 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 -- 172.21.15.122:0/4181133707 shutdown_connections 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 --2- 172.21.15.122:0/4181133707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e00101fc0 0x7f1e0010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 --2- 172.21.15.122:0/4181133707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00101a80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 --2- 172.21.15.122:0/4181133707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e00105a80 0x7f1e00105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 -- 172.21.15.122:0/4181133707 >> 172.21.15.122:0/4181133707 conn(0x7f1e000fba90 msgr2=0x7f1e000fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 -- 172.21.15.122:0/4181133707 shutdown_connections 2024-09-12T01:43:49.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.974+0000 7f1e07f82640 1 -- 172.21.15.122:0/4181133707 wait complete. 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 Processor -- start 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 -- start start 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e00101fc0 0x7f1e00198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e00105a80 0x7f1e0019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00110ef0 con 0x7f1e00101600 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00110d70 con 0x7f1e00101fc0 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e07f82640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00111070 con 0x7f1e00105a80 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37824/0 (socket says 172.21.15.122:37824) 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 -- 172.21.15.122:0/3712061858 learned_addr learned my addr 172.21.15.122:0/3712061858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:49.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 -- 172.21.15.122:0/3712061858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e00105a80 msgr2=0x7f1e0019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e00105a80 0x7f1e0019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 -- 172.21.15.122:0/3712061858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e00101fc0 msgr2=0x7f1e00198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e00101fc0 0x7f1e00198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 -- 172.21.15.122:0/3712061858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0019d530 con 0x7f1e00101600 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1e05d1e640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00198580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b4a0 tx=0x7f1dfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.975+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc014070 con 0x7f1e00101600 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.976+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dfc010af0 con 0x7f1e00101600 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.976+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0019d820 con 0x7f1e00101600 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.976+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc017d10 con 0x7f1e00101600 2024-09-12T01:43:49.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.976+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a5100 con 0x7f1e00101600 2024-09-12T01:43:49.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.976+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e00101b00 con 0x7f1e00101600 2024-09-12T01:43:49.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.978+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1dfc017070 con 0x7f1e00101600 2024-09-12T01:43:49.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.978+0000 7f1df6ffd640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dec0a2c00 0x7f1dec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:49.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.978+0000 7f1e0551d640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dec0a2c00 0x7f1dec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:49.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.979+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f1dfc0ce3b0 con 0x7f1e00101600 2024-09-12T01:43:49.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.979+0000 7f1e0551d640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dec0a2c00 0x7f1dec0a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1df00096c0 tx=0x7f1df0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:49.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:49.982+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1dfc09b7b0 con 0x7f1e00101600 2024-09-12T01:43:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.147+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e0002d7c0 con 0x7f1e00101600 2024-09-12T01:43:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.148+0000 7f1df6ffd640 1 -- 172.21.15.122:0/3712061858 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f1dfc09b0b0 con 0x7f1e00101600 2024-09-12T01:43:50.148 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.149+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dec0a2c00 msgr2=0x7f1dec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.149+0000 7f1e07f82640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dec0a2c00 0x7f1dec0a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1df00096c0 tx=0x7f1df0000bc0 comp rx=0 tx=0).stop 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.149+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 msgr2=0x7f1e00198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.149+0000 7f1e07f82640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00198580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b4a0 tx=0x7f1dfc00b960 comp rx=0 tx=0).stop 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 shutdown_connections 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1dec0a2c00 0x7f1dec0a50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e00105a80 0x7f1e0019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e00101fc0 0x7f1e00198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 --2- 172.21.15.122:0/3712061858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e00101600 0x7f1e00198580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 >> 172.21.15.122:0/3712061858 conn(0x7f1e000fba90 msgr2=0x7f1e00100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 shutdown_connections 2024-09-12T01:43:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:50.150+0000 7f1e07f82640 1 -- 172.21.15.122:0/3712061858 wait complete. 2024-09-12T01:43:50.864 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:50 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3712061858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:43:50.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:50 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3712061858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.a on smithi122 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.d on smithi122 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.g on smithi122 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.j on smithi122 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.b on smithi151 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.e on smithi151 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.h on smithi151 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.k on smithi151 2024-09-12T01:43:50.891 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-09-12T01:43:50.892 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-09-12T01:43:50.892 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-09-12T01:43:50.892 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-09-12T01:43:50.892 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-09-12T01:43:50.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:50 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3712061858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:43:51.149 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 -- 172.21.15.183:0/597736230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130102690 msgr2=0x7f4130108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 --2- 172.21.15.183:0/597736230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130102690 0x7f4130108f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4128009ca0 tx=0x7f412802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 -- 172.21.15.183:0/597736230 shutdown_connections 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 --2- 172.21.15.183:0/597736230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130102690 0x7f4130108f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 --2- 172.21.15.183:0/597736230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130101cd0 0x7f4130102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 --2- 172.21.15.183:0/597736230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 0x7f4130100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 -- 172.21.15.183:0/597736230 >> 172.21.15.183:0/597736230 conn(0x7f41300fc280 msgr2=0x7f41300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:51.664 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 -- 172.21.15.183:0/597736230 shutdown_connections 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.648+0000 7f4138033640 1 -- 172.21.15.183:0/597736230 wait complete. 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 Processor -- start 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 -- start start 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 0x7f4130196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 0x7f4130197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130102690 0x7f413019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f413010c7b0 con 0x7f4130100ad0 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f413010c630 con 0x7f4130102690 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f4138033640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f413010c930 con 0x7f4130101cd0 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f41355ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 0x7f4130197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f41355ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 0x7f4130197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:33962/0 (socket says 172.21.15.183:33962) 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f41355ce640 1 -- 172.21.15.183:0/4034623303 learned_addr learned my addr 172.21.15.183:0/4034623303 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T01:43:51.665 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.649+0000 7f41365d0640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130102690 0x7f413019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4135dcf640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 0x7f4130196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41355ce640 1 -- 172.21.15.183:0/4034623303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130102690 msgr2=0x7f413019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41355ce640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130102690 0x7f413019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41355ce640 1 -- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 msgr2=0x7f4130196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41355ce640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 0x7f4130196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41355ce640 1 -- 172.21.15.183:0/4034623303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413019ea30 con 0x7f4130101cd0 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4135dcf640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 0x7f4130196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41365d0640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130102690 0x7f413019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f41355ce640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 0x7f4130197340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f412000b4a0 tx=0x7f412000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120014070 con 0x7f4130101cd0 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4120010af0 con 0x7f4130101cd0 2024-09-12T01:43:51.666 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413019ed20 con 0x7f4130101cd0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120017d10 con 0x7f4130101cd0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.650+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413019f280 con 0x7f4130101cd0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.651+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4130107e70 con 0x7f4130101cd0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.652+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4120017070 con 0x7f4130101cd0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.653+0000 7f4126ffd640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f411c0a2bb0 0x7f411c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.653+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f41200cdfb0 con 0x7f4130101cd0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.653+0000 7f4135dcf640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f411c0a2bb0 0x7f411c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.654+0000 7f4135dcf640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f411c0a2bb0 0x7f411c0a5070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4130101b30 tx=0x7f412c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:51.667 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.658+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f412009b3b0 con 0x7f4130101cd0 2024-09-12T01:43:51.811 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:51 smithi183 ceph-mon[35127]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:51.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.842+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) -- 0x7f4130062730 con 0x7f411c0a2bb0 2024-09-12T01:43:51.854 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.854+0000 7f4126ffd640 1 -- 172.21.15.183:0/4034623303 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4130062730 con 0x7f411c0a2bb0 2024-09-12T01:43:51.854 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f411c0a2bb0 msgr2=0x7f411c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f411c0a2bb0 0x7f411c0a5070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4130101b30 tx=0x7f412c005f90 comp rx=0 tx=0).stop 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 msgr2=0x7f4130197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 0x7f4130197340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f412000b4a0 tx=0x7f412000b960 comp rx=0 tx=0).stop 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 shutdown_connections 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f411c0a2bb0 0x7f411c0a5070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4130102690 0x7f413019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4130101cd0 0x7f4130197340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.856 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 --2- 172.21.15.183:0/4034623303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130100ad0 0x7f4130196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:51.857 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 >> 172.21.15.183:0/4034623303 conn(0x7f41300fc280 msgr2=0x7f4130105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:51.857 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 shutdown_connections 2024-09-12T01:43:51.857 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T01:43:51.856+0000 7f4138033640 1 -- 172.21.15.183:0/4034623303 wait complete. 2024-09-12T01:43:51.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:51 smithi151 ceph-mon[35229]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:51.879 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:51 smithi122 ceph-mon[30925]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:52.588 DEBUG:teuthology.orchestra.run.smithi122:mds.a> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T01:43:52.591 DEBUG:teuthology.orchestra.run.smithi122:mds.d> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T01:43:52.633 DEBUG:teuthology.orchestra.run.smithi122:mds.g> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T01:43:52.677 DEBUG:teuthology.orchestra.run.smithi122:mds.j> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T01:43:52.720 DEBUG:teuthology.orchestra.run.smithi151:mds.b> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T01:43:52.723 DEBUG:teuthology.orchestra.run.smithi151:mds.e> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T01:43:52.768 DEBUG:teuthology.orchestra.run.smithi151:mds.h> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T01:43:52.813 DEBUG:teuthology.orchestra.run.smithi151:mds.k> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T01:43:52.857 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T01:43:52.860 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T01:43:52.905 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T01:43:52.947 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T01:43:52.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:43:52.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T01:43:52.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:52.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:52.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:52.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:52.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:52.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:52.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:52.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:52.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: Deploying daemon mds.l on smithi183 2024-09-12T01:43:52.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:52 smithi122 ceph-mon[30925]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:52.993 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T01:43:52.993 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:43:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:43:53.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: Deploying daemon mds.l on smithi183 2024-09-12T01:43:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:52 smithi151 ceph-mon[35229]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:53.156 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:53.157 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:53.157 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: Deploying daemon mds.l on smithi183 2024-09-12T01:43:53.157 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:52 smithi183 ceph-mon[35127]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:53.244 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:53.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.717+0000 7f978dc1e640 1 -- 172.21.15.122:0/2930181515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788106a80 msgr2=0x7f9788106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:53.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.717+0000 7f978dc1e640 1 --2- 172.21.15.122:0/2930181515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788106a80 0x7f9788106e60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9778009930 tx=0x7f977802a420 comp rx=0 tx=0).stop 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 -- 172.21.15.122:0/2930181515 shutdown_connections 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 --2- 172.21.15.122:0/2930181515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788101420 0x7f978810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 --2- 172.21.15.122:0/2930181515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9788100a80 0x7f9788100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 --2- 172.21.15.122:0/2930181515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788106a80 0x7f9788106e60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 -- 172.21.15.122:0/2930181515 >> 172.21.15.122:0/2930181515 conn(0x7f97880fc7a0 msgr2=0x7f97880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 -- 172.21.15.122:0/2930181515 shutdown_connections 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 -- 172.21.15.122:0/2930181515 wait complete. 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 Processor -- start 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.718+0000 7f978dc1e640 1 -- start start 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9788100a80 0x7f9788198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:53.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 0x7f9788198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788106a80 0x7f978819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9788110ed0 con 0x7f9788101420 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9788110d50 con 0x7f9788100a80 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9788111050 con 0x7f9788106a80 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 0x7f9788198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 0x7f9788198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60038/0 (socket says 172.21.15.122:60038) 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 -- 172.21.15.122:0/1378298884 learned_addr learned my addr 172.21.15.122:0/1378298884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f97877fe640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9788100a80 0x7f9788198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f9787fff640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788106a80 0x7f978819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 -- 172.21.15.122:0/1378298884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788106a80 msgr2=0x7f978819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788106a80 0x7f978819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 -- 172.21.15.122:0/1378298884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9788100a80 msgr2=0x7f9788198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9788100a80 0x7f9788198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 -- 172.21.15.122:0/1378298884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978819d3d0 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f9787fff640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788106a80 0x7f978819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f977effd640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 0x7f9788198960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f977000d950 tx=0x7f977000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770017030 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9770004020 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978819d6c0 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97700047a0 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.719+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97881a4fa0 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.720+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9788100ee0 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.722+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f97700041d0 con 0x7f9788101420 2024-09-12T01:43:53.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.722+0000 7f97857fa640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f97740a2b80 0x7f97740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:53.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.722+0000 7f97877fe640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f97740a2b80 0x7f97740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:53.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.722+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f97700ce580 con 0x7f9788101420 2024-09-12T01:43:53.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.723+0000 7f97877fe640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f97740a2b80 0x7f97740a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9778006010 tx=0x7f9778000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:53.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.726+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f977009b980 con 0x7f9788101420 2024-09-12T01:43:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.926+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9788199780 con 0x7f9788101420 2024-09-12T01:43:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.927+0000 7f97857fa640 1 -- 172.21.15.122:0/1378298884 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f9788199780 con 0x7f9788101420 2024-09-12T01:43:53.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:43:53.928 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"btime":"2024-09-12T01:36:52:283630+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T01:43:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 1 2024-09-12T01:43:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f97740a2b80 msgr2=0x7f97740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f97740a2b80 0x7f97740a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9778006010 tx=0x7f9778000bc0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 msgr2=0x7f9788198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 0x7f9788198960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f977000d950 tx=0x7f977000de10 comp rx=0 tx=0).stop 2024-09-12T01:43:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 shutdown_connections 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f97740a2b80 0x7f97740a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9788106a80 0x7f978819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9788101420 0x7f9788198960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 --2- 172.21.15.122:0/1378298884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9788100a80 0x7f9788198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 >> 172.21.15.122:0/1378298884 conn(0x7f97880fc7a0 msgr2=0x7f9788104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 shutdown_connections 2024-09-12T01:43:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:53.929+0000 7f978dc1e640 1 -- 172.21.15.122:0/1378298884 wait complete. 2024-09-12T01:43:54.102 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:53 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1378298884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:43:54.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:53 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1378298884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:43:54.325 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:53 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1378298884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:43:54.452 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 systemd[1]: Starting Ceph mds.l for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:43:54.756 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 podman[64118]: 2024-09-12 01:43:54.393302025 +0000 UTC m=+0.015612955 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:43:54.756 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 podman[64118]: 2024-09-12 01:43:54.505245146 +0000 UTC m=+0.127556064 container create 669df5a4698d1b7081fc968d9b16a74d5403064bbd72d08cabf23cb5637fe173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:43:54.756 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 podman[64118]: 2024-09-12 01:43:54.756515032 +0000 UTC m=+0.378825951 container init 669df5a4698d1b7081fc968d9b16a74d5403064bbd72d08cabf23cb5637fe173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:43:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6832/3975453954,v1:172.21.15.183:6833/3975453954] up:boot 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: fsmap 1 up:standby 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:55.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 podman[64118]: 2024-09-12 01:43:54.759407087 +0000 UTC m=+0.381718011 container start 669df5a4698d1b7081fc968d9b16a74d5403064bbd72d08cabf23cb5637fe173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:43:55.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-mds[64137]: -- 172.21.15.183:0/1438437190 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f834c1dc20 con 0x55f834bfb000 2024-09-12T01:43:55.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l[64133]: starting mds.l at 2024-09-12T01:43:55.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 bash[64118]: 669df5a4698d1b7081fc968d9b16a74d5403064bbd72d08cabf23cb5637fe173 2024-09-12T01:43:55.204 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 01:43:54 smithi183 systemd[1]: Started Ceph mds.l for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:43:55.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6832/3975453954,v1:172.21.15.183:6833/3975453954] up:boot 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: fsmap 1 up:standby 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:55.341 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:55.341 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6832/3975453954,v1:172.21.15.183:6833/3975453954] up:boot 2024-09-12T01:43:55.341 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: fsmap 1 up:standby 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:55.342 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:56.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:55 smithi183 ceph-mon[35127]: Deploying daemon mds.k on smithi151 2024-09-12T01:43:56.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:55 smithi151 ceph-mon[35229]: Deploying daemon mds.k on smithi151 2024-09-12T01:43:56.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:55 smithi122 ceph-mon[30925]: Deploying daemon mds.k on smithi151 2024-09-12T01:43:56.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:43:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:43:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:56 smithi151 ceph-mon[35229]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:57.286 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:56 smithi122 ceph-mon[30925]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:57.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 -- 172.21.15.122:0/4157860797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790102380 msgr2=0x7fe79010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 --2- 172.21.15.122:0/4157860797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790102380 0x7fe79010e9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe788009cd0 tx=0x7fe78802a910 comp rx=0 tx=0).stop 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 -- 172.21.15.122:0/4157860797 shutdown_connections 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 --2- 172.21.15.122:0/4157860797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790102380 0x7fe79010e9c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 --2- 172.21.15.122:0/4157860797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7901019e0 0x7fe790101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 --2- 172.21.15.122:0/4157860797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe790107af0 0x7fe790107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 -- 172.21.15.122:0/4157860797 >> 172.21.15.122:0/4157860797 conn(0x7fe7900fd700 msgr2=0x7fe7900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 -- 172.21.15.122:0/4157860797 shutdown_connections 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.366+0000 7fe7961a0640 1 -- 172.21.15.122:0/4157860797 wait complete. 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 Processor -- start 2024-09-12T01:43:57.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 -- start start 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 0x7fe7901984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe790102380 0x7fe790198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe790107af0 0x7fe79019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe790111fb0 con 0x7fe7901019e0 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe790111e30 con 0x7fe790102380 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe7961a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe790112130 con 0x7fe790107af0 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 0x7fe7901984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 0x7fe7901984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60046/0 (socket says 172.21.15.122:60046) 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 -- 172.21.15.122:0/3506152480 learned_addr learned my addr 172.21.15.122:0/3506152480 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 -- 172.21.15.122:0/3506152480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe790107af0 msgr2=0x7fe79019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe790107af0 0x7fe79019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 -- 172.21.15.122:0/3506152480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe790102380 msgr2=0x7fe790198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:57.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe790102380 0x7fe790198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.367+0000 7fe79519e640 1 -- 172.21.15.122:0/3506152480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79019d480 con 0x7fe7901019e0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.368+0000 7fe79519e640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 0x7fe7901984d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00b4a0 tx=0x7fe78c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.368+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c014070 con 0x7fe7901019e0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.368+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78c010af0 con 0x7fe7901019e0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.368+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79019d770 con 0x7fe7901019e0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.368+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c017d10 con 0x7fe7901019e0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.368+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901a5050 con 0x7fe7901019e0 2024-09-12T01:43:57.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.369+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790101ec0 con 0x7fe7901019e0 2024-09-12T01:43:57.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.371+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe78c017070 con 0x7fe7901019e0 2024-09-12T01:43:57.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.371+0000 7fe7867fc640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe77c0a2b80 0x7fe77c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:43:57.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.372+0000 7fe79499d640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe77c0a2b80 0x7fe77c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:43:57.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.372+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe78c0ce430 con 0x7fe7901019e0 2024-09-12T01:43:57.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.372+0000 7fe79499d640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe77c0a2b80 0x7fe77c0a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7900ffce0 tx=0x7fe780006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:43:57.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.375+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe78c09b830 con 0x7fe7901019e0 2024-09-12T01:43:57.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:56 smithi183 ceph-mon[35127]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:57.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.577+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe790199830 con 0x7fe7901019e0 2024-09-12T01:43:57.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.577+0000 7fe7867fc640 1 -- 172.21.15.122:0/3506152480 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fe78c09b130 con 0x7fe7901019e0 2024-09-12T01:43:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:43:57.578 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"btime":"2024-09-12T01:43:54:788030+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:43:57.578 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 2 2024-09-12T01:43:57.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe77c0a2b80 msgr2=0x7fe77c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:57.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe77c0a2b80 0x7fe77c0a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7900ffce0 tx=0x7fe780006c40 comp rx=0 tx=0).stop 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 msgr2=0x7fe7901984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 0x7fe7901984d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00b4a0 tx=0x7fe78c00b960 comp rx=0 tx=0).stop 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 shutdown_connections 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe77c0a2b80 0x7fe77c0a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe790107af0 0x7fe79019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe790102380 0x7fe790198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 --2- 172.21.15.122:0/3506152480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7901019e0 0x7fe7901984d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 >> 172.21.15.122:0/3506152480 conn(0x7fe7900fd700 msgr2=0x7fe7900fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 shutdown_connections 2024-09-12T01:43:57.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:43:57.580+0000 7fe7961a0640 1 -- 172.21.15.122:0/3506152480 wait complete. 2024-09-12T01:43:57.865 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 01:43:57 smithi151 systemd[1]: Started Ceph mds.k for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:43:58.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3506152480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:43:58.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:58.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:58.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:58.402 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3506152480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:43:58.403 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.403 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.403 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.403 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:58.403 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:58.403 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:58.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3506152480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:43:58.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:43:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:43:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:43:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:43:59.286 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:59 smithi122 ceph-mon[30925]: Deploying daemon mds.j on smithi122 2024-09-12T01:43:59.286 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:59 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6832/3451432768,v1:172.21.15.151:6833/3451432768] up:boot 2024-09-12T01:43:59.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:59 smithi122 ceph-mon[30925]: fsmap 2 up:standby 2024-09-12T01:43:59.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:43:59.287 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:43:59 smithi122 ceph-mon[30925]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:59.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:59 smithi151 ceph-mon[35229]: Deploying daemon mds.j on smithi122 2024-09-12T01:43:59.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:59 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6832/3451432768,v1:172.21.15.151:6833/3451432768] up:boot 2024-09-12T01:43:59.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:59 smithi151 ceph-mon[35229]: fsmap 2 up:standby 2024-09-12T01:43:59.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:43:59.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:43:59 smithi151 ceph-mon[35229]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:43:59.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:59 smithi183 ceph-mon[35127]: Deploying daemon mds.j on smithi122 2024-09-12T01:43:59.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:59 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6832/3451432768,v1:172.21.15.151:6833/3451432768] up:boot 2024-09-12T01:43:59.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:59 smithi183 ceph-mon[35127]: fsmap 2 up:standby 2024-09-12T01:43:59.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:43:59.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:43:59 smithi183 ceph-mon[35127]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:00.210 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 systemd[1]: Starting Ceph mds.j for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:00.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:00.475 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 podman[70293]: 2024-09-12 01:44:00.113721472 +0000 UTC m=+0.015475430 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:00.475 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 podman[70293]: 2024-09-12 01:44:00.231770133 +0000 UTC m=+0.133524090 container create 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T01:44:00.475 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 podman[70293]: 2024-09-12 01:44:00.476323301 +0000 UTC m=+0.378077260 container init 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-12T01:44:00.703 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:00.738 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 podman[70293]: 2024-09-12 01:44:00.479479865 +0000 UTC m=+0.381233823 container start 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T01:44:00.739 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 ceph-mds[70330]: -- 172.21.15.122:0/1077653478 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579b70ed860 con 0x5579b70ca800 2024-09-12T01:44:00.739 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j[70308]: starting mds.j at 2024-09-12T01:44:00.739 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 bash[70293]: 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 2024-09-12T01:44:00.739 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 01:44:00 smithi122 systemd[1]: Started Ceph mds.j for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 -- 172.21.15.122:0/2682754074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 msgr2=0x7f715c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 --2- 172.21.15.122:0/2682754074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c10ea80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7154009ca0 tx=0x7f715402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 -- 172.21.15.122:0/2682754074 shutdown_connections 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 --2- 172.21.15.122:0/2682754074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c10ea80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 --2- 172.21.15.122:0/2682754074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f715c102780 0x7f715c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 --2- 172.21.15.122:0/2682754074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 0x7f715c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 -- 172.21.15.122:0/2682754074 >> 172.21.15.122:0/2682754074 conn(0x7f715c0fd890 msgr2=0x7f715c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 -- 172.21.15.122:0/2682754074 shutdown_connections 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 -- 172.21.15.122:0/2682754074 wait complete. 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 Processor -- start 2024-09-12T01:44:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.177+0000 7f7163183640 1 -- start start 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 0x7f715c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f715c102780 0x7f715c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111f80 con 0x7f715c106f00 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111e00 con 0x7f715c102780 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f715c112100 con 0x7f715c101dd0 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50846/0 (socket says 172.21.15.122:50846) 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 -- 172.21.15.122:0/403709858 learned_addr learned my addr 172.21.15.122:0/403709858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7160f1f640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 0x7f715c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 -- 172.21.15.122:0/403709858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 msgr2=0x7f715c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 0x7f715c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 -- 172.21.15.122:0/403709858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f715c102780 msgr2=0x7f715c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f715c102780 0x7f715c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 -- 172.21.15.122:0/403709858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c1a6920 con 0x7f715c106f00 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7160f1f640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 0x7f715c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7161720640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c074f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7154031a60 tx=0x7f7154031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154003c30 con 0x7f715c106f00 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7154003dd0 con 0x7f715c106f00 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540047e0 con 0x7f715c106f00 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.178+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c1a6bb0 con 0x7f715c106f00 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.179+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1a70a0 con 0x7f715c106f00 2024-09-12T01:44:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.180+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c1021b0 con 0x7f715c106f00 2024-09-12T01:44:01.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.181+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f715402f790 con 0x7f715c106f00 2024-09-12T01:44:01.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.181+0000 7f7159ffb640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f714c0a2b80 0x7f714c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:01.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.182+0000 7f7160f1f640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f714c0a2b80 0x7f714c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:01.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.182+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f71540f0030 con 0x7f715c106f00 2024-09-12T01:44:01.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.182+0000 7f7160f1f640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f714c0a2b80 0x7f714c0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7150006fd0 tx=0x7f7150008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:01.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.186+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71540bd430 con 0x7f715c106f00 2024-09-12T01:44:01.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.402+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f715c02d7c0 con 0x7f715c106f00 2024-09-12T01:44:01.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.402+0000 7f7159ffb640 1 -- 172.21.15.122:0/403709858 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f71540bcd30 con 0x7f715c106f00 2024-09-12T01:44:01.403 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:01.403 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"btime":"2024-09-12T01:43:57:995622+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:44:01.403 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 3 2024-09-12T01:44:01.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.404+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f714c0a2b80 msgr2=0x7f714c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.404+0000 7f7163183640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f714c0a2b80 0x7f714c0a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7150006fd0 tx=0x7f7150008040 comp rx=0 tx=0).stop 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.404+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 msgr2=0x7f715c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.404+0000 7f7163183640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c074f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7154031a60 tx=0x7f7154031a90 comp rx=0 tx=0).stop 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 shutdown_connections 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f714c0a2b80 0x7f714c0a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f715c106f00 0x7f715c074f70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f715c102780 0x7f715c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 --2- 172.21.15.122:0/403709858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f715c101dd0 0x7f715c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 >> 172.21.15.122:0/403709858 conn(0x7f715c0fd890 msgr2=0x7f715c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 shutdown_connections 2024-09-12T01:44:01.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:01.405+0000 7f7163183640 1 -- 172.21.15.122:0/403709858 wait complete. 2024-09-12T01:44:01.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:01.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:01.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:01.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:01.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:01 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/403709858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:01.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:01 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/403709858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:01 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/403709858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:02.756 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:02 smithi122 ceph-mon[30925]: Deploying daemon mds.d on smithi122 2024-09-12T01:44:02.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:02 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.122:6834/3513109229,v1:172.21.15.122:6835/3513109229] up:boot 2024-09-12T01:44:02.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:02 smithi122 ceph-mon[30925]: fsmap 3 up:standby 2024-09-12T01:44:02.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:02 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:44:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:02 smithi151 ceph-mon[35229]: Deploying daemon mds.d on smithi122 2024-09-12T01:44:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:02 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.122:6834/3513109229,v1:172.21.15.122:6835/3513109229] up:boot 2024-09-12T01:44:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:02 smithi151 ceph-mon[35229]: fsmap 3 up:standby 2024-09-12T01:44:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:02 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:44:02.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:02 smithi183 ceph-mon[35127]: Deploying daemon mds.d on smithi122 2024-09-12T01:44:02.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:02 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.122:6834/3513109229,v1:172.21.15.122:6835/3513109229] up:boot 2024-09-12T01:44:02.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:02 smithi183 ceph-mon[35127]: fsmap 3 up:standby 2024-09-12T01:44:02.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:02 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T01:44:03.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:03 smithi122 ceph-mon[30925]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:03 smithi151 ceph-mon[35229]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:03 smithi183 ceph-mon[35127]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:04.239 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 01:44:03 smithi122 systemd[1]: Started Ceph mds.d for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: Deploying daemon mds.e on smithi151 2024-09-12T01:44:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:05.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.122:6836/6361172,v1:172.21.15.122:6837/6361172] up:boot 2024-09-12T01:44:05.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: fsmap 4 up:standby 2024-09-12T01:44:05.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:04 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:44:05.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: Deploying daemon mds.e on smithi151 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.122:6836/6361172,v1:172.21.15.122:6837/6361172] up:boot 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: fsmap 4 up:standby 2024-09-12T01:44:05.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:04 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:44:05.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: Deploying daemon mds.e on smithi151 2024-09-12T01:44:05.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.122:6836/6361172,v1:172.21.15.122:6837/6361172] up:boot 2024-09-12T01:44:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: fsmap 4 up:standby 2024-09-12T01:44:05.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:04 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T01:44:05.630 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 -- 172.21.15.122:0/279110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 msgr2=0x7fee6410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 --2- 172.21.15.122:0/279110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee6410ea90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009ca0 tx=0x7fee5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 -- 172.21.15.122:0/279110440 shutdown_connections 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 --2- 172.21.15.122:0/279110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee6410ea90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 --2- 172.21.15.122:0/279110440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee64102790 0x7fee641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 --2- 172.21.15.122:0/279110440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee64101de0 0x7fee641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 -- 172.21.15.122:0/279110440 >> 172.21.15.122:0/279110440 conn(0x7fee640fd8c0 msgr2=0x7fee640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 -- 172.21.15.122:0/279110440 shutdown_connections 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.076+0000 7fee6c074640 1 -- 172.21.15.122:0/279110440 wait complete. 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 Processor -- start 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 -- start start 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee64101de0 0x7fee640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee64102790 0x7fee64074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee64074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fee64111f70 con 0x7fee64106f10 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee64111df0 con 0x7fee64102790 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6c074640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee641120f0 con 0x7fee64101de0 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee64074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:06.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee64074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50870/0 (socket says 172.21.15.122:50870) 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 -- 172.21.15.122:0/2932867772 learned_addr learned my addr 172.21.15.122:0/2932867772 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6960f640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee64102790 0x7fee64074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee69e10640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee64101de0 0x7fee640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 -- 172.21.15.122:0/2932867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee64101de0 msgr2=0x7fee640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee64101de0 0x7fee640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 -- 172.21.15.122:0/2932867772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee64102790 msgr2=0x7fee64074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee64102790 0x7fee64074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.077+0000 7fee6a611640 1 -- 172.21.15.122:0/2932867772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee641aad80 con 0x7fee64106f10 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.078+0000 7fee6a611640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee64074f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fee5c031a20 tx=0x7fee5c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.078+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c003b70 con 0x7fee64106f10 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.078+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c003d10 con 0x7fee64106f10 2024-09-12T01:44:06.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.078+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee641aafb0 con 0x7fee64106f10 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.078+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c02fc40 con 0x7fee64106f10 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.078+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641ab4b0 con 0x7fee64106f10 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.079+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee641032c0 con 0x7fee64106f10 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.080+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fee5c004b40 con 0x7fee64106f10 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.080+0000 7fee56ffd640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fee500a2c00 0x7fee500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.081+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fee5c0efdc0 con 0x7fee64106f10 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.081+0000 7fee69e10640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fee500a2c00 0x7fee500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.081+0000 7fee69e10640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fee500a2c00 0x7fee500a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee58009b60 tx=0x7fee58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:06.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.084+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee5c0bd250 con 0x7fee64106f10 2024-09-12T01:44:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.283+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee641021c0 con 0x7fee64106f10 2024-09-12T01:44:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.283+0000 7fee56ffd640 1 -- 172.21.15.122:0/2932867772 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fee5c0bcb50 con 0x7fee64106f10 2024-09-12T01:44:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:06.284 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":5,"btime":"2024-09-12T01:44:04:504501+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T01:44:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 5 2024-09-12T01:44:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fee500a2c00 msgr2=0x7fee500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fee500a2c00 0x7fee500a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee58009b60 tx=0x7fee58005f90 comp rx=0 tx=0).stop 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 msgr2=0x7fee64074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee64074f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fee5c031a20 tx=0x7fee5c031a50 comp rx=0 tx=0).stop 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 shutdown_connections 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fee500a2c00 0x7fee500a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee64106f10 0x7fee64074f70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee64102790 0x7fee64074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 --2- 172.21.15.122:0/2932867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee64101de0 0x7fee640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.285+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 >> 172.21.15.122:0/2932867772 conn(0x7fee640fd8c0 msgr2=0x7fee640feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.286+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 shutdown_connections 2024-09-12T01:44:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:06.286+0000 7fee6c074640 1 -- 172.21.15.122:0/2932867772 wait complete. 2024-09-12T01:44:06.336 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 systemd[1]: Starting Ceph mds.e for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:06.599 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:06 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2932867772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:06.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:06 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2932867772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:06.615 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 podman[64389]: 2024-09-12 01:44:06.25642938 +0000 UTC m=+0.016103445 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:06.615 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 podman[64389]: 2024-09-12 01:44:06.383640606 +0000 UTC m=+0.143314725 container create ac2e85ffdfb7b5d82758d38867ca97708b75eb2dadcf999019f9966ba122d071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T01:44:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:06 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2932867772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:07.115 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 podman[64389]: 2024-09-12 01:44:06.653376756 +0000 UTC m=+0.413050813 container init ac2e85ffdfb7b5d82758d38867ca97708b75eb2dadcf999019f9966ba122d071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T01:44:07.115 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 podman[64389]: 2024-09-12 01:44:06.656469899 +0000 UTC m=+0.416143957 container start ac2e85ffdfb7b5d82758d38867ca97708b75eb2dadcf999019f9966ba122d071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T01:44:07.115 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 ceph-mds[64408]: -- 172.21.15.151:0/2438213830 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1551d7860 con 0x55c1551b5000 2024-09-12T01:44:07.115 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e[64404]: starting mds.e at 2024-09-12T01:44:07.116 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 bash[64389]: ac2e85ffdfb7b5d82758d38867ca97708b75eb2dadcf999019f9966ba122d071 2024-09-12T01:44:07.116 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 01:44:06 smithi151 systemd[1]: Started Ceph mds.e for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:07.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:07.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:07.525 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:07.526 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:07 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:07.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:07.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:07.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:07.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:07 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:07.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:07 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:08.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:08 smithi183 ceph-mon[35127]: Deploying daemon mds.f on smithi183 2024-09-12T01:44:08.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:08 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6834/2236567181,v1:172.21.15.151:6835/2236567181] up:boot 2024-09-12T01:44:08.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:08 smithi183 ceph-mon[35127]: fsmap 5 up:standby 2024-09-12T01:44:08.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:08 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:44:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:08 smithi151 ceph-mon[35229]: Deploying daemon mds.f on smithi183 2024-09-12T01:44:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:08 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6834/2236567181,v1:172.21.15.151:6835/2236567181] up:boot 2024-09-12T01:44:08.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:08 smithi151 ceph-mon[35229]: fsmap 5 up:standby 2024-09-12T01:44:08.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:08 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:44:08.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:08 smithi122 ceph-mon[30925]: Deploying daemon mds.f on smithi183 2024-09-12T01:44:08.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:08 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6834/2236567181,v1:172.21.15.151:6835/2236567181] up:boot 2024-09-12T01:44:08.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:08 smithi122 ceph-mon[30925]: fsmap 5 up:standby 2024-09-12T01:44:08.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:44:08.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:08 smithi183 systemd[1]: Starting Ceph mds.f for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:09.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:09.234 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:08 smithi183 podman[64421]: 2024-09-12 01:44:08.866600945 +0000 UTC m=+0.015540204 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:09.235 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:08 smithi183 podman[64421]: 2024-09-12 01:44:08.991123498 +0000 UTC m=+0.140062758 container create 00a7be5dffbc5a959c26c5b106e74b671a93fa8ab5fe799a25dd8c5021ea578f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T01:44:09.235 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:09 smithi183 podman[64421]: 2024-09-12 01:44:09.235506091 +0000 UTC m=+0.384445351 container init 00a7be5dffbc5a959c26c5b106e74b671a93fa8ab5fe799a25dd8c5021ea578f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T01:44:09.292 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:09.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:09 smithi151 ceph-mon[35229]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:09.681 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:09 smithi122 ceph-mon[30925]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:09.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:09 smithi183 ceph-mon[35127]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:09.702 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:09 smithi183 podman[64421]: 2024-09-12 01:44:09.238425388 +0000 UTC m=+0.387364649 container start 00a7be5dffbc5a959c26c5b106e74b671a93fa8ab5fe799a25dd8c5021ea578f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:44:09.702 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:09 smithi183 ceph-mds[64439]: -- 172.21.15.183:0/793685622 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56359915b860 con 0x563599139400 2024-09-12T01:44:09.703 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: starting mds.f at 2024-09-12T01:44:09.703 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:09 smithi183 bash[64421]: 00a7be5dffbc5a959c26c5b106e74b671a93fa8ab5fe799a25dd8c5021ea578f 2024-09-12T01:44:09.703 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 01:44:09 smithi183 systemd[1]: Started Ceph mds.f for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 -- 172.21.15.122:0/2762964819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 msgr2=0x7fa594106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/2762964819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa594106e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009900 tx=0x7fa58c02a410 comp rx=0 tx=0).stop 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 -- 172.21.15.122:0/2762964819 shutdown_connections 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/2762964819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa594101420 0x7fa59410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/2762964819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa594100a80 0x7fa594100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/2762964819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa594106e60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 -- 172.21.15.122:0/2762964819 >> 172.21.15.122:0/2762964819 conn(0x7fa5940fc7a0 msgr2=0x7fa5940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 -- 172.21.15.122:0/2762964819 shutdown_connections 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 -- 172.21.15.122:0/2762964819 wait complete. 2024-09-12T01:44:09.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 Processor -- start 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.762+0000 7fa59b6e0640 1 -- start start 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59b6e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa594100a80 0x7fa59410e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59b6e0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa594101420 0x7fa59410e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59b6e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa59410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59b6e0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa594111f60 con 0x7fa594106a80 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59b6e0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa594111de0 con 0x7fa594101420 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59b6e0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941120e0 con 0x7fa594100a80 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa59410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa59410eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50874/0 (socket says 172.21.15.122:50874) 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 -- 172.21.15.122:0/4262324282 learned_addr learned my addr 172.21.15.122:0/4262324282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa598c7b640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa594101420 0x7fa59410e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa59947c640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa594100a80 0x7fa59410e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:09.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 -- 172.21.15.122:0/4262324282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa594100a80 msgr2=0x7fa59410e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa594100a80 0x7fa59410e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 -- 172.21.15.122:0/4262324282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa594101420 msgr2=0x7fa59410e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa594101420 0x7fa59410e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 -- 172.21.15.122:0/4262324282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5941a6900 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa599c7d640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa59410eb60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa58400d750 tx=0x7fa58400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.763+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584017070 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.764+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5941a6bf0 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.764+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa584005500 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.764+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584004d90 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.764+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5941a7180 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.764+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa594100ee0 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.766+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa584004f30 con 0x7fa594106a80 2024-09-12T01:44:09.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.766+0000 7fa5927fc640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa57c0a2b80 0x7fa57c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:09.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.766+0000 7fa59947c640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa57c0a2b80 0x7fa57c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:09.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.766+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa5840ce890 con 0x7fa594106a80 2024-09-12T01:44:09.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.767+0000 7fa59947c640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa57c0a2b80 0x7fa57c0a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005ab0 tx=0x7fa58c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:09.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.769+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa58409bc90 con 0x7fa594106a80 2024-09-12T01:44:09.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.970+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa59402d7c0 con 0x7fa594106a80 2024-09-12T01:44:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.971+0000 7fa5927fc640 1 -- 172.21.15.122:0/4262324282 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7fa58409b590 con 0x7fa594106a80 2024-09-12T01:44:09.971 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:09.971 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":7,"btime":"2024-09-12T01:44:09:342461+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-12T01:44:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 7 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa57c0a2b80 msgr2=0x7fa57c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa57c0a2b80 0x7fa57c0a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005ab0 tx=0x7fa58c005a20 comp rx=0 tx=0).stop 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 msgr2=0x7fa59410eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa59410eb60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa58400d750 tx=0x7fa58400dc10 comp rx=0 tx=0).stop 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 shutdown_connections 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa57c0a2b80 0x7fa57c0a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa594106a80 0x7fa59410eb60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa594101420 0x7fa59410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 --2- 172.21.15.122:0/4262324282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa594100a80 0x7fa59410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 >> 172.21.15.122:0/4262324282 conn(0x7fa5940fc7a0 msgr2=0x7fa5940fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 shutdown_connections 2024-09-12T01:44:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:09.973+0000 7fa59b6e0640 1 -- 172.21.15.122:0/4262324282 wait complete. 2024-09-12T01:44:10.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6834/1190566748,v1:172.21.15.183:6835/1190566748] up:boot 2024-09-12T01:44:10.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: fsmap 6 up:standby 2024-09-12T01:44:10.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: Deploying daemon mds.c on smithi183 2024-09-12T01:44:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:10 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4262324282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:10.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6834/1190566748,v1:172.21.15.183:6835/1190566748] up:boot 2024-09-12T01:44:10.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: fsmap 6 up:standby 2024-09-12T01:44:10.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:44:10.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: Deploying daemon mds.c on smithi183 2024-09-12T01:44:10.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:10 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4262324282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6834/1190566748,v1:172.21.15.183:6835/1190566748] up:boot 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: fsmap 6 up:standby 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:10.653 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:10.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:10.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:10.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: Deploying daemon mds.c on smithi183 2024-09-12T01:44:10.654 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:10 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4262324282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:11 smithi183 ceph-mon[35127]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:11.453 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 systemd[1]: Starting Ceph mds.c for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:11 smithi151 ceph-mon[35229]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:11 smithi122 ceph-mon[30925]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:11.875 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 podman[64719]: 2024-09-12 01:44:11.500621693 +0000 UTC m=+0.015875314 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:11.875 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 podman[64719]: 2024-09-12 01:44:11.615239069 +0000 UTC m=+0.130492683 container create 9902db5727df5c4ee0b641a7fb98ab6aa39b8c2d4927b5fbb085158386dcb3cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T01:44:11.875 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 podman[64719]: 2024-09-12 01:44:11.876270219 +0000 UTC m=+0.391523840 container init 9902db5727df5c4ee0b641a7fb98ab6aa39b8c2d4927b5fbb085158386dcb3cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T01:44:12.202 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 podman[64719]: 2024-09-12 01:44:11.879368029 +0000 UTC m=+0.394621644 container start 9902db5727df5c4ee0b641a7fb98ab6aa39b8c2d4927b5fbb085158386dcb3cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T01:44:12.203 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 ceph-mds[64739]: -- 172.21.15.183:0/2218363032 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618e4ff3860 con 0x5618e4fd1000 2024-09-12T01:44:12.203 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c[64735]: starting mds.c at 2024-09-12T01:44:12.203 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 bash[64719]: 9902db5727df5c4ee0b641a7fb98ab6aa39b8c2d4927b5fbb085158386dcb3cc 2024-09-12T01:44:12.203 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 01:44:11 smithi183 systemd[1]: Started Ceph mds.c for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:12.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:12.950 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:13.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: Deploying daemon mds.h on smithi151 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] up:boot 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: fsmap 7 up:standby 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:44:13.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:13 smithi122 ceph-mon[30925]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:13.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:13.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:13.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:13.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: Deploying daemon mds.h on smithi151 2024-09-12T01:44:13.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] up:boot 2024-09-12T01:44:13.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: fsmap 7 up:standby 2024-09-12T01:44:13.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:44:13.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:13 smithi151 ceph-mon[35229]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:13.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 -- 172.21.15.122:0/4242602691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101200 msgr2=0x7f1408101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 --2- 172.21.15.122:0/4242602691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101200 0x7f1408101660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f140402a410 comp rx=0 tx=0).stop 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 -- 172.21.15.122:0/4242602691 shutdown_connections 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 --2- 172.21.15.122:0/4242602691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101ba0 0x7f140810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 --2- 172.21.15.122:0/4242602691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101200 0x7f1408101660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 --2- 172.21.15.122:0/4242602691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408105ad0 0x7f1408105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 -- 172.21.15.122:0/4242602691 >> 172.21.15.122:0/4242602691 conn(0x7f1408077210 msgr2=0x7f14080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 -- 172.21.15.122:0/4242602691 shutdown_connections 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 -- 172.21.15.122:0/4242602691 wait complete. 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 Processor -- start 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.411+0000 7f140f4f1640 1 -- start start 2024-09-12T01:44:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140f4f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101200 0x7f1408199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140f4f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 0x7f1408199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140f4f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408105ad0 0x7f14081a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140f4f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1408110ef0 con 0x7f1408101ba0 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140f4f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1408110d70 con 0x7f1408101200 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140f4f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1408111070 con 0x7f1408105ad0 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 0x7f1408199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 0x7f1408199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33838/0 (socket says 172.21.15.122:33838) 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 -- 172.21.15.122:0/550075176 learned_addr learned my addr 172.21.15.122:0/550075176 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140d28d640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101200 0x7f1408199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140da8e640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408105ad0 0x7f14081a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 -- 172.21.15.122:0/550075176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408105ad0 msgr2=0x7f14081a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:13.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408105ad0 0x7f14081a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 -- 172.21.15.122:0/550075176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101200 msgr2=0x7f1408199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101200 0x7f1408199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 -- 172.21.15.122:0/550075176 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14081a48c0 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140d28d640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101200 0x7f1408199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f140ca8c640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 0x7f1408199840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1404006010 tx=0x7f1404003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.412+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140403f070 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.413+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1404031c30 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.413+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404035030 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.413+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14081a4a90 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.413+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14081a4ee0 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.414+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1408101660 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.415+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1404042070 con 0x7f1408101ba0 2024-09-12T01:44:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.415+0000 7f13fe7fc640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f40a2b80 0x7f13f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.415+0000 7f140d28d640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f40a2b80 0x7f13f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.416+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f14040ef8d0 con 0x7f1408101ba0 2024-09-12T01:44:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.416+0000 7f140d28d640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f40a2b80 0x7f13f40a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005d70 tx=0x7f13f8005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.419+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1404031de0 con 0x7f1408101ba0 2024-09-12T01:44:13.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: Deploying daemon mds.h on smithi151 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] up:boot 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: fsmap 7 up:standby 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:44:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:13 smithi183 ceph-mon[35127]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:13.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.622+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14081a5110 con 0x7f1408101ba0 2024-09-12T01:44:13.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.622+0000 7f13fe7fc640 1 -- 172.21.15.122:0/550075176 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f14040bccd0 con 0x7f1408101ba0 2024-09-12T01:44:13.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:13.623 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-12T01:44:12:355362+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T01:44:13.623 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-12T01:44:13.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f40a2b80 msgr2=0x7f13f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f40a2b80 0x7f13f40a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005d70 tx=0x7f13f8005d00 comp rx=0 tx=0).stop 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 msgr2=0x7f1408199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 0x7f1408199840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1404006010 tx=0x7f1404003c10 comp rx=0 tx=0).stop 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 shutdown_connections 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f40a2b80 0x7f13f40a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.624+0000 7f140f4f1640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408105ad0 0x7f14081a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.625+0000 7f140f4f1640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408101ba0 0x7f1408199840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.625+0000 7f140f4f1640 1 --2- 172.21.15.122:0/550075176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101200 0x7f1408199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.625+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 >> 172.21.15.122:0/550075176 conn(0x7f1408077210 msgr2=0x7f1408103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.625+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 shutdown_connections 2024-09-12T01:44:13.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:13.625+0000 7f140f4f1640 1 -- 172.21.15.122:0/550075176 wait complete. 2024-09-12T01:44:14.341 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 systemd[1]: Starting Ceph mds.h for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:14.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:14 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/550075176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:14.615 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 podman[64690]: 2024-09-12 01:44:14.2444121 +0000 UTC m=+0.016168990 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:14.615 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 podman[64690]: 2024-09-12 01:44:14.380809298 +0000 UTC m=+0.152566175 container create 7c09818ca91006a61d36727367f1e06c838e5dca42a23ec4daaaaedc24afc494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T01:44:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:14 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/550075176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:14.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:14 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/550075176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:15.115 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 podman[64690]: 2024-09-12 01:44:14.650406631 +0000 UTC m=+0.422163508 container init 7c09818ca91006a61d36727367f1e06c838e5dca42a23ec4daaaaedc24afc494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T01:44:15.116 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 podman[64690]: 2024-09-12 01:44:14.653197495 +0000 UTC m=+0.424954374 container start 7c09818ca91006a61d36727367f1e06c838e5dca42a23ec4daaaaedc24afc494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:44:15.116 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 ceph-mds[64709]: -- 172.21.15.151:0/3182363658 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56004d7b7860 con 0x56004d794800 2024-09-12T01:44:15.116 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: starting mds.h at 2024-09-12T01:44:15.116 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 bash[64690]: 7c09818ca91006a61d36727367f1e06c838e5dca42a23ec4daaaaedc24afc494 2024-09-12T01:44:15.116 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:44:14 smithi151 systemd[1]: Started Ceph mds.h for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:15.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:15.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:15.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:15.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:15 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:15 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:15 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:16.354 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:16.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:16 smithi183 ceph-mon[35127]: Deploying daemon mds.i on smithi183 2024-09-12T01:44:16.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:16 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:boot 2024-09-12T01:44:16.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:16 smithi183 ceph-mon[35127]: fsmap 8 up:standby 2024-09-12T01:44:16.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:16 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:44:16.615 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:16.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:16 smithi122 ceph-mon[30925]: Deploying daemon mds.i on smithi183 2024-09-12T01:44:16.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:16 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:boot 2024-09-12T01:44:16.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:16 smithi122 ceph-mon[30925]: fsmap 8 up:standby 2024-09-12T01:44:16.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:16 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:44:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:16 smithi151 ceph-mon[35229]: Deploying daemon mds.i on smithi183 2024-09-12T01:44:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:16 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:boot 2024-09-12T01:44:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:16 smithi151 ceph-mon[35229]: fsmap 8 up:standby 2024-09-12T01:44:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:16 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 -- 172.21.15.122:0/2550387788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0101420 msgr2=0x7f86f010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/2550387788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0101420 0x7f86f010d950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009cd0 tx=0x7f86e802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 -- 172.21.15.122:0/2550387788 shutdown_connections 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/2550387788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0101420 0x7f86f010d950 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/2550387788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f0100a80 0x7f86f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/2550387788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0106a80 0x7f86f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 -- 172.21.15.122:0/2550387788 >> 172.21.15.122:0/2550387788 conn(0x7f86f00fc7a0 msgr2=0x7f86f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 -- 172.21.15.122:0/2550387788 shutdown_connections 2024-09-12T01:44:17.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.088+0000 7f86f4bf9640 1 -- 172.21.15.122:0/2550387788 wait complete. 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 Processor -- start 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 -- start start 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 0x7f86f0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0101420 0x7f86f0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f0106a80 0x7f86f019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0110ea0 con 0x7f86f0100a80 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0110d20 con 0x7f86f0106a80 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86f4bf9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111020 con 0x7f86f0101420 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86ee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 0x7f86f0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86ee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 0x7f86f0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33852/0 (socket says 172.21.15.122:33852) 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86ee59c640 1 -- 172.21.15.122:0/4214135057 learned_addr learned my addr 172.21.15.122:0/4214135057 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86eed9d640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f0106a80 0x7f86f019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:17.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86edd9b640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0101420 0x7f86f0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86ee59c640 1 -- 172.21.15.122:0/4214135057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0101420 msgr2=0x7f86f0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86ee59c640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0101420 0x7f86f0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.089+0000 7f86ee59c640 1 -- 172.21.15.122:0/4214135057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f0106a80 msgr2=0x7f86f019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86ee59c640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f0106a80 0x7f86f019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86ee59c640 1 -- 172.21.15.122:0/4214135057 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f019d3e0 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86ee59c640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 0x7f86f0198430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b4a0 tx=0x7f86e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0014070 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0010af0 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f019d6d0 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0017d10 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.090+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a4fb0 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.091+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0100ee0 con 0x7f86f0100a80 2024-09-12T01:44:17.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.092+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f86e0017070 con 0x7f86f0100a80 2024-09-12T01:44:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.093+0000 7f86e77fe640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86dc0a2b80 0x7f86dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.093+0000 7f86edd9b640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86dc0a2b80 0x7f86dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.093+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f86e00cdf70 con 0x7f86f0100a80 2024-09-12T01:44:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.093+0000 7f86edd9b640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86dc0a2b80 0x7f86dc0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005fd0 tx=0x7f86d8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.097+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86e009b370 con 0x7f86f0100a80 2024-09-12T01:44:17.203 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 podman[65020]: 2024-09-12 01:44:17.153657602 +0000 UTC m=+0.118303280 container create 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T01:44:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.297+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86f01a5250 con 0x7f86f0100a80 2024-09-12T01:44:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.298+0000 7f86e77fe640 1 -- 172.21.15.122:0/4214135057 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f86e009ac70 con 0x7f86f0100a80 2024-09-12T01:44:17.298 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":9,"btime":"2024-09-12T01:44:15:362635+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T01:44:17.299 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 9 2024-09-12T01:44:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86dc0a2b80 msgr2=0x7f86dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86dc0a2b80 0x7f86dc0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005fd0 tx=0x7f86d8005850 comp rx=0 tx=0).stop 2024-09-12T01:44:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 msgr2=0x7f86f0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 0x7f86f0198430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b4a0 tx=0x7f86e000b960 comp rx=0 tx=0).stop 2024-09-12T01:44:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 shutdown_connections 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86dc0a2b80 0x7f86dc0a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f0106a80 0x7f86f019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0101420 0x7f86f0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 --2- 172.21.15.122:0/4214135057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f0100a80 0x7f86f0198430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 >> 172.21.15.122:0/4214135057 conn(0x7f86f00fc7a0 msgr2=0x7f86f01047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 shutdown_connections 2024-09-12T01:44:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:17.300+0000 7f86f4bf9640 1 -- 172.21.15.122:0/4214135057 wait complete. 2024-09-12T01:44:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:17 smithi183 ceph-mon[35127]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:17 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4214135057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:17.703 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 podman[65020]: 2024-09-12 01:44:17.397885777 +0000 UTC m=+0.362531413 container init 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T01:44:17.703 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 podman[65020]: 2024-09-12 01:44:17.40079276 +0000 UTC m=+0.365438396 container start 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T01:44:17.703 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 ceph-mds[65040]: -- 172.21.15.183:0/2507625086 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7aae81860 con 0x55e7aae5f000 2024-09-12T01:44:17.703 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i[65036]: starting mds.i at 2024-09-12T01:44:17.703 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 bash[65020]: 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 2024-09-12T01:44:17.703 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 01:44:17 smithi183 systemd[1]: Started Ceph mds.i for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:17.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:17 smithi122 ceph-mon[30925]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:17 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4214135057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:17 smithi151 ceph-mon[35229]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:17 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4214135057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: Deploying daemon mds.g on smithi122 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6838/3259334666,v1:172.21.15.183:6839/3259334666] up:boot 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: fsmap 9 up:standby 2024-09-12T01:44:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:18 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:44:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: Deploying daemon mds.g on smithi122 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6838/3259334666,v1:172.21.15.183:6839/3259334666] up:boot 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: fsmap 9 up:standby 2024-09-12T01:44:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:18 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:44:18.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: Deploying daemon mds.g on smithi122 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6838/3259334666,v1:172.21.15.183:6839/3259334666] up:boot 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: fsmap 9 up:standby 2024-09-12T01:44:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:18 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:44:19.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:19 smithi122 ceph-mon[30925]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:19 smithi151 ceph-mon[35229]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:19 smithi183 ceph-mon[35127]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:19.957 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:19.981 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:19 smithi122 podman[71483]: 2024-09-12 01:44:19.891522128 +0000 UTC m=+0.132377323 container create 4aa22800bb3467c5ba9ac5070abb9380e0380e2486e9f6cf9705d25c31633b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T01:44:20.239 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:20 smithi122 podman[71483]: 2024-09-12 01:44:20.12787351 +0000 UTC m=+0.368728681 container init 4aa22800bb3467c5ba9ac5070abb9380e0380e2486e9f6cf9705d25c31633b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T01:44:20.239 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:20 smithi122 podman[71483]: 2024-09-12 01:44:20.130993624 +0000 UTC m=+0.371848777 container start 4aa22800bb3467c5ba9ac5070abb9380e0380e2486e9f6cf9705d25c31633b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T01:44:20.239 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:20 smithi122 ceph-mds[71521]: -- 172.21.15.122:0/1080601221 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d55a85860 con 0x557d55a62800 2024-09-12T01:44:20.239 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g[71516]: starting mds.g at 2024-09-12T01:44:20.239 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:20 smithi122 bash[71483]: 4aa22800bb3467c5ba9ac5070abb9380e0380e2486e9f6cf9705d25c31633b87 2024-09-12T01:44:20.239 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 01:44:20 smithi122 systemd[1]: Started Ceph mds.g for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:20.297 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:20.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 -- 172.21.15.122:0/3288018114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac1027a0 msgr2=0x7fbbac1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 --2- 172.21.15.122:0/3288018114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac1027a0 0x7fbbac1067f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02a410 comp rx=0 tx=0).stop 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 -- 172.21.15.122:0/3288018114 shutdown_connections 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 --2- 172.21.15.122:0/3288018114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbac106f20 0x7fbbac10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 --2- 172.21.15.122:0/3288018114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac1027a0 0x7fbbac1067f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 --2- 172.21.15.122:0/3288018114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbac101df0 0x7fbbac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 -- 172.21.15.122:0/3288018114 >> 172.21.15.122:0/3288018114 conn(0x7fbbac0fd8f0 msgr2=0x7fbbac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 -- 172.21.15.122:0/3288018114 shutdown_connections 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.782+0000 7fbbb4343640 1 -- 172.21.15.122:0/3288018114 wait complete. 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 Processor -- start 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 -- start start 2024-09-12T01:44:20.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbac101df0 0x7fbbac1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbac1027a0 0x7fbbac1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 0x7fbbac19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111fa0 con 0x7fbbac106f20 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111e20 con 0x7fbbac1027a0 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb4343640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac112120 con 0x7fbbac101df0 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 0x7fbbac19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 0x7fbbac19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57578/0 (socket says 172.21.15.122:57578) 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 -- 172.21.15.122:0/1921596248 learned_addr learned my addr 172.21.15.122:0/1921596248 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb18de640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbac1027a0 0x7fbbac1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb20df640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbac101df0 0x7fbbac1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 -- 172.21.15.122:0/1921596248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbac101df0 msgr2=0x7fbbac1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:20.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbac101df0 0x7fbbac1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 -- 172.21.15.122:0/1921596248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbac1027a0 msgr2=0x7fbbac1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.783+0000 7fbbb28e0640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbac1027a0 0x7fbbac1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbbb28e0640 1 -- 172.21.15.122:0/1921596248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19d450 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbbb28e0640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 0x7fbbac19cd70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b970 tx=0x7fbba400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4019070 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba400cb40 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac19d740 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4007650 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.784+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a5020 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.785+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbac1021d0 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.786+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbba401c070 con 0x7fbbac106f20 2024-09-12T01:44:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.787+0000 7fbba37fe640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb980a2b80 0x7fbb980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:20.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.787+0000 7fbbb20df640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb980a2b80 0x7fbb980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:20.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.787+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbba40ce660 con 0x7fbbac106f20 2024-09-12T01:44:20.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.787+0000 7fbbb20df640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb980a2b80 0x7fbb980a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbba80058c0 tx=0x7fbba800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:20.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:20.791+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbba409b3f0 con 0x7fbbac106f20 2024-09-12T01:44:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.000+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbac10a1c0 con 0x7fbbac106f20 2024-09-12T01:44:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.001+0000 7fbba37fe640 1 -- 172.21.15.122:0/1921596248 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7fbba409f6f0 con 0x7fbbac106f20 2024-09-12T01:44:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:21.001 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":11,"btime":"2024-09-12T01:44:20:551203+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T01:44:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 11 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb980a2b80 msgr2=0x7fbb980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb980a2b80 0x7fbb980a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbba80058c0 tx=0x7fbba800a4c0 comp rx=0 tx=0).stop 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 msgr2=0x7fbbac19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 0x7fbbac19cd70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b970 tx=0x7fbba400be30 comp rx=0 tx=0).stop 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 shutdown_connections 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb980a2b80 0x7fbb980a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbac106f20 0x7fbbac19cd70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbac1027a0 0x7fbbac1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 --2- 172.21.15.122:0/1921596248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbac101df0 0x7fbbac1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 >> 172.21.15.122:0/1921596248 conn(0x7fbbac0fd8f0 msgr2=0x7fbbac0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 shutdown_connections 2024-09-12T01:44:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:21.003+0000 7fbbb4343640 1 -- 172.21.15.122:0/1921596248 wait complete. 2024-09-12T01:44:21.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: Deploying daemon mds.a on smithi122 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.122:6838/2056437657,v1:172.21.15.122:6839/2056437657] up:boot 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: fsmap 10 up:standby 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:44:21.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:21 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1921596248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: Deploying daemon mds.a on smithi122 2024-09-12T01:44:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.122:6838/2056437657,v1:172.21.15.122:6839/2056437657] up:boot 2024-09-12T01:44:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: fsmap 10 up:standby 2024-09-12T01:44:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:44:21.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:21 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1921596248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: Deploying daemon mds.a on smithi122 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.122:6838/2056437657,v1:172.21.15.122:6839/2056437657] up:boot 2024-09-12T01:44:21.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: fsmap 10 up:standby 2024-09-12T01:44:21.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:44:21.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:21 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1921596248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:23.200 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:22 smithi122 systemd[1]: Starting Ceph mds.a for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:23.460 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 podman[71898]: 2024-09-12 01:44:23.103458142 +0000 UTC m=+0.015250596 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:23.460 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 podman[71898]: 2024-09-12 01:44:23.21607325 +0000 UTC m=+0.127865704 container create b2eb61ef28b76fcdbe18f98e14210bb5555a112192abed87c7bdd39aab9d6f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T01:44:23.460 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 podman[71898]: 2024-09-12 01:44:23.460749946 +0000 UTC m=+0.372542400 container init b2eb61ef28b76fcdbe18f98e14210bb5555a112192abed87c7bdd39aab9d6f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T01:44:23.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:23 smithi122 ceph-mon[30925]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:23.739 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 podman[71898]: 2024-09-12 01:44:23.463831301 +0000 UTC m=+0.375623761 container start b2eb61ef28b76fcdbe18f98e14210bb5555a112192abed87c7bdd39aab9d6f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T01:44:23.739 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 ceph-mds[71918]: -- 172.21.15.122:0/234157226 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f71ea8b860 con 0x55f71ea68800 2024-09-12T01:44:23.739 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a[71914]: starting mds.a at 2024-09-12T01:44:23.739 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 bash[71898]: b2eb61ef28b76fcdbe18f98e14210bb5555a112192abed87c7bdd39aab9d6f2e 2024-09-12T01:44:23.739 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 01:44:23 smithi122 systemd[1]: Started Ceph mds.a for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:23.776 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:23 smithi183 ceph-mon[35127]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:23 smithi151 ceph-mon[35229]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:24.105 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:24.352 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.122:6840/1896979097,v1:172.21.15.122:6841/1896979097] up:boot 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: fsmap 11 up:standby 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:24 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 -- 172.21.15.122:0/3383694789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 msgr2=0x7fbd8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 --2- 172.21.15.122:0/3383694789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1021c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009ca0 tx=0x7fbd8402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 -- 172.21.15.122:0/3383694789 shutdown_connections 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 --2- 172.21.15.122:0/3383694789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 0x7fbd8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 --2- 172.21.15.122:0/3383694789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd8c102790 0x7fbd8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 --2- 172.21.15.122:0/3383694789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1021c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 -- 172.21.15.122:0/3383694789 >> 172.21.15.122:0/3383694789 conn(0x7fbd8c0fd8c0 msgr2=0x7fbd8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 -- 172.21.15.122:0/3383694789 shutdown_connections 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 -- 172.21.15.122:0/3383694789 wait complete. 2024-09-12T01:44:24.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.818+0000 7fbd9200b640 1 Processor -- start 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 -- start start 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd8c102790 0x7fbd8c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 0x7fbd8c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c112010 con 0x7fbd8c101de0 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c111e90 con 0x7fbd8c102790 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd9200b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c112190 con 0x7fbd8c106f10 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57622/0 (socket says 172.21.15.122:57622) 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 -- 172.21.15.122:0/259938349 learned_addr learned my addr 172.21.15.122:0/259938349 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8affd640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd8c102790 0x7fbd8c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:24.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8bfff640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 0x7fbd8c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 -- 172.21.15.122:0/259938349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 msgr2=0x7fbd8c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 0x7fbd8c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 -- 172.21.15.122:0/259938349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd8c102790 msgr2=0x7fbd8c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd8c102790 0x7fbd8c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8b7fe640 1 -- 172.21.15.122:0/259938349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c19d560 con 0x7fbd8c101de0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.819+0000 7fbd8bfff640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 0x7fbd8c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.820+0000 7fbd8b7fe640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1985b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbd840097b0 tx=0x7fbd84031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.820+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8402fba0 con 0x7fbd8c101de0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.820+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8402fd40 con 0x7fbd8c101de0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.820+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c19d7f0 con 0x7fbd8c101de0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.820+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8403b860 con 0x7fbd8c101de0 2024-09-12T01:44:24.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.820+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c1a5130 con 0x7fbd8c101de0 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.821+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c102240 con 0x7fbd8c101de0 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.822+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbd84030c30 con 0x7fbd8c101de0 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.823+0000 7fbd88ff9640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.823+0000 7fbd8affd640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.823+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbd840f02d0 con 0x7fbd8c101de0 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.823+0000 7fbd8affd640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbd780a2b80 0x7fbd780a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0ffde0 tx=0x7fbd74009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:24.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:24.827+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd84044380 con 0x7fbd8c101de0 2024-09-12T01:44:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.122:6840/1896979097,v1:172.21.15.122:6841/1896979097] up:boot 2024-09-12T01:44:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: fsmap 11 up:standby 2024-09-12T01:44:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:44:24.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:24.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:24.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:24 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.122:6840/1896979097,v1:172.21.15.122:6841/1896979097] up:boot 2024-09-12T01:44:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: fsmap 11 up:standby 2024-09-12T01:44:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:44:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T01:44:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T01:44:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:24 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.027+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd8c10a0f0 con 0x7fbd8c101de0 2024-09-12T01:44:25.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.028+0000 7fbd88ff9640 1 -- 172.21.15.122:0/259938349 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7fbd840bd6d0 con 0x7fbd8c101de0 2024-09-12T01:44:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:25.029 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":12,"btime":"2024-09-12T01:44:23:493940+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T01:44:25.029 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 12 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbd780a2b80 msgr2=0x7fbd780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbd780a2b80 0x7fbd780a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0ffde0 tx=0x7fbd74009110 comp rx=0 tx=0).stop 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 msgr2=0x7fbd8c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1985b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbd840097b0 tx=0x7fbd84031b10 comp rx=0 tx=0).stop 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 shutdown_connections 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd8c106f10 0x7fbd8c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd8c102790 0x7fbd8c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 --2- 172.21.15.122:0/259938349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd8c101de0 0x7fbd8c1985b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 >> 172.21.15.122:0/259938349 conn(0x7fbd8c0fd8c0 msgr2=0x7fbd8c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 shutdown_connections 2024-09-12T01:44:25.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:25.030+0000 7fbd9200b640 1 -- 172.21.15.122:0/259938349 wait complete. 2024-09-12T01:44:25.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:25 smithi122 ceph-mon[30925]: Deploying daemon mds.b on smithi151 2024-09-12T01:44:25.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:25 smithi122 ceph-mon[30925]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:25.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:25 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/259938349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:25 smithi151 ceph-mon[35229]: Deploying daemon mds.b on smithi151 2024-09-12T01:44:25.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:25 smithi151 ceph-mon[35229]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:25.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/259938349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:25.866 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:25 smithi151 systemd[1]: Starting Ceph mds.b for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T01:44:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:25 smithi183 ceph-mon[35127]: Deploying daemon mds.b on smithi151 2024-09-12T01:44:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:25 smithi183 ceph-mon[35127]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:25 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/259938349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:26.170 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:25 smithi151 podman[65012]: 2024-09-12 01:44:25.900642956 +0000 UTC m=+0.106117373 container create f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T01:44:26.171 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:25 smithi151 podman[65012]: 2024-09-12 01:44:25.810814167 +0000 UTC m=+0.016288595 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:44:26.171 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:26 smithi151 podman[65012]: 2024-09-12 01:44:26.14524414 +0000 UTC m=+0.350718557 container init f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:44:26.171 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:26 smithi151 podman[65012]: 2024-09-12 01:44:26.148035691 +0000 UTC m=+0.353510117 container start f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T01:44:26.615 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:26 smithi151 ceph-mds[65031]: -- 172.21.15.151:0/577085750 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e2b811860 con 0x563e2b7ef000 2024-09-12T01:44:26.615 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: starting mds.b at 2024-09-12T01:44:26.615 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:26 smithi151 bash[65012]: f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e 2024-09-12T01:44:26.615 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:26 smithi151 systemd[1]: Started Ceph mds.b for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:44:27.325 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:27.326 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] up:boot 2024-09-12T01:44:27.326 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: fsmap 12 up:standby 2024-09-12T01:44:27.326 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:27 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:44:27.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] up:boot 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: fsmap 12 up:standby 2024-09-12T01:44:27.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:27 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:44:27.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] up:boot 2024-09-12T01:44:27.732 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: fsmap 12 up:standby 2024-09-12T01:44:27.733 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:27 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:44:27.969 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:28.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 -- 172.21.15.122:0/2137165262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 msgr2=0x7fbf4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:28.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 --2- 172.21.15.122:0/2137165262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4810ea90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009ca0 tx=0x7fbf4002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 -- 172.21.15.122:0/2137165262 shutdown_connections 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 --2- 172.21.15.122:0/2137165262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4810ea90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 --2- 172.21.15.122:0/2137165262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 0x7fbf481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 --2- 172.21.15.122:0/2137165262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf48101de0 0x7fbf481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 -- 172.21.15.122:0/2137165262 >> 172.21.15.122:0/2137165262 conn(0x7fbf480fd8c0 msgr2=0x7fbf480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 -- 172.21.15.122:0/2137165262 shutdown_connections 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.438+0000 7fbf50245640 1 -- 172.21.15.122:0/2137165262 wait complete. 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 Processor -- start 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 -- start start 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf48101de0 0x7fbf481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:28.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 0x7fbf48198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48111fc0 con 0x7fbf48106f10 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48111e40 con 0x7fbf48102790 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf50245640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48112140 con 0x7fbf48101de0 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4dfe1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf48101de0 0x7fbf481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57640/0 (socket says 172.21.15.122:57640) 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 -- 172.21.15.122:0/2385267298 learned_addr learned my addr 172.21.15.122:0/2385267298 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4d7e0640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 0x7fbf48198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 -- 172.21.15.122:0/2385267298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf48101de0 msgr2=0x7fbf481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf48101de0 0x7fbf481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 -- 172.21.15.122:0/2385267298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 msgr2=0x7fbf48198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 0x7fbf48198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4e7e2640 1 -- 172.21.15.122:0/2385267298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4819d380 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.439+0000 7fbf4d7e0640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 0x7fbf48198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.440+0000 7fbf4e7e2640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4819cca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf40033040 tx=0x7fbf40003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.440+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40031650 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.440+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf400317f0 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.440+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4002fa40 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.440+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4819d610 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.440+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481a4f50 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.441+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf481021c0 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.442+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbf4002fc80 con 0x7fbf48106f10 2024-09-12T01:44:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.443+0000 7fbf3effd640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf340a2b80 0x7fbf340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:28.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.443+0000 7fbf4dfe1640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf340a2b80 0x7fbf340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:28.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.443+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbf400effe0 con 0x7fbf48106f10 2024-09-12T01:44:28.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.443+0000 7fbf4dfe1640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf340a2b80 0x7fbf340a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf440058c0 tx=0x7fbf4400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:28.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.448+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf400bd3c0 con 0x7fbf48106f10 2024-09-12T01:44:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.673+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf4810a150 con 0x7fbf48106f10 2024-09-12T01:44:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.674+0000 7fbf3effd640 1 -- 172.21.15.122:0/2385267298 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7fbf400bccc0 con 0x7fbf48106f10 2024-09-12T01:44:28.674 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:28.674 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"btime":"2024-09-12T01:44:26:624110+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T01:44:28.674 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-09-12T01:44:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf340a2b80 msgr2=0x7fbf340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf340a2b80 0x7fbf340a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf440058c0 tx=0x7fbf4400a250 comp rx=0 tx=0).stop 2024-09-12T01:44:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 msgr2=0x7fbf4819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4819cca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf40033040 tx=0x7fbf40003f10 comp rx=0 tx=0).stop 2024-09-12T01:44:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 shutdown_connections 2024-09-12T01:44:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf340a2b80 0x7fbf340a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf48106f10 0x7fbf4819cca0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48102790 0x7fbf48198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 --2- 172.21.15.122:0/2385267298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf48101de0 0x7fbf481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:28.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 >> 172.21.15.122:0/2385267298 conn(0x7fbf480fd8c0 msgr2=0x7fbf480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:28.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 shutdown_connections 2024-09-12T01:44:28.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:28.677+0000 7fbf50245640 1 -- 172.21.15.122:0/2385267298 wait complete. 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:44:28.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:28.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:28 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2385267298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:29.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:44:29.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:44:29.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:29.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:28 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2385267298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:29.204 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:28 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2385267298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:29.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T01:44:30.137 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:30.576 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.577 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:30 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 -- 172.21.15.122:0/3156870062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 msgr2=0x7f6ac4106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 --2- 172.21.15.122:0/3156870062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac4106f90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009ca0 tx=0x7f6ab402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 -- 172.21.15.122:0/3156870062 shutdown_connections 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 --2- 172.21.15.122:0/3156870062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ac4100970 0x7f6ac410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 --2- 172.21.15.122:0/3156870062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac40fffd0 0x7f6ac4100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 --2- 172.21.15.122:0/3156870062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac4106f90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 -- 172.21.15.122:0/3156870062 >> 172.21.15.122:0/3156870062 conn(0x7f6ac40fbee0 msgr2=0x7f6ac40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:30.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 -- 172.21.15.122:0/3156870062 shutdown_connections 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.655+0000 7f6ac905d640 1 -- 172.21.15.122:0/3156870062 wait complete. 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 Processor -- start 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 -- start start 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ac40fffd0 0x7f6ac4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac4100970 0x7f6ac4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac41120c0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac4111f40 con 0x7f6ac4100970 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac905d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac4112240 con 0x7f6ac40fffd0 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47990/0 (socket says 172.21.15.122:47990) 2024-09-12T01:44:30.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 -- 172.21.15.122:0/1652083434 learned_addr learned my addr 172.21.15.122:0/1652083434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac2d9d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ac40fffd0 0x7f6ac4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac259c640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac4100970 0x7f6ac4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 -- 172.21.15.122:0/1652083434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ac40fffd0 msgr2=0x7f6ac4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ac40fffd0 0x7f6ac4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 -- 172.21.15.122:0/1652083434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac4100970 msgr2=0x7f6ac4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac4100970 0x7f6ac4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac359e640 1 -- 172.21.15.122:0/1652083434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac419d510 con 0x7f6ac4106bb0 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.656+0000 7f6ac259c640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac4100970 0x7f6ac4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.657+0000 7f6ac359e640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac419ce30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00e970 tx=0x7f6abc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:30.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.657+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00ccc0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.657+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac419d800 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.657+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6abc005500 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.657+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc010690 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.657+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41a50e0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.658+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac4100430 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.659+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6abc0046c0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.660+0000 7f6aabfff640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6aac0a2b80 0x7f6aac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.660+0000 7f6ac2d9d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6aac0a2b80 0x7f6aac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.660+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f6abc0cf1f0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.660+0000 7f6ac2d9d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6aac0a2b80 0x7f6aac0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4006010 tx=0x7f6ab4009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:30.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.663+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6abc0541b0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:30 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.864+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6ac41a5370 con 0x7f6ac4106bb0 2024-09-12T01:44:30.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.865+0000 7f6aabfff640 1 -- 172.21.15.122:0/1652083434 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f6abc09c5f0 con 0x7f6ac4106bb0 2024-09-12T01:44:30.865 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:30.865 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-12T01:44:30.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:44:30.865 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-12T01:44:30.866 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-12T01:44:30.866 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:44:30.866 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-12T01:44:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:30 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6aac0a2b80 msgr2=0x7f6aac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6aac0a2b80 0x7f6aac0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4006010 tx=0x7f6ab4009360 comp rx=0 tx=0).stop 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 msgr2=0x7f6ac419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac419ce30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00e970 tx=0x7f6abc00ee30 comp rx=0 tx=0).stop 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 shutdown_connections 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6aac0a2b80 0x7f6aac0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ac4106bb0 0x7f6ac419ce30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac4100970 0x7f6ac4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 --2- 172.21.15.122:0/1652083434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ac40fffd0 0x7f6ac4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 >> 172.21.15.122:0/1652083434 conn(0x7f6ac40fbee0 msgr2=0x7f6ac40fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 shutdown_connections 2024-09-12T01:44:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:30.868+0000 7f6ac905d640 1 -- 172.21.15.122:0/1652083434 wait complete. 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1652083434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:44:31.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.558 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T01:44:31.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T01:44:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1652083434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:44:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:44:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.802 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1652083434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T01:44:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:31.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:44:31.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:44:31.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 -- 172.21.15.122:0/984700845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 msgr2=0x7fc7ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/984700845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac100ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009930 tx=0x7fc7a002a420 comp rx=0 tx=0).stop 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 -- 172.21.15.122:0/984700845 shutdown_connections 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/984700845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 0x7fc7ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/984700845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac100ee0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/984700845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7ac106a80 0x7fc7ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 -- 172.21.15.122:0/984700845 >> 172.21.15.122:0/984700845 conn(0x7fc7ac0fc7a0 msgr2=0x7fc7ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:32.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 -- 172.21.15.122:0/984700845 shutdown_connections 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 -- 172.21.15.122:0/984700845 wait complete. 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 Processor -- start 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.268+0000 7fc7b13e3640 1 -- start start 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 0x7fc7ac199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7ac106a80 0x7fc7ac1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac110e80 con 0x7fc7ac100a80 2024-09-12T01:44:32.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac110d00 con 0x7fc7ac106a80 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac111000 con 0x7fc7ac101420 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48000/0 (socket says 172.21.15.122:48000) 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 -- 172.21.15.122:0/2115551661 learned_addr learned my addr 172.21.15.122:0/2115551661 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7ab7fe640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7ac106a80 0x7fc7ac1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aa7fc640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 0x7fc7ac199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 -- 172.21.15.122:0/2115551661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 msgr2=0x7fc7ac199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 0x7fc7ac199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 -- 172.21.15.122:0/2115551661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7ac106a80 msgr2=0x7fc7ac1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7ac106a80 0x7fc7ac1a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 -- 172.21.15.122:0/2115551661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac1a48d0 con 0x7fc7ac100a80 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aa7fc640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 0x7fc7ac199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7aaffd640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac199310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c890 tx=0x7fc79c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004220 con 0x7fc7ac100a80 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1a4aa0 con 0x7fc7ac100a80 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.269+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c0043c0 con 0x7fc7ac100a80 2024-09-12T01:44:32.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.270+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004a00 con 0x7fc7ac100a80 2024-09-12T01:44:32.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.270+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac1a4fe0 con 0x7fc7ac100a80 2024-09-12T01:44:32.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.270+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac100ee0 con 0x7fc7ac100a80 2024-09-12T01:44:32.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.272+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc79c0079b0 con 0x7fc7ac100a80 2024-09-12T01:44:32.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.272+0000 7fc793fff640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc7940a2b80 0x7fc7940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:32.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.272+0000 7fc7aa7fc640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc7940a2b80 0x7fc7940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:32.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.272+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc79c0ce830 con 0x7fc7ac100a80 2024-09-12T01:44:32.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.273+0000 7fc7aa7fc640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc7940a2b80 0x7fc7940a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0000c00 tx=0x7fc7a0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:32.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.276+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc79c014030 con 0x7fc7ac100a80 2024-09-12T01:44:32.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.446+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc7ac108ff0 con 0x7fc7ac100a80 2024-09-12T01:44:32.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.734+0000 7fc793fff640 1 -- 172.21.15.122:0/2115551661 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fc79c09bc30 con 0x7fc7ac100a80 2024-09-12T01:44:32.735 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-12T01:44:32.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc7940a2b80 msgr2=0x7fc7940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:32.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc7940a2b80 0x7fc7940a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0000c00 tx=0x7fc7a0032c70 comp rx=0 tx=0).stop 2024-09-12T01:44:32.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 msgr2=0x7fc7ac199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:32.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac199310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c890 tx=0x7fc79c00cd50 comp rx=0 tx=0).stop 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 shutdown_connections 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc7940a2b80 0x7fc7940a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7ac106a80 0x7fc7ac1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ac101420 0x7fc7ac199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 --2- 172.21.15.122:0/2115551661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac100a80 0x7fc7ac199310 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 >> 172.21.15.122:0/2115551661 conn(0x7fc7ac0fc7a0 msgr2=0x7fc7ac0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 shutdown_connections 2024-09-12T01:44:32.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:32.737+0000 7fc7b13e3640 1 -- 172.21.15.122:0/2115551661 wait complete. 2024-09-12T01:44:32.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:32 smithi122 ceph-mon[30925]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:32.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:32 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2115551661' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:44:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:32 smithi151 ceph-mon[35229]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:32 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2115551661' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:44:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:32 smithi183 ceph-mon[35127]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:32 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2115551661' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T01:44:33.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T01:44:33.653 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:34.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:33 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2115551661' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:44:34.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:33 smithi122 ceph-mon[30925]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T01:44:34.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:33 smithi122 ceph-mon[30925]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.089+0000 7f32118a2640 1 -- 172.21.15.122:0/517642816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c10c2f0 msgr2=0x7f320c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.089+0000 7f32118a2640 1 --2- 172.21.15.122:0/517642816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c10c2f0 0x7f320c10e790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3204009ca0 tx=0x7f320402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 -- 172.21.15.122:0/517642816 shutdown_connections 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 --2- 172.21.15.122:0/517642816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c10c2f0 0x7f320c10e790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 --2- 172.21.15.122:0/517642816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 0x7f320c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 --2- 172.21.15.122:0/517642816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f320c102df0 0x7f320c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 -- 172.21.15.122:0/517642816 >> 172.21.15.122:0/517642816 conn(0x7f320c0fc7d0 msgr2=0x7f320c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 -- 172.21.15.122:0/517642816 shutdown_connections 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 -- 172.21.15.122:0/517642816 wait complete. 2024-09-12T01:44:34.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 Processor -- start 2024-09-12T01:44:34.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.090+0000 7f32118a2640 1 -- start start 2024-09-12T01:44:34.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 0x7f320c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:34.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 0x7f320c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:34.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f320c10c2f0 0x7f320c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112170 con 0x7f320c102df0 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f320c111ff0 con 0x7f320c1037a0 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1122f0 con 0x7f320c10c2f0 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 0x7f320c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 0x7f320c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48018/0 (socket says 172.21.15.122:48018) 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 -- 172.21.15.122:0/4268333552 learned_addr learned my addr 172.21.15.122:0/4268333552 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320b7fe640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f320c10c2f0 0x7f320c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320a7fc640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 0x7f320c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 -- 172.21.15.122:0/4268333552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f320c10c2f0 msgr2=0x7f320c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f320c10c2f0 0x7f320c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 -- 172.21.15.122:0/4268333552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 msgr2=0x7f320c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:34.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 0x7f320c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 -- 172.21.15.122:0/4268333552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c19d5c0 con 0x7f320c102df0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320a7fc640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 0x7f320c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f320affd640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 0x7f320c198610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00b4a0 tx=0x7f31fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc014070 con 0x7f320c102df0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31fc010af0 con 0x7f320c102df0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c19d8b0 con 0x7f320c102df0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc017d10 con 0x7f320c102df0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.091+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c1a5190 con 0x7f320c102df0 2024-09-12T01:44:34.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.092+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320c06a790 con 0x7f320c102df0 2024-09-12T01:44:34.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.094+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f31fc017070 con 0x7f320c102df0 2024-09-12T01:44:34.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.095+0000 7f32108a0640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b80 0x7f31f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:34.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.095+0000 7f320a7fc640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b80 0x7f31f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:34.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.095+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f31fc0ce640 con 0x7f320c102df0 2024-09-12T01:44:34.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.095+0000 7f320a7fc640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b80 0x7f31f40a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f320c10d610 tx=0x7f3200005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:34.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.098+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31fc09b830 con 0x7f320c102df0 2024-09-12T01:44:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:33 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2115551661' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:44:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:33 smithi151 ceph-mon[35229]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T01:44:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:33 smithi151 ceph-mon[35229]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T01:44:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:33 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2115551661' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T01:44:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:33 smithi183 ceph-mon[35127]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T01:44:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:33 smithi183 ceph-mon[35127]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T01:44:34.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.266+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f320c1031d0 con 0x7f320c102df0 2024-09-12T01:44:34.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.738+0000 7f32108a0640 1 -- 172.21.15.122:0/4268333552 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f31fc09b130 con 0x7f320c102df0 2024-09-12T01:44:34.739 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-12T01:44:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b80 msgr2=0x7f31f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b80 0x7f31f40a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f320c10d610 tx=0x7f3200005e50 comp rx=0 tx=0).stop 2024-09-12T01:44:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 msgr2=0x7f320c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 0x7f320c198610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00b4a0 tx=0x7f31fc00b960 comp rx=0 tx=0).stop 2024-09-12T01:44:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 shutdown_connections 2024-09-12T01:44:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b80 0x7f31f40a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f320c10c2f0 0x7f320c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f320c1037a0 0x7f320c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 --2- 172.21.15.122:0/4268333552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f320c102df0 0x7f320c198610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 >> 172.21.15.122:0/4268333552 conn(0x7f320c0fc7d0 msgr2=0x7f320c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.741+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 shutdown_connections 2024-09-12T01:44:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:34.742+0000 7f32118a2640 1 -- 172.21.15.122:0/4268333552 wait complete. 2024-09-12T01:44:35.056 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:34 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4268333552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:44:35.056 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:34 smithi122 ceph-mon[30925]: pgmap v273: 33 pgs: 8 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:35.056 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:34 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4268333552' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:44:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:34 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4268333552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:44:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:34 smithi151 ceph-mon[35229]: pgmap v273: 33 pgs: 8 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:34 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4268333552' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:44:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:34 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4268333552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T01:44:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:34 smithi183 ceph-mon[35127]: pgmap v273: 33 pgs: 8 creating+peering, 24 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:34 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4268333552' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T01:44:35.376 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T01:44:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:35.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:35 smithi122 ceph-mon[30925]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:44:35.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:35 smithi122 ceph-mon[30925]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:44:35.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:35 smithi122 ceph-mon[30925]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:44:36.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 -- 172.21.15.122:0/3690551457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 msgr2=0x7f48e8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:36.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 --2- 172.21.15.122:0/3690551457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e8074970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009900 tx=0x7f48e402a480 comp rx=0 tx=0).stop 2024-09-12T01:44:36.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 -- 172.21.15.122:0/3690551457 shutdown_connections 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 --2- 172.21.15.122:0/3690551457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8112b90 0x7f48e8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 --2- 172.21.15.122:0/3690551457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e8074eb0 0x7f48e8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 --2- 172.21.15.122:0/3690551457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e8074970 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 -- 172.21.15.122:0/3690551457 >> 172.21.15.122:0/3690551457 conn(0x7f48e80fd8c0 msgr2=0x7f48e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 -- 172.21.15.122:0/3690551457 shutdown_connections 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 -- 172.21.15.122:0/3690551457 wait complete. 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 Processor -- start 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 -- start start 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e81a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.069+0000 7f48efd31640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8074eb0 0x7f48e81a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48efd31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e8112b90 0x7f48e819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:36.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48efd31640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48e8118550 con 0x7f48e8074590 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48efd31640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48e81183d0 con 0x7f48e8074eb0 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48efd31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48e81186d0 con 0x7f48e8112b90 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e81a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e81a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48034/0 (socket says 172.21.15.122:48034) 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 -- 172.21.15.122:0/1168178210 learned_addr learned my addr 172.21.15.122:0/1168178210 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48ed2cc640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8074eb0 0x7f48e81a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48ee2ce640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e8112b90 0x7f48e819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 -- 172.21.15.122:0/1168178210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e8112b90 msgr2=0x7f48e819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e8112b90 0x7f48e819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 -- 172.21.15.122:0/1168178210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8074eb0 msgr2=0x7f48e81a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8074eb0 0x7f48e81a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 -- 172.21.15.122:0/1168178210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e819f010 con 0x7f48e8074590 2024-09-12T01:44:36.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48ed2cc640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8074eb0 0x7f48e81a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48edacd640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e81a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e4006010 tx=0x7f48e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e403f030 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e819f2a0 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e4003ce0 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4035070 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.070+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e81ab5f0 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.071+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e8074970 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.073+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f48e4042030 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.073+0000 7f48deffd640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f48d40a2b80 0x7f48d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.073+0000 7f48ed2cc640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f48d40a2b80 0x7f48d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.073+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f48e40ef8a0 con 0x7f48e8074590 2024-09-12T01:44:36.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.074+0000 7f48ed2cc640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f48d40a2b80 0x7f48d40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f48d80058c0 tx=0x7f48d8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:36.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.077+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48e40bca70 con 0x7f48e8074590 2024-09-12T01:44:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:35 smithi151 ceph-mon[35229]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:44:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:35 smithi151 ceph-mon[35229]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:44:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:35 smithi151 ceph-mon[35229]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:44:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:35 smithi183 ceph-mon[35127]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T01:44:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:35 smithi183 ceph-mon[35127]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T01:44:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:35 smithi183 ceph-mon[35127]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T01:44:36.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.279+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f48e819fe10 con 0x7f48e8074590 2024-09-12T01:44:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.767+0000 7f48deffd640 1 -- 172.21.15.122:0/1168178210 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f48e40bc370 con 0x7f48e8074590 2024-09-12T01:44:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f48d40a2b80 msgr2=0x7f48d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f48d40a2b80 0x7f48d40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f48d80058c0 tx=0x7f48d8005850 comp rx=0 tx=0).stop 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 msgr2=0x7f48e81a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e81a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e4006010 tx=0x7f48e40042f0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 shutdown_connections 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f48d40a2b80 0x7f48d40a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e8112b90 0x7f48e819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8074eb0 0x7f48e81a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 --2- 172.21.15.122:0/1168178210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48e8074590 0x7f48e81a4720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 >> 172.21.15.122:0/1168178210 conn(0x7f48e80fd8c0 msgr2=0x7f48e80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 shutdown_connections 2024-09-12T01:44:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:36.770+0000 7f48efd31640 1 -- 172.21.15.122:0/1168178210 wait complete. 2024-09-12T01:44:37.091 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:36 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1168178210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:44:37.091 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:36 smithi122 ceph-mon[30925]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:44:37.091 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:36 smithi122 ceph-mon[30925]: pgmap v276: 97 pgs: 19 creating+peering, 57 unknown, 21 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:36 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1168178210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:44:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:36 smithi151 ceph-mon[35229]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:44:37.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:36 smithi151 ceph-mon[35229]: pgmap v276: 97 pgs: 19 creating+peering, 57 unknown, 21 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:36 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1168178210' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T01:44:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:36 smithi183 ceph-mon[35127]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T01:44:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:36 smithi183 ceph-mon[35127]: pgmap v276: 97 pgs: 19 creating+peering, 57 unknown, 21 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:44:37.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:44:37.674 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:38.037 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:37 smithi122 ceph-mon[30925]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T01:44:38.037 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:37 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1168178210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:44:38.037 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:37 smithi122 ceph-mon[30925]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:44:38.037 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:37 smithi122 ceph-mon[30925]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:44:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:37 smithi151 ceph-mon[35229]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T01:44:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:37 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1168178210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:44:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:37 smithi151 ceph-mon[35229]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:44:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:37 smithi151 ceph-mon[35229]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 -- 172.21.15.122:0/1033682170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4106a80 msgr2=0x7fa6c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 --2- 172.21.15.122:0/1033682170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4106a80 0x7fa6c4106e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009930 tx=0x7fa6b402a420 comp rx=0 tx=0).stop 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 -- 172.21.15.122:0/1033682170 shutdown_connections 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 --2- 172.21.15.122:0/1033682170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 0x7fa6c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 --2- 172.21.15.122:0/1033682170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c4100a80 0x7fa6c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 --2- 172.21.15.122:0/1033682170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4106a80 0x7fa6c4106e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 -- 172.21.15.122:0/1033682170 >> 172.21.15.122:0/1033682170 conn(0x7fa6c40fc7a0 msgr2=0x7fa6c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 -- 172.21.15.122:0/1033682170 shutdown_connections 2024-09-12T01:44:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 -- 172.21.15.122:0/1033682170 wait complete. 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.116+0000 7fa6c990a640 1 Processor -- start 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 -- start start 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 0x7fa6c419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 0x7fa6c419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c4106a80 0x7fa6c419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4110ea0 con 0x7fa6c4100a80 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4110d20 con 0x7fa6c4101420 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c990a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4111020 con 0x7fa6c4106a80 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 0x7fa6c419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 0x7fa6c419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48046/0 (socket says 172.21.15.122:48046) 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 -- 172.21.15.122:0/3433612617 learned_addr learned my addr 172.21.15.122:0/3433612617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c37fe640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c4106a80 0x7fa6c419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c27fc640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 0x7fa6c419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 -- 172.21.15.122:0/3433612617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c4106a80 msgr2=0x7fa6c419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c4106a80 0x7fa6c419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 -- 172.21.15.122:0/3433612617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 msgr2=0x7fa6c419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 0x7fa6c419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 -- 172.21.15.122:0/3433612617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c419f6a0 con 0x7fa6c4100a80 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c27fc640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 0x7fa6c419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c2ffd640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 0x7fa6c419a6f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009230 tx=0x7fa6b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.117+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b403f030 con 0x7fa6c4100a80 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.118+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c419f930 con 0x7fa6c4100a80 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.118+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b4003ad0 con 0x7fa6c4100a80 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.118+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4035070 con 0x7fa6c4100a80 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.118+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c41a7270 con 0x7fa6c4100a80 2024-09-12T01:44:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.119+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c41021c0 con 0x7fa6c4100a80 2024-09-12T01:44:38.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.120+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa6b4042030 con 0x7fa6c4100a80 2024-09-12T01:44:38.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.120+0000 7fa6c8908640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6b00a2b80 0x7fa6b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:38.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.121+0000 7fa6c27fc640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6b00a2b80 0x7fa6b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:38.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.121+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa6b40ef800 con 0x7fa6c4100a80 2024-09-12T01:44:38.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.121+0000 7fa6c27fc640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6b00a2b80 0x7fa6b00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006fd0 tx=0x7fa6ac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:38.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.124+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6b40bc8b0 con 0x7fa6c4100a80 2024-09-12T01:44:38.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:37 smithi183 ceph-mon[35127]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T01:44:38.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:37 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1168178210' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T01:44:38.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:37 smithi183 ceph-mon[35127]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T01:44:38.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:37 smithi183 ceph-mon[35127]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T01:44:38.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.291+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6c41a7440 con 0x7fa6c4100a80 2024-09-12T01:44:38.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.292+0000 7fa6c8908640 1 -- 172.21.15.122:0/3433612617 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22302 (secure 0 0 0) 0x7fa6b40bc1b0 con 0x7fa6c4100a80 2024-09-12T01:44:38.293 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:38.293 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":81,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:36.280492+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6b00a2b80 msgr2=0x7fa6b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6b00a2b80 0x7fa6b00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006fd0 tx=0x7fa6ac008040 comp rx=0 tx=0).stop 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 msgr2=0x7fa6c419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 0x7fa6c419a6f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009230 tx=0x7fa6b4004270 comp rx=0 tx=0).stop 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 shutdown_connections 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6b00a2b80 0x7fa6b00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c4106a80 0x7fa6c419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c4101420 0x7fa6c419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 --2- 172.21.15.122:0/3433612617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c4100a80 0x7fa6c419a6f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:38.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 >> 172.21.15.122:0/3433612617 conn(0x7fa6c40fc7a0 msgr2=0x7fa6c406a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:38.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 shutdown_connections 2024-09-12T01:44:38.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:38.295+0000 7fa6c990a640 1 -- 172.21.15.122:0/3433612617 wait complete. 2024-09-12T01:44:38.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:38 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] up:active 2024-09-12T01:44:38.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:38 smithi122 ceph-mon[30925]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:38.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:38 smithi122 ceph-mon[30925]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:38.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:38 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3433612617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:38.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:38 smithi122 ceph-mon[30925]: pgmap v278: 97 pgs: 19 creating+peering, 17 unknown, 61 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-12T01:44:38.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:44:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:38 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] up:active 2024-09-12T01:44:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:38 smithi151 ceph-mon[35229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:38 smithi151 ceph-mon[35229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:38 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3433612617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:38 smithi151 ceph-mon[35229]: pgmap v278: 97 pgs: 19 creating+peering, 17 unknown, 61 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-12T01:44:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:38 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] up:active 2024-09-12T01:44:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:38 smithi183 ceph-mon[35127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:38 smithi183 ceph-mon[35127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:38 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3433612617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:38 smithi183 ceph-mon[35127]: pgmap v278: 97 pgs: 19 creating+peering, 17 unknown, 61 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-12T01:44:39.232 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 -- 172.21.15.122:0/934401536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c106f00 msgr2=0x7fea3c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 --2- 172.21.15.122:0/934401536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c106f00 0x7fea3c10ea80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea34009ca0 tx=0x7fea3402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 -- 172.21.15.122:0/934401536 shutdown_connections 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 --2- 172.21.15.122:0/934401536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c106f00 0x7fea3c10ea80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 --2- 172.21.15.122:0/934401536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c102780 0x7fea3c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 --2- 172.21.15.122:0/934401536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea3c101dd0 0x7fea3c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 -- 172.21.15.122:0/934401536 >> 172.21.15.122:0/934401536 conn(0x7fea3c0fd8b0 msgr2=0x7fea3c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:39.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 -- 172.21.15.122:0/934401536 shutdown_connections 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.709+0000 7fea41663640 1 -- 172.21.15.122:0/934401536 wait complete. 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 Processor -- start 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 -- start start 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c101dd0 0x7fea3c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 0x7fea3c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea3c106f00 0x7fea3c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c112000 con 0x7fea3c102780 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c111e80 con 0x7fea3c101dd0 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea41663640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c112180 con 0x7fea3c106f00 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 0x7fea3c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 0x7fea3c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48074/0 (socket says 172.21.15.122:48074) 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 -- 172.21.15.122:0/1657436827 learned_addr learned my addr 172.21.15.122:0/1657436827 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:39.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 -- 172.21.15.122:0/1657436827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea3c106f00 msgr2=0x7fea3c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea3c106f00 0x7fea3c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 -- 172.21.15.122:0/1657436827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c101dd0 msgr2=0x7fea3c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3affd640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c101dd0 0x7fea3c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c101dd0 0x7fea3c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3a7fc640 1 -- 172.21.15.122:0/1657436827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c19d510 con 0x7fea3c102780 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.710+0000 7fea3affd640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c101dd0 0x7fea3c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.711+0000 7fea3a7fc640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 0x7fea3c198aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b4a0 tx=0x7fea3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.711+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30014070 con 0x7fea3c102780 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.711+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c19d800 con 0x7fea3c102780 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.711+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30010af0 con 0x7fea3c102780 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.711+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30017d10 con 0x7fea3c102780 2024-09-12T01:44:39.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.711+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1a50e0 con 0x7fea3c102780 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.712+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c102230 con 0x7fea3c102780 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.713+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fea30017070 con 0x7fea3c102780 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.713+0000 7fea23fff640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea240a2b80 0x7fea240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.714+0000 7fea3affd640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea240a2b80 0x7fea240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.714+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fea300ce700 con 0x7fea3c102780 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.714+0000 7fea3affd640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea240a2b80 0x7fea240a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006fd0 tx=0x7fea2c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.717+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea3009b7b0 con 0x7fea3c102780 2024-09-12T01:44:39.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.888+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea3c1998c0 con 0x7fea3c102780 2024-09-12T01:44:39.888 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:39 smithi122 ceph-mon[30925]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:44:39.888 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:39 smithi122 ceph-mon[30925]: Cluster is now healthy 2024-09-12T01:44:39.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.889+0000 7fea23fff640 1 -- 172.21.15.122:0/1657436827 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22302 (secure 0 0 0) 0x7fea3c1998c0 con 0x7fea3c102780 2024-09-12T01:44:39.889 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:39.889 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":81,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:36.280492+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:44:39.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea240a2b80 msgr2=0x7fea240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea240a2b80 0x7fea240a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006fd0 tx=0x7fea2c008040 comp rx=0 tx=0).stop 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 msgr2=0x7fea3c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 0x7fea3c198aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b4a0 tx=0x7fea3000b960 comp rx=0 tx=0).stop 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 shutdown_connections 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea240a2b80 0x7fea240a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea3c106f00 0x7fea3c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea3c102780 0x7fea3c198aa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 --2- 172.21.15.122:0/1657436827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea3c101dd0 0x7fea3c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.891+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 >> 172.21.15.122:0/1657436827 conn(0x7fea3c0fd8b0 msgr2=0x7fea3c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.892+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 shutdown_connections 2024-09-12T01:44:39.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:39.892+0000 7fea41663640 1 -- 172.21.15.122:0/1657436827 wait complete. 2024-09-12T01:44:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:39 smithi151 ceph-mon[35229]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:44:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:39 smithi151 ceph-mon[35229]: Cluster is now healthy 2024-09-12T01:44:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:39 smithi183 ceph-mon[35127]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T01:44:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:39 smithi183 ceph-mon[35127]: Cluster is now healthy 2024-09-12T01:44:40.608 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T01:44:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:40.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:40 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1657436827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:40.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:40 smithi122 ceph-mon[30925]: pgmap v279: 97 pgs: 11 creating+peering, 86 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-12T01:44:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:40 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1657436827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:40 smithi151 ceph-mon[35229]: pgmap v279: 97 pgs: 11 creating+peering, 86 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-12T01:44:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:40 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1657436827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:40 smithi183 ceph-mon[35127]: pgmap v279: 97 pgs: 11 creating+peering, 86 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 -- 172.21.15.122:0/2432591526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60102d40 msgr2=0x7fdc60103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 --2- 172.21.15.122:0/2432591526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60102d40 0x7fdc60103120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009900 tx=0x7fdc5c02a480 comp rx=0 tx=0).stop 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 -- 172.21.15.122:0/2432591526 shutdown_connections 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 --2- 172.21.15.122:0/2432591526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc6010c170 0x7fdc6010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 --2- 172.21.15.122:0/2432591526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60103660 0x7fdc6010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 --2- 172.21.15.122:0/2432591526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60102d40 0x7fdc60103120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.312+0000 7fdc68330640 1 -- 172.21.15.122:0/2432591526 >> 172.21.15.122:0/2432591526 conn(0x7fdc600fc7a0 msgr2=0x7fdc600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 -- 172.21.15.122:0/2432591526 shutdown_connections 2024-09-12T01:44:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 -- 172.21.15.122:0/2432591526 wait complete. 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 Processor -- start 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 -- start start 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60102d40 0x7fdc601009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 0x7fdc60100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc6010c170 0x7fdc60101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111fb0 con 0x7fdc60103660 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111e30 con 0x7fdc60102d40 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc68330640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60112130 con 0x7fdc6010c170 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc658cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 0x7fdc60100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.313+0000 7fdc658cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 0x7fdc60100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42184/0 (socket says 172.21.15.122:42184) 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 -- 172.21.15.122:0/464037569 learned_addr learned my addr 172.21.15.122:0/464037569 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:41.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc660cc640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60102d40 0x7fdc601009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc668cd640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc6010c170 0x7fdc60101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 -- 172.21.15.122:0/464037569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc6010c170 msgr2=0x7fdc60101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc6010c170 0x7fdc60101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 -- 172.21.15.122:0/464037569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60102d40 msgr2=0x7fdc601009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60102d40 0x7fdc601009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 -- 172.21.15.122:0/464037569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601a6930 con 0x7fdc60103660 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc658cb640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 0x7fdc60100ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c970 tx=0x7fdc5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50007bb0 con 0x7fdc60103660 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601a6c20 con 0x7fdc60103660 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc50007d50 con 0x7fdc60103660 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50018820 con 0x7fdc60103660 2024-09-12T01:44:41.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.314+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a71b0 con 0x7fdc60103660 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.315+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc60103120 con 0x7fdc60103660 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.316+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdc500189c0 con 0x7fdc60103660 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.317+0000 7fdc577fe640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc4c0a2b80 0x7fdc4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.317+0000 7fdc660cc640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc4c0a2b80 0x7fdc4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.317+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fdc500ceed0 con 0x7fdc60103660 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.317+0000 7fdc660cc640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc4c0a2b80 0x7fdc4c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000c00 tx=0x7fdc5c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:41.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.321+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc50014030 con 0x7fdc60103660 2024-09-12T01:44:41.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.526+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdc60107240 con 0x7fdc60103660 2024-09-12T01:44:41.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.782+0000 7fdc577fe640 1 -- 172.21.15.122:0/464037569 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fdc5009bf80 con 0x7fdc60103660 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.784+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc4c0a2b80 msgr2=0x7fdc4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.784+0000 7fdc68330640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc4c0a2b80 0x7fdc4c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000c00 tx=0x7fdc5c0058d0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.784+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 msgr2=0x7fdc60100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.784+0000 7fdc68330640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 0x7fdc60100ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c970 tx=0x7fdc5000ce30 comp rx=0 tx=0).stop 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 shutdown_connections 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc4c0a2b80 0x7fdc4c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc6010c170 0x7fdc60101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc60103660 0x7fdc60100ee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 --2- 172.21.15.122:0/464037569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60102d40 0x7fdc601009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:41.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 >> 172.21.15.122:0/464037569 conn(0x7fdc600fc7a0 msgr2=0x7fdc600fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:41.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 shutdown_connections 2024-09-12T01:44:41.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:41.785+0000 7fdc68330640 1 -- 172.21.15.122:0/464037569 wait complete. 2024-09-12T01:44:42.093 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:41 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/464037569' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T01:44:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:41 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/464037569' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T01:44:42.115 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:41 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: 2024-09-12T01:44:41.631+0000 7f79f5926640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T01:44:42.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:41 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/464037569' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T01:44:42.389 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-12T01:44:42.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T01:44:42.634 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:43.042 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:42 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/464037569' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T01:44:43.042 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:42 smithi122 ceph-mon[30925]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:43.042 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:42 smithi122 ceph-mon[30925]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T01:44:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:42 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/464037569' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T01:44:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:42 smithi151 ceph-mon[35229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:42 smithi151 ceph-mon[35229]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T01:44:43.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 -- 172.21.15.122:0/708132380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768101420 msgr2=0x7f576810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:43.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 --2- 172.21.15.122:0/708132380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768101420 0x7f576810d950 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5760009cd0 tx=0x7f576002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 -- 172.21.15.122:0/708132380 shutdown_connections 2024-09-12T01:44:43.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 --2- 172.21.15.122:0/708132380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768101420 0x7f576810d950 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 --2- 172.21.15.122:0/708132380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 0x7f5768100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 --2- 172.21.15.122:0/708132380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5768106a80 0x7f5768106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 -- 172.21.15.122:0/708132380 >> 172.21.15.122:0/708132380 conn(0x7f57680fc7a0 msgr2=0x7f57680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 -- 172.21.15.122:0/708132380 shutdown_connections 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.126+0000 7f576ed18640 1 -- 172.21.15.122:0/708132380 wait complete. 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.127+0000 7f576ed18640 1 Processor -- start 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.127+0000 7f576ed18640 1 -- start start 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 0x7f5768197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5768101420 0x7f5768197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 0x7f576819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5768110e80 con 0x7f5768106a80 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5768110d00 con 0x7f5768100a80 2024-09-12T01:44:43.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5768111000 con 0x7f5768101420 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 0x7f576819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 0x7f576819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42190/0 (socket says 172.21.15.122:42190) 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 -- 172.21.15.122:0/1364564934 learned_addr learned my addr 172.21.15.122:0/1364564934 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f5767fff640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5768101420 0x7f5768197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576cab4640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 0x7f5768197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 -- 172.21.15.122:0/1364564934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5768101420 msgr2=0x7f5768197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5768101420 0x7f5768197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 -- 172.21.15.122:0/1364564934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 msgr2=0x7f5768197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 0x7f5768197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 -- 172.21.15.122:0/1364564934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576819c2b0 con 0x7f5768106a80 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576cab4640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 0x7f5768197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576d2b5640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 0x7f576819bbd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5760005cc0 tx=0x7f5760003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760004040 con 0x7f5768106a80 2024-09-12T01:44:43.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57600046c0 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760046480 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.128+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576819c540 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.129+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57681a3e80 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.130+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5768100ee0 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.131+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f57600305a0 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.131+0000 7f5765ffb640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f57580a2b80 0x7f57580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.131+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f576002e030 con 0x7f5768106a80 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.131+0000 7f576cab4640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f57580a2b80 0x7f57580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.132+0000 7f576cab4640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f57580a2b80 0x7f57580a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f575c005fd0 tx=0x7f575c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:43.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.136+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f576004b060 con 0x7f5768106a80 2024-09-12T01:44:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:42 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/464037569' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T01:44:43.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:42 smithi183 ceph-mon[35127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:43.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:42 smithi183 ceph-mon[35127]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T01:44:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.345+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f57681a40e0 con 0x7f5768106a80 2024-09-12T01:44:43.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.793+0000 7f5765ffb640 1 -- 172.21.15.122:0/1364564934 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f57600bd8b0 con 0x7f5768106a80 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f57580a2b80 msgr2=0x7f57580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f57580a2b80 0x7f57580a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f575c005fd0 tx=0x7f575c0094d0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 msgr2=0x7f576819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 0x7f576819bbd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5760005cc0 tx=0x7f5760003970 comp rx=0 tx=0).stop 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 shutdown_connections 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f57580a2b80 0x7f57580a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5768106a80 0x7f576819bbd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5768101420 0x7f5768197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 --2- 172.21.15.122:0/1364564934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5768100a80 0x7f5768197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:43.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 >> 172.21.15.122:0/1364564934 conn(0x7f57680fc7a0 msgr2=0x7f57680fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:43.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 shutdown_connections 2024-09-12T01:44:43.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:43.796+0000 7f576ed18640 1 -- 172.21.15.122:0/1364564934 wait complete. 2024-09-12T01:44:44.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:43 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1364564934' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T01:44:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:43 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1364564934' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T01:44:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:43 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1364564934' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T01:44:44.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T01:44:44.654 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1364564934' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: fsmap cephfs:2 {0=b=up:active,1=c=up:creating} 10 up:standby 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T01:44:45.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T01:44:45.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:44 smithi122 ceph-mon[30925]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T01:44:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T01:44:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1364564934' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T01:44:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: fsmap cephfs:2 {0=b=up:active,1=c=up:creating} 10 up:standby 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T01:44:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:44 smithi151 ceph-mon[35229]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 -- 172.21.15.122:0/1362133943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74410c170 msgr2=0x7fd74410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 --2- 172.21.15.122:0/1362133943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74410c170 0x7fd74410e610 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009ca0 tx=0x7fd73c02a950 comp rx=0 tx=0).stop 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 -- 172.21.15.122:0/1362133943 shutdown_connections 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 --2- 172.21.15.122:0/1362133943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74410c170 0x7fd74410e610 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 --2- 172.21.15.122:0/1362133943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd744103660 0x7fd74410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 --2- 172.21.15.122:0/1362133943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd744102d40 0x7fd744103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 -- 172.21.15.122:0/1362133943 >> 172.21.15.122:0/1362133943 conn(0x7fd7440fc7a0 msgr2=0x7fd7440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 -- 172.21.15.122:0/1362133943 shutdown_connections 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 -- 172.21.15.122:0/1362133943 wait complete. 2024-09-12T01:44:45.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 Processor -- start 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.154+0000 7fd749a79640 1 -- start start 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 0x7fd7441009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd744103660 0x7fd744100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74410c170 0x7fd744101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd744111f70 con 0x7fd744102d40 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd744111df0 con 0x7fd744103660 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441120f0 con 0x7fd74410c170 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 0x7fd7441009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 0x7fd7441009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42198/0 (socket says 172.21.15.122:42198) 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 -- 172.21.15.122:0/2876928219 learned_addr learned my addr 172.21.15.122:0/2876928219 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd742ffd640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd744103660 0x7fd744100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd743fff640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74410c170 0x7fd744101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:45.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 -- 172.21.15.122:0/2876928219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74410c170 msgr2=0x7fd744101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74410c170 0x7fd744101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 -- 172.21.15.122:0/2876928219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd744103660 msgr2=0x7fd744100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd744103660 0x7fd744100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 -- 172.21.15.122:0/2876928219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441a6850 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd743fff640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74410c170 0x7fd744101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd7437fe640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 0x7fd7441009a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c970 tx=0x7fd73400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734007bb0 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd734007d50 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400f450 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7441a6b40 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.155+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441a7030 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.156+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd744103120 con 0x7fd744102d40 2024-09-12T01:44:45.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.157+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd734005020 con 0x7fd744102d40 2024-09-12T01:44:45.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.158+0000 7fd740ff9640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd72c0a2b80 0x7fd72c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:45.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.158+0000 7fd742ffd640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd72c0a2b80 0x7fd72c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:45.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.158+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd7340cee30 con 0x7fd744102d40 2024-09-12T01:44:45.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.158+0000 7fd742ffd640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd72c0a2b80 0x7fd72c0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd744102430 tx=0x7fd73800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:45.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.162+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd734014030 con 0x7fd744102d40 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1364564934' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: fsmap cephfs:2 {0=b=up:active,1=c=up:creating} 10 up:standby 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]: dispatch 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T01:44:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T01:44:45.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-12T01:44:45.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-12T01:44:45.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-12T01:44:45.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T01:44:45.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:44 smithi183 ceph-mon[35127]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T01:44:45.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.366+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fd7441a7200 con 0x7fd744102d40 2024-09-12T01:44:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.818+0000 7fd740ff9640 1 -- 172.21.15.122:0/2876928219 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fd73409bd90 con 0x7fd744102d40 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.821+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd72c0a2b80 msgr2=0x7fd72c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.821+0000 7fd749a79640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd72c0a2b80 0x7fd72c0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd744102430 tx=0x7fd73800a250 comp rx=0 tx=0).stop 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.821+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 msgr2=0x7fd7441009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.821+0000 7fd749a79640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 0x7fd7441009a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c970 tx=0x7fd73400ce30 comp rx=0 tx=0).stop 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 shutdown_connections 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd72c0a2b80 0x7fd72c0a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd74410c170 0x7fd744101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd744103660 0x7fd744100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 --2- 172.21.15.122:0/2876928219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd744102d40 0x7fd7441009a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 >> 172.21.15.122:0/2876928219 conn(0x7fd7440fc7a0 msgr2=0x7fd7440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 shutdown_connections 2024-09-12T01:44:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:45.822+0000 7fd749a79640 1 -- 172.21.15.122:0/2876928219 wait complete. 2024-09-12T01:44:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] up:active 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T01:44:46.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: Cluster is now healthy 2024-09-12T01:44:46.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: fsmap cephfs:2 {0=b=up:active,1=c=up:active} 10 up:standby 2024-09-12T01:44:46.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:44:46.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:45 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2876928219' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-12T01:44:46.138 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] up:active 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: Cluster is now healthy 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: fsmap cephfs:2 {0=b=up:active,1=c=up:active} 10 up:standby 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:44:46.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:45 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2876928219' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:44:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] up:active 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T01:44:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T01:44:46.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: Cluster is now healthy 2024-09-12T01:44:46.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: fsmap cephfs:2 {0=b=up:active,1=c=up:active} 10 up:standby 2024-09-12T01:44:46.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:44:46.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:45 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2876928219' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T01:44:46.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T01:44:46.763 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:46 smithi151 ceph-mon[35229]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T01:44:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:46 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2876928219' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T01:44:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:46 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:44:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:46 smithi151 ceph-mon[35229]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T01:44:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:46 smithi151 ceph-mon[35229]: pgmap v284: 97 pgs: 3 peering, 94 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-12T01:44:47.116 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 01:44:46 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: 2024-09-12T01:44:46.632+0000 7f79f5926640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T01:44:47.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:46 smithi122 ceph-mon[30925]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T01:44:47.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:46 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2876928219' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T01:44:47.145 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:46 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:44:47.146 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:46 smithi122 ceph-mon[30925]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T01:44:47.146 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:46 smithi122 ceph-mon[30925]: pgmap v284: 97 pgs: 3 peering, 94 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-12T01:44:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:46 smithi183 ceph-mon[35127]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T01:44:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:46 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2876928219' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T01:44:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:46 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:creating} 9 up:standby 2024-09-12T01:44:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:46 smithi183 ceph-mon[35127]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T01:44:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:46 smithi183 ceph-mon[35127]: pgmap v284: 97 pgs: 3 peering, 94 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-12T01:44:47.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 -- 172.21.15.122:0/3931946678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102570 msgr2=0x7f1c201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 --2- 172.21.15.122:0/3931946678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102570 0x7f1c201029b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009900 tx=0x7f1c1002a480 comp rx=0 tx=0).stop 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 -- 172.21.15.122:0/3931946678 shutdown_connections 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 --2- 172.21.15.122:0/3931946678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20102ef0 0x7f1c2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 --2- 172.21.15.122:0/3931946678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102570 0x7f1c201029b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 --2- 172.21.15.122:0/3931946678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20100e20 0x7f1c20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 -- 172.21.15.122:0/3931946678 >> 172.21.15.122:0/3931946678 conn(0x7f1c200fc7d0 msgr2=0x7f1c200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 -- 172.21.15.122:0/3931946678 shutdown_connections 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 -- 172.21.15.122:0/3931946678 wait complete. 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.227+0000 7f1c27a69640 1 Processor -- start 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 -- start start 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20100e20 0x7f1c20199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20102570 0x7f1c20199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 0x7f1c201a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20110f80 con 0x7f1c20102ef0 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20110e00 con 0x7f1c20102570 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c27a69640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20111100 con 0x7f1c20100e20 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 0x7f1c201a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 0x7f1c201a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42202/0 (socket says 172.21.15.122:42202) 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 -- 172.21.15.122:0/4073366823 learned_addr learned my addr 172.21.15.122:0/4073366823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c25004640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20102570 0x7f1c20199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c25805640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20100e20 0x7f1c20199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 -- 172.21.15.122:0/4073366823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20100e20 msgr2=0x7f1c20199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20100e20 0x7f1c20199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 -- 172.21.15.122:0/4073366823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20102570 msgr2=0x7f1c20199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20102570 0x7f1c20199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 -- 172.21.15.122:0/4073366823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c201a4910 con 0x7f1c20102ef0 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c25805640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20100e20 0x7f1c20199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.228+0000 7f1c26006640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 0x7f1c201a43d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800eaa0 tx=0x7f1c1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.229+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800cdb0 con 0x7f1c20102ef0 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.229+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c201a4ae0 con 0x7f1c20102ef0 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.229+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18005020 con 0x7f1c20102ef0 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.229+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180116d0 con 0x7f1c20102ef0 2024-09-12T01:44:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.229+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c201a4fa0 con 0x7f1c20102ef0 2024-09-12T01:44:47.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.230+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c20101200 con 0x7f1c20102ef0 2024-09-12T01:44:47.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.231+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1c180054d0 con 0x7f1c20102ef0 2024-09-12T01:44:47.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.231+0000 7f1c16ffd640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c0c0a2b80 0x7f1c0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:47.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.231+0000 7f1c25805640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c0c0a2b80 0x7f1c0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:47.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.232+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1c180cf220 con 0x7f1c20102ef0 2024-09-12T01:44:47.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.232+0000 7f1c25805640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c0c0a2b80 0x7f1c0c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0058c0 tx=0x7f1c1c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:47.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.236+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c18016030 con 0x7f1c20102ef0 2024-09-12T01:44:47.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.438+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f1c20109090 con 0x7f1c20102ef0 2024-09-12T01:44:47.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.832+0000 7f1c16ffd640 1 -- 172.21.15.122:0/4073366823 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f1c1809c180 con 0x7f1c20102ef0 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c0c0a2b80 msgr2=0x7f1c0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c0c0a2b80 0x7f1c0c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0058c0 tx=0x7f1c1c005850 comp rx=0 tx=0).stop 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 msgr2=0x7f1c201a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 0x7f1c201a43d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800eaa0 tx=0x7f1c1800ef60 comp rx=0 tx=0).stop 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 shutdown_connections 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c0c0a2b80 0x7f1c0c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c20102ef0 0x7f1c201a43d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20102570 0x7f1c20199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 --2- 172.21.15.122:0/4073366823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20100e20 0x7f1c20199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 >> 172.21.15.122:0/4073366823 conn(0x7f1c200fc7d0 msgr2=0x7f1c201068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 shutdown_connections 2024-09-12T01:44:47.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:47.834+0000 7f1c27a69640 1 -- 172.21.15.122:0/4073366823 wait complete. 2024-09-12T01:44:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:47 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:active 2024-09-12T01:44:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:47 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 9 up:standby 2024-09-12T01:44:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:47 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:44:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:47 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4073366823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T01:44:48.149 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:47 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:active 2024-09-12T01:44:48.150 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:47 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 9 up:standby 2024-09-12T01:44:48.150 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:47 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:44:48.150 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:47 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4073366823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T01:44:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:47 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:active 2024-09-12T01:44:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:47 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 9 up:standby 2024-09-12T01:44:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:47 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:44:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:47 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4073366823' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T01:44:48.470 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-12T01:44:48.470 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T01:44:48.713 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:49.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:48 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4073366823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T01:44:49.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:48 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:44:49.087 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:48 smithi122 ceph-mon[30925]: pgmap v285: 97 pgs: 3 peering, 94 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T01:44:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:48 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4073366823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T01:44:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:48 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:44:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:48 smithi151 ceph-mon[35229]: pgmap v285: 97 pgs: 3 peering, 94 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T01:44:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:48 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4073366823' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T01:44:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:48 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:44:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:48 smithi183 ceph-mon[35127]: pgmap v285: 97 pgs: 3 peering, 94 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T01:44:49.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 -- 172.21.15.122:0/161126889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4106f10 msgr2=0x7f59f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 --2- 172.21.15.122:0/161126889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4106f10 0x7f59f410ea90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009ca0 tx=0x7f59ec02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 -- 172.21.15.122:0/161126889 shutdown_connections 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 --2- 172.21.15.122:0/161126889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4106f10 0x7f59f410ea90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 --2- 172.21.15.122:0/161126889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59f4102790 0x7f59f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 --2- 172.21.15.122:0/161126889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 0x7f59f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 -- 172.21.15.122:0/161126889 >> 172.21.15.122:0/161126889 conn(0x7f59f40fd8c0 msgr2=0x7f59f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 -- 172.21.15.122:0/161126889 shutdown_connections 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 -- 172.21.15.122:0/161126889 wait complete. 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.170+0000 7f59f9a55640 1 Processor -- start 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 -- start start 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 0x7f59f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 0x7f59f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:49.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59f4106f10 0x7f59f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4112010 con 0x7f59f4102790 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4111e90 con 0x7f59f4106f10 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f9a55640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4112190 con 0x7f59f4101de0 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 0x7f59f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 0x7f59f4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42228/0 (socket says 172.21.15.122:42228) 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 -- 172.21.15.122:0/785802310 learned_addr learned my addr 172.21.15.122:0/785802310 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 -- 172.21.15.122:0/785802310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 msgr2=0x7f59f40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f2ffd640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 0x7f59f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 0x7f59f40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 -- 172.21.15.122:0/785802310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59f4106f10 msgr2=0x7f59f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59f4106f10 0x7f59f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 -- 172.21.15.122:0/785802310 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f41a6ab0 con 0x7f59f4102790 2024-09-12T01:44:49.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f2ffd640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 0x7f59f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.171+0000 7f59f27fc640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 0x7f59f4074a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b4a0 tx=0x7f59e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.172+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8014070 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.172+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f41a6da0 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.172+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e8010af0 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.172+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8017d10 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.172+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f41a7260 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.173+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f4102240 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.174+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f59e8017070 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.174+0000 7f59f8a53640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59dc0a2b80 0x7f59dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.175+0000 7f59f2ffd640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59dc0a2b80 0x7f59dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.175+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f59e80ce840 con 0x7f59f4102790 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.175+0000 7f59f2ffd640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59dc0a2b80 0x7f59dc0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f59e4006fd0 tx=0x7f59e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:49.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.178+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59e809b7a0 con 0x7f59f4102790 2024-09-12T01:44:49.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.370+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f59f4062630 con 0x7f59dc0a2b80 2024-09-12T01:44:49.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.394+0000 7f59f8a53640 1 -- 172.21.15.122:0/785802310 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59f4062630 con 0x7f59dc0a2b80 2024-09-12T01:44:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.395+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59dc0a2b80 msgr2=0x7f59dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:49.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.395+0000 7f59f9a55640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59dc0a2b80 0x7f59dc0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f59e4006fd0 tx=0x7f59e4008040 comp rx=0 tx=0).stop 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.395+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 msgr2=0x7f59f4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.395+0000 7f59f9a55640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 0x7f59f4074a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b4a0 tx=0x7f59e800b960 comp rx=0 tx=0).stop 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 shutdown_connections 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59dc0a2b80 0x7f59dc0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59f4106f10 0x7f59f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59f4102790 0x7f59f4074a30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 --2- 172.21.15.122:0/785802310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59f4101de0 0x7f59f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 >> 172.21.15.122:0/785802310 conn(0x7f59f40fd8c0 msgr2=0x7f59f40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 shutdown_connections 2024-09-12T01:44:49.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:49.396+0000 7f59f9a55640 1 -- 172.21.15.122:0/785802310 wait complete. 2024-09-12T01:44:49.739 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:44:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:44:49.375+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:44:49.739 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:44:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:44:49.375+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:44:49.739 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:44:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:44:49.375+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:44:49.739 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:44:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:44:49.375+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:44:49.739 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:44:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:44:49.375+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:44:50.059 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:49 smithi122 ceph-mon[30925]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:50.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T01:44:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:49 smithi151 ceph-mon[35229]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:49 smithi183 ceph-mon[35127]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:50.327 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 -- 172.21.15.122:0/1483723522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0106f10 msgr2=0x7f28f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 --2- 172.21.15.122:0/1483723522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0106f10 0x7f28f010ea90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009ca0 tx=0x7f28e802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 -- 172.21.15.122:0/1483723522 shutdown_connections 2024-09-12T01:44:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 --2- 172.21.15.122:0/1483723522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0106f10 0x7f28f010ea90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 --2- 172.21.15.122:0/1483723522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f0102790 0x7f28f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 --2- 172.21.15.122:0/1483723522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28f0101de0 0x7f28f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 -- 172.21.15.122:0/1483723522 >> 172.21.15.122:0/1483723522 conn(0x7f28f00fd8c0 msgr2=0x7f28f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 -- 172.21.15.122:0/1483723522 shutdown_connections 2024-09-12T01:44:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.948+0000 7f28f836f640 1 -- 172.21.15.122:0/1483723522 wait complete. 2024-09-12T01:44:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 Processor -- start 2024-09-12T01:44:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 -- start start 2024-09-12T01:44:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28f0101de0 0x7f28f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 0x7f28f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f0106f10 0x7f28f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0111f70 con 0x7f28f0102790 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0111df0 con 0x7f28f0101de0 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f836f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01120f0 con 0x7f28f0106f10 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f590a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 0x7f28f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f590a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 0x7f28f0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57000/0 (socket says 172.21.15.122:57000) 2024-09-12T01:44:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f590a640 1 -- 172.21.15.122:0/2814674526 learned_addr learned my addr 172.21.15.122:0/2814674526 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.949+0000 7f28f610b640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28f0101de0 0x7f28f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f690c640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f0106f10 0x7f28f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f590a640 1 -- 172.21.15.122:0/2814674526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f0106f10 msgr2=0x7f28f0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f590a640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f0106f10 0x7f28f0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f590a640 1 -- 172.21.15.122:0/2814674526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28f0101de0 msgr2=0x7f28f00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f590a640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28f0101de0 0x7f28f00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f590a640 1 -- 172.21.15.122:0/2814674526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f01a6800 con 0x7f28f0102790 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f590a640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 0x7f28f0074a30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c920 tx=0x7f28e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0007c70 con 0x7f28f0102790 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0007e10 con 0x7f28f0102790 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01a6af0 con 0x7f28f0102790 2024-09-12T01:44:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0018890 con 0x7f28f0102790 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.950+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a7030 con 0x7f28f0102790 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.951+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f01021c0 con 0x7f28f0102790 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.952+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f28e0018a70 con 0x7f28f0102790 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.953+0000 7f28e77fe640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28dc0a2c10 0x7f28dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.953+0000 7f28f610b640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28dc0a2c10 0x7f28dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.953+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f28e00cf1f0 con 0x7f28f0102790 2024-09-12T01:44:50.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.953+0000 7f28f610b640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28dc0a2c10 0x7f28dc0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0058c0 tx=0x7f28ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:50.957+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28e0014030 con 0x7f28f0102790 2024-09-12T01:44:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:50 smithi151 ceph-mon[35229]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-12T01:44:51.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.146+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f28f0062630 con 0x7f28dc0a2c10 2024-09-12T01:44:51.146 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:50 smithi122 ceph-mon[30925]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-12T01:44:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.171+0000 7f28e77fe640 1 -- 172.21.15.122:0/2814674526 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f28f0062630 con 0x7f28dc0a2c10 2024-09-12T01:44:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28dc0a2c10 msgr2=0x7f28dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28dc0a2c10 0x7f28dc0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0058c0 tx=0x7f28ec00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 msgr2=0x7f28f0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 0x7f28f0074a30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c920 tx=0x7f28e000cde0 comp rx=0 tx=0).stop 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 shutdown_connections 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28dc0a2c10 0x7f28dc0a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28f0106f10 0x7f28f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28f0102790 0x7f28f0074a30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 --2- 172.21.15.122:0/2814674526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28f0101de0 0x7f28f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.173+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 >> 172.21.15.122:0/2814674526 conn(0x7f28f00fd8c0 msgr2=0x7f28f00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.174+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 shutdown_connections 2024-09-12T01:44:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:51.174+0000 7f28f836f640 1 -- 172.21.15.122:0/2814674526 wait complete. 2024-09-12T01:44:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:50 smithi183 ceph-mon[35127]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-12T01:44:51.790 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T01:44:52.038 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:51 smithi151 ceph-mon[35229]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T01:44:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:51 smithi151 ceph-mon[35229]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:52.158 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:51 smithi122 ceph-mon[30925]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T01:44:52.158 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:51 smithi122 ceph-mon[30925]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:51 smithi183 ceph-mon[35127]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T01:44:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:51 smithi183 ceph-mon[35127]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- 172.21.15.122:0/1704500338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c102ef0 msgr2=0x7f056c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- 172.21.15.122:0/1704500338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c102ef0 0x7f056c10da40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0564009ca0 tx=0x7f056402a950 comp rx=0 tx=0).stop 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- 172.21.15.122:0/1704500338 shutdown_connections 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- 172.21.15.122:0/1704500338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c102ef0 0x7f056c10da40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- 172.21.15.122:0/1704500338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056c102570 0x7f056c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- 172.21.15.122:0/1704500338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056c100e20 0x7f056c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- 172.21.15.122:0/1704500338 >> 172.21.15.122:0/1704500338 conn(0x7f056c0fc7d0 msgr2=0x7f056c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- 172.21.15.122:0/1704500338 shutdown_connections 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- 172.21.15.122:0/1704500338 wait complete. 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 Processor -- start 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- start start 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 0x7f056c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056c102570 0x7f056c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056c102ef0 0x7f056c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f056c110fa0 con 0x7f056c100e20 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f056c110e20 con 0x7f056c102ef0 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.550+0000 7f05740c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f056c111120 con 0x7f056c102570 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 0x7f056c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 0x7f056c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57010/0 (socket says 172.21.15.122:57010) 2024-09-12T01:44:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 -- 172.21.15.122:0/3885264081 learned_addr learned my addr 172.21.15.122:0/3885264081 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f057165c640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056c102570 0x7f056c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 -- 172.21.15.122:0/3885264081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056c102570 msgr2=0x7f056c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056c102570 0x7f056c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 -- 172.21.15.122:0/3885264081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056c102ef0 msgr2=0x7f056c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056c102ef0 0x7f056c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 -- 172.21.15.122:0/3885264081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c19d400 con 0x7f056c100e20 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0571e5d640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 0x7f056c198450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f056800c920 tx=0x7f056800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568007c70 con 0x7f056c100e20 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0568007e10 con 0x7f056c100e20 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c19d6f0 con 0x7f056c100e20 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568018890 con 0x7f056c100e20 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.551+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c1a4fd0 con 0x7f056c100e20 2024-09-12T01:44:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.552+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c101200 con 0x7f056c100e20 2024-09-12T01:44:52.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.553+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0568018a70 con 0x7f056c100e20 2024-09-12T01:44:52.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.554+0000 7f0562ffd640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05580a2c10 0x7f05580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:52.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.554+0000 7f057165c640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05580a2c10 0x7f05580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:52.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.554+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f05680cf3d0 con 0x7f056c100e20 2024-09-12T01:44:52.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.554+0000 7f057165c640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05580a2c10 0x7f05580a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f056c105190 tx=0x7f055c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:52.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.558+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0568014030 con 0x7f056c100e20 2024-09-12T01:44:52.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.753+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f056c062630 con 0x7f05580a2c10 2024-09-12T01:44:52.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.758+0000 7f0562ffd640 1 -- 172.21.15.122:0/3885264081 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f056c062630 con 0x7f05580a2c10 2024-09-12T01:44:52.758 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/qa/sv_0/ec3d3797-98df-4995-bee5-efa47dacdbd1 2024-09-12T01:44:52.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05580a2c10 msgr2=0x7f05580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:52.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05580a2c10 0x7f05580a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f056c105190 tx=0x7f055c008040 comp rx=0 tx=0).stop 2024-09-12T01:44:52.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 msgr2=0x7f056c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 0x7f056c198450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f056800c920 tx=0x7f056800cde0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 shutdown_connections 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05580a2c10 0x7f05580a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f056c102ef0 0x7f056c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056c102570 0x7f056c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 --2- 172.21.15.122:0/3885264081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c100e20 0x7f056c198450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 >> 172.21.15.122:0/3885264081 conn(0x7f056c0fc7d0 msgr2=0x7f056c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 shutdown_connections 2024-09-12T01:44:52.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:52.760+0000 7f05740c1640 1 -- 172.21.15.122:0/3885264081 wait complete. 2024-09-12T01:44:53.069 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:52 smithi122 ceph-mon[30925]: pgmap v287: 97 pgs: 97 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T01:44:53.104 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:52 smithi183 ceph-mon[35127]: pgmap v287: 97 pgs: 97 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T01:44:53.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:52 smithi151 ceph-mon[35229]: pgmap v287: 97 pgs: 97 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T01:44:53.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T01:44:53.691 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:54.107 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:53 smithi122 ceph-mon[30925]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:53 smithi151 ceph-mon[35229]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:54.121 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:53 smithi183 ceph-mon[35127]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 -- 172.21.15.122:0/333992078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 msgr2=0x7f0cdc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/333992078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc106840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009930 tx=0x7f0cc402a420 comp rx=0 tx=0).stop 2024-09-12T01:44:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 -- 172.21.15.122:0/333992078 shutdown_connections 2024-09-12T01:44:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/333992078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc106f50 0x7f0cdc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/333992078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc106840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/333992078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cdc1002f0 0x7f0cdc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 -- 172.21.15.122:0/333992078 >> 172.21.15.122:0/333992078 conn(0x7f0cdc077210 msgr2=0x7f0cdc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 -- 172.21.15.122:0/333992078 shutdown_connections 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 -- 172.21.15.122:0/333992078 wait complete. 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.192+0000 7f0ce25c5640 1 Processor -- start 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 -- start start 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc1002f0 0x7f0cdc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cdc106f50 0x7f0cdc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc111fa0 con 0x7f0cdc100ca0 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc111e20 con 0x7f0cdc1002f0 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce25c5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc112120 con 0x7f0cdc106f50 2024-09-12T01:44:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57028/0 (socket says 172.21.15.122:57028) 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 -- 172.21.15.122:0/3094443841 learned_addr learned my addr 172.21.15.122:0/3094443841 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdbfff640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc1002f0 0x7f0cdc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0ce0b62640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cdc106f50 0x7f0cdc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 -- 172.21.15.122:0/3094443841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cdc106f50 msgr2=0x7f0cdc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cdc106f50 0x7f0cdc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 -- 172.21.15.122:0/3094443841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc1002f0 msgr2=0x7f0cdc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc1002f0 0x7f0cdc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdb7fe640 1 -- 172.21.15.122:0/3094443841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc19d4d0 con 0x7f0cdc100ca0 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.193+0000 7f0cdbfff640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc1002f0 0x7f0cdc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.194+0000 7f0cdb7fe640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc198a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005940 tx=0x7f0cc40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:54.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.194+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc403f030 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.194+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc19d760 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.194+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4003ce0 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.194+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4035070 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.194+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc1011a0 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.195+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc1006d0 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.196+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0cc4042030 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.196+0000 7f0cd97fa640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0cc80a2c10 0x7f0cc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.197+0000 7f0cdbfff640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0cc80a2c10 0x7f0cc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.197+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0cc40ef640 con 0x7f0cdc100ca0 2024-09-12T01:44:54.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.197+0000 7f0cdbfff640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0cc80a2c10 0x7f0cc80a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc005fd0 tx=0x7f0ccc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:54.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.201+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cc40bbdf0 con 0x7f0cdc100ca0 2024-09-12T01:44:54.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.392+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0cdc062630 con 0x7f0cc80a2c10 2024-09-12T01:44:54.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.411+0000 7f0cd97fa640 1 -- 172.21.15.122:0/3094443841 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0cdc062630 con 0x7f0cc80a2c10 2024-09-12T01:44:54.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.413+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0cc80a2c10 msgr2=0x7f0cc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.413+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0cc80a2c10 0x7f0cc80a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc005fd0 tx=0x7f0ccc00b390 comp rx=0 tx=0).stop 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.413+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 msgr2=0x7f0cdc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.413+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc198a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005940 tx=0x7f0cc40042f0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 shutdown_connections 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0cc80a2c10 0x7f0cc80a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cdc106f50 0x7f0cdc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cdc100ca0 0x7f0cdc198a60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 --2- 172.21.15.122:0/3094443841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cdc1002f0 0x7f0cdc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:54.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 >> 172.21.15.122:0/3094443841 conn(0x7f0cdc077210 msgr2=0x7f0cdc105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:54.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 shutdown_connections 2024-09-12T01:44:54.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:54.414+0000 7f0ce25c5640 1 -- 172.21.15.122:0/3094443841 wait complete. 2024-09-12T01:44:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:54 smithi151 ceph-mon[35229]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:54 smithi151 ceph-mon[35229]: pgmap v288: 97 pgs: 97 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.3 KiB/s wr, 13 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T01:44:55.147 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:54 smithi122 ceph-mon[30925]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:55.147 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:54 smithi122 ceph-mon[30925]: pgmap v288: 97 pgs: 97 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.3 KiB/s wr, 13 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T01:44:55.175 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T01:44:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:54 smithi183 ceph-mon[35127]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:54 smithi183 ceph-mon[35127]: pgmap v288: 97 pgs: 97 active+clean; 588 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.3 KiB/s wr, 13 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-12T01:44:55.418 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:55.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 -- 172.21.15.122:0/4116354095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 msgr2=0x7f8e0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 --2- 172.21.15.122:0/4116354095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1021c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009ca0 tx=0x7f8dfc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 -- 172.21.15.122:0/4116354095 shutdown_connections 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 --2- 172.21.15.122:0/4116354095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e0c106f10 0x7f8e0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 --2- 172.21.15.122:0/4116354095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 0x7f8e0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 --2- 172.21.15.122:0/4116354095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1021c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 -- 172.21.15.122:0/4116354095 >> 172.21.15.122:0/4116354095 conn(0x7f8e0c0fd8c0 msgr2=0x7f8e0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:55.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 -- 172.21.15.122:0/4116354095 shutdown_connections 2024-09-12T01:44:55.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.878+0000 7f8e10b40640 1 -- 172.21.15.122:0/4116354095 wait complete. 2024-09-12T01:44:55.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 Processor -- start 2024-09-12T01:44:55.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 -- start start 2024-09-12T01:44:55.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:55.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 0x7f8e0c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:55.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e0c106f10 0x7f8e0c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:55.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c111f70 con 0x7f8e0c101de0 2024-09-12T01:44:55.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c111df0 con 0x7f8e0c102790 2024-09-12T01:44:55.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e10b40640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1120f0 con 0x7f8e0c106f10 2024-09-12T01:44:55.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e0a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:55.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e0a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57042/0 (socket says 172.21.15.122:57042) 2024-09-12T01:44:55.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e0a59c640 1 -- 172.21.15.122:0/3032056672 learned_addr learned my addr 172.21.15.122:0/3032056672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:55.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e0ad9d640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e0c106f10 0x7f8e0c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:55.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e09d9b640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 0x7f8e0c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:55.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e0a59c640 1 -- 172.21.15.122:0/3032056672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e0c106f10 msgr2=0x7f8e0c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:55.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.879+0000 7f8e0a59c640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e0c106f10 0x7f8e0c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:55.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e0a59c640 1 -- 172.21.15.122:0/3032056672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 msgr2=0x7f8e0c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e0a59c640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 0x7f8e0c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e0a59c640 1 -- 172.21.15.122:0/3032056672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c19d480 con 0x7f8e0c101de0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e09d9b640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 0x7f8e0c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e0a59c640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1984d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc02ade0 tx=0x7f8dfc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc02fa20 con 0x7f8e0c101de0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc02fbc0 con 0x7f8e0c101de0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c19d710 con 0x7f8e0c101de0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0467b0 con 0x7f8e0c101de0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.880+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1a5050 con 0x7f8e0c101de0 2024-09-12T01:44:55.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.881+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c1021c0 con 0x7f8e0c101de0 2024-09-12T01:44:55.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.882+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8dfc046950 con 0x7f8e0c101de0 2024-09-12T01:44:55.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.883+0000 7f8dfb7fe640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8df40a2c10 0x7f8df40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:55.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.883+0000 7f8e09d9b640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8df40a2c10 0x7f8df40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:55.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.883+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8dfc0f07a0 con 0x7f8e0c101de0 2024-09-12T01:44:55.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.883+0000 7f8e09d9b640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8df40a2c10 0x7f8df40a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0ff840 tx=0x7f8e0000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:55.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:55.887+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8dfc0bd740 con 0x7f8e0c101de0 2024-09-12T01:44:56.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.078+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8e0c062630 con 0x7f8df40a2c10 2024-09-12T01:44:56.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.083+0000 7f8dfb7fe640 1 -- 172.21.15.122:0/3032056672 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8e0c062630 con 0x7f8df40a2c10 2024-09-12T01:44:56.083 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8df40a2c10 msgr2=0x7f8df40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8df40a2c10 0x7f8df40a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0ff840 tx=0x7f8e0000b5f0 comp rx=0 tx=0).stop 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 msgr2=0x7f8e0c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1984d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc02ade0 tx=0x7f8dfc0319f0 comp rx=0 tx=0).stop 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 shutdown_connections 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8df40a2c10 0x7f8df40a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e0c106f10 0x7f8e0c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e0c102790 0x7f8e0c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 --2- 172.21.15.122:0/3032056672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c101de0 0x7f8e0c1984d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:56.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 >> 172.21.15.122:0/3032056672 conn(0x7f8e0c0fd8c0 msgr2=0x7f8e0c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:56.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 shutdown_connections 2024-09-12T01:44:56.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:56.086+0000 7f8e10b40640 1 -- 172.21.15.122:0/3032056672 wait complete. 2024-09-12T01:44:56.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:44:57.144 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:57.582 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:57 smithi122 ceph-mon[30925]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:57.582 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:57 smithi122 ceph-mon[30925]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 KiB/s wr, 14 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:44:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.581+0000 7f55df214640 1 -- 172.21.15.122:0/3372755378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8108300 msgr2=0x7f55d810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.581+0000 7f55df214640 1 --2- 172.21.15.122:0/3372755378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8108300 0x7f55d810ec70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009930 tx=0x7f55c002a440 comp rx=0 tx=0).stop 2024-09-12T01:44:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 -- 172.21.15.122:0/3372755378 shutdown_connections 2024-09-12T01:44:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 --2- 172.21.15.122:0/3372755378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8108300 0x7f55d810ec70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 --2- 172.21.15.122:0/3372755378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8103780 0x7f55d8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 --2- 172.21.15.122:0/3372755378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55d8102dd0 0x7f55d81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 -- 172.21.15.122:0/3372755378 >> 172.21.15.122:0/3372755378 conn(0x7f55d80fc7d0 msgr2=0x7f55d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 -- 172.21.15.122:0/3372755378 shutdown_connections 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 -- 172.21.15.122:0/3372755378 wait complete. 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 Processor -- start 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 -- start start 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55d8102dd0 0x7f55d81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.582+0000 7f55df214640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 0x7f55d8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55df214640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8108300 0x7f55d819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55df214640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8112170 con 0x7f55d8103780 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55df214640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8111ff0 con 0x7f55d8102dd0 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55df214640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81122f0 con 0x7f55d8108300 2024-09-12T01:44:57.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 0x7f55d8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 0x7f55d8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57058/0 (socket says 172.21.15.122:57058) 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 -- 172.21.15.122:0/2123938481 learned_addr learned my addr 172.21.15.122:0/2123938481 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55dcfb0640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55d8102dd0 0x7f55d81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55dd7b1640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8108300 0x7f55d819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 -- 172.21.15.122:0/2123938481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8108300 msgr2=0x7f55d819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8108300 0x7f55d819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 -- 172.21.15.122:0/2123938481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55d8102dd0 msgr2=0x7f55d81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55d8102dd0 0x7f55d81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 -- 172.21.15.122:0/2123938481 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d819d680 con 0x7f55d8103780 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55dd7b1640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8108300 0x7f55d819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:44:57.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d7fff640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 0x7f55d8198c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55c800b990 tx=0x7f55c800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8019070 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c8014440 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d819d970 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.583+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c80125a0 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.584+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a5250 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.585+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d8103230 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.586+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f55c801c070 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.587+0000 7f55d5ffb640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55c40a2c10 0x7f55c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.587+0000 7f55dcfb0640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55c40a2c10 0x7f55c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.587+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f55c80ce6e0 con 0x7f55d8103780 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.587+0000 7f55dcfb0640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55c40a2c10 0x7f55c40a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009990 tx=0x7f55d0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.591+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55c809b680 con 0x7f55d8103780 2024-09-12T01:44:57.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.795+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55d810a290 con 0x7f55d8103780 2024-09-12T01:44:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.796+0000 7f55d5ffb640 1 -- 172.21.15.122:0/2123938481 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f55c809af80 con 0x7f55d8103780 2024-09-12T01:44:57.796 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:57.796 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"btime":"2024-09-12T01:44:47:831953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:44:47.439571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:44:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:44:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55c40a2c10 msgr2=0x7f55c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55c40a2c10 0x7f55c40a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009990 tx=0x7f55d0009110 comp rx=0 tx=0).stop 2024-09-12T01:44:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 msgr2=0x7f55d8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 0x7f55d8198c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55c800b990 tx=0x7f55c800bd90 comp rx=0 tx=0).stop 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 shutdown_connections 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55c40a2c10 0x7f55c40a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55d8108300 0x7f55d819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d8103780 0x7f55d8198c10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 --2- 172.21.15.122:0/2123938481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55d8102dd0 0x7f55d81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 >> 172.21.15.122:0/2123938481 conn(0x7f55d80fc7d0 msgr2=0x7f55d8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 shutdown_connections 2024-09-12T01:44:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:57.798+0000 7f55df214640 1 -- 172.21.15.122:0/2123938481 wait complete. 2024-09-12T01:44:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:57 smithi151 ceph-mon[35229]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:57 smithi151 ceph-mon[35229]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 KiB/s wr, 14 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:44:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:57 smithi183 ceph-mon[35127]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:44:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:57 smithi183 ceph-mon[35127]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 KiB/s wr, 14 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:44:58.470 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:44:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:44:58.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:58 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2123938481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:58 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2123938481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:58.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:58 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2123938481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 -- 172.21.15.122:0/1309913558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c106f50 msgr2=0x7f814c10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 --2- 172.21.15.122:0/1309913558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c106f50 0x7f814c10eaf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8144009930 tx=0x7f814402a420 comp rx=0 tx=0).stop 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 -- 172.21.15.122:0/1309913558 shutdown_connections 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 --2- 172.21.15.122:0/1309913558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c106f50 0x7f814c10eaf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 --2- 172.21.15.122:0/1309913558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 0x7f814c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 --2- 172.21.15.122:0/1309913558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c100350 0x7f814c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 -- 172.21.15.122:0/1309913558 >> 172.21.15.122:0/1309913558 conn(0x7f814c077210 msgr2=0x7f814c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:59.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 -- 172.21.15.122:0/1309913558 shutdown_connections 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.221+0000 7f815159e640 1 -- 172.21.15.122:0/1309913558 wait complete. 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 Processor -- start 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 -- start start 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 0x7f814c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 0x7f814c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c106f50 0x7f814c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f814c111fa0 con 0x7f814c100350 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f814c111e20 con 0x7f814c106f50 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f815159e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f814c112120 con 0x7f814c100d00 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f814affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 0x7f814c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f814affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 0x7f814c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57072/0 (socket says 172.21.15.122:57072) 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.222+0000 7f814affd640 1 -- 172.21.15.122:0/1660899038 learned_addr learned my addr 172.21.15.122:0/1660899038 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814a7fc640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 0x7f814c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:59.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814b7fe640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c106f50 0x7f814c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814affd640 1 -- 172.21.15.122:0/1660899038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 msgr2=0x7f814c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814affd640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 0x7f814c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814affd640 1 -- 172.21.15.122:0/1660899038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c106f50 msgr2=0x7f814c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814affd640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c106f50 0x7f814c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814affd640 1 -- 172.21.15.122:0/1660899038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c19d310 con 0x7f814c100350 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814a7fc640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 0x7f814c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814b7fe640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c106f50 0x7f814c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f814affd640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 0x7f814c198360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f813c00cce0 tx=0x7f813c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c00f650 con 0x7f814c100350 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c19d600 con 0x7f814c100350 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c00fcd0 con 0x7f814c100350 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c018c70 con 0x7f814c100350 2024-09-12T01:44:59.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.223+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1a4ee0 con 0x7f814c100350 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.224+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c100730 con 0x7f814c100350 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.225+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f813c00f7f0 con 0x7f814c100350 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.226+0000 7f8143fff640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81380a2c10 0x7f81380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.226+0000 7f814a7fc640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81380a2c10 0x7f81380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.226+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f813c0cf740 con 0x7f814c100350 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.226+0000 7f814a7fc640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81380a2c10 0x7f81380a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8134005e10 tx=0x7f8134005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:44:59.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.230+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f813c014030 con 0x7f814c100350 2024-09-12T01:44:59.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.400+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f814c1996c0 con 0x7f814c100350 2024-09-12T01:44:59.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.401+0000 7f8143fff640 1 -- 172.21.15.122:0/1660899038 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7f813c09c540 con 0x7f814c100350 2024-09-12T01:44:59.402 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:44:59.402 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81380a2c10 msgr2=0x7f81380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81380a2c10 0x7f81380a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8134005e10 tx=0x7f8134005da0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 msgr2=0x7f814c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 0x7f814c198360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f813c00cce0 tx=0x7f813c00a620 comp rx=0 tx=0).stop 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 shutdown_connections 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81380a2c10 0x7f81380a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f814c106f50 0x7f814c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f814c100d00 0x7f814c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 --2- 172.21.15.122:0/1660899038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c100350 0x7f814c198360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:44:59.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 >> 172.21.15.122:0/1660899038 conn(0x7f814c077210 msgr2=0x7f814c105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:44:59.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 shutdown_connections 2024-09-12T01:44:59.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:44:59.404+0000 7f815159e640 1 -- 172.21.15.122:0/1660899038 wait complete. 2024-09-12T01:44:59.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:59 smithi122 ceph-mon[30925]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:44:59.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:44:59 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1660899038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:59 smithi151 ceph-mon[35229]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:44:59.868 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:44:59 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1660899038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:44:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:59 smithi183 ceph-mon[35127]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:44:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:44:59 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1660899038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:00.082 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T01:45:00.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:45:00.325 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:00.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 -- 172.21.15.122:0/3943528615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 msgr2=0x7fcb34106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:00.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3943528615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb34106e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009930 tx=0x7fcb2802a420 comp rx=0 tx=0).stop 2024-09-12T01:45:00.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 -- 172.21.15.122:0/3943528615 shutdown_connections 2024-09-12T01:45:00.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3943528615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34101420 0x7fcb3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3943528615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34100a80 0x7fcb34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3943528615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb34106e60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 -- 172.21.15.122:0/3943528615 >> 172.21.15.122:0/3943528615 conn(0x7fcb340fc7a0 msgr2=0x7fcb340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 -- 172.21.15.122:0/3943528615 shutdown_connections 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.785+0000 7fcb3abad640 1 -- 172.21.15.122:0/3943528615 wait complete. 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 Processor -- start 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 -- start start 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34100a80 0x7fcb34102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34101420 0x7fcb34102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb341030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34110f20 con 0x7fcb34106a80 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34110da0 con 0x7fcb34101420 2024-09-12T01:45:00.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3abad640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb341110a0 con 0x7fcb34100a80 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb341030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb341030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48778/0 (socket says 172.21.15.122:48778) 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 -- 172.21.15.122:0/3029561707 learned_addr learned my addr 172.21.15.122:0/3029561707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb33fff640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34101420 0x7fcb34102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb38949640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34100a80 0x7fcb34102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 -- 172.21.15.122:0/3029561707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34100a80 msgr2=0x7fcb34102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34100a80 0x7fcb34102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 -- 172.21.15.122:0/3029561707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34101420 msgr2=0x7fcb34102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34101420 0x7fcb34102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 -- 172.21.15.122:0/3029561707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb341a6a80 con 0x7fcb34106a80 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb33fff640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34101420 0x7fcb34102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:00.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb38949640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34100a80 0x7fcb34102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.786+0000 7fcb3914a640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb341030d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b970 tx=0x7fcb2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.787+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c019070 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.787+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb2c00cb40 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.787+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb341a6d70 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.787+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c007650 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.787+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341a72d0 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.788+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb34100f60 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.789+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcb2c01c070 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.790+0000 7fcb31ffb640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcb240a2c90 0x7fcb240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.790+0000 7fcb38949640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcb240a2c90 0x7fcb240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.790+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcb2c0ce620 con 0x7fcb34106a80 2024-09-12T01:45:00.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.790+0000 7fcb38949640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcb240a2c90 0x7fcb240a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb28032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:00.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.794+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb2c09b530 con 0x7fcb34106a80 2024-09-12T01:45:00.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.957+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb340fce70 con 0x7fcb240a2c90 2024-09-12T01:45:00.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.963+0000 7fcb31ffb640 1 -- 172.21.15.122:0/3029561707 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258951 (secure 0 0 0) 0x7fcb340fce70 con 0x7fcb240a2c90 2024-09-12T01:45:00.966 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:45:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcb240a2c90 msgr2=0x7fcb240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcb240a2c90 0x7fcb240a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb28032d50 comp rx=0 tx=0).stop 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 msgr2=0x7fcb341030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb341030d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b970 tx=0x7fcb2c00be30 comp rx=0 tx=0).stop 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 shutdown_connections 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcb240a2c90 0x7fcb240a5150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb34106a80 0x7fcb341030d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb34101420 0x7fcb34102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 --2- 172.21.15.122:0/3029561707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb34100a80 0x7fcb34102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 >> 172.21.15.122:0/3029561707 conn(0x7fcb340fc7a0 msgr2=0x7fcb341048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 shutdown_connections 2024-09-12T01:45:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:00.966+0000 7fcb3abad640 1 -- 172.21.15.122:0/3029561707 wait complete. 2024-09-12T01:45:01.705 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-12T01:45:00.450514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635670,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":212,"num_read_kb":200,"num_write":333,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360260,"kb_used_data":11004,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581564,"statfs":{"total":1151940427776,"available":1151571521536,"internally_reserved":0,"allocated":11268096,"data_stored":7175636,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41486,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":-4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":78,"num_read_kb":84,"num_write":60,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003506"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818499+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:44:45.818499+0000","last_peered":"2024-09-12T01:44:45.818499+0000","last_clean":"2024-09-12T01:44:45.818499+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:44:45.818499+0000","last_undegraded":"2024-09-12T01:44:45.818499+0000","last_fullsized":"2024-09-12T01:44:45.818499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076814+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:44:46.076814+0000","last_peered":"2024-09-12T01:44:46.076814+0000","last_clean":"2024-09-12T01:44:46.076814+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:44:46.076814+0000","last_undegraded":"2024-09-12T01:44:46.076814+0000","last_fullsized":"2024-09-12T01:44:46.076814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076808+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:44:46.076808+0000","last_peered":"2024-09-12T01:44:46.076808+0000","last_clean":"2024-09-12T01:44:46.076808+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:44:46.076808+0000","last_undegraded":"2024-09-12T01:44:46.076808+0000","last_fullsized":"2024-09-12T01:44:46.076808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815947+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:44:45.815947+0000","last_peered":"2024-09-12T01:44:45.815947+0000","last_clean":"2024-09-12T01:44:45.815947+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:44:45.815947+0000","last_undegraded":"2024-09-12T01:44:45.815947+0000","last_fullsized":"2024-09-12T01:44:45.815947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819617+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:44:45.819617+0000","last_peered":"2024-09-12T01:44:45.819617+0000","last_clean":"2024-09-12T01:44:45.819617+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:44:45.819617+0000","last_undegraded":"2024-09-12T01:44:45.819617+0000","last_fullsized":"2024-09-12T01:44:45.819617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818302+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:44:45.818302+0000","last_peered":"2024-09-12T01:44:45.818302+0000","last_clean":"2024-09-12T01:44:45.818302+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:44:45.818302+0000","last_undegraded":"2024-09-12T01:44:45.818302+0000","last_fullsized":"2024-09-12T01:44:45.818302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077967+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:44:46.077967+0000","last_peered":"2024-09-12T01:44:46.077967+0000","last_clean":"2024-09-12T01:44:46.077967+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:44:46.077967+0000","last_undegraded":"2024-09-12T01:44:46.077967+0000","last_fullsized":"2024-09-12T01:44:46.077967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821645+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:44:45.821645+0000","last_peered":"2024-09-12T01:44:45.821645+0000","last_clean":"2024-09-12T01:44:45.821645+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:44:45.821645+0000","last_undegraded":"2024-09-12T01:44:45.821645+0000","last_fullsized":"2024-09-12T01:44:45.821645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821329+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:44:45.821329+0000","last_peered":"2024-09-12T01:44:45.821329+0000","last_clean":"2024-09-12T01:44:45.821329+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:44:45.821329+0000","last_undegraded":"2024-09-12T01:44:45.821329+0000","last_fullsized":"2024-09-12T01:44:45.821329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824934+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:44:45.824934+0000","last_peered":"2024-09-12T01:44:45.824934+0000","last_clean":"2024-09-12T01:44:45.824934+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:44:45.824934+0000","last_undegraded":"2024-09-12T01:44:45.824934+0000","last_fullsized":"2024-09-12T01:44:45.824934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818038+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:44:45.818038+0000","last_peered":"2024-09-12T01:44:45.818038+0000","last_clean":"2024-09-12T01:44:45.818038+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:44:45.818038+0000","last_undegraded":"2024-09-12T01:44:45.818038+0000","last_fullsized":"2024-09-12T01:44:45.818038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822055+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:44:45.822055+0000","last_peered":"2024-09-12T01:44:45.822055+0000","last_clean":"2024-09-12T01:44:45.822055+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:44:45.822055+0000","last_undegraded":"2024-09-12T01:44:45.822055+0000","last_fullsized":"2024-09-12T01:44:45.822055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077339+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:44:46.077339+0000","last_peered":"2024-09-12T01:44:46.077339+0000","last_clean":"2024-09-12T01:44:46.077339+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:44:46.077339+0000","last_undegraded":"2024-09-12T01:44:46.077339+0000","last_fullsized":"2024-09-12T01:44:46.077339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076657+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:44:46.076657+0000","last_peered":"2024-09-12T01:44:46.076657+0000","last_clean":"2024-09-12T01:44:46.076657+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:44:46.076657+0000","last_undegraded":"2024-09-12T01:44:46.076657+0000","last_fullsized":"2024-09-12T01:44:46.076657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076627+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:44:46.076627+0000","last_peered":"2024-09-12T01:44:46.076627+0000","last_clean":"2024-09-12T01:44:46.076627+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:44:46.076627+0000","last_undegraded":"2024-09-12T01:44:46.076627+0000","last_fullsized":"2024-09-12T01:44:46.076627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820478+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:44:45.820478+0000","last_peered":"2024-09-12T01:44:45.820478+0000","last_clean":"2024-09-12T01:44:45.820478+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:44:45.820478+0000","last_undegraded":"2024-09-12T01:44:45.820478+0000","last_fullsized":"2024-09-12T01:44:45.820478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818598+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:44:45.818598+0000","last_peered":"2024-09-12T01:44:45.818598+0000","last_clean":"2024-09-12T01:44:45.818598+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:44:45.818598+0000","last_undegraded":"2024-09-12T01:44:45.818598+0000","last_fullsized":"2024-09-12T01:44:45.818598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816391+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:44:45.816391+0000","last_peered":"2024-09-12T01:44:45.816391+0000","last_clean":"2024-09-12T01:44:45.816391+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:44:45.816391+0000","last_undegraded":"2024-09-12T01:44:45.816391+0000","last_fullsized":"2024-09-12T01:44:45.816391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821743+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:44:45.821743+0000","last_peered":"2024-09-12T01:44:45.821743+0000","last_clean":"2024-09-12T01:44:45.821743+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:44:45.821743+0000","last_undegraded":"2024-09-12T01:44:45.821743+0000","last_fullsized":"2024-09-12T01:44:45.821743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818574+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:44:45.818574+0000","last_peered":"2024-09-12T01:44:45.818574+0000","last_clean":"2024-09-12T01:44:45.818574+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:44:45.818574+0000","last_undegraded":"2024-09-12T01:44:45.818574+0000","last_fullsized":"2024-09-12T01:44:45.818574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077436+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:44:46.077436+0000","last_peered":"2024-09-12T01:44:46.077436+0000","last_clean":"2024-09-12T01:44:46.077436+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:44:46.077436+0000","last_undegraded":"2024-09-12T01:44:46.077436+0000","last_fullsized":"2024-09-12T01:44:46.077436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820692+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:44:45.820692+0000","last_peered":"2024-09-12T01:44:45.820692+0000","last_clean":"2024-09-12T01:44:45.820692+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:44:45.820692+0000","last_undegraded":"2024-09-12T01:44:45.820692+0000","last_fullsized":"2024-09-12T01:44:45.820692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821444+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:44:45.821444+0000","last_peered":"2024-09-12T01:44:45.821444+0000","last_clean":"2024-09-12T01:44:45.821444+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:44:45.821444+0000","last_undegraded":"2024-09-12T01:44:45.821444+0000","last_fullsized":"2024-09-12T01:44:45.821444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824121+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:44:45.824121+0000","last_peered":"2024-09-12T01:44:45.824121+0000","last_clean":"2024-09-12T01:44:45.824121+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:44:45.824121+0000","last_undegraded":"2024-09-12T01:44:45.824121+0000","last_fullsized":"2024-09-12T01:44:45.824121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821120+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:44:45.821120+0000","last_peered":"2024-09-12T01:44:45.821120+0000","last_clean":"2024-09-12T01:44:45.821120+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:44:45.821120+0000","last_undegraded":"2024-09-12T01:44:45.821120+0000","last_fullsized":"2024-09-12T01:44:45.821120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822863+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:44:45.822863+0000","last_peered":"2024-09-12T01:44:45.822863+0000","last_clean":"2024-09-12T01:44:45.822863+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:44:45.822863+0000","last_undegraded":"2024-09-12T01:44:45.822863+0000","last_fullsized":"2024-09-12T01:44:45.822863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817853+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:44:45.817853+0000","last_peered":"2024-09-12T01:44:45.817853+0000","last_clean":"2024-09-12T01:44:45.817853+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:44:45.817853+0000","last_undegraded":"2024-09-12T01:44:45.817853+0000","last_fullsized":"2024-09-12T01:44:45.817853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077201+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:44:46.077201+0000","last_peered":"2024-09-12T01:44:46.077201+0000","last_clean":"2024-09-12T01:44:46.077201+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:44:46.077201+0000","last_undegraded":"2024-09-12T01:44:46.077201+0000","last_fullsized":"2024-09-12T01:44:46.077201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076827+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:44:46.076827+0000","last_peered":"2024-09-12T01:44:46.076827+0000","last_clean":"2024-09-12T01:44:46.076827+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:44:46.076827+0000","last_undegraded":"2024-09-12T01:44:46.076827+0000","last_fullsized":"2024-09-12T01:44:46.076827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820905+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:44:45.820905+0000","last_peered":"2024-09-12T01:44:45.820905+0000","last_clean":"2024-09-12T01:44:45.820905+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:44:45.820905+0000","last_undegraded":"2024-09-12T01:44:45.820905+0000","last_fullsized":"2024-09-12T01:44:45.820905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821482+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:44:45.821482+0000","last_peered":"2024-09-12T01:44:45.821482+0000","last_clean":"2024-09-12T01:44:45.821482+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:44:45.821482+0000","last_undegraded":"2024-09-12T01:44:45.821482+0000","last_fullsized":"2024-09-12T01:44:45.821482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817474+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:44:45.817474+0000","last_peered":"2024-09-12T01:44:45.817474+0000","last_clean":"2024-09-12T01:44:45.817474+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:44:45.817474+0000","last_undegraded":"2024-09-12T01:44:45.817474+0000","last_fullsized":"2024-09-12T01:44:45.817474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819263+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:44:45.819263+0000","last_peered":"2024-09-12T01:44:45.819263+0000","last_clean":"2024-09-12T01:44:45.819263+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:44:45.819263+0000","last_undegraded":"2024-09-12T01:44:45.819263+0000","last_fullsized":"2024-09-12T01:44:45.819263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:56.853306+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:44:56.853306+0000","last_peered":"2024-09-12T01:44:56.853306+0000","last_clean":"2024-09-12T01:44:56.853306+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:44:56.853306+0000","last_undegraded":"2024-09-12T01:44:56.853306+0000","last_fullsized":"2024-09-12T01:44:56.853306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815476+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:44:45.815476+0000","last_peered":"2024-09-12T01:44:45.815476+0000","last_clean":"2024-09-12T01:44:45.815476+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:44:45.815476+0000","last_undegraded":"2024-09-12T01:44:45.815476+0000","last_fullsized":"2024-09-12T01:44:45.815476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821664+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:44:45.821664+0000","last_peered":"2024-09-12T01:44:45.821664+0000","last_clean":"2024-09-12T01:44:45.821664+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:44:45.821664+0000","last_undegraded":"2024-09-12T01:44:45.821664+0000","last_fullsized":"2024-09-12T01:44:45.821664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816283+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:44:45.816283+0000","last_peered":"2024-09-12T01:44:45.816283+0000","last_clean":"2024-09-12T01:44:45.816283+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:44:45.816283+0000","last_undegraded":"2024-09-12T01:44:45.816283+0000","last_fullsized":"2024-09-12T01:44:45.816283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:53.847882+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:44:53.847882+0000","last_peered":"2024-09-12T01:44:53.847882+0000","last_clean":"2024-09-12T01:44:53.847882+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:44:53.847882+0000","last_undegraded":"2024-09-12T01:44:53.847882+0000","last_fullsized":"2024-09-12T01:44:53.847882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821400+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:44:45.821400+0000","last_peered":"2024-09-12T01:44:45.821400+0000","last_clean":"2024-09-12T01:44:45.821400+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:44:45.821400+0000","last_undegraded":"2024-09-12T01:44:45.821400+0000","last_fullsized":"2024-09-12T01:44:45.821400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077017+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:44:46.077017+0000","last_peered":"2024-09-12T01:44:46.077017+0000","last_clean":"2024-09-12T01:44:46.077017+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:44:46.077017+0000","last_undegraded":"2024-09-12T01:44:46.077017+0000","last_fullsized":"2024-09-12T01:44:46.077017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820850+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:44:45.820850+0000","last_peered":"2024-09-12T01:44:45.820850+0000","last_clean":"2024-09-12T01:44:45.820850+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:44:45.820850+0000","last_undegraded":"2024-09-12T01:44:45.820850+0000","last_fullsized":"2024-09-12T01:44:45.820850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'21","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:54.853410+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:44:54.853410+0000","last_peered":"2024-09-12T01:44:54.853410+0000","last_clean":"2024-09-12T01:44:54.853410+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:44:54.853410+0000","last_undegraded":"2024-09-12T01:44:54.853410+0000","last_fullsized":"2024-09-12T01:44:54.853410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":51,"num_write":22,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820311+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:44:45.820311+0000","last_peered":"2024-09-12T01:44:45.820311+0000","last_clean":"2024-09-12T01:44:45.820311+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:44:45.820311+0000","last_undegraded":"2024-09-12T01:44:45.820311+0000","last_fullsized":"2024-09-12T01:44:45.820311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077080+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:44:46.077080+0000","last_peered":"2024-09-12T01:44:46.077080+0000","last_clean":"2024-09-12T01:44:46.077080+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:44:46.077080+0000","last_undegraded":"2024-09-12T01:44:46.077080+0000","last_fullsized":"2024-09-12T01:44:46.077080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817662+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:44:45.817662+0000","last_peered":"2024-09-12T01:44:45.817662+0000","last_clean":"2024-09-12T01:44:45.817662+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:44:45.817662+0000","last_undegraded":"2024-09-12T01:44:45.817662+0000","last_fullsized":"2024-09-12T01:44:45.817662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:55.854850+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:44:55.854850+0000","last_peered":"2024-09-12T01:44:55.854850+0000","last_clean":"2024-09-12T01:44:55.854850+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:44:55.854850+0000","last_undegraded":"2024-09-12T01:44:55.854850+0000","last_fullsized":"2024-09-12T01:44:55.854850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817679+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:44:45.817679+0000","last_peered":"2024-09-12T01:44:45.817679+0000","last_clean":"2024-09-12T01:44:45.817679+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:44:45.817679+0000","last_undegraded":"2024-09-12T01:44:45.817679+0000","last_fullsized":"2024-09-12T01:44:45.817679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822535+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:44:45.822535+0000","last_peered":"2024-09-12T01:44:45.822535+0000","last_clean":"2024-09-12T01:44:45.822535+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:44:45.822535+0000","last_undegraded":"2024-09-12T01:44:45.822535+0000","last_fullsized":"2024-09-12T01:44:45.822535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818060+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:44:45.818060+0000","last_peered":"2024-09-12T01:44:45.818060+0000","last_clean":"2024-09-12T01:44:45.818060+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:44:45.818060+0000","last_undegraded":"2024-09-12T01:44:45.818060+0000","last_fullsized":"2024-09-12T01:44:45.818060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818393+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:44:45.818393+0000","last_peered":"2024-09-12T01:44:45.818393+0000","last_clean":"2024-09-12T01:44:45.818393+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:44:45.818393+0000","last_undegraded":"2024-09-12T01:44:45.818393+0000","last_fullsized":"2024-09-12T01:44:45.818393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820335+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:44:45.820335+0000","last_peered":"2024-09-12T01:44:45.820335+0000","last_clean":"2024-09-12T01:44:45.820335+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:44:45.820335+0000","last_undegraded":"2024-09-12T01:44:45.820335+0000","last_fullsized":"2024-09-12T01:44:45.820335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818057+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:44:45.818057+0000","last_peered":"2024-09-12T01:44:45.818057+0000","last_clean":"2024-09-12T01:44:45.818057+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:44:45.818057+0000","last_undegraded":"2024-09-12T01:44:45.818057+0000","last_fullsized":"2024-09-12T01:44:45.818057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818530+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818530+0000","last_peered":"2024-09-12T01:44:45.818530+0000","last_clean":"2024-09-12T01:44:45.818530+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818530+0000","last_undegraded":"2024-09-12T01:44:45.818530+0000","last_fullsized":"2024-09-12T01:44:45.818530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821278+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:44:45.821278+0000","last_peered":"2024-09-12T01:44:45.821278+0000","last_clean":"2024-09-12T01:44:45.821278+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:44:45.821278+0000","last_undegraded":"2024-09-12T01:44:45.821278+0000","last_fullsized":"2024-09-12T01:44:45.821278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077069+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:44:46.077069+0000","last_peered":"2024-09-12T01:44:46.077069+0000","last_clean":"2024-09-12T01:44:46.077069+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:44:46.077069+0000","last_undegraded":"2024-09-12T01:44:46.077069+0000","last_fullsized":"2024-09-12T01:44:46.077069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:56.856042+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:44:56.856042+0000","last_peered":"2024-09-12T01:44:56.856042+0000","last_clean":"2024-09-12T01:44:56.856042+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:44:56.856042+0000","last_undegraded":"2024-09-12T01:44:56.856042+0000","last_fullsized":"2024-09-12T01:44:56.856042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824468+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:44:45.824468+0000","last_peered":"2024-09-12T01:44:45.824468+0000","last_clean":"2024-09-12T01:44:45.824468+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:44:45.824468+0000","last_undegraded":"2024-09-12T01:44:45.824468+0000","last_fullsized":"2024-09-12T01:44:45.824468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818919+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:44:45.818919+0000","last_peered":"2024-09-12T01:44:45.818919+0000","last_clean":"2024-09-12T01:44:45.818919+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:44:45.818919+0000","last_undegraded":"2024-09-12T01:44:45.818919+0000","last_fullsized":"2024-09-12T01:44:45.818919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819406+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:44:45.819406+0000","last_peered":"2024-09-12T01:44:45.819406+0000","last_clean":"2024-09-12T01:44:45.819406+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:44:45.819406+0000","last_undegraded":"2024-09-12T01:44:45.819406+0000","last_fullsized":"2024-09-12T01:44:45.819406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077325+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:44:46.077325+0000","last_peered":"2024-09-12T01:44:46.077325+0000","last_clean":"2024-09-12T01:44:46.077325+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:44:46.077325+0000","last_undegraded":"2024-09-12T01:44:46.077325+0000","last_fullsized":"2024-09-12T01:44:46.077325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:56.780804+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:44:56.780804+0000","last_peered":"2024-09-12T01:44:56.780804+0000","last_clean":"2024-09-12T01:44:56.780804+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:44:56.780804+0000","last_undegraded":"2024-09-12T01:44:56.780804+0000","last_fullsized":"2024-09-12T01:44:56.780804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821213+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:44:45.821213+0000","last_peered":"2024-09-12T01:44:45.821213+0000","last_clean":"2024-09-12T01:44:45.821213+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:44:45.821213+0000","last_undegraded":"2024-09-12T01:44:45.821213+0000","last_fullsized":"2024-09-12T01:44:45.821213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076938+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:44:46.076938+0000","last_peered":"2024-09-12T01:44:46.076938+0000","last_clean":"2024-09-12T01:44:46.076938+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:44:46.076938+0000","last_undegraded":"2024-09-12T01:44:46.076938+0000","last_fullsized":"2024-09-12T01:44:46.076938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819237+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:44:45.819237+0000","last_peered":"2024-09-12T01:44:45.819237+0000","last_clean":"2024-09-12T01:44:45.819237+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:44:45.819237+0000","last_undegraded":"2024-09-12T01:44:45.819237+0000","last_fullsized":"2024-09-12T01:44:45.819237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817496+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:44:45.817496+0000","last_peered":"2024-09-12T01:44:45.817496+0000","last_clean":"2024-09-12T01:44:45.817496+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:44:45.817496+0000","last_undegraded":"2024-09-12T01:44:45.817496+0000","last_fullsized":"2024-09-12T01:44:45.817496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821064+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:44:45.821064+0000","last_peered":"2024-09-12T01:44:45.821064+0000","last_clean":"2024-09-12T01:44:45.821064+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:44:45.821064+0000","last_undegraded":"2024-09-12T01:44:45.821064+0000","last_fullsized":"2024-09-12T01:44:45.821064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818032+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:44:45.818032+0000","last_peered":"2024-09-12T01:44:45.818032+0000","last_clean":"2024-09-12T01:44:45.818032+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:44:45.818032+0000","last_undegraded":"2024-09-12T01:44:45.818032+0000","last_fullsized":"2024-09-12T01:44:45.818032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816211+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:44:45.816211+0000","last_peered":"2024-09-12T01:44:45.816211+0000","last_clean":"2024-09-12T01:44:45.816211+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:44:45.816211+0000","last_undegraded":"2024-09-12T01:44:45.816211+0000","last_fullsized":"2024-09-12T01:44:45.816211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820739+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:44:45.820739+0000","last_peered":"2024-09-12T01:44:45.820739+0000","last_clean":"2024-09-12T01:44:45.820739+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:44:45.820739+0000","last_undegraded":"2024-09-12T01:44:45.820739+0000","last_fullsized":"2024-09-12T01:44:45.820739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076981+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:44:46.076981+0000","last_peered":"2024-09-12T01:44:46.076981+0000","last_clean":"2024-09-12T01:44:46.076981+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:44:46.076981+0000","last_undegraded":"2024-09-12T01:44:46.076981+0000","last_fullsized":"2024-09-12T01:44:46.076981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820929+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:44:45.820929+0000","last_peered":"2024-09-12T01:44:45.820929+0000","last_clean":"2024-09-12T01:44:45.820929+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:44:45.820929+0000","last_undegraded":"2024-09-12T01:44:45.820929+0000","last_fullsized":"2024-09-12T01:44:45.820929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":45068,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":86,"num_read_kb":91,"num_write":150,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":157274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":1070734,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029883,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438250,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444095,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":490629,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":489313,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":53416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:45:01.710 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:45:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:01 smithi151 ceph-mon[35229]: pgmap v291: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:45:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:01 smithi183 ceph-mon[35127]: pgmap v291: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:45:01.956 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:01.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:01 smithi122 ceph-mon[30925]: pgmap v291: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 -- 172.21.15.122:0/245436189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100480 msgr2=0x7fe6e0100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/245436189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100480 0x7fe6e0100860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009ca0 tx=0x7fe6d802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 -- 172.21.15.122:0/245436189 shutdown_connections 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/245436189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e01055b0 0x7fe6e010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/245436189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6e0100e30 0x7fe6e0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/245436189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100480 0x7fe6e0100860 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 -- 172.21.15.122:0/245436189 >> 172.21.15.122:0/245436189 conn(0x7fe6e0077210 msgr2=0x7fe6e00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 -- 172.21.15.122:0/245436189 shutdown_connections 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.434+0000 7fe6e7a71640 1 -- 172.21.15.122:0/245436189 wait complete. 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 Processor -- start 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 -- start start 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e0100480 0x7fe6e019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 0x7fe6e019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6e01055b0 0x7fe6e019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010fd90 con 0x7fe6e0100e30 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010fc10 con 0x7fe6e0100480 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e7a71640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010ff10 con 0x7fe6e01055b0 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 0x7fe6e019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 0x7fe6e019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48784/0 (socket says 172.21.15.122:48784) 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 -- 172.21.15.122:0/287836130 learned_addr learned my addr 172.21.15.122:0/287836130 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:02.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e600e640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6e01055b0 0x7fe6e019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e580d640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e0100480 0x7fe6e019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 -- 172.21.15.122:0/287836130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6e01055b0 msgr2=0x7fe6e019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6e01055b0 0x7fe6e019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 -- 172.21.15.122:0/287836130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e0100480 msgr2=0x7fe6e019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e0100480 0x7fe6e019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.435+0000 7fe6e500c640 1 -- 172.21.15.122:0/287836130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e019f610 con 0x7fe6e0100e30 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.436+0000 7fe6e500c640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 0x7fe6e019aba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400dbc0 tx=0x7fe6d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.436+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4005220 con 0x7fe6e0100e30 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.436+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d400ae50 con 0x7fe6e0100e30 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.436+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400e870 con 0x7fe6e0100e30 2024-09-12T01:45:02.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.436+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e019f900 con 0x7fe6e0100e30 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.436+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01a71e0 con 0x7fe6e0100e30 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.437+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e0100860 con 0x7fe6e0100e30 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.438+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe6d400ea10 con 0x7fe6e0100e30 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.438+0000 7fe6deffd640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6c80a2c00 0x7fe6c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.439+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe6d40cf620 con 0x7fe6e0100e30 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.439+0000 7fe6e580d640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6c80a2c00 0x7fe6c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.439+0000 7fe6e580d640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6c80a2c00 0x7fe6c80a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000f80 tx=0x7fe6d80093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:02.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.442+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6d409c530 con 0x7fe6e0100e30 2024-09-12T01:45:02.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.607+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6e0062630 con 0x7fe6c80a2c00 2024-09-12T01:45:02.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:02 smithi122 ceph-mon[30925]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:02.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.614+0000 7fe6deffd640 1 -- 172.21.15.122:0/287836130 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258951 (secure 0 0 0) 0x7fe6e0062630 con 0x7fe6c80a2c00 2024-09-12T01:45:02.616 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:02.617 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6c80a2c00 msgr2=0x7fe6c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6c80a2c00 0x7fe6c80a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000f80 tx=0x7fe6d80093f0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 msgr2=0x7fe6e019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 0x7fe6e019aba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400dbc0 tx=0x7fe6d400a620 comp rx=0 tx=0).stop 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 shutdown_connections 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6c80a2c00 0x7fe6c80a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6e01055b0 0x7fe6e019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6e0100e30 0x7fe6e019aba0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 --2- 172.21.15.122:0/287836130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e0100480 0x7fe6e019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.616+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 >> 172.21.15.122:0/287836130 conn(0x7fe6e0077210 msgr2=0x7fe6e0103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.617+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 shutdown_connections 2024-09-12T01:45:02.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:02.617+0000 7fe6e7a71640 1 -- 172.21.15.122:0/287836130 wait complete. 2024-09-12T01:45:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:02 smithi151 ceph-mon[35229]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:02.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:02 smithi183 ceph-mon[35127]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:03.418 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-12T01:45:02.450941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646705,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":228,"num_read_kb":221,"num_write":338,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360276,"kb_used_data":11020,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581548,"statfs":{"total":1151940427776,"available":1151571505152,"internally_reserved":0,"allocated":11284480,"data_stored":7190928,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51115,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":-4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":93,"num_write":35,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003380"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818499+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:44:45.818499+0000","last_peered":"2024-09-12T01:44:45.818499+0000","last_clean":"2024-09-12T01:44:45.818499+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:44:45.818499+0000","last_undegraded":"2024-09-12T01:44:45.818499+0000","last_fullsized":"2024-09-12T01:44:45.818499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076814+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:44:46.076814+0000","last_peered":"2024-09-12T01:44:46.076814+0000","last_clean":"2024-09-12T01:44:46.076814+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:44:46.076814+0000","last_undegraded":"2024-09-12T01:44:46.076814+0000","last_fullsized":"2024-09-12T01:44:46.076814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076808+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:44:46.076808+0000","last_peered":"2024-09-12T01:44:46.076808+0000","last_clean":"2024-09-12T01:44:46.076808+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:44:46.076808+0000","last_undegraded":"2024-09-12T01:44:46.076808+0000","last_fullsized":"2024-09-12T01:44:46.076808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815947+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:44:45.815947+0000","last_peered":"2024-09-12T01:44:45.815947+0000","last_clean":"2024-09-12T01:44:45.815947+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:44:45.815947+0000","last_undegraded":"2024-09-12T01:44:45.815947+0000","last_fullsized":"2024-09-12T01:44:45.815947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819617+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:44:45.819617+0000","last_peered":"2024-09-12T01:44:45.819617+0000","last_clean":"2024-09-12T01:44:45.819617+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:44:45.819617+0000","last_undegraded":"2024-09-12T01:44:45.819617+0000","last_fullsized":"2024-09-12T01:44:45.819617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818302+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:44:45.818302+0000","last_peered":"2024-09-12T01:44:45.818302+0000","last_clean":"2024-09-12T01:44:45.818302+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:44:45.818302+0000","last_undegraded":"2024-09-12T01:44:45.818302+0000","last_fullsized":"2024-09-12T01:44:45.818302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077967+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:44:46.077967+0000","last_peered":"2024-09-12T01:44:46.077967+0000","last_clean":"2024-09-12T01:44:46.077967+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:44:46.077967+0000","last_undegraded":"2024-09-12T01:44:46.077967+0000","last_fullsized":"2024-09-12T01:44:46.077967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821645+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:44:45.821645+0000","last_peered":"2024-09-12T01:44:45.821645+0000","last_clean":"2024-09-12T01:44:45.821645+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:44:45.821645+0000","last_undegraded":"2024-09-12T01:44:45.821645+0000","last_fullsized":"2024-09-12T01:44:45.821645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821329+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:44:45.821329+0000","last_peered":"2024-09-12T01:44:45.821329+0000","last_clean":"2024-09-12T01:44:45.821329+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:44:45.821329+0000","last_undegraded":"2024-09-12T01:44:45.821329+0000","last_fullsized":"2024-09-12T01:44:45.821329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824934+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:44:45.824934+0000","last_peered":"2024-09-12T01:44:45.824934+0000","last_clean":"2024-09-12T01:44:45.824934+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:44:45.824934+0000","last_undegraded":"2024-09-12T01:44:45.824934+0000","last_fullsized":"2024-09-12T01:44:45.824934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818038+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:44:45.818038+0000","last_peered":"2024-09-12T01:44:45.818038+0000","last_clean":"2024-09-12T01:44:45.818038+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:44:45.818038+0000","last_undegraded":"2024-09-12T01:44:45.818038+0000","last_fullsized":"2024-09-12T01:44:45.818038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822055+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:44:45.822055+0000","last_peered":"2024-09-12T01:44:45.822055+0000","last_clean":"2024-09-12T01:44:45.822055+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:44:45.822055+0000","last_undegraded":"2024-09-12T01:44:45.822055+0000","last_fullsized":"2024-09-12T01:44:45.822055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077339+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:44:46.077339+0000","last_peered":"2024-09-12T01:44:46.077339+0000","last_clean":"2024-09-12T01:44:46.077339+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:44:46.077339+0000","last_undegraded":"2024-09-12T01:44:46.077339+0000","last_fullsized":"2024-09-12T01:44:46.077339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076657+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:44:46.076657+0000","last_peered":"2024-09-12T01:44:46.076657+0000","last_clean":"2024-09-12T01:44:46.076657+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:44:46.076657+0000","last_undegraded":"2024-09-12T01:44:46.076657+0000","last_fullsized":"2024-09-12T01:44:46.076657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076627+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:44:46.076627+0000","last_peered":"2024-09-12T01:44:46.076627+0000","last_clean":"2024-09-12T01:44:46.076627+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:44:46.076627+0000","last_undegraded":"2024-09-12T01:44:46.076627+0000","last_fullsized":"2024-09-12T01:44:46.076627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820478+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:44:45.820478+0000","last_peered":"2024-09-12T01:44:45.820478+0000","last_clean":"2024-09-12T01:44:45.820478+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:44:45.820478+0000","last_undegraded":"2024-09-12T01:44:45.820478+0000","last_fullsized":"2024-09-12T01:44:45.820478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818598+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:44:45.818598+0000","last_peered":"2024-09-12T01:44:45.818598+0000","last_clean":"2024-09-12T01:44:45.818598+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:44:45.818598+0000","last_undegraded":"2024-09-12T01:44:45.818598+0000","last_fullsized":"2024-09-12T01:44:45.818598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816391+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:44:45.816391+0000","last_peered":"2024-09-12T01:44:45.816391+0000","last_clean":"2024-09-12T01:44:45.816391+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:44:45.816391+0000","last_undegraded":"2024-09-12T01:44:45.816391+0000","last_fullsized":"2024-09-12T01:44:45.816391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821743+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:44:45.821743+0000","last_peered":"2024-09-12T01:44:45.821743+0000","last_clean":"2024-09-12T01:44:45.821743+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:44:45.821743+0000","last_undegraded":"2024-09-12T01:44:45.821743+0000","last_fullsized":"2024-09-12T01:44:45.821743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818574+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:44:45.818574+0000","last_peered":"2024-09-12T01:44:45.818574+0000","last_clean":"2024-09-12T01:44:45.818574+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:44:45.818574+0000","last_undegraded":"2024-09-12T01:44:45.818574+0000","last_fullsized":"2024-09-12T01:44:45.818574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077436+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:44:46.077436+0000","last_peered":"2024-09-12T01:44:46.077436+0000","last_clean":"2024-09-12T01:44:46.077436+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:44:46.077436+0000","last_undegraded":"2024-09-12T01:44:46.077436+0000","last_fullsized":"2024-09-12T01:44:46.077436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820692+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:44:45.820692+0000","last_peered":"2024-09-12T01:44:45.820692+0000","last_clean":"2024-09-12T01:44:45.820692+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:44:45.820692+0000","last_undegraded":"2024-09-12T01:44:45.820692+0000","last_fullsized":"2024-09-12T01:44:45.820692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821444+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:44:45.821444+0000","last_peered":"2024-09-12T01:44:45.821444+0000","last_clean":"2024-09-12T01:44:45.821444+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:44:45.821444+0000","last_undegraded":"2024-09-12T01:44:45.821444+0000","last_fullsized":"2024-09-12T01:44:45.821444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824121+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:44:45.824121+0000","last_peered":"2024-09-12T01:44:45.824121+0000","last_clean":"2024-09-12T01:44:45.824121+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:44:45.824121+0000","last_undegraded":"2024-09-12T01:44:45.824121+0000","last_fullsized":"2024-09-12T01:44:45.824121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821120+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:44:45.821120+0000","last_peered":"2024-09-12T01:44:45.821120+0000","last_clean":"2024-09-12T01:44:45.821120+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:44:45.821120+0000","last_undegraded":"2024-09-12T01:44:45.821120+0000","last_fullsized":"2024-09-12T01:44:45.821120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822863+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:44:45.822863+0000","last_peered":"2024-09-12T01:44:45.822863+0000","last_clean":"2024-09-12T01:44:45.822863+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:44:45.822863+0000","last_undegraded":"2024-09-12T01:44:45.822863+0000","last_fullsized":"2024-09-12T01:44:45.822863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817853+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:44:45.817853+0000","last_peered":"2024-09-12T01:44:45.817853+0000","last_clean":"2024-09-12T01:44:45.817853+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:44:45.817853+0000","last_undegraded":"2024-09-12T01:44:45.817853+0000","last_fullsized":"2024-09-12T01:44:45.817853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077201+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:44:46.077201+0000","last_peered":"2024-09-12T01:44:46.077201+0000","last_clean":"2024-09-12T01:44:46.077201+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:44:46.077201+0000","last_undegraded":"2024-09-12T01:44:46.077201+0000","last_fullsized":"2024-09-12T01:44:46.077201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076827+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:44:46.076827+0000","last_peered":"2024-09-12T01:44:46.076827+0000","last_clean":"2024-09-12T01:44:46.076827+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:44:46.076827+0000","last_undegraded":"2024-09-12T01:44:46.076827+0000","last_fullsized":"2024-09-12T01:44:46.076827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820905+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:44:45.820905+0000","last_peered":"2024-09-12T01:44:45.820905+0000","last_clean":"2024-09-12T01:44:45.820905+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:44:45.820905+0000","last_undegraded":"2024-09-12T01:44:45.820905+0000","last_fullsized":"2024-09-12T01:44:45.820905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821482+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:44:45.821482+0000","last_peered":"2024-09-12T01:44:45.821482+0000","last_clean":"2024-09-12T01:44:45.821482+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:44:45.821482+0000","last_undegraded":"2024-09-12T01:44:45.821482+0000","last_fullsized":"2024-09-12T01:44:45.821482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817474+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:44:45.817474+0000","last_peered":"2024-09-12T01:44:45.817474+0000","last_clean":"2024-09-12T01:44:45.817474+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:44:45.817474+0000","last_undegraded":"2024-09-12T01:44:45.817474+0000","last_fullsized":"2024-09-12T01:44:45.817474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819263+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:44:45.819263+0000","last_peered":"2024-09-12T01:44:45.819263+0000","last_clean":"2024-09-12T01:44:45.819263+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:44:45.819263+0000","last_undegraded":"2024-09-12T01:44:45.819263+0000","last_fullsized":"2024-09-12T01:44:45.819263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:56.853306+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:44:56.853306+0000","last_peered":"2024-09-12T01:44:56.853306+0000","last_clean":"2024-09-12T01:44:56.853306+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:44:56.853306+0000","last_undegraded":"2024-09-12T01:44:56.853306+0000","last_fullsized":"2024-09-12T01:44:56.853306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815476+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:44:45.815476+0000","last_peered":"2024-09-12T01:44:45.815476+0000","last_clean":"2024-09-12T01:44:45.815476+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:44:45.815476+0000","last_undegraded":"2024-09-12T01:44:45.815476+0000","last_fullsized":"2024-09-12T01:44:45.815476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821664+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:44:45.821664+0000","last_peered":"2024-09-12T01:44:45.821664+0000","last_clean":"2024-09-12T01:44:45.821664+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:44:45.821664+0000","last_undegraded":"2024-09-12T01:44:45.821664+0000","last_fullsized":"2024-09-12T01:44:45.821664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816283+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:44:45.816283+0000","last_peered":"2024-09-12T01:44:45.816283+0000","last_clean":"2024-09-12T01:44:45.816283+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:44:45.816283+0000","last_undegraded":"2024-09-12T01:44:45.816283+0000","last_fullsized":"2024-09-12T01:44:45.816283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:58.855237+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:44:58.855237+0000","last_peered":"2024-09-12T01:44:58.855237+0000","last_clean":"2024-09-12T01:44:58.855237+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:44:58.855237+0000","last_undegraded":"2024-09-12T01:44:58.855237+0000","last_fullsized":"2024-09-12T01:44:58.855237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821400+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:44:45.821400+0000","last_peered":"2024-09-12T01:44:45.821400+0000","last_clean":"2024-09-12T01:44:45.821400+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:44:45.821400+0000","last_undegraded":"2024-09-12T01:44:45.821400+0000","last_fullsized":"2024-09-12T01:44:45.821400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077017+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:44:46.077017+0000","last_peered":"2024-09-12T01:44:46.077017+0000","last_clean":"2024-09-12T01:44:46.077017+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:44:46.077017+0000","last_undegraded":"2024-09-12T01:44:46.077017+0000","last_fullsized":"2024-09-12T01:44:46.077017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820850+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:44:45.820850+0000","last_peered":"2024-09-12T01:44:45.820850+0000","last_clean":"2024-09-12T01:44:45.820850+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:44:45.820850+0000","last_undegraded":"2024-09-12T01:44:45.820850+0000","last_fullsized":"2024-09-12T01:44:45.820850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:59.861240+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:44:59.861240+0000","last_peered":"2024-09-12T01:44:59.861240+0000","last_clean":"2024-09-12T01:44:59.861240+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:44:59.861240+0000","last_undegraded":"2024-09-12T01:44:59.861240+0000","last_fullsized":"2024-09-12T01:44:59.861240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":67,"num_write":27,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820311+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:44:45.820311+0000","last_peered":"2024-09-12T01:44:45.820311+0000","last_clean":"2024-09-12T01:44:45.820311+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:44:45.820311+0000","last_undegraded":"2024-09-12T01:44:45.820311+0000","last_fullsized":"2024-09-12T01:44:45.820311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077080+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:44:46.077080+0000","last_peered":"2024-09-12T01:44:46.077080+0000","last_clean":"2024-09-12T01:44:46.077080+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:44:46.077080+0000","last_undegraded":"2024-09-12T01:44:46.077080+0000","last_fullsized":"2024-09-12T01:44:46.077080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817662+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:44:45.817662+0000","last_peered":"2024-09-12T01:44:45.817662+0000","last_clean":"2024-09-12T01:44:45.817662+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:44:45.817662+0000","last_undegraded":"2024-09-12T01:44:45.817662+0000","last_fullsized":"2024-09-12T01:44:45.817662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:55.854850+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:44:55.854850+0000","last_peered":"2024-09-12T01:44:55.854850+0000","last_clean":"2024-09-12T01:44:55.854850+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:44:55.854850+0000","last_undegraded":"2024-09-12T01:44:55.854850+0000","last_fullsized":"2024-09-12T01:44:55.854850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817679+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:44:45.817679+0000","last_peered":"2024-09-12T01:44:45.817679+0000","last_clean":"2024-09-12T01:44:45.817679+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:44:45.817679+0000","last_undegraded":"2024-09-12T01:44:45.817679+0000","last_fullsized":"2024-09-12T01:44:45.817679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822535+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:44:45.822535+0000","last_peered":"2024-09-12T01:44:45.822535+0000","last_clean":"2024-09-12T01:44:45.822535+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:44:45.822535+0000","last_undegraded":"2024-09-12T01:44:45.822535+0000","last_fullsized":"2024-09-12T01:44:45.822535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818060+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:44:45.818060+0000","last_peered":"2024-09-12T01:44:45.818060+0000","last_clean":"2024-09-12T01:44:45.818060+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:44:45.818060+0000","last_undegraded":"2024-09-12T01:44:45.818060+0000","last_fullsized":"2024-09-12T01:44:45.818060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818393+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:44:45.818393+0000","last_peered":"2024-09-12T01:44:45.818393+0000","last_clean":"2024-09-12T01:44:45.818393+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:44:45.818393+0000","last_undegraded":"2024-09-12T01:44:45.818393+0000","last_fullsized":"2024-09-12T01:44:45.818393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820335+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:44:45.820335+0000","last_peered":"2024-09-12T01:44:45.820335+0000","last_clean":"2024-09-12T01:44:45.820335+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:44:45.820335+0000","last_undegraded":"2024-09-12T01:44:45.820335+0000","last_fullsized":"2024-09-12T01:44:45.820335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818057+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:44:45.818057+0000","last_peered":"2024-09-12T01:44:45.818057+0000","last_clean":"2024-09-12T01:44:45.818057+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:44:45.818057+0000","last_undegraded":"2024-09-12T01:44:45.818057+0000","last_fullsized":"2024-09-12T01:44:45.818057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818530+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818530+0000","last_peered":"2024-09-12T01:44:45.818530+0000","last_clean":"2024-09-12T01:44:45.818530+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818530+0000","last_undegraded":"2024-09-12T01:44:45.818530+0000","last_fullsized":"2024-09-12T01:44:45.818530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821278+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:44:45.821278+0000","last_peered":"2024-09-12T01:44:45.821278+0000","last_clean":"2024-09-12T01:44:45.821278+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:44:45.821278+0000","last_undegraded":"2024-09-12T01:44:45.821278+0000","last_fullsized":"2024-09-12T01:44:45.821278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077069+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:44:46.077069+0000","last_peered":"2024-09-12T01:44:46.077069+0000","last_clean":"2024-09-12T01:44:46.077069+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:44:46.077069+0000","last_undegraded":"2024-09-12T01:44:46.077069+0000","last_fullsized":"2024-09-12T01:44:46.077069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:56.856042+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:44:56.856042+0000","last_peered":"2024-09-12T01:44:56.856042+0000","last_clean":"2024-09-12T01:44:56.856042+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:44:56.856042+0000","last_undegraded":"2024-09-12T01:44:56.856042+0000","last_fullsized":"2024-09-12T01:44:56.856042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824468+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:44:45.824468+0000","last_peered":"2024-09-12T01:44:45.824468+0000","last_clean":"2024-09-12T01:44:45.824468+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:44:45.824468+0000","last_undegraded":"2024-09-12T01:44:45.824468+0000","last_fullsized":"2024-09-12T01:44:45.824468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818919+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:44:45.818919+0000","last_peered":"2024-09-12T01:44:45.818919+0000","last_clean":"2024-09-12T01:44:45.818919+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:44:45.818919+0000","last_undegraded":"2024-09-12T01:44:45.818919+0000","last_fullsized":"2024-09-12T01:44:45.818919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819406+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:44:45.819406+0000","last_peered":"2024-09-12T01:44:45.819406+0000","last_clean":"2024-09-12T01:44:45.819406+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:44:45.819406+0000","last_undegraded":"2024-09-12T01:44:45.819406+0000","last_fullsized":"2024-09-12T01:44:45.819406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077325+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:44:46.077325+0000","last_peered":"2024-09-12T01:44:46.077325+0000","last_clean":"2024-09-12T01:44:46.077325+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:44:46.077325+0000","last_undegraded":"2024-09-12T01:44:46.077325+0000","last_fullsized":"2024-09-12T01:44:46.077325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:56.780804+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:44:56.780804+0000","last_peered":"2024-09-12T01:44:56.780804+0000","last_clean":"2024-09-12T01:44:56.780804+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:44:56.780804+0000","last_undegraded":"2024-09-12T01:44:56.780804+0000","last_fullsized":"2024-09-12T01:44:56.780804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821213+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:44:45.821213+0000","last_peered":"2024-09-12T01:44:45.821213+0000","last_clean":"2024-09-12T01:44:45.821213+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:44:45.821213+0000","last_undegraded":"2024-09-12T01:44:45.821213+0000","last_fullsized":"2024-09-12T01:44:45.821213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076938+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:44:46.076938+0000","last_peered":"2024-09-12T01:44:46.076938+0000","last_clean":"2024-09-12T01:44:46.076938+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:44:46.076938+0000","last_undegraded":"2024-09-12T01:44:46.076938+0000","last_fullsized":"2024-09-12T01:44:46.076938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819237+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:44:45.819237+0000","last_peered":"2024-09-12T01:44:45.819237+0000","last_clean":"2024-09-12T01:44:45.819237+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:44:45.819237+0000","last_undegraded":"2024-09-12T01:44:45.819237+0000","last_fullsized":"2024-09-12T01:44:45.819237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817496+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:44:45.817496+0000","last_peered":"2024-09-12T01:44:45.817496+0000","last_clean":"2024-09-12T01:44:45.817496+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:44:45.817496+0000","last_undegraded":"2024-09-12T01:44:45.817496+0000","last_fullsized":"2024-09-12T01:44:45.817496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821064+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:44:45.821064+0000","last_peered":"2024-09-12T01:44:45.821064+0000","last_clean":"2024-09-12T01:44:45.821064+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:44:45.821064+0000","last_undegraded":"2024-09-12T01:44:45.821064+0000","last_fullsized":"2024-09-12T01:44:45.821064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818032+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:44:45.818032+0000","last_peered":"2024-09-12T01:44:45.818032+0000","last_clean":"2024-09-12T01:44:45.818032+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:44:45.818032+0000","last_undegraded":"2024-09-12T01:44:45.818032+0000","last_fullsized":"2024-09-12T01:44:45.818032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816211+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:44:45.816211+0000","last_peered":"2024-09-12T01:44:45.816211+0000","last_clean":"2024-09-12T01:44:45.816211+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:44:45.816211+0000","last_undegraded":"2024-09-12T01:44:45.816211+0000","last_fullsized":"2024-09-12T01:44:45.816211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820739+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:44:45.820739+0000","last_peered":"2024-09-12T01:44:45.820739+0000","last_clean":"2024-09-12T01:44:45.820739+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:44:45.820739+0000","last_undegraded":"2024-09-12T01:44:45.820739+0000","last_fullsized":"2024-09-12T01:44:45.820739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076981+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:44:46.076981+0000","last_peered":"2024-09-12T01:44:46.076981+0000","last_clean":"2024-09-12T01:44:46.076981+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:44:46.076981+0000","last_undegraded":"2024-09-12T01:44:46.076981+0000","last_fullsized":"2024-09-12T01:44:46.076981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820929+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:44:45.820929+0000","last_peered":"2024-09-12T01:44:45.820929+0000","last_clean":"2024-09-12T01:44:45.820929+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:44:45.820929+0000","last_undegraded":"2024-09-12T01:44:45.820929+0000","last_fullsized":"2024-09-12T01:44:45.820929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56103,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":102,"num_read_kb":112,"num_write":155,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":172566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1081769,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029883,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438250,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":494886,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":489313,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:45:03.422 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T01:45:03.423 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T01:45:03.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T01:45:03.675 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:03.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:03 smithi122 ceph-mon[30925]: pgmap v292: 97 pgs: 97 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:45:03.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:03 smithi122 ceph-mon[30925]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:03 smithi151 ceph-mon[35229]: pgmap v292: 97 pgs: 97 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:45:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:03 smithi151 ceph-mon[35229]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:03 smithi183 ceph-mon[35127]: pgmap v292: 97 pgs: 97 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-09-12T01:45:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:03 smithi183 ceph-mon[35127]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 -- 172.21.15.122:0/3216054134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328102e00 msgr2=0x7fb328103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 --2- 172.21.15.122:0/3216054134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328102e00 0x7fb328103280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009900 tx=0x7fb31c02a410 comp rx=0 tx=0).stop 2024-09-12T01:45:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 -- 172.21.15.122:0/3216054134 shutdown_connections 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 --2- 172.21.15.122:0/3216054134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3281037c0 0x7fb32810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 --2- 172.21.15.122:0/3216054134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328102e00 0x7fb328103280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 --2- 172.21.15.122:0/3216054134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb328101c00 0x7fb328102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 -- 172.21.15.122:0/3216054134 >> 172.21.15.122:0/3216054134 conn(0x7fb3280fd390 msgr2=0x7fb3280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 -- 172.21.15.122:0/3216054134 shutdown_connections 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.131+0000 7fb32ce67640 1 -- 172.21.15.122:0/3216054134 wait complete. 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 Processor -- start 2024-09-12T01:45:04.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 -- start start 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 0x7fb328198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb328102e00 0x7fb3281985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3281037c0 0x7fb32819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810d980 con 0x7fb328101c00 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810d800 con 0x7fb3281037c0 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32ce67640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810db00 con 0x7fb328102e00 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 0x7fb328198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 0x7fb328198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48798/0 (socket says 172.21.15.122:48798) 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb32659c640 1 -- 172.21.15.122:0/1590994559 learned_addr learned my addr 172.21.15.122:0/1590994559 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:04.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb325d9b640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb328102e00 0x7fb3281985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.132+0000 7fb326d9d640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3281037c0 0x7fb32819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32659c640 1 -- 172.21.15.122:0/1590994559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb328102e00 msgr2=0x7fb3281985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32659c640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb328102e00 0x7fb3281985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32659c640 1 -- 172.21.15.122:0/1590994559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3281037c0 msgr2=0x7fb32819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32659c640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3281037c0 0x7fb32819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32659c640 1 -- 172.21.15.122:0/1590994559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32819fcc0 con 0x7fb328101c00 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32659c640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 0x7fb328198090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb31800b450 tx=0x7fb31800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318015020 con 0x7fb328101c00 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb318011d70 con 0x7fb328101c00 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32819ffb0 con 0x7fb328101c00 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800f340 con 0x7fb328101c00 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.133+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3281a0430 con 0x7fb328101c00 2024-09-12T01:45:04.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.134+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb328109070 con 0x7fb328101c00 2024-09-12T01:45:04.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.135+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb318018020 con 0x7fb328101c00 2024-09-12T01:45:04.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.136+0000 7fb3177fe640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3100a2c40 0x7fb3100a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:04.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.136+0000 7fb325d9b640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3100a2c40 0x7fb3100a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:04.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.136+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb3180cec20 con 0x7fb328101c00 2024-09-12T01:45:04.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.136+0000 7fb325d9b640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3100a2c40 0x7fb3100a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb31c000c00 tx=0x7fb31c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:04.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.140+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb31801f020 con 0x7fb328101c00 2024-09-12T01:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.370+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb32802da10 con 0x7fb328101c00 2024-09-12T01:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.372+0000 7fb3177fe640 1 -- 172.21.15.122:0/1590994559 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb31800d040 con 0x7fb328101c00 2024-09-12T01:45:04.372 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-12T01:45:04.373 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCgR+JmAScpFhAANhH6uZRDHk4KHvOc52cIlg== 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3100a2c40 msgr2=0x7fb3100a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3100a2c40 0x7fb3100a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb31c000c00 tx=0x7fb31c031400 comp rx=0 tx=0).stop 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 msgr2=0x7fb328198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 0x7fb328198090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb31800b450 tx=0x7fb31800b910 comp rx=0 tx=0).stop 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 shutdown_connections 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3100a2c40 0x7fb3100a5100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3281037c0 0x7fb32819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb328102e00 0x7fb3281985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 --2- 172.21.15.122:0/1590994559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb328101c00 0x7fb328198090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 >> 172.21.15.122:0/1590994559 conn(0x7fb3280fd390 msgr2=0x7fb328106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:04.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 shutdown_connections 2024-09-12T01:45:04.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:04.375+0000 7fb32ce67640 1 -- 172.21.15.122:0/1590994559 wait complete. 2024-09-12T01:45:04.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:04 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1590994559' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:45:04.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:04 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1590994559' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T01:45:04.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:04 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1590994559' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:45:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:04 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1590994559' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T01:45:04.844 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:04 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1590994559' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T01:45:04.845 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:04 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1590994559' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T01:45:05.169 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:45:05.169 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T01:45:05.169 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T01:45:05.206 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T01:45:05.206 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T01:45:05.207 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T01:45:05.490 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:05 smithi183 ceph-mon[35127]: pgmap v293: 97 pgs: 97 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-12T01:45:05.861 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:05 smithi122 ceph-mon[30925]: pgmap v293: 97 pgs: 97 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-12T01:45:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:05 smithi151 ceph-mon[35229]: pgmap v293: 97 pgs: 97 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-12T01:45:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- 172.21.15.122:0/1597544840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 msgr2=0x7f07a40748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- 172.21.15.122:0/1597544840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a40748f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009900 tx=0x7f07a002a480 comp rx=0 tx=0).stop 2024-09-12T01:45:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- 172.21.15.122:0/1597544840 shutdown_connections 2024-09-12T01:45:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- 172.21.15.122:0/1597544840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a410e450 0x7f07a4110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- 172.21.15.122:0/1597544840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a4074e30 0x7f07a410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- 172.21.15.122:0/1597544840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a40748f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- 172.21.15.122:0/1597544840 >> 172.21.15.122:0/1597544840 conn(0x7f07a40fd8f0 msgr2=0x7f07a40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- 172.21.15.122:0/1597544840 shutdown_connections 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- 172.21.15.122:0/1597544840 wait complete. 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 Processor -- start 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- start start 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a4074e30 0x7f07a41a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a410e450 0x7f07a419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4114260 con 0x7f07a4074510 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.940+0000 7f07ab638640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41140e0 con 0x7f07a410e450 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07ab638640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41143e0 con 0x7f07a4074e30 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a41a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48804/0 (socket says 172.21.15.122:48804) 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 -- 172.21.15.122:0/2111688148 learned_addr learned my addr 172.21.15.122:0/2111688148 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a8bd3640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a4074e30 0x7f07a41a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a9bd5640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a410e450 0x7f07a419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 -- 172.21.15.122:0/2111688148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a4074e30 msgr2=0x7f07a41a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a4074e30 0x7f07a41a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 -- 172.21.15.122:0/2111688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a410e450 msgr2=0x7f07a419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a410e450 0x7f07a419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 -- 172.21.15.122:0/2111688148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a419ad30 con 0x7f07a4074510 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a8bd3640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a4074e30 0x7f07a41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07a93d4640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a41a0440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f07a0005ab0 tx=0x7f07a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a003f030 con 0x7f07a4074510 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a419afc0 con 0x7f07a4074510 2024-09-12T01:45:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a0003ce0 con 0x7f07a4074510 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0035070 con 0x7f07a4074510 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.941+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41a7310 con 0x7f07a4074510 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.942+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a4074970 con 0x7f07a4074510 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.943+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f07a0042030 con 0x7f07a4074510 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.944+0000 7f079a7fc640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07900a2c10 0x7f07900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.944+0000 7f07a8bd3640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07900a2c10 0x7f07900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.944+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f07a00efb70 con 0x7f07a4074510 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.944+0000 7f07a8bd3640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07900a2c10 0x7f07900a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07a40fff60 tx=0x7f0794008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:05.948+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07a00bc970 con 0x7f07a4074510 2024-09-12T01:45:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.160+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f07a41095d0 con 0x7f07a4074510 2024-09-12T01:45:06.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.165+0000 7f079a7fc640 1 -- 172.21.15.122:0/2111688148 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263208 (secure 0 0 0) 0x7f07a00bc270 con 0x7f07a4074510 2024-09-12T01:45:06.169 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:06.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.167+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07900a2c10 msgr2=0x7f07900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:06.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.167+0000 7f07ab638640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07900a2c10 0x7f07900a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07a40fff60 tx=0x7f0794008040 comp rx=0 tx=0).stop 2024-09-12T01:45:06.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.167+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 msgr2=0x7f07a41a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:06.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.167+0000 7f07ab638640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a41a0440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f07a0005ab0 tx=0x7f07a00042f0 comp rx=0 tx=0).stop 2024-09-12T01:45:06.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.167+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 shutdown_connections 2024-09-12T01:45:06.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.167+0000 7f07ab638640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07900a2c10 0x7f07900a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:06.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.168+0000 7f07ab638640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a410e450 0x7f07a419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:06.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.168+0000 7f07ab638640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a4074e30 0x7f07a41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:06.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.168+0000 7f07ab638640 1 --2- 172.21.15.122:0/2111688148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a4074510 0x7f07a41a0440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:06.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.168+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 >> 172.21.15.122:0/2111688148 conn(0x7f07a40fd8f0 msgr2=0x7f07a40ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:06.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.168+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 shutdown_connections 2024-09-12T01:45:06.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:06.168+0000 7f07ab638640 1 -- 172.21.15.122:0/2111688148 wait complete. 2024-09-12T01:45:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:06 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2111688148' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T01:45:06.886 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:06 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2111688148' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T01:45:06.914 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3245830997},{"type":"v1","addr":"172.21.15.122:6801","nonce":3245830997}]},"active_addr":"172.21.15.122:6801/3245830997","active_change":"2024-09-12T01:37:44.367455+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.122:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3110672081}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3549926238}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2349071867}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":730445688}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":256757236}]}]} 2024-09-12T01:45:06.919 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T01:45:06.919 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T01:45:06.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:45:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:06 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2111688148' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T01:45:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:07.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 -- 172.21.15.122:0/125040959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690100a80 msgr2=0x7f2690100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:07.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 --2- 172.21.15.122:0/125040959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690100a80 0x7f2690100ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2680009930 tx=0x7f268002a420 comp rx=0 tx=0).stop 2024-09-12T01:45:07.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 -- 172.21.15.122:0/125040959 shutdown_connections 2024-09-12T01:45:07.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 --2- 172.21.15.122:0/125040959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2690101420 0x7f269010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 --2- 172.21.15.122:0/125040959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690100a80 0x7f2690100ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 --2- 172.21.15.122:0/125040959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 0x7f2690106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 -- 172.21.15.122:0/125040959 >> 172.21.15.122:0/125040959 conn(0x7f26900fc7a0 msgr2=0x7f26900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 -- 172.21.15.122:0/125040959 shutdown_connections 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.602+0000 7f2697886640 1 -- 172.21.15.122:0/125040959 wait complete. 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 Processor -- start 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 -- start start 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2690100a80 0x7f26901974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 0x7f2690197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 0x7f269019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110f20 con 0x7f2690101420 2024-09-12T01:45:07.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110da0 con 0x7f2690106a80 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2697886640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26901110a0 con 0x7f2690100a80 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 0x7f2690197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 0x7f2690197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48826/0 (socket says 172.21.15.122:48826) 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 -- 172.21.15.122:0/1195542209 learned_addr learned my addr 172.21.15.122:0/1195542209 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2695e23640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 0x7f269019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 -- 172.21.15.122:0/1195542209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2690100a80 msgr2=0x7f26901974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2690100a80 0x7f26901974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 -- 172.21.15.122:0/1195542209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 msgr2=0x7f269019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 0x7f269019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2694e21640 1 -- 172.21.15.122:0/1195542209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269019c470 con 0x7f2690101420 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.603+0000 7f2695e23640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 0x7f269019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.604+0000 7f2694e21640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 0x7f2690197a00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2680009a50 tx=0x7f2680004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.604+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268003f030 con 0x7f2690101420 2024-09-12T01:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.604+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2680003ce0 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.604+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269019c700 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.604+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680035070 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.604+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a4040 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.605+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2690100f60 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.606+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2680042030 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.606+0000 7f26867fc640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f267c0a2c90 0x7f267c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.607+0000 7f2695622640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f267c0a2c90 0x7f267c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.607+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f26800efbe0 con 0x7f2690101420 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.607+0000 7f2695622640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f267c0a2c90 0x7f267c0a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f268c009710 tx=0x7f268c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.610+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26800bc9e0 con 0x7f2690101420 2024-09-12T01:45:07.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.780+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2690109040 con 0x7f2690101420 2024-09-12T01:45:07.782 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:07 smithi122 ceph-mon[30925]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-12T01:45:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.782+0000 7f26867fc640 1 -- 172.21.15.122:0/1195542209 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7f26800bc2e0 con 0x7f2690101420 2024-09-12T01:45:07.783 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:07.783 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f267c0a2c90 msgr2=0x7f267c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f267c0a2c90 0x7f267c0a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f268c009710 tx=0x7f268c006c90 comp rx=0 tx=0).stop 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 msgr2=0x7f2690197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 0x7f2690197a00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2680009a50 tx=0x7f2680004270 comp rx=0 tx=0).stop 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 shutdown_connections 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f267c0a2c90 0x7f267c0a5150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2690106a80 0x7f269019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2690101420 0x7f2690197a00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 --2- 172.21.15.122:0/1195542209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2690100a80 0x7f26901974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:07.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 >> 172.21.15.122:0/1195542209 conn(0x7f26900fc7a0 msgr2=0x7f26900fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:07.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 shutdown_connections 2024-09-12T01:45:07.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:07.784+0000 7f2697886640 1 -- 172.21.15.122:0/1195542209 wait complete. 2024-09-12T01:45:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:07 smithi151 ceph-mon[35229]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-12T01:45:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:07 smithi183 ceph-mon[35127]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-09-12T01:45:08.554 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T01:45:08.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:45:08.798 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:08 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1195542209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:08.918 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:08 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1195542209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:08.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:08 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1195542209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:09.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 -- 172.21.15.122:0/2621701376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100460 msgr2=0x7f8c90100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:09.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 --2- 172.21.15.122:0/2621701376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100460 0x7f8c90100840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009900 tx=0x7f8c8002a410 comp rx=0 tx=0).stop 2024-09-12T01:45:09.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 -- 172.21.15.122:0/2621701376 shutdown_connections 2024-09-12T01:45:09.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 --2- 172.21.15.122:0/2621701376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c90105590 0x7f8c9010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 --2- 172.21.15.122:0/2621701376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c90100e10 0x7f8c90104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 --2- 172.21.15.122:0/2621701376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100460 0x7f8c90100840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 -- 172.21.15.122:0/2621701376 >> 172.21.15.122:0/2621701376 conn(0x7f8c900fbee0 msgr2=0x7f8c900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 -- 172.21.15.122:0/2621701376 shutdown_connections 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.303+0000 7f8c95300640 1 -- 172.21.15.122:0/2621701376 wait complete. 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 Processor -- start 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 -- start start 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c90100460 0x7f8c90198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 0x7f8c90198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c90105590 0x7f8c9019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c90111f70 con 0x7f8c90100e10 2024-09-12T01:45:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c90111df0 con 0x7f8c90100460 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c95300640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c901120f0 con 0x7f8c90105590 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 0x7f8c90198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 0x7f8c90198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48842/0 (socket says 172.21.15.122:48842) 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 -- 172.21.15.122:0/3532236913 learned_addr learned my addr 172.21.15.122:0/3532236913 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c8effd640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c90100460 0x7f8c90198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 -- 172.21.15.122:0/3532236913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c90105590 msgr2=0x7f8c9019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c90105590 0x7f8c9019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 -- 172.21.15.122:0/3532236913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c90100460 msgr2=0x7f8c90198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c90100460 0x7f8c90198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 -- 172.21.15.122:0/3532236913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9019d3d0 con 0x7f8c90100e10 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.304+0000 7f8c87fff640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 0x7f8c90198960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800d950 tx=0x7f8c7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.305+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78017070 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.305+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9019d6c0 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.305+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c78009c30 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.305+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800f6b0 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.305+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c901a4fa0 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.306+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c90100840 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.307+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8c780109d0 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.307+0000 7f8c8cff9640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c7c0a2c10 0x7f8c7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.307+0000 7f8c8effd640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c7c0a2c10 0x7f8c7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.307+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8c780ceeb0 con 0x7f8c90100e10 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.308+0000 7f8c8effd640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c7c0a2c10 0x7f8c7c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005ab0 tx=0x7f8c80005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:09.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.311+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c78009de0 con 0x7f8c90100e10 2024-09-12T01:45:09.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.477+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c901081e0 con 0x7f8c90100e10 2024-09-12T01:45:09.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.479+0000 7f8c8cff9640 1 -- 172.21.15.122:0/3532236913 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7f8c7809bcb0 con 0x7f8c90100e10 2024-09-12T01:45:09.479 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:09.479 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c7c0a2c10 msgr2=0x7f8c7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c7c0a2c10 0x7f8c7c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c80005ab0 tx=0x7f8c80005a20 comp rx=0 tx=0).stop 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 msgr2=0x7f8c90198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 0x7f8c90198960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800d950 tx=0x7f8c7800de10 comp rx=0 tx=0).stop 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 shutdown_connections 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c7c0a2c10 0x7f8c7c0a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c90105590 0x7f8c9019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c90100e10 0x7f8c90198960 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 --2- 172.21.15.122:0/3532236913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c90100460 0x7f8c90198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 >> 172.21.15.122:0/3532236913 conn(0x7f8c900fbee0 msgr2=0x7f8c900fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 shutdown_connections 2024-09-12T01:45:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:09.481+0000 7f8c95300640 1 -- 172.21.15.122:0/3532236913 wait complete. 2024-09-12T01:45:09.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:09 smithi122 ceph-mon[30925]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T01:45:09.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:09 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3532236913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:09 smithi151 ceph-mon[35229]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T01:45:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:09 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3532236913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:09 smithi183 ceph-mon[35127]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T01:45:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:09 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3532236913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:10.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T01:45:10.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T01:45:10.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T01:45:10.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T01:45:10.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T01:45:10.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T01:45:10.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T01:45:10.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T01:45:10.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T01:45:10.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T01:45:10.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T01:45:10.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T01:45:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:10.906 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:10.907 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:10.912 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:10.912 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:11.647 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:11 smithi122 ceph-mon[30925]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.646+0000 7fc639fc7640 1 -- 172.21.15.122:0/1934745153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 msgr2=0x7fc6341021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.646+0000 7fc639fc7640 1 --2- 172.21.15.122:0/1934745153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341021d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc624009ca0 tx=0x7fc62402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.646+0000 7fc639fc7640 1 -- 172.21.15.122:0/1934745153 shutdown_connections 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.646+0000 7fc639fc7640 1 --2- 172.21.15.122:0/1934745153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634106f20 0x7fc63410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.646+0000 7fc639fc7640 1 --2- 172.21.15.122:0/1934745153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6341027a0 0x7fc6341067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.646+0000 7fc639fc7640 1 --2- 172.21.15.122:0/1934745153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341021d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.647+0000 7fc639fc7640 1 -- 172.21.15.122:0/1934745153 >> 172.21.15.122:0/1934745153 conn(0x7fc6340fd8f0 msgr2=0x7fc6340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.647+0000 7fc639fc7640 1 -- 172.21.15.122:0/1934745153 shutdown_connections 2024-09-12T01:45:11.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.647+0000 7fc639fc7640 1 -- 172.21.15.122:0/1934745153 wait complete. 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.647+0000 7fc639fc7640 1 Processor -- start 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.647+0000 7fc639fc7640 1 -- start start 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc639fc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc639fc7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6341027a0 0x7fc6341a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc639fc7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634106f20 0x7fc63419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc639fc7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc634111fc0 con 0x7fc634101df0 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc639fc7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc634111e40 con 0x7fc6341027a0 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc639fc7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112140 con 0x7fc634106f20 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35112/0 (socket says 172.21.15.122:35112) 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 -- 172.21.15.122:0/2451976723 learned_addr learned my addr 172.21.15.122:0/2451976723 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6341027a0 0x7fc6341a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:11.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 -- 172.21.15.122:0/2451976723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634106f20 msgr2=0x7fc63419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:11.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc633fff640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634106f20 0x7fc63419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:11.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634106f20 0x7fc63419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 -- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6341027a0 msgr2=0x7fc6341a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:11.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6341027a0 0x7fc6341a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 -- 172.21.15.122:0/2451976723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419ace0 con 0x7fc634101df0 2024-09-12T01:45:11.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc633fff640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc634106f20 0x7fc63419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6341027a0 0x7fc6341a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.648+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341a03f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6240097b0 tx=0x7fc624031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.649+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62402fba0 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.649+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419af70 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.649+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62402fd40 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.649+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62403b930 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.649+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341a72c0 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.649+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc634075700 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.651+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc624030cc0 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.651+0000 7fc630ff9640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6200a2c10 0x7fc6200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6200a2c10 0x7fc6200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc6240f0a30 con 0x7fc634101df0 2024-09-12T01:45:11.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc630ff9640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] conn(0x7fc6200ac830 0x7fc6200aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 --> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6200af3a0 con 0x7fc6200ac830 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fc6240f0e60 con 0x7fc634101df0 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6200a2c10 0x7fc6200a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc61c006fd0 tx=0x7fc61c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.652+0000 7fc633fff640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] conn(0x7fc6200ac830 0x7fc6200aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.653+0000 7fc633fff640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] conn(0x7fc6200ac830 0x7fc6200aecd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.654+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== osd.10 v2:172.21.15.151:6824/1544038944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6200af3a0 con 0x7fc6200ac830 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.667+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 --> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6341032d0 con 0x7fc6200ac830 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc630ff9640 1 -- 172.21.15.122:0/2451976723 <== osd.10 v2:172.21.15.151:6824/1544038944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc6341032d0 con 0x7fc6200ac830 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] conn(0x7fc6200ac830 msgr2=0x7fc6200aecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] conn(0x7fc6200ac830 0x7fc6200aecd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6200a2c10 msgr2=0x7fc6200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6200a2c10 0x7fc6200a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc61c006fd0 tx=0x7fc61c008040 comp rx=0 tx=0).stop 2024-09-12T01:45:11.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 msgr2=0x7fc6341a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:11.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 --2- 172.21.15.122:0/2451976723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634101df0 0x7fc6341a03f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6240097b0 tx=0x7fc624031b10 comp rx=0 tx=0).stop 2024-09-12T01:45:11.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc633fff640 1 -- 172.21.15.122:0/2451976723 reap_dead start 2024-09-12T01:45:11.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 shutdown_connections 2024-09-12T01:45:11.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 >> 172.21.15.122:0/2451976723 conn(0x7fc6340fd8f0 msgr2=0x7fc6340fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:11.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 shutdown_connections 2024-09-12T01:45:11.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:11.668+0000 7fc639fc7640 1 -- 172.21.15.122:0/2451976723 wait complete. 2024-09-12T01:45:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:11 smithi151 ceph-mon[35229]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T01:45:11.895 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:11 smithi183 ceph-mon[35127]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T01:45:12.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 -- 172.21.15.122:0/97354626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a28106f20 msgr2=0x7f3a2810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 --2- 172.21.15.122:0/97354626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a28106f20 0x7f3a2810eaa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009ca0 tx=0x7f3a2002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 -- 172.21.15.122:0/97354626 shutdown_connections 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 --2- 172.21.15.122:0/97354626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a28106f20 0x7f3a2810eaa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 --2- 172.21.15.122:0/97354626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a281027a0 0x7f3a281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 --2- 172.21.15.122:0/97354626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a28101df0 0x7f3a281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 -- 172.21.15.122:0/97354626 >> 172.21.15.122:0/97354626 conn(0x7f3a280fd8b0 msgr2=0x7f3a280ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 -- 172.21.15.122:0/97354626 shutdown_connections 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.522+0000 7f3a2d223640 1 -- 172.21.15.122:0/97354626 wait complete. 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 Processor -- start 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 -- start start 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a28101df0 0x7f3a28198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a281027a0 0x7f3a28198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a28106f20 0x7f3a2819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28112020 con 0x7f3a281027a0 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111ea0 con 0x7f3a28101df0 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2d223640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281121a0 con 0x7f3a28106f20 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.523+0000 7f3a2659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a281027a0 0x7f3a28198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a281027a0 0x7f3a28198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35118/0 (socket says 172.21.15.122:35118) 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 -- 172.21.15.122:0/3700195438 learned_addr learned my addr 172.21.15.122:0/3700195438 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a26d9d640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a28101df0 0x7f3a28198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2759e640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a28106f20 0x7f3a2819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 -- 172.21.15.122:0/3700195438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a28106f20 msgr2=0x7f3a2819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a28106f20 0x7f3a2819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 -- 172.21.15.122:0/3700195438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a28101df0 msgr2=0x7f3a28198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a28101df0 0x7f3a28198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 -- 172.21.15.122:0/3700195438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2819d5f0 con 0x7f3a281027a0 2024-09-12T01:45:12.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2759e640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a28106f20 0x7f3a2819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2659c640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a281027a0 0x7f3a28198b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00b4a0 tx=0x7f3a1c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c015020 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2819d8e0 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c011d10 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00f2e0 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.524+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a281a51c0 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.525+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a28062900 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.526+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3a1c018020 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a0ffff640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3a100a2c10 0x7f3a100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a26d9d640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3a100a2c10 0x7f3a100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3a1c0cefc0 con 0x7f3a281027a0 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a0ffff640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] conn(0x7f3a100ac840 0x7f3a100aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 --> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a1c004450 con 0x7f3a100ac840 2024-09-12T01:45:12.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f3a1c0cf4a0 con 0x7f3a281027a0 2024-09-12T01:45:12.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a2759e640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] conn(0x7f3a100ac840 0x7f3a100aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.527+0000 7f3a26d9d640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3a100a2c10 0x7f3a100a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009990 tx=0x7f3a18009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:12.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.528+0000 7f3a2759e640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] conn(0x7f3a100ac840 0x7f3a100aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:12.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.528+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== osd.6 v2:172.21.15.122:6818/913950299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a1c004450 con 0x7f3a100ac840 2024-09-12T01:45:12.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.542+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 --> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a28062d00 con 0x7f3a100ac840 2024-09-12T01:45:12.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.542+0000 7f3a0ffff640 1 -- 172.21.15.122:0/3700195438 <== osd.6 v2:172.21.15.122:6818/913950299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3a28062d00 con 0x7f3a100ac840 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.542+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] conn(0x7f3a100ac840 msgr2=0x7f3a100aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.542+0000 7f3a2d223640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] conn(0x7f3a100ac840 0x7f3a100aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.542+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3a100a2c10 msgr2=0x7f3a100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.542+0000 7f3a2d223640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3a100a2c10 0x7f3a100a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009990 tx=0x7f3a18009110 comp rx=0 tx=0).stop 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a281027a0 msgr2=0x7f3a28198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2d223640 1 --2- 172.21.15.122:0/3700195438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a281027a0 0x7f3a28198b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00b4a0 tx=0x7f3a1c00b8a0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2759e640 1 -- 172.21.15.122:0/3700195438 reap_dead start 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 shutdown_connections 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 >> 172.21.15.122:0/3700195438 conn(0x7f3a280fd8b0 msgr2=0x7f3a280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 shutdown_connections 2024-09-12T01:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.543+0000 7f3a2d223640 1 -- 172.21.15.122:0/3700195438 wait complete. 2024-09-12T01:45:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.698+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/152136430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 msgr2=0x7f3d74103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.698+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/152136430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74103110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009930 tx=0x7f3d6402a420 comp rx=0 tx=0).stop 2024-09-12T01:45:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/152136430 shutdown_connections 2024-09-12T01:45:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/152136430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d741082a0 0x7f3d7410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/152136430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d741036e0 0x7f3d74107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/152136430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74103110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/152136430 >> 172.21.15.122:0/152136430 conn(0x7f3d740fc7d0 msgr2=0x7f3d740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/152136430 shutdown_connections 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/152136430 wait complete. 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 Processor -- start 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.699+0000 7f3d7c0aa640 1 -- start start 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7c0aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7c0aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d741036e0 0x7f3d74198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7c0aa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d741082a0 0x7f3d7419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7c0aa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d74112120 con 0x7f3d74102d30 2024-09-12T01:45:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7c0aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d74111fa0 con 0x7f3d741036e0 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7c0aa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d741122a0 con 0x7f3d741082a0 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35134/0 (socket says 172.21.15.122:35134) 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 -- 172.21.15.122:0/696313940 learned_addr learned my addr 172.21.15.122:0/696313940 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d7a647640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d741082a0 0x7f3d7419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79645640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d741036e0 0x7f3d74198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 -- 172.21.15.122:0/696313940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d741082a0 msgr2=0x7f3d7419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d741082a0 0x7f3d7419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 -- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d741036e0 msgr2=0x7f3d74198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d741036e0 0x7f3d74198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 -- 172.21.15.122:0/696313940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7419d5f0 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d79e46640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74198640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009230 tx=0x7f3d64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.700+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6403f030 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.701+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7419d880 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.701+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d64003ad0 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.701+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64035070 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.701+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d741a51c0 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.701+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d74062980 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3d64042030 con 0x7f3d74102d30 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d6affd640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d600a2c10 0x7f3d600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d79645640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d600a2c10 0x7f3d600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3d640ef820 con 0x7f3d74102d30 2024-09-12T01:45:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d6affd640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] conn(0x7f3d600ac840 0x7f3d600aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 --> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d600af3b0 con 0x7f3d600ac840 2024-09-12T01:45:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.703+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f3d640efc50 con 0x7f3d74102d30 2024-09-12T01:45:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.704+0000 7f3d79645640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d600a2c10 0x7f3d600a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005fd0 tx=0x7f3d70005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.704+0000 7f3d7a647640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] conn(0x7f3d600ac840 0x7f3d600aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.704+0000 7f3d7a647640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] conn(0x7f3d600ac840 0x7f3d600aece0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.705+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== osd.4 v2:172.21.15.151:6808/190394298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d600af3b0 con 0x7f3d600ac840 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.718+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 --> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d74062e00 con 0x7f3d600ac840 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d6affd640 1 -- 172.21.15.122:0/696313940 <== osd.4 v2:172.21.15.151:6808/190394298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d74062e00 con 0x7f3d600ac840 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] conn(0x7f3d600ac840 msgr2=0x7f3d600aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] conn(0x7f3d600ac840 0x7f3d600aece0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d600a2c10 msgr2=0x7f3d600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d600a2c10 0x7f3d600a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005fd0 tx=0x7f3d70005850 comp rx=0 tx=0).stop 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 msgr2=0x7f3d74198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 --2- 172.21.15.122:0/696313940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d74102d30 0x7f3d74198640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009230 tx=0x7f3d64004270 comp rx=0 tx=0).stop 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7a647640 1 -- 172.21.15.122:0/696313940 reap_dead start 2024-09-12T01:45:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 shutdown_connections 2024-09-12T01:45:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 >> 172.21.15.122:0/696313940 conn(0x7f3d740fc7d0 msgr2=0x7f3d74105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 shutdown_connections 2024-09-12T01:45:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:12.719+0000 7f3d7c0aa640 1 -- 172.21.15.122:0/696313940 wait complete. 2024-09-12T01:45:13.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 -- 172.21.15.122:0/3615911288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308102570 msgr2=0x7f73081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 --2- 172.21.15.122:0/3615911288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308102570 0x7f73081029b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009900 tx=0x7f72fc02a480 comp rx=0 tx=0).stop 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 -- 172.21.15.122:0/3615911288 shutdown_connections 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 --2- 172.21.15.122:0/3615911288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7308102ef0 0x7f730810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 --2- 172.21.15.122:0/3615911288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308102570 0x7f73081029b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 --2- 172.21.15.122:0/3615911288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7308100e20 0x7f7308101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 -- 172.21.15.122:0/3615911288 >> 172.21.15.122:0/3615911288 conn(0x7f73080fc7d0 msgr2=0x7f73080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 -- 172.21.15.122:0/3615911288 shutdown_connections 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.101+0000 7f730e310640 1 -- 172.21.15.122:0/3615911288 wait complete. 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 Processor -- start 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 -- start start 2024-09-12T01:45:13.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308100e20 0x7f73081973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7308102570 0x7f7308197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7308102ef0 0x7f730819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7308110f80 con 0x7f7308100e20 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7308110e00 con 0x7f7308102570 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f730e310640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7308111100 con 0x7f7308102ef0 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f7307fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308100e20 0x7f73081973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f7307fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308100e20 0x7f73081973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35150/0 (socket says 172.21.15.122:35150) 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.102+0000 7f7307fff640 1 -- 172.21.15.122:0/3811351798 learned_addr learned my addr 172.21.15.122:0/3811351798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f730c8ad640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7308102ef0 0x7f730819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f7307fff640 1 -- 172.21.15.122:0/3811351798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7308102ef0 msgr2=0x7f730819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:13.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f7307fff640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7308102ef0 0x7f730819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f73077fe640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7308102570 0x7f7308197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f7307fff640 1 -- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7308102570 msgr2=0x7f7308197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f7307fff640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7308102570 0x7f7308197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f7307fff640 1 -- 172.21.15.122:0/3811351798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730819c390 con 0x7f7308100e20 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f730c8ad640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7308102ef0 0x7f730819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f7307fff640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308100e20 0x7f73081973e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72f800d950 tx=0x7f72f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8017070 con 0x7f7308100e20 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730819c680 con 0x7f7308100e20 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f8009c30 con 0x7f7308100e20 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f800f6b0 con 0x7f7308100e20 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.103+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73081a3f60 con 0x7f7308100e20 2024-09-12T01:45:13.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.104+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7308062980 con 0x7f7308100e20 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.105+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f72f80109d0 con 0x7f7308100e20 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.106+0000 7f73057fa640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f72f00a2c90 0x7f72f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.106+0000 7f73077fe640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f72f00a2c90 0x7f72f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.106+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f72f80cf160 con 0x7f7308100e20 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.106+0000 7f73057fa640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] conn(0x7f72f00ac8c0 0x7f72f00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.106+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 --> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72f00af430 con 0x7f72f00ac8c0 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.106+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f72f80cf590 con 0x7f7308100e20 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.107+0000 7f730c8ad640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] conn(0x7f72f00ac8c0 0x7f72f00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.107+0000 7f73077fe640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f72f00a2c90 0x7f72f00a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0096c0 tx=0x7f72fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.107+0000 7f730c8ad640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] conn(0x7f72f00ac8c0 0x7f72f00aed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:13.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.108+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== osd.1 v2:172.21.15.151:6800/148046823 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72f00af430 con 0x7f72f00ac8c0 2024-09-12T01:45:13.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.121+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 --> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7308062d80 con 0x7f72f00ac8c0 2024-09-12T01:45:13.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f73057fa640 1 -- 172.21.15.122:0/3811351798 <== osd.1 v2:172.21.15.151:6800/148046823 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7308062d80 con 0x7f72f00ac8c0 2024-09-12T01:45:13.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] conn(0x7f72f00ac8c0 msgr2=0x7f72f00aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:13.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] conn(0x7f72f00ac8c0 0x7f72f00aed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f72f00a2c90 msgr2=0x7f72f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f72f00a2c90 0x7f72f00a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0096c0 tx=0x7f72fc000bc0 comp rx=0 tx=0).stop 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308100e20 msgr2=0x7f73081973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 --2- 172.21.15.122:0/3811351798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308100e20 0x7f73081973e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72f800d950 tx=0x7f72f800de10 comp rx=0 tx=0).stop 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730c8ad640 1 -- 172.21.15.122:0/3811351798 reap_dead start 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 shutdown_connections 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 >> 172.21.15.122:0/3811351798 conn(0x7f73080fc7d0 msgr2=0x7f73081068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 shutdown_connections 2024-09-12T01:45:13.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:13.122+0000 7f730e310640 1 -- 172.21.15.122:0/3811351798 wait complete. 2024-09-12T01:45:13.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:13 smithi122 ceph-mon[30925]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T01:45:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:13 smithi151 ceph-mon[35229]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T01:45:13.899 INFO:teuthology.orchestra.run.smithi122.stdout:55834574920 2024-09-12T01:45:13.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T01:45:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:13 smithi183 ceph-mon[35127]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T01:45:14.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/107592897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 msgr2=0x7f5a34106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:14.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/107592897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a34106e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009930 tx=0x7f5a2802a420 comp rx=0 tx=0).stop 2024-09-12T01:45:14.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/107592897 shutdown_connections 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/107592897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a34101420 0x7f5a3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/107592897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34100a80 0x7f5a34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/107592897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a34106e60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/107592897 >> 172.21.15.122:0/107592897 conn(0x7f5a340fc7a0 msgr2=0x7f5a340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/107592897 shutdown_connections 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.457+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/107592897 wait complete. 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 Processor -- start 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 -- start start 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34100a80 0x7f5a341985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a34101420 0x7f5a34198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a3419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:14.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34112020 con 0x7f5a34106a80 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111ea0 con 0x7f5a34100a80 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a3b9a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341121a0 con 0x7f5a34101420 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a3419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a3419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35168/0 (socket says 172.21.15.122:35168) 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 -- 172.21.15.122:0/2345849689 learned_addr learned my addr 172.21.15.122:0/2345849689 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39745640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34100a80 0x7f5a341985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a38f44640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a34101420 0x7f5a34198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 -- 172.21.15.122:0/2345849689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a34101420 msgr2=0x7f5a34198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a34101420 0x7f5a34198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 -- 172.21.15.122:0/2345849689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34100a80 msgr2=0x7f5a341985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a34100a80 0x7f5a341985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 -- 172.21.15.122:0/2345849689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3419d590 con 0x7f5a34106a80 2024-09-12T01:45:14.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a38f44640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a34101420 0x7f5a34198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a39f46640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a3419ceb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b970 tx=0x7f5a2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.458+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c019070 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.459+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c00cb40 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.459+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c007650 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.459+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3419d880 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.459+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a341a5160 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.459+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a34062980 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.461+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5a2c01c070 con 0x7f5a34106a80 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.461+0000 7f5a267fc640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5a200a2c90 0x7f5a200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.461+0000 7f5a39745640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5a200a2c90 0x7f5a200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:14.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.461+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5a2c0ceee0 con 0x7f5a34106a80 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.462+0000 7f5a267fc640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] conn(0x7f5a200ac8c0 0x7f5a200aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.462+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 --> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a2c0047d0 con 0x7f5a200ac8c0 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.462+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c0cf310 con 0x7f5a34106a80 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.462+0000 7f5a38f44640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] conn(0x7f5a200ac8c0 0x7f5a200aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.462+0000 7f5a39745640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5a200a2c90 0x7f5a200a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.462+0000 7f5a38f44640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] conn(0x7f5a200ac8c0 0x7f5a200aed60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.463+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== osd.0 v2:172.21.15.122:6802/1924566162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a2c0047d0 con 0x7f5a200ac8c0 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.476+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 --> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a34062e00 con 0x7f5a200ac8c0 2024-09-12T01:45:14.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.476+0000 7f5a267fc640 1 -- 172.21.15.122:0/2345849689 <== osd.0 v2:172.21.15.122:6802/1924566162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5a34062e00 con 0x7f5a200ac8c0 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] conn(0x7f5a200ac8c0 msgr2=0x7f5a200aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] conn(0x7f5a200ac8c0 0x7f5a200aed60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5a200a2c90 msgr2=0x7f5a200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5a200a2c90 0x7f5a200a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28031490 comp rx=0 tx=0).stop 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 msgr2=0x7f5a3419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 --2- 172.21.15.122:0/2345849689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a34106a80 0x7f5a3419ceb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b970 tx=0x7f5a2c00be30 comp rx=0 tx=0).stop 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a39f46640 1 -- 172.21.15.122:0/2345849689 reap_dead start 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 shutdown_connections 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 >> 172.21.15.122:0/2345849689 conn(0x7f5a340fc7a0 msgr2=0x7f5a340fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 shutdown_connections 2024-09-12T01:45:14.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:14.477+0000 7f5a3b9a9640 1 -- 172.21.15.122:0/2345849689 wait complete. 2024-09-12T01:45:15.218 INFO:teuthology.orchestra.run.smithi122.stdout:34359738444 2024-09-12T01:45:15.218 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T01:45:15.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:15 smithi122 ceph-mon[30925]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T01:45:15.796 INFO:teuthology.orchestra.run.smithi122.stdout:137438953529 2024-09-12T01:45:15.796 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T01:45:15.865 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:15 smithi151 ceph-mon[35229]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T01:45:15.866 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:15.866 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:15.872 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:15 smithi183 ceph-mon[35127]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T01:45:16.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3305294181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100e10 msgr2=0x7f4b54104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:16.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3305294181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100e10 0x7f4b54104e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400aa10 tx=0x7f4b4402b720 comp rx=0 tx=0).stop 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3305294181 shutdown_connections 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3305294181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b54105590 0x7f4b5410d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3305294181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100e10 0x7f4b54104e60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3305294181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100460 0x7f4b54100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3305294181 >> 172.21.15.122:0/3305294181 conn(0x7f4b540fbee0 msgr2=0x7f4b540fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3305294181 shutdown_connections 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.747+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3305294181 wait complete. 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 Processor -- start 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 -- start start 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100460 0x7f4b54101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100e10 0x7f4b54102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b54105590 0x7f4b54102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5410fd90 con 0x7f4b54100460 2024-09-12T01:45:16.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5410fc10 con 0x7f4b54105590 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5bd20640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5410ff10 con 0x7f4b54100e10 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b59abc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100460 0x7f4b54101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b59abc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100460 0x7f4b54101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35184/0 (socket says 172.21.15.122:35184) 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b59abc640 1 -- 172.21.15.122:0/3353193046 learned_addr learned my addr 172.21.15.122:0/3353193046 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b5a2bd640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b54105590 0x7f4b54102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b59abc640 1 -- 172.21.15.122:0/3353193046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100e10 msgr2=0x7f4b54102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.748+0000 7f4b592bb640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100e10 0x7f4b54102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b59abc640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100e10 0x7f4b54102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b59abc640 1 -- 172.21.15.122:0/3353193046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b54105590 msgr2=0x7f4b54102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b59abc640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b54105590 0x7f4b54102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b59abc640 1 -- 172.21.15.122:0/3353193046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b541a6980 con 0x7f4b54100460 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b5a2bd640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b54105590 0x7f4b54102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:16.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b592bb640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b54100e10 0x7f4b54102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b59abc640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100460 0x7f4b54101f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b4a0 tx=0x7f4b5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50014070 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b50010af0 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b541a6c70 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50017d10 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.749+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541a7160 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.750+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b54104290 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.751+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4b50017070 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b4affd640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b400a2c10 0x7f4b400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b592bb640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b400a2c10 0x7f4b400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4b500ce910 con 0x7f4b54100460 2024-09-12T01:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b4affd640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] conn(0x7f4b400ac840 0x7f4b400aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 --> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b400af3b0 con 0x7f4b400ac840 2024-09-12T01:45:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f4b500ced40 con 0x7f4b54100460 2024-09-12T01:45:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b5a2bd640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] conn(0x7f4b400ac840 0x7f4b400aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b592bb640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b400a2c10 0x7f4b400a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000c00 tx=0x7f4b44030e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.752+0000 7f4b5a2bd640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] conn(0x7f4b400ac840 0x7f4b400aece0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.753+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== osd.3 v2:172.21.15.122:6810/533436685 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b400af3b0 con 0x7f4b400ac840 2024-09-12T01:45:16.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.767+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 --> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4b54101940 con 0x7f4b400ac840 2024-09-12T01:45:16.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.767+0000 7f4b4affd640 1 -- 172.21.15.122:0/3353193046 <== osd.3 v2:172.21.15.122:6810/533436685 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4b54101940 con 0x7f4b400ac840 2024-09-12T01:45:16.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] conn(0x7f4b400ac840 msgr2=0x7f4b400aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:16.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] conn(0x7f4b400ac840 0x7f4b400aece0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:16.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b400a2c10 msgr2=0x7f4b400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:16.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b400a2c10 0x7f4b400a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000c00 tx=0x7f4b44030e50 comp rx=0 tx=0).stop 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100460 msgr2=0x7f4b54101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 --2- 172.21.15.122:0/3353193046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b54100460 0x7f4b54101f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b4a0 tx=0x7f4b5000b960 comp rx=0 tx=0).stop 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5a2bd640 1 -- 172.21.15.122:0/3353193046 reap_dead start 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 shutdown_connections 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 >> 172.21.15.122:0/3353193046 conn(0x7f4b540fbee0 msgr2=0x7f4b540fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 shutdown_connections 2024-09-12T01:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:16.768+0000 7f4b5bd20640 1 -- 172.21.15.122:0/3353193046 wait complete. 2024-09-12T01:45:17.642 INFO:teuthology.orchestra.run.smithi122.stdout:292057776154 2024-09-12T01:45:17.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T01:45:17.831 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:17 smithi122 ceph-mon[30925]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T01:45:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:17 smithi151 ceph-mon[35229]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T01:45:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:17 smithi183 ceph-mon[35127]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T01:45:18.111 INFO:teuthology.orchestra.run.smithi122.stdout:188978561069 2024-09-12T01:45:18.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T01:45:18.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 -- 172.21.15.122:0/34664638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 msgr2=0x7f30fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 --2- 172.21.15.122:0/34664638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc101200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009ca0 tx=0x7f30f802a950 comp rx=0 tx=0).stop 2024-09-12T01:45:18.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 -- 172.21.15.122:0/34664638 shutdown_connections 2024-09-12T01:45:18.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 --2- 172.21.15.122:0/34664638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30fc102ef0 0x7f30fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 --2- 172.21.15.122:0/34664638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30fc102570 0x7f30fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 --2- 172.21.15.122:0/34664638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc101200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 -- 172.21.15.122:0/34664638 >> 172.21.15.122:0/34664638 conn(0x7f30fc0fc7d0 msgr2=0x7f30fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 -- 172.21.15.122:0/34664638 shutdown_connections 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.286+0000 7f3102be4640 1 -- 172.21.15.122:0/34664638 wait complete. 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 Processor -- start 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 -- start start 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30fc102570 0x7f30fc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30fc102ef0 0x7f30fc1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc110fc0 con 0x7f30fc100e20 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc110e40 con 0x7f30fc102570 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.287+0000 7f3102be4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc111140 con 0x7f30fc102ef0 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35200/0 (socket says 172.21.15.122:35200) 2024-09-12T01:45:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 -- 172.21.15.122:0/3121482490 learned_addr learned my addr 172.21.15.122:0/3121482490 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f30f3fff640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30fc102570 0x7f30fc1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3101181640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30fc102ef0 0x7f30fc1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 -- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30fc102ef0 msgr2=0x7f30fc1a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30fc102ef0 0x7f30fc1a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 -- 172.21.15.122:0/3121482490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30fc102570 msgr2=0x7f30fc1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30fc102570 0x7f30fc1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 -- 172.21.15.122:0/3121482490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc1a4960 con 0x7f30fc100e20 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f30f3fff640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30fc102570 0x7f30fc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3100980640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc1993a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005cc0 tx=0x7f30f802f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8034040 con 0x7f30fc100e20 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8030920 con 0x7f30fc100e20 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc1a4b30 con 0x7f30fc100e20 2024-09-12T01:45:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.288+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8004830 con 0x7f30fc100e20 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.289+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a4f80 con 0x7f30fc100e20 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.289+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30fc062980 con 0x7f30fc100e20 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.291+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f30f8037020 con 0x7f30fc100e20 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.291+0000 7f30f1ffb640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30e40a2c10 0x7f30e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.291+0000 7f30f3fff640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30e40a2c10 0x7f30e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.291+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f30f80f0320 con 0x7f30fc100e20 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.292+0000 7f30f1ffb640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] conn(0x7f30e40ac840 0x7f30e40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.292+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 --> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30f802c030 con 0x7f30e40ac840 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.292+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f30f80f0750 con 0x7f30fc100e20 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.292+0000 7f30f3fff640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30e40a2c10 0x7f30e40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0058c0 tx=0x7f30ec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.292+0000 7f3101181640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] conn(0x7f30e40ac840 0x7f30e40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.292+0000 7f3101181640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] conn(0x7f30e40ac840 0x7f30e40aece0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:18.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.293+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== osd.2 v2:172.21.15.183:6800/3556820790 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30f802c030 con 0x7f30e40ac840 2024-09-12T01:45:18.310 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:18.311 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:18.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.313+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 --> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30fc062e00 con 0x7f30e40ac840 2024-09-12T01:45:18.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.314+0000 7f30f1ffb640 1 -- 172.21.15.122:0/3121482490 <== osd.2 v2:172.21.15.183:6800/3556820790 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30fc062e00 con 0x7f30e40ac840 2024-09-12T01:45:18.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] conn(0x7f30e40ac840 msgr2=0x7f30e40aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] conn(0x7f30e40ac840 0x7f30e40aece0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30e40a2c10 msgr2=0x7f30e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30e40a2c10 0x7f30e40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0058c0 tx=0x7f30ec005850 comp rx=0 tx=0).stop 2024-09-12T01:45:18.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 msgr2=0x7f30fc1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 --2- 172.21.15.122:0/3121482490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30fc100e20 0x7f30fc1993a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005cc0 tx=0x7f30f802f650 comp rx=0 tx=0).stop 2024-09-12T01:45:18.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3101181640 1 -- 172.21.15.122:0/3121482490 reap_dead start 2024-09-12T01:45:18.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 shutdown_connections 2024-09-12T01:45:18.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 >> 172.21.15.122:0/3121482490 conn(0x7f30fc0fc7d0 msgr2=0x7f30fc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:18.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 shutdown_connections 2024-09-12T01:45:18.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.315+0000 7f3102be4640 1 -- 172.21.15.122:0/3121482490 wait complete. 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 -- 172.21.15.122:0/11117249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 msgr2=0x7fe99c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/11117249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c10da40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe994009ca0 tx=0x7fe99402a950 comp rx=0 tx=0).stop 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 -- 172.21.15.122:0/11117249 shutdown_connections 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/11117249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c10da40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/11117249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe99c102570 0x7fe99c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/11117249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe99c100e20 0x7fe99c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 -- 172.21.15.122:0/11117249 >> 172.21.15.122:0/11117249 conn(0x7fe99c0fc7d0 msgr2=0x7fe99c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 -- 172.21.15.122:0/11117249 shutdown_connections 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 -- 172.21.15.122:0/11117249 wait complete. 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 Processor -- start 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 -- start start 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.722+0000 7fe9a27ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe99c100e20 0x7fe99c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe99c102570 0x7fe99c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c110f80 con 0x7fe99c102ef0 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c110e00 con 0x7fe99c102570 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c111100 con 0x7fe99c100e20 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c1a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35206/0 (socket says 172.21.15.122:35206) 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 -- 172.21.15.122:0/4201582234 learned_addr learned my addr 172.21.15.122:0/4201582234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe99bfff640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe99c100e20 0x7fe99c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe993fff640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe99c102570 0x7fe99c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 -- 172.21.15.122:0/4201582234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe99c100e20 msgr2=0x7fe99c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe99c100e20 0x7fe99c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 -- 172.21.15.122:0/4201582234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe99c102570 msgr2=0x7fe99c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe99c102570 0x7fe99c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 -- 172.21.15.122:0/4201582234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c1a4930 con 0x7fe99c102ef0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a0d8a640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c1a43f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe99402ae70 tx=0x7fe99402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994034040 con 0x7fe99c102ef0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c1a4b00 con 0x7fe99c102ef0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994030920 con 0x7fe99c102ef0 2024-09-12T01:45:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994045480 con 0x7fe99c102ef0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.723+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1a4fb0 con 0x7fe99c102ef0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.724+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe99c1072a0 con 0x7fe99c102ef0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.726+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe994037020 con 0x7fe99c102ef0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.726+0000 7fe999ffb640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe9880a2c90 0x7fe9880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.726+0000 7fe99bfff640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe9880a2c90 0x7fe9880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.726+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe9940f0420 con 0x7fe99c102ef0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.727+0000 7fe999ffb640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] conn(0x7fe9880ac8c0 0x7fe9880aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.727+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 --> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9880af430 con 0x7fe9880ac8c0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.727+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe9940f0850 con 0x7fe99c102ef0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.727+0000 7fe993fff640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] conn(0x7fe9880ac8c0 0x7fe9880aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.727+0000 7fe99bfff640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe9880a2c90 0x7fe9880a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005fd0 tx=0x7fe98c005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.727+0000 7fe993fff640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] conn(0x7fe9880ac8c0 0x7fe9880aed60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:18.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.728+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== osd.9 v2:172.21.15.122:6826/163584006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9880af430 con 0x7fe9880ac8c0 2024-09-12T01:45:18.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.741+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 --> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe99c1021b0 con 0x7fe9880ac8c0 2024-09-12T01:45:18.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.741+0000 7fe999ffb640 1 -- 172.21.15.122:0/4201582234 <== osd.9 v2:172.21.15.122:6826/163584006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe99c1021b0 con 0x7fe9880ac8c0 2024-09-12T01:45:18.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] conn(0x7fe9880ac8c0 msgr2=0x7fe9880aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] conn(0x7fe9880ac8c0 0x7fe9880aed60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe9880a2c90 msgr2=0x7fe9880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe9880a2c90 0x7fe9880a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005fd0 tx=0x7fe98c005e20 comp rx=0 tx=0).stop 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 msgr2=0x7fe99c1a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 --2- 172.21.15.122:0/4201582234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c102ef0 0x7fe99c1a43f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe99402ae70 tx=0x7fe99402f5d0 comp rx=0 tx=0).stop 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a0d8a640 1 -- 172.21.15.122:0/4201582234 reap_dead start 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 shutdown_connections 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 >> 172.21.15.122:0/4201582234 conn(0x7fe99c0fc7d0 msgr2=0x7fe99c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 shutdown_connections 2024-09-12T01:45:18.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:18.742+0000 7fe9a27ed640 1 -- 172.21.15.122:0/4201582234 wait complete. 2024-09-12T01:45:19.459 INFO:teuthology.orchestra.run.smithi122.stdout:270582939680 2024-09-12T01:45:19.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T01:45:19.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:19 smithi122 ceph-mon[30925]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T01:45:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:19 smithi151 ceph-mon[35229]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T01:45:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:19 smithi183 ceph-mon[35127]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T01:45:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 -- 172.21.15.122:0/1956754854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 msgr2=0x7f74c4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 --2- 172.21.15.122:0/1956754854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c4074970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009900 tx=0x7f74c002a480 comp rx=0 tx=0).stop 2024-09-12T01:45:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 -- 172.21.15.122:0/1956754854 shutdown_connections 2024-09-12T01:45:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 --2- 172.21.15.122:0/1956754854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c4112b80 0x7f74c4114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 --2- 172.21.15.122:0/1956754854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74c4074eb0 0x7f74c4112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 --2- 172.21.15.122:0/1956754854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c4074970 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 -- 172.21.15.122:0/1956754854 >> 172.21.15.122:0/1956754854 conn(0x7f74c40fd8b0 msgr2=0x7f74c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 -- 172.21.15.122:0/1956754854 shutdown_connections 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.184+0000 7f74cb355640 1 -- 172.21.15.122:0/1956754854 wait complete. 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 Processor -- start 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 -- start start 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c4074eb0 0x7f74c419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74c4112b80 0x7f74c41a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4118560 con 0x7f74c4074590 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74c41183e0 con 0x7f74c4112b80 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74cb355640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74c41186e0 con 0x7f74c4074eb0 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74c90f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74c90f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35220/0 (socket says 172.21.15.122:35220) 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74c90f1640 1 -- 172.21.15.122:0/338786768 learned_addr learned my addr 172.21.15.122:0/338786768 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74c98f2640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74c4112b80 0x7f74c41a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.185+0000 7f74c88f0640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c4074eb0 0x7f74c419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c90f1640 1 -- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c4074eb0 msgr2=0x7f74c419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c90f1640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c4074eb0 0x7f74c419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c90f1640 1 -- 172.21.15.122:0/338786768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74c4112b80 msgr2=0x7f74c41a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c90f1640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74c4112b80 0x7f74c41a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c90f1640 1 -- 172.21.15.122:0/338786768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c41a3bb0 con 0x7f74c4074590 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c88f0640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c4074eb0 0x7f74c419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74c90f1640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c419eb80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005ab0 tx=0x7f74c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c003f030 con 0x7f74c4074590 2024-09-12T01:45:20.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c41a3e40 con 0x7f74c4074590 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c0003ce0 con 0x7f74c4074590 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0035070 con 0x7f74c4074590 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.186+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c41ab730 con 0x7f74c4074590 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.187+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f74c40628c0 con 0x7f74c4074590 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.188+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f74c0042030 con 0x7f74c4074590 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74ba7fc640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74b00a2c10 0x7f74b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74c88f0640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74b00a2c10 0x7f74b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f74c00efe10 con 0x7f74c4074590 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74ba7fc640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] conn(0x7f74b00ac840 0x7f74b00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 --> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74c0001d30 con 0x7f74b00ac840 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f74c00f0240 con 0x7f74c4074590 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74c88f0640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74b00a2c10 0x7f74b00a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74c40fff80 tx=0x7f74b4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.189+0000 7f74c98f2640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] conn(0x7f74b00ac840 0x7f74b00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.190+0000 7f74c98f2640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] conn(0x7f74b00ac840 0x7f74b00aece0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.191+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== osd.11 v2:172.21.15.183:6824/2785958349 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74c0001d30 con 0x7f74b00ac840 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.204+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 --> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f74c4062cc0 con 0x7f74b00ac840 2024-09-12T01:45:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74ba7fc640 1 -- 172.21.15.122:0/338786768 <== osd.11 v2:172.21.15.183:6824/2785958349 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f74c4062cc0 con 0x7f74b00ac840 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] conn(0x7f74b00ac840 msgr2=0x7f74b00aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] conn(0x7f74b00ac840 0x7f74b00aece0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74b00a2c10 msgr2=0x7f74b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74b00a2c10 0x7f74b00a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74c40fff80 tx=0x7f74b4005d90 comp rx=0 tx=0).stop 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 msgr2=0x7f74c419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 --2- 172.21.15.122:0/338786768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74c4074590 0x7f74c419eb80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005ab0 tx=0x7f74c00042f0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74c98f2640 1 -- 172.21.15.122:0/338786768 reap_dead start 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 shutdown_connections 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 >> 172.21.15.122:0/338786768 conn(0x7f74c40fd8b0 msgr2=0x7f74c40ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 shutdown_connections 2024-09-12T01:45:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.205+0000 7f74cb355640 1 -- 172.21.15.122:0/338786768 wait complete. 2024-09-12T01:45:20.652 INFO:teuthology.orchestra.run.smithi122.stdout:73014444100 2024-09-12T01:45:20.653 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T01:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 -- 172.21.15.122:0/3208349508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518102570 msgr2=0x7f55181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 --2- 172.21.15.122:0/3208349508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518102570 0x7f55181029b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f550c009900 tx=0x7f550c02a480 comp rx=0 tx=0).stop 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 -- 172.21.15.122:0/3208349508 shutdown_connections 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 --2- 172.21.15.122:0/3208349508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5518102ef0 0x7f551810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 --2- 172.21.15.122:0/3208349508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518102570 0x7f55181029b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 --2- 172.21.15.122:0/3208349508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5518100e20 0x7f5518101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 -- 172.21.15.122:0/3208349508 >> 172.21.15.122:0/3208349508 conn(0x7f55180fc7d0 msgr2=0x7f55180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 -- 172.21.15.122:0/3208349508 shutdown_connections 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 -- 172.21.15.122:0/3208349508 wait complete. 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 Processor -- start 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.808+0000 7f551f2bc640 1 -- start start 2024-09-12T01:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518100e20 0x7f5518199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5518102570 0x7f55181998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5518102ef0 0x7f55181a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5518110f80 con 0x7f5518100e20 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5518110e00 con 0x7f5518102570 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5518111100 con 0x7f5518102ef0 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518100e20 0x7f5518199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518100e20 0x7f5518199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36858/0 (socket says 172.21.15.122:36858) 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 -- 172.21.15.122:0/2995649049 learned_addr learned my addr 172.21.15.122:0/2995649049 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551c857640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5518102570 0x7f55181998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d859640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5518102ef0 0x7f55181a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 -- 172.21.15.122:0/2995649049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5518102ef0 msgr2=0x7f55181a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5518102ef0 0x7f55181a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 -- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5518102570 msgr2=0x7f55181998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5518102570 0x7f55181998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 -- 172.21.15.122:0/2995649049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55181a4950 con 0x7f5518100e20 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d859640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5518102ef0 0x7f55181a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551d058640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518100e20 0x7f5518199390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f551400d770 tx=0x7f551400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514017070 con 0x7f5518100e20 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5514005500 con 0x7f5518100e20 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55181a4b20 con 0x7f5518100e20 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.809+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514004e00 con 0x7f5518100e20 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.810+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55181a4fe0 con 0x7f5518100e20 2024-09-12T01:45:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.811+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5518062900 con 0x7f5518100e20 2024-09-12T01:45:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.812+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5514004fa0 con 0x7f5518100e20 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f550a7fc640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55040a2c10 0x7f55040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f551c857640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55040a2c10 0x7f55040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f55140cf100 con 0x7f5518100e20 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f550a7fc640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] conn(0x7f55040ac840 0x7f55040aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 --> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f551400fec0 con 0x7f55040ac840 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f55140cf530 con 0x7f5518100e20 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f551c857640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55040a2c10 0x7f55040a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f551806aac0 tx=0x7f550c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.813+0000 7f551d859640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] conn(0x7f55040ac840 0x7f55040aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.814+0000 7f551d859640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] conn(0x7f55040ac840 0x7f55040aece0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.815+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== osd.7 v2:172.21.15.151:6816/1163877446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f551400fec0 con 0x7f55040ac840 2024-09-12T01:45:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.828+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 --> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5518062d80 con 0x7f55040ac840 2024-09-12T01:45:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f550a7fc640 1 -- 172.21.15.122:0/2995649049 <== osd.7 v2:172.21.15.151:6816/1163877446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5518062d80 con 0x7f55040ac840 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] conn(0x7f55040ac840 msgr2=0x7f55040aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] conn(0x7f55040ac840 0x7f55040aece0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55040a2c10 msgr2=0x7f55040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f55040a2c10 0x7f55040a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f551806aac0 tx=0x7f550c005a20 comp rx=0 tx=0).stop 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518100e20 msgr2=0x7f5518199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 --2- 172.21.15.122:0/2995649049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5518100e20 0x7f5518199390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f551400d770 tx=0x7f551400dc30 comp rx=0 tx=0).stop 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551d859640 1 -- 172.21.15.122:0/2995649049 reap_dead start 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 shutdown_connections 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 >> 172.21.15.122:0/2995649049 conn(0x7f55180fc7d0 msgr2=0x7f5518105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.829+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 shutdown_connections 2024-09-12T01:45:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:20.830+0000 7f551f2bc640 1 -- 172.21.15.122:0/2995649049 wait complete. 2024-09-12T01:45:21.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 -- 172.21.15.122:0/680630084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 msgr2=0x7f0f1c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 --2- 172.21.15.122:0/680630084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c10ea90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009ca0 tx=0x7f0f1402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 -- 172.21.15.122:0/680630084 shutdown_connections 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 --2- 172.21.15.122:0/680630084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c10ea90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 --2- 172.21.15.122:0/680630084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f1c102790 0x7f0f1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 --2- 172.21.15.122:0/680630084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f1c101de0 0x7f0f1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 -- 172.21.15.122:0/680630084 >> 172.21.15.122:0/680630084 conn(0x7f0f1c0fd8c0 msgr2=0x7f0f1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 -- 172.21.15.122:0/680630084 shutdown_connections 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 -- 172.21.15.122:0/680630084 wait complete. 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.016+0000 7f0f220d5640 1 Processor -- start 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 -- start start 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f1c101de0 0x7f0f1c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:21.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f1c102790 0x7f0f1c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c111fb0 con 0x7f0f1c106f10 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c111e30 con 0x7f0f1c101de0 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f220d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c112130 con 0x7f0f1c102790 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36872/0 (socket says 172.21.15.122:36872) 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 -- 172.21.15.122:0/3542567551 learned_addr learned my addr 172.21.15.122:0/3542567551 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1b7fe640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f1c101de0 0x7f0f1c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1affd640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f1c102790 0x7f0f1c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 -- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f1c102790 msgr2=0x7f0f1c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f1c102790 0x7f0f1c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 -- 172.21.15.122:0/3542567551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f1c101de0 msgr2=0x7f0f1c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f1c101de0 0x7f0f1c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1bfff640 1 -- 172.21.15.122:0/3542567551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c19d420 con 0x7f0f1c106f10 2024-09-12T01:45:21.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1affd640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f1c102790 0x7f0f1c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:21.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.017+0000 7f0f1b7fe640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f1c101de0 0x7f0f1c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.018+0000 7f0f1bfff640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c19cd40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f14031a60 tx=0x7f0f14031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.018+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14003c30 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.018+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f14003dd0 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.018+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140047e0 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.018+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c19d6b0 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.018+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c1a4ff0 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.019+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f1c062980 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.020+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0f1402f790 con 0x7f0f1c106f10 2024-09-12T01:45:21.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.020+0000 7f0f18ff9640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f040a2c10 0x7f0f040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.020+0000 7f0f1b7fe640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f040a2c10 0x7f0f040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.020+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0f140f07f0 con 0x7f0f1c106f10 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.021+0000 7f0f18ff9640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] conn(0x7f0f040ac710 0x7f0f040aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.021+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 --> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f040af280 con 0x7f0f040ac710 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.021+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0f140f0c20 con 0x7f0f1c106f10 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.021+0000 7f0f1b7fe640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f040a2c10 0x7f0f040a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009930 tx=0x7f0f0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.021+0000 7f0f1affd640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] conn(0x7f0f040ac710 0x7f0f040aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.021+0000 7f0f1affd640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] conn(0x7f0f040ac710 0x7f0f040aebb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.022+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== osd.5 v2:172.21.15.183:6808/685154842 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f040af280 con 0x7f0f040ac710 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.035+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 --> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f1c062e00 con 0x7f0f040ac710 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.036+0000 7f0f18ff9640 1 -- 172.21.15.122:0/3542567551 <== osd.5 v2:172.21.15.183:6808/685154842 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f1c062e00 con 0x7f0f040ac710 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] conn(0x7f0f040ac710 msgr2=0x7f0f040aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:21.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] conn(0x7f0f040ac710 0x7f0f040aebb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f040a2c10 msgr2=0x7f0f040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f040a2c10 0x7f0f040a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009930 tx=0x7f0f0c008040 comp rx=0 tx=0).stop 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 msgr2=0x7f0f1c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 --2- 172.21.15.122:0/3542567551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f1c106f10 0x7f0f1c19cd40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f14031a60 tx=0x7f0f14031a90 comp rx=0 tx=0).stop 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f1bfff640 1 -- 172.21.15.122:0/3542567551 reap_dead start 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 shutdown_connections 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 >> 172.21.15.122:0/3542567551 conn(0x7f0f1c0fd8c0 msgr2=0x7f0f1c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 shutdown_connections 2024-09-12T01:45:21.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:21.037+0000 7f0f220d5640 1 -- 172.21.15.122:0/3542567551 wait complete. 2024-09-12T01:45:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:21 smithi151 ceph-mon[35229]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T01:45:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:21 smithi183 ceph-mon[35127]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T01:45:21.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:21 smithi122 ceph-mon[30925]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T01:45:22.070 INFO:teuthology.orchestra.run.smithi122.stdout:210453397546 2024-09-12T01:45:22.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T01:45:22.822 INFO:teuthology.orchestra.run.smithi122.stdout:322122547221 2024-09-12T01:45:22.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T01:45:23.390 INFO:teuthology.orchestra.run.smithi122.stdout:107374182461 2024-09-12T01:45:23.391 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T01:45:23.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:23 smithi122 ceph-mon[30925]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T01:45:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:23 smithi151 ceph-mon[35229]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T01:45:23.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 -- 172.21.15.122:0/3879741631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e8102d40 msgr2=0x7f59e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 --2- 172.21.15.122:0/3879741631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e8102d40 0x7f59e8103120 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009900 tx=0x7f59e402a480 comp rx=0 tx=0).stop 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 -- 172.21.15.122:0/3879741631 shutdown_connections 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 --2- 172.21.15.122:0/3879741631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59e810c170 0x7f59e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 --2- 172.21.15.122:0/3879741631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e8103660 0x7f59e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 --2- 172.21.15.122:0/3879741631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e8102d40 0x7f59e8103120 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 -- 172.21.15.122:0/3879741631 >> 172.21.15.122:0/3879741631 conn(0x7f59e80fc7a0 msgr2=0x7f59e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 -- 172.21.15.122:0/3879741631 shutdown_connections 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 -- 172.21.15.122:0/3879741631 wait complete. 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.925+0000 7f59ef4af640 1 Processor -- start 2024-09-12T01:45:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 -- start start 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59e8102d40 0x7f59e81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e8103660 0x7f59e8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e810c170 0x7f59e8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8112030 con 0x7f59e810c170 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8111eb0 con 0x7f59e8102d40 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ef4af640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81121b0 con 0x7f59e8103660 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e810c170 0x7f59e8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e810c170 0x7f59e8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36888/0 (socket says 172.21.15.122:36888) 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 -- 172.21.15.122:0/76350697 learned_addr learned my addr 172.21.15.122:0/76350697 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eca4a640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e8103660 0x7f59e8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59ed24b640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59e8102d40 0x7f59e81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 -- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e8103660 msgr2=0x7f59e8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e8103660 0x7f59e8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 -- 172.21.15.122:0/76350697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59e8102d40 msgr2=0x7f59e81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59e8102d40 0x7f59e81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 -- 172.21.15.122:0/76350697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e81a6a60 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.926+0000 7f59eda4c640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e810c170 0x7f59e8101420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59e000b970 tx=0x7f59e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.927+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0019070 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.927+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e000cb40 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.927+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e81a6d50 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.927+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0007650 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.927+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81a7290 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.927+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59e80628c0 con 0x7f59e810c170 2024-09-12T01:45:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.929+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f59e001c070 con 0x7f59e810c170 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.929+0000 7f59de7fc640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59d40a2c90 0x7f59d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.929+0000 7f59ed24b640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59d40a2c90 0x7f59d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.929+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f59e00cee20 con 0x7f59e810c170 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.930+0000 7f59de7fc640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] conn(0x7f59d40ac8c0 0x7f59d40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.930+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 --> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59e00047d0 con 0x7f59d40ac8c0 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.930+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f59e00cf250 con 0x7f59e810c170 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.930+0000 7f59ed24b640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59d40a2c90 0x7f59d40a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f59e40096c0 tx=0x7f59e4031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.930+0000 7f59eca4a640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] conn(0x7f59d40ac8c0 0x7f59d40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.930+0000 7f59eca4a640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] conn(0x7f59d40ac8c0 0x7f59d40aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.931+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== osd.8 v2:172.21.15.183:6816/830128598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59e00047d0 con 0x7f59d40ac8c0 2024-09-12T01:45:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.945+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 --> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59e8062d40 con 0x7f59d40ac8c0 2024-09-12T01:45:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59de7fc640 1 -- 172.21.15.122:0/76350697 <== osd.8 v2:172.21.15.183:6816/830128598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59e8062d40 con 0x7f59d40ac8c0 2024-09-12T01:45:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] conn(0x7f59d40ac8c0 msgr2=0x7f59d40aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] conn(0x7f59d40ac8c0 0x7f59d40aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59d40a2c90 msgr2=0x7f59d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f59d40a2c90 0x7f59d40a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f59e40096c0 tx=0x7f59e4031490 comp rx=0 tx=0).stop 2024-09-12T01:45:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e810c170 msgr2=0x7f59e8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 --2- 172.21.15.122:0/76350697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59e810c170 0x7f59e8101420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59e000b970 tx=0x7f59e000be30 comp rx=0 tx=0).stop 2024-09-12T01:45:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59eda4c640 1 -- 172.21.15.122:0/76350697 reap_dead start 2024-09-12T01:45:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 shutdown_connections 2024-09-12T01:45:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 >> 172.21.15.122:0/76350697 conn(0x7f59e80fc7a0 msgr2=0x7f59e80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 shutdown_connections 2024-09-12T01:45:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:23.946+0000 7f59ef4af640 1 -- 172.21.15.122:0/76350697 wait complete. 2024-09-12T01:45:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:23 smithi183 ceph-mon[35127]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T01:45:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:24.429 INFO:teuthology.orchestra.run.smithi122.stdout:158913790005 2024-09-12T01:45:24.432 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T01:45:24.689 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:24.689 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:24.733 INFO:teuthology.orchestra.run.smithi122.stdout:240518168613 2024-09-12T01:45:24.733 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 -- 172.21.15.122:0/3362480696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508100e20 msgr2=0x7f3508101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 --2- 172.21.15.122:0/3362480696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508100e20 0x7f3508101200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009900 tx=0x7f34f802a480 comp rx=0 tx=0).stop 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 -- 172.21.15.122:0/3362480696 shutdown_connections 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 --2- 172.21.15.122:0/3362480696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3508102ef0 0x7f350810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 --2- 172.21.15.122:0/3362480696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508102570 0x7f35081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 --2- 172.21.15.122:0/3362480696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508100e20 0x7f3508101200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 -- 172.21.15.122:0/3362480696 >> 172.21.15.122:0/3362480696 conn(0x7f35080fc7d0 msgr2=0x7f35080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 -- 172.21.15.122:0/3362480696 shutdown_connections 2024-09-12T01:45:25.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.360+0000 7f350e74a640 1 -- 172.21.15.122:0/3362480696 wait complete. 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 Processor -- start 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 -- start start 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508100e20 0x7f3508103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3508102570 0x7f3508103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 0x7f35081041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3508110f80 con 0x7f3508102ef0 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3508110e00 con 0x7f3508100e20 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350e74a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3508111100 con 0x7f3508102570 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 0x7f35081041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 0x7f35081041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36912/0 (socket says 172.21.15.122:36912) 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 -- 172.21.15.122:0/2500635040 learned_addr learned my addr 172.21.15.122:0/2500635040 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:25.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f34fffff640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3508102570 0x7f3508103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f3507fff640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508100e20 0x7f3508103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 -- 172.21.15.122:0/2500635040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3508102570 msgr2=0x7f3508103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3508102570 0x7f3508103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 -- 172.21.15.122:0/2500635040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508100e20 msgr2=0x7f3508103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508100e20 0x7f3508103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 -- 172.21.15.122:0/2500635040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081a69b0 con 0x7f3508102ef0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f3507fff640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508100e20 0x7f3508103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.361+0000 7f350cce7640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 0x7f35081041d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f350000eb10 tx=0x7f350000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.362+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500004b20 con 0x7f3508102ef0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.362+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3500004cc0 con 0x7f3508102ef0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.362+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35081a6ca0 con 0x7f3508102ef0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.362+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500010760 con 0x7f3508102ef0 2024-09-12T01:45:25.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.362+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a71e0 con 0x7f3508102ef0 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.363+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3508101200 con 0x7f3508102ef0 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.364+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3500005000 con 0x7f3508102ef0 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.364+0000 7f3505ffb640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34f40a2c10 0x7f34f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.365+0000 7f3507fff640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34f40a2c10 0x7f34f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.365+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f35000cfc10 con 0x7f3508102ef0 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.365+0000 7f3507fff640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34f40a2c10 0x7f34f40a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f8004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:25.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.368+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f350009ca10 con 0x7f3508102ef0 2024-09-12T01:45:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.539+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3508104c80 con 0x7f3508102ef0 2024-09-12T01:45:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.540+0000 7f3505ffb640 1 -- 172.21.15.122:0/2500635040 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3508104c80 con 0x7f3508102ef0 2024-09-12T01:45:25.540 INFO:teuthology.orchestra.run.smithi122.stdout:137438953531 2024-09-12T01:45:25.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34f40a2c10 msgr2=0x7f34f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34f40a2c10 0x7f34f40a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f8004100 comp rx=0 tx=0).stop 2024-09-12T01:45:25.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 msgr2=0x7f35081041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 0x7f35081041d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f350000eb10 tx=0x7f350000c2d0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 shutdown_connections 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34f40a2c10 0x7f34f40a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3508102ef0 0x7f35081041d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3508102570 0x7f3508103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 --2- 172.21.15.122:0/2500635040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3508100e20 0x7f3508103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 >> 172.21.15.122:0/2500635040 conn(0x7f35080fc7d0 msgr2=0x7f350806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 shutdown_connections 2024-09-12T01:45:25.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.542+0000 7f350e74a640 1 -- 172.21.15.122:0/2500635040 wait complete. 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.625+0000 7ff64ac37640 1 -- 172.21.15.122:0/893743458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 msgr2=0x7ff6441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.625+0000 7ff64ac37640 1 --2- 172.21.15.122:0/893743458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6441029b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff630009900 tx=0x7ff63002a480 comp rx=0 tx=0).stop 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- 172.21.15.122:0/893743458 shutdown_connections 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 --2- 172.21.15.122:0/893743458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff644102ef0 0x7ff64410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 --2- 172.21.15.122:0/893743458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6441029b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 --2- 172.21.15.122:0/893743458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff644100e20 0x7ff644101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- 172.21.15.122:0/893743458 >> 172.21.15.122:0/893743458 conn(0x7ff6440fc7d0 msgr2=0x7ff6440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- 172.21.15.122:0/893743458 shutdown_connections 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- 172.21.15.122:0/893743458 wait complete. 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 Processor -- start 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- start start 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff644100e20 0x7ff6440fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6440fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff644102ef0 0x7ff6440fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff644110fa0 con 0x7ff644102570 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff644110e20 con 0x7ff644102ef0 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff64ac37640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff644111120 con 0x7ff644100e20 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff643fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6440fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff643fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6440fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36924/0 (socket says 172.21.15.122:36924) 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.626+0000 7ff643fff640 1 -- 172.21.15.122:0/494210303 learned_addr learned my addr 172.21.15.122:0/494210303 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff6491d4640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff644102ef0 0x7ff6440fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff6489d3640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff644100e20 0x7ff6440fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff643fff640 1 -- 172.21.15.122:0/494210303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff644100e20 msgr2=0x7ff6440fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff643fff640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff644100e20 0x7ff6440fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff643fff640 1 -- 172.21.15.122:0/494210303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff644102ef0 msgr2=0x7ff6440fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff643fff640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff644102ef0 0x7ff6440fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff643fff640 1 -- 172.21.15.122:0/494210303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6441a6a50 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff643fff640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6440fd1c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff630006010 tx=0x7ff630003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63003f070 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff630031e30 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6441a6ce0 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630035030 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.627+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6441a71c0 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.628+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff644101200 con 0x7ff644102570 2024-09-12T01:45:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.629+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff630042070 con 0x7ff644102570 2024-09-12T01:45:25.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.630+0000 7ff641ffb640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6340a2c10 0x7ff6340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:25.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.630+0000 7ff6489d3640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6340a2c10 0x7ff6340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:25.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.630+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff6300efb20 con 0x7ff644102570 2024-09-12T01:45:25.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.630+0000 7ff6489d3640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6340a2c10 0x7ff6340a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff63800a810 tx=0x7ff638008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:25.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.634+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6300bc920 con 0x7ff644102570 2024-09-12T01:45:25.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.806+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff64410a0a0 con 0x7ff644102570 2024-09-12T01:45:25.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:25 smithi122 ceph-mon[30925]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T01:45:25.806 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:25 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2500635040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T01:45:25.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.806+0000 7ff641ffb640 1 -- 172.21.15.122:0/494210303 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6300bc220 con 0x7ff644102570 2024-09-12T01:45:25.807 INFO:teuthology.orchestra.run.smithi122.stdout:34359738446 2024-09-12T01:45:25.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6340a2c10 msgr2=0x7ff6340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6340a2c10 0x7ff6340a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff63800a810 tx=0x7ff638008040 comp rx=0 tx=0).stop 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 msgr2=0x7ff6440fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6440fd1c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff630006010 tx=0x7ff630003c80 comp rx=0 tx=0).stop 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 shutdown_connections 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6340a2c10 0x7ff6340a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff644102ef0 0x7ff6440fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff644102570 0x7ff6440fd1c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 --2- 172.21.15.122:0/494210303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff644100e20 0x7ff6440fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.808+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 >> 172.21.15.122:0/494210303 conn(0x7ff6440fc7d0 msgr2=0x7ff64406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.809+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 shutdown_connections 2024-09-12T01:45:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:25.809+0000 7ff64ac37640 1 -- 172.21.15.122:0/494210303 wait complete. 2024-09-12T01:45:25.868 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:25 smithi151 ceph-mon[35229]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T01:45:25.868 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2500635040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T01:45:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:25 smithi183 ceph-mon[35127]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T01:45:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:25 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2500635040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T01:45:26.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953529 got 137438953531 for osd.4 2024-09-12T01:45:26.265 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:26 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/494210303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T01:45:26.884 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:26 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/494210303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T01:45:26.914 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738446 for osd.0 2024-09-12T01:45:26.915 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:26 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/494210303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T01:45:27.126 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.128 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.128 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.130 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.132 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 -- 172.21.15.122:0/106947013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0102790 msgr2=0x7f4ad01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/106947013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0102790 0x7f4ad01067e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009900 tx=0x7f4ac402a410 comp rx=0 tx=0).stop 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 -- 172.21.15.122:0/106947013 shutdown_connections 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/106947013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ad0106f10 0x7f4ad010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/106947013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0102790 0x7f4ad01067e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/106947013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0101de0 0x7f4ad01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 -- 172.21.15.122:0/106947013 >> 172.21.15.122:0/106947013 conn(0x7f4ad00fd8c0 msgr2=0x7f4ad00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 -- 172.21.15.122:0/106947013 shutdown_connections 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.335+0000 7f4ad60b7640 1 -- 172.21.15.122:0/106947013 wait complete. 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 Processor -- start 2024-09-12T01:45:27.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 -- start start 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 0x7f4ad01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0102790 0x7f4ad0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ad0106f10 0x7f4ad019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0111fb0 con 0x7f4ad0101de0 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0111e30 con 0x7f4ad0102790 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4ad60b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112130 con 0x7f4ad0106f10 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 0x7f4ad01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 0x7f4ad01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36948/0 (socket says 172.21.15.122:36948) 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 -- 172.21.15.122:0/2335992839 learned_addr learned my addr 172.21.15.122:0/2335992839 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4aceffd640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0102790 0x7f4ad0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acffff640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ad0106f10 0x7f4ad019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:27.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 -- 172.21.15.122:0/2335992839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ad0106f10 msgr2=0x7f4ad019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ad0106f10 0x7f4ad019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 -- 172.21.15.122:0/2335992839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0102790 msgr2=0x7f4ad0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0102790 0x7f4ad0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4acf7fe640 1 -- 172.21.15.122:0/2335992839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad019d4a0 con 0x7f4ad0101de0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.336+0000 7f4aceffd640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0102790 0x7f4ad0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.337+0000 7f4acf7fe640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 0x7f4ad01984f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000b4a0 tx=0x7f4ac000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.337+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0014070 con 0x7f4ad0101de0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.337+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0010af0 con 0x7f4ad0101de0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.337+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0017d10 con 0x7f4ad0101de0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.337+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad019d790 con 0x7f4ad0101de0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.337+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a5070 con 0x7f4ad0101de0 2024-09-12T01:45:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.338+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad01021c0 con 0x7f4ad0101de0 2024-09-12T01:45:27.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.339+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4ac0017070 con 0x7f4ad0101de0 2024-09-12T01:45:27.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.339+0000 7f4accff9640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ab80a2c90 0x7f4ab80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:27.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.340+0000 7f4aceffd640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ab80a2c90 0x7f4ab80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:27.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.340+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4ac00ce460 con 0x7f4ad0101de0 2024-09-12T01:45:27.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.340+0000 7f4aceffd640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ab80a2c90 0x7f4ab80a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ad01088e0 tx=0x7f4ac4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:27.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.344+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ac009b370 con 0x7f4ad0101de0 2024-09-12T01:45:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.362 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:27.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.531+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4ad0199850 con 0x7f4ad0101de0 2024-09-12T01:45:27.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.532+0000 7f4accff9640 1 -- 172.21.15.122:0/2335992839 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ac009ac70 con 0x7f4ad0101de0 2024-09-12T01:45:27.532 INFO:teuthology.orchestra.run.smithi122.stdout:55834574922 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ab80a2c90 msgr2=0x7f4ab80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ab80a2c90 0x7f4ab80a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ad01088e0 tx=0x7f4ac4005a20 comp rx=0 tx=0).stop 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 msgr2=0x7f4ad01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 0x7f4ad01984f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000b4a0 tx=0x7f4ac000b960 comp rx=0 tx=0).stop 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 shutdown_connections 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4ab80a2c90 0x7f4ab80a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ad0106f10 0x7f4ad019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ad0102790 0x7f4ad0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 --2- 172.21.15.122:0/2335992839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad0101de0 0x7f4ad01984f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 >> 172.21.15.122:0/2335992839 conn(0x7f4ad00fd8c0 msgr2=0x7f4ad00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 shutdown_connections 2024-09-12T01:45:27.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:27.534+0000 7f4ad60b7640 1 -- 172.21.15.122:0/2335992839 wait complete. 2024-09-12T01:45:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:27 smithi151 ceph-mon[35229]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-12T01:45:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:27 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2335992839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T01:45:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:27 smithi183 ceph-mon[35127]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-12T01:45:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:27 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2335992839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T01:45:27.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:27 smithi122 ceph-mon[30925]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-12T01:45:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:27 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2335992839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T01:45:28.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574922 for osd.1 2024-09-12T01:45:28.199 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:29.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 -- 172.21.15.122:0/1205605481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640102e00 msgr2=0x7fb6401031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 --2- 172.21.15.122:0/1205605481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640102e00 0x7fb6401031e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009900 tx=0x7fb63c02a480 comp rx=0 tx=0).stop 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 -- 172.21.15.122:0/1205605481 shutdown_connections 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 --2- 172.21.15.122:0/1205605481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb64010c1a0 0x7fb64010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 --2- 172.21.15.122:0/1205605481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640103720 0x7fb64010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 --2- 172.21.15.122:0/1205605481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640102e00 0x7fb6401031e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 -- 172.21.15.122:0/1205605481 >> 172.21.15.122:0/1205605481 conn(0x7fb6400fc7a0 msgr2=0x7fb6400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 -- 172.21.15.122:0/1205605481 shutdown_connections 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.397+0000 7fb647cc5640 1 -- 172.21.15.122:0/1205605481 wait complete. 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 Processor -- start 2024-09-12T01:45:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 -- start start 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640102e00 0x7fb640198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 0x7fb640198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb64010c1a0 0x7fb64019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb640111fb0 con 0x7fb640103720 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb640111e30 con 0x7fb64010c1a0 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb647cc5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb640112130 con 0x7fb640102e00 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 0x7fb640198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 0x7fb640198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36960/0 (socket says 172.21.15.122:36960) 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 -- 172.21.15.122:0/3006245972 learned_addr learned my addr 172.21.15.122:0/3006245972 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb646262640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb64010c1a0 0x7fb64019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645a61640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640102e00 0x7fb640198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 -- 172.21.15.122:0/3006245972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640102e00 msgr2=0x7fb640198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640102e00 0x7fb640198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 -- 172.21.15.122:0/3006245972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb64010c1a0 msgr2=0x7fb64019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb64010c1a0 0x7fb64019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645260640 1 -- 172.21.15.122:0/3006245972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64019d500 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.398+0000 7fb645a61640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640102e00 0x7fb640198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.399+0000 7fb645260640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 0x7fb640198a90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb63000b4a0 tx=0x7fb63000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.399+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630014070 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.399+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64019d7f0 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.399+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb630010af0 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.399+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630017d10 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.399+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6401a50d0 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.400+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6401031e0 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.401+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb630017070 con 0x7fb640103720 2024-09-12T01:45:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.401+0000 7fb636ffd640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb62c0a2c10 0x7fb62c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stdout:322122547223 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.401+0000 7fb645a61640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb62c0a2c10 0x7fb62c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.402+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb6300ce4c0 con 0x7fb640103720 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.402+0000 7fb645a61640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb62c0a2c10 0x7fb62c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005ab0 tx=0x7fb63c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.405+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb63009b3d0 con 0x7fb640103720 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.576+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb640107280 con 0x7fb640103720 2024-09-12T01:45:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.577+0000 7fb636ffd640 1 -- 172.21.15.122:0/3006245972 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb63009acd0 con 0x7fb640103720 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb62c0a2c10 msgr2=0x7fb62c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb62c0a2c10 0x7fb62c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005ab0 tx=0x7fb63c005a20 comp rx=0 tx=0).stop 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 msgr2=0x7fb640198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 0x7fb640198a90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb63000b4a0 tx=0x7fb63000b960 comp rx=0 tx=0).stop 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 shutdown_connections 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb62c0a2c10 0x7fb62c0a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb64010c1a0 0x7fb64019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb640103720 0x7fb640198a90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 --2- 172.21.15.122:0/3006245972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb640102e00 0x7fb640198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 >> 172.21.15.122:0/3006245972 conn(0x7fb6400fc7a0 msgr2=0x7fb6400fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 shutdown_connections 2024-09-12T01:45:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:28.579+0000 7fb647cc5640 1 -- 172.21.15.122:0/3006245972 wait complete. 2024-09-12T01:45:29.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:29 smithi122 ceph-mon[30925]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:29.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:29 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3006245972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T01:45:29.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:29 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2104995147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 -- 172.21.15.122:0/174130748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 msgr2=0x7f862010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 --2- 172.21.15.122:0/174130748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f862010bb50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8610009900 tx=0x7f861002a480 comp rx=0 tx=0).stop 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 -- 172.21.15.122:0/174130748 shutdown_connections 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 --2- 172.21.15.122:0/174130748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f862010c170 0x7f862010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 --2- 172.21.15.122:0/174130748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f862010bb50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 --2- 172.21.15.122:0/174130748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 0x7f8620103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 -- 172.21.15.122:0/174130748 >> 172.21.15.122:0/174130748 conn(0x7f86200fc7a0 msgr2=0x7f86200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 -- 172.21.15.122:0/174130748 shutdown_connections 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.823+0000 7f86279ef640 1 -- 172.21.15.122:0/174130748 wait complete. 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 Processor -- start 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 -- start start 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 0x7f8620198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f8620198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f862010c170 0x7f862019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8620111fd0 con 0x7f8620103660 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8620111e50 con 0x7f8620102d40 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f86279ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8620112150 con 0x7f862010c170 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f8620198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f8620198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37002/0 (socket says 172.21.15.122:37002) 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 -- 172.21.15.122:0/2413198146 learned_addr learned my addr 172.21.15.122:0/2413198146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8625f8c640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f862010c170 0x7f862019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f862578b640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 0x7f8620198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 -- 172.21.15.122:0/2413198146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f862010c170 msgr2=0x7f862019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f862010c170 0x7f862019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 -- 172.21.15.122:0/2413198146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 msgr2=0x7f8620198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 0x7f8620198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 -- 172.21.15.122:0/2413198146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862019d500 con 0x7f8620103660 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f862578b640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 0x7f8620198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.824+0000 7f8624f8a640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f8620198a90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8610005ab0 tx=0x7f86100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.825+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861003f030 con 0x7f8620103660 2024-09-12T01:45:29.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.825+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610003ce0 con 0x7f8620103660 2024-09-12T01:45:29.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.825+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862019d790 con 0x7f8620103660 2024-09-12T01:45:29.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.825+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610035070 con 0x7f8620103660 2024-09-12T01:45:29.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.825+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86201a50d0 con 0x7f8620103660 2024-09-12T01:45:29.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.826+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8620103120 con 0x7f8620103660 2024-09-12T01:45:29.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.827+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8610042030 con 0x7f8620103660 2024-09-12T01:45:29.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.828+0000 7f86167fc640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f860c0a2c90 0x7f860c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:29.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.828+0000 7f862578b640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f860c0a2c90 0x7f860c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:29.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.828+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86100efd00 con 0x7f8620103660 2024-09-12T01:45:29.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.828+0000 7f862578b640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f860c0a2c90 0x7f860c0a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f861c006fd0 tx=0x7f861c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:29.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.833+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86100bcac0 con 0x7f8620103660 2024-09-12T01:45:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:29 smithi151 ceph-mon[35229]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:29 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3006245972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T01:45:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:29 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2104995147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T01:45:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:29 smithi183 ceph-mon[35127]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:29 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3006245972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T01:45:29.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:29 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2104995147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T01:45:29.965 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547223 for osd.11 2024-09-12T01:45:29.966 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.005+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f86201071e0 con 0x7f8620103660 2024-09-12T01:45:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.005+0000 7f86167fc640 1 -- 172.21.15.122:0/2413198146 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f86100bc3c0 con 0x7f8620103660 2024-09-12T01:45:30.006 INFO:teuthology.orchestra.run.smithi122.stdout:210453397548 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.007+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f860c0a2c90 msgr2=0x7f860c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.007+0000 7f86279ef640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f860c0a2c90 0x7f860c0a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f861c006fd0 tx=0x7f861c008040 comp rx=0 tx=0).stop 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.007+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 msgr2=0x7f8620198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.007+0000 7f86279ef640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f8620198a90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8610005ab0 tx=0x7f86100042f0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 shutdown_connections 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f860c0a2c90 0x7f860c0a5150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f862010c170 0x7f862019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8620103660 0x7f8620198a90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 --2- 172.21.15.122:0/2413198146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8620102d40 0x7f8620198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 >> 172.21.15.122:0/2413198146 conn(0x7f86200fc7a0 msgr2=0x7f86200fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:30.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 shutdown_connections 2024-09-12T01:45:30.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.008+0000 7f86279ef640 1 -- 172.21.15.122:0/2413198146 wait complete. 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 -- 172.21.15.122:0/2220556678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 msgr2=0x7f373c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 --2- 172.21.15.122:0/2220556678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c100ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3724009930 tx=0x7f372402a420 comp rx=0 tx=0).stop 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 -- 172.21.15.122:0/2220556678 shutdown_connections 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 --2- 172.21.15.122:0/2220556678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373c101420 0x7f373c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 --2- 172.21.15.122:0/2220556678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c100ee0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 --2- 172.21.15.122:0/2220556678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f373c106a80 0x7f373c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 -- 172.21.15.122:0/2220556678 >> 172.21.15.122:0/2220556678 conn(0x7f373c0fc7a0 msgr2=0x7f373c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 -- 172.21.15.122:0/2220556678 shutdown_connections 2024-09-12T01:45:30.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.429+0000 7f3740be4640 1 -- 172.21.15.122:0/2220556678 wait complete. 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 Processor -- start 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 -- start start 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f373c101420 0x7f373c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373c106a80 0x7f373c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f373c110ea0 con 0x7f373c100a80 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f373c110d20 con 0x7f373c101420 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3740be4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f373c111020 con 0x7f373c106a80 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33534/0 (socket says 172.21.15.122:33534) 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 -- 172.21.15.122:0/270224618 learned_addr learned my addr 172.21.15.122:0/270224618 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f3733fff640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f373c101420 0x7f373c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373ad9d640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373c106a80 0x7f373c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 -- 172.21.15.122:0/270224618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373c106a80 msgr2=0x7f373c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373c106a80 0x7f373c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 -- 172.21.15.122:0/270224618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f373c101420 msgr2=0x7f373c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f373c101420 0x7f373c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.430+0000 7f373a59c640 1 -- 172.21.15.122:0/270224618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c19d420 con 0x7f373c100a80 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.431+0000 7f373a59c640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c198470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f372c00c890 tx=0x7f372c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.431+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c004220 con 0x7f373c100a80 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.431+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c19d710 con 0x7f373c100a80 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.431+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c0043c0 con 0x7f373c100a80 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.431+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c004a00 con 0x7f373c100a80 2024-09-12T01:45:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.431+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c1a4ff0 con 0x7f373c100a80 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.432+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373c100ee0 con 0x7f373c100a80 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.433+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f372c0079b0 con 0x7f373c100a80 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.434+0000 7f37337fe640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f37280a2c10 0x7f37280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.434+0000 7f3733fff640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f37280a2c10 0x7f37280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.434+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f372c0cef80 con 0x7f373c100a80 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.434+0000 7f3733fff640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f37280a2c10 0x7f37280a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3724000c00 tx=0x7f3724031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.438+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f372c014030 con 0x7f373c100a80 2024-09-12T01:45:30.461 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:30.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.617+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f373c109a90 con 0x7f373c100a80 2024-09-12T01:45:30.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.617+0000 7f37337fe640 1 -- 172.21.15.122:0/270224618 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f372c09bd80 con 0x7f373c100a80 2024-09-12T01:45:30.618 INFO:teuthology.orchestra.run.smithi122.stdout:292057776157 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f37280a2c10 msgr2=0x7f37280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f37280a2c10 0x7f37280a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3724000c00 tx=0x7f3724031490 comp rx=0 tx=0).stop 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 msgr2=0x7f373c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c198470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f372c00c890 tx=0x7f372c00cd50 comp rx=0 tx=0).stop 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 shutdown_connections 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f37280a2c10 0x7f37280a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373c106a80 0x7f373c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f373c101420 0x7f373c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 --2- 172.21.15.122:0/270224618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f373c100a80 0x7f373c198470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 >> 172.21.15.122:0/270224618 conn(0x7f373c0fc7a0 msgr2=0x7f373c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:30.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 shutdown_connections 2024-09-12T01:45:30.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:30.620+0000 7f3740be4640 1 -- 172.21.15.122:0/270224618 wait complete. 2024-09-12T01:45:30.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:30 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2413198146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T01:45:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:30 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2413198146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T01:45:30.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:30 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2413198146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T01:45:31.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 -- 172.21.15.122:0/3954422416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 msgr2=0x7f3ba410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 --2- 172.21.15.122:0/3954422416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba410ec30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009cd0 tx=0x7f3b9c02a8e0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 -- 172.21.15.122:0/3954422416 shutdown_connections 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 --2- 172.21.15.122:0/3954422416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba410ec30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 --2- 172.21.15.122:0/3954422416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ba4103740 0x7f3ba4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 --2- 172.21.15.122:0/3954422416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4102d90 0x7f3ba4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 -- 172.21.15.122:0/3954422416 >> 172.21.15.122:0/3954422416 conn(0x7f3ba40fc7d0 msgr2=0x7f3ba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 -- 172.21.15.122:0/3954422416 shutdown_connections 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.109+0000 7f3bac695640 1 -- 172.21.15.122:0/3954422416 wait complete. 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 Processor -- start 2024-09-12T01:45:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 -- start start 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ba4102d90 0x7f3ba41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4103740 0x7f3ba4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba41120d0 con 0x7f3ba4108290 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4111f50 con 0x7f3ba4103740 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3bac695640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112250 con 0x7f3ba4102d90 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3baac32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3baac32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba4101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33546/0 (socket says 172.21.15.122:33546) 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.110+0000 7f3baac32640 1 -- 172.21.15.122:0/961438020 learned_addr learned my addr 172.21.15.122:0/961438020 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3ba9c30640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4103740 0x7f3ba4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baa431640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ba4102d90 0x7f3ba41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:31.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baac32640 1 -- 172.21.15.122:0/961438020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ba4102d90 msgr2=0x7f3ba41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baac32640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ba4102d90 0x7f3ba41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baac32640 1 -- 172.21.15.122:0/961438020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4103740 msgr2=0x7f3ba4100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baac32640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4103740 0x7f3ba4100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baac32640 1 -- 172.21.15.122:0/961438020 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba41a6a70 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3baac32640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba4101470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c006010 tx=0x7f3b9c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c004450 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c003e10 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba41a6d00 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0466e0 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.111+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba41a7240 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.112+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba4103170 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.113+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3b9c0305a0 con 0x7f3ba4108290 2024-09-12T01:45:31.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.114+0000 7f3b9b7fe640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3b900a2c10 0x7f3b900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:31.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.114+0000 7f3baa431640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3b900a2c10 0x7f3b900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:31.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.114+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3b9c02e030 con 0x7f3ba4108290 2024-09-12T01:45:31.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.114+0000 7f3baa431640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3b900a2c10 0x7f3b900a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00058c0 tx=0x7f3ba000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:31.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.118+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b9c04b060 con 0x7f3ba4108290 2024-09-12T01:45:31.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.291+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3ba410a2a0 con 0x7f3ba4108290 2024-09-12T01:45:31.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.291+0000 7f3b9b7fe640 1 -- 172.21.15.122:0/961438020 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b9c0bda20 con 0x7f3ba4108290 2024-09-12T01:45:31.291 INFO:teuthology.orchestra.run.smithi122.stdout:188978561073 2024-09-12T01:45:31.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3b900a2c10 msgr2=0x7f3b900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3b900a2c10 0x7f3b900a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00058c0 tx=0x7f3ba000a4c0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 msgr2=0x7f3ba4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba4101470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c006010 tx=0x7f3b9c003c20 comp rx=0 tx=0).stop 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 shutdown_connections 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3b900a2c10 0x7f3b900a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ba4108290 0x7f3ba4101470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4103740 0x7f3ba4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 --2- 172.21.15.122:0/961438020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ba4102d90 0x7f3ba41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.293+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 >> 172.21.15.122:0/961438020 conn(0x7f3ba40fc7d0 msgr2=0x7f3ba4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.294+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 shutdown_connections 2024-09-12T01:45:31.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:31.294+0000 7f3bac695640 1 -- 172.21.15.122:0/961438020 wait complete. 2024-09-12T01:45:31.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:31 smithi122 ceph-mon[30925]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:31.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/270224618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T01:45:31.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:31 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/961438020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T01:45:31.857 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:31 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:45:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:31 smithi151 ceph-mon[35229]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/270224618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T01:45:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:31 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/961438020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T01:45:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:31 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:45:31.921 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:31 smithi183 ceph-mon[35127]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:31.921 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/270224618' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T01:45:31.921 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:31 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/961438020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T01:45:31.921 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:31 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:45:32.220 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561069 got 188978561073 for osd.6 2024-09-12T01:45:32.220 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:32.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397546 got 210453397548 for osd.7 2024-09-12T01:45:32.612 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:32.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2634209259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 msgr2=0x7fd99c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2634209259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c1067e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd984009900 tx=0x7fd98402a410 comp rx=0 tx=0).stop 2024-09-12T01:45:32.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2634209259 shutdown_connections 2024-09-12T01:45:32.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2634209259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd99c106f10 0x7fd99c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2634209259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c1067e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2634209259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c101de0 0x7fd99c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2634209259 >> 172.21.15.122:0/2634209259 conn(0x7fd99c0fd8c0 msgr2=0x7fd99c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2634209259 shutdown_connections 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2634209259 wait complete. 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.795+0000 7fd9a13b1640 1 Processor -- start 2024-09-12T01:45:32.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 -- start start 2024-09-12T01:45:32.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c101de0 0x7fd99c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd99c106f10 0x7fd99c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111f70 con 0x7fd99c102790 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111df0 con 0x7fd99c106f10 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd9a13b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1120f0 con 0x7fd99c101de0 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33560/0 (socket says 172.21.15.122:33560) 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 -- 172.21.15.122:0/2464999245 learned_addr learned my addr 172.21.15.122:0/2464999245 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd99affd640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c101de0 0x7fd99c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd99b7fe640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd99c106f10 0x7fd99c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 -- 172.21.15.122:0/2464999245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c101de0 msgr2=0x7fd99c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c101de0 0x7fd99c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 -- 172.21.15.122:0/2464999245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd99c106f10 msgr2=0x7fd99c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd99c106f10 0x7fd99c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.796+0000 7fd993fff640 1 -- 172.21.15.122:0/2464999245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1a6950 con 0x7fd99c102790 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.797+0000 7fd993fff640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c074a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd984006010 tx=0x7fd9840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.797+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98403f030 con 0x7fd99c102790 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.797+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd984003ce0 con 0x7fd99c102790 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.797+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1a6be0 con 0x7fd99c102790 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.797+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a70c0 con 0x7fd99c102790 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.797+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984035070 con 0x7fd99c102790 2024-09-12T01:45:32.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.798+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c1021c0 con 0x7fd99c102790 2024-09-12T01:45:32.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.799+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd984042030 con 0x7fd99c102790 2024-09-12T01:45:32.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.800+0000 7fd998ff9640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9880a2c10 0x7fd9880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.800+0000 7fd99affd640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9880a2c10 0x7fd9880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.800+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd9840ef630 con 0x7fd99c102790 2024-09-12T01:45:32.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.800+0000 7fd99affd640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9880a2c10 0x7fd9880a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:32.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.804+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9840bbea0 con 0x7fd99c102790 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.126+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/3447692950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4100e20 msgr2=0x7f4bf4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.126+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/3447692950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4100e20 0x7f4bf4101200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009900 tx=0x7f4bf002a480 comp rx=0 tx=0).stop 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/3447692950 shutdown_connections 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/3447692950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bf4102ef0 0x7f4bf410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/3447692950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4102570 0x7f4bf41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/3447692950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4100e20 0x7f4bf4101200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/3447692950 >> 172.21.15.122:0/3447692950 conn(0x7f4bf40fc7d0 msgr2=0x7f4bf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/3447692950 shutdown_connections 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/3447692950 wait complete. 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 Processor -- start 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- start start 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4100e20 0x7f4bf41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 0x7f4bf4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bf4102ef0 0x7f4bf419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4110f50 con 0x7f4bf4102570 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4110dd0 con 0x7f4bf4102ef0 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.127+0000 7f4bfc5c9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf41110d0 con 0x7f4bf4100e20 2024-09-12T01:45:32.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 0x7f4bf4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 0x7f4bf4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36988/0 (socket says 172.21.15.122:36988) 2024-09-12T01:45:32.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 -- 172.21.15.122:0/2104995147 learned_addr learned my addr 172.21.15.122:0/2104995147 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:32.974 INFO:teuthology.orchestra.run.smithi122.stdout:270582939682 2024-09-12T01:45:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bfa365640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4100e20 0x7f4bf41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bfab66640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bf4102ef0 0x7f4bf419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 -- 172.21.15.122:0/2104995147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4100e20 msgr2=0x7f4bf41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4100e20 0x7f4bf41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 -- 172.21.15.122:0/2104995147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bf4102ef0 msgr2=0x7f4bf419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bf4102ef0 0x7f4bf419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 -- 172.21.15.122:0/2104995147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf419c280 con 0x7f4bf4102570 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bf9b64640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 0x7f4bf4197810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c920 tx=0x7f4be400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007c70 con 0x7f4bf4102570 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be4007e10 con 0x7f4bf4102570 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4018890 con 0x7f4bf4102570 2024-09-12T01:45:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf419c570 con 0x7f4bf4102570 2024-09-12T01:45:32.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.128+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a3e50 con 0x7f4bf4102570 2024-09-12T01:45:32.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.129+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf4101200 con 0x7f4bf4102570 2024-09-12T01:45:32.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.131+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4be4018a50 con 0x7f4bf4102570 2024-09-12T01:45:32.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.132+0000 7f4beb7fe640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4be00a2c10 0x7f4be00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:32.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.132+0000 7f4bfa365640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4be00a2c10 0x7f4be00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.132+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4be40cf5b0 con 0x7f4bf4102570 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.132+0000 7f4bfa365640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4be00a2c10 0x7f4be00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.135+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4be4014030 con 0x7f4bf4102570 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.305+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f4bf4198630 con 0x7f4bf4102570 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.305+0000 7f4beb7fe640 1 -- 172.21.15.122:0/2104995147 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4bf4198630 con 0x7f4bf4102570 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4be00a2c10 msgr2=0x7f4be00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4be00a2c10 0x7f4be00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf00058d0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 msgr2=0x7f4bf4197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 0x7f4bf4197810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c920 tx=0x7f4be400cde0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 shutdown_connections 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4be00a2c10 0x7f4be00a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bf4102ef0 0x7f4bf419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf4102570 0x7f4bf4197810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 --2- 172.21.15.122:0/2104995147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4100e20 0x7f4bf41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 >> 172.21.15.122:0/2104995147 conn(0x7f4bf40fc7d0 msgr2=0x7f4bf4106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 shutdown_connections 2024-09-12T01:45:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:29.307+0000 7f4bfc5c9640 1 -- 172.21.15.122:0/2104995147 wait complete. 2024-09-12T01:45:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.977+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd99c10a150 con 0x7fd99c102790 2024-09-12T01:45:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.977+0000 7fd998ff9640 1 -- 172.21.15.122:0/2464999245 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd9840c01a0 con 0x7fd99c102790 2024-09-12T01:45:32.981 INFO:teuthology.orchestra.run.smithi122.stdout:73014444103 2024-09-12T01:45:32.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9880a2c10 msgr2=0x7fd9880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9880a2c10 0x7fd9880a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c00b390 comp rx=0 tx=0).stop 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 msgr2=0x7fd99c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c074a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd984006010 tx=0x7fd9840042f0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 shutdown_connections 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9880a2c10 0x7fd9880a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd99c106f10 0x7fd99c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd99c102790 0x7fd99c074a30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 --2- 172.21.15.122:0/2464999245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c101de0 0x7fd99c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 >> 172.21.15.122:0/2464999245 conn(0x7fd99c0fd8c0 msgr2=0x7fd99c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.979+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 shutdown_connections 2024-09-12T01:45:32.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:32.980+0000 7fd9a13b1640 1 -- 172.21.15.122:0/2464999245 wait complete. 2024-09-12T01:45:33.651 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:33.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776157 for osd.10 2024-09-12T01:45:33.659 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:33.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:33 smithi151 ceph-mon[35229]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:33.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:33 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:33 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2464999245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T01:45:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:33 smithi183 ceph-mon[35127]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:33 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:33 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2464999245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T01:45:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:33 smithi122 ceph-mon[30925]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:33 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:33 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2464999245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T01:45:34.083 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939682 for osd.9 2024-09-12T01:45:34.083 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 -- 172.21.15.122:0/3624161989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 msgr2=0x7f6de4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 --2- 172.21.15.122:0/3624161989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de4103110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009930 tx=0x7f6de002a420 comp rx=0 tx=0).stop 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 -- 172.21.15.122:0/3624161989 shutdown_connections 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 --2- 172.21.15.122:0/3624161989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6de41082a0 0x7f6de410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 --2- 172.21.15.122:0/3624161989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 0x7f6de4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 --2- 172.21.15.122:0/3624161989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de4103110 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.709+0000 7f6debfac640 1 -- 172.21.15.122:0/3624161989 >> 172.21.15.122:0/3624161989 conn(0x7f6de40fc7d0 msgr2=0x7f6de40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.710+0000 7f6debfac640 1 -- 172.21.15.122:0/3624161989 shutdown_connections 2024-09-12T01:45:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.710+0000 7f6debfac640 1 -- 172.21.15.122:0/3624161989 wait complete. 2024-09-12T01:45:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.710+0000 7f6debfac640 1 Processor -- start 2024-09-12T01:45:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.710+0000 7f6debfac640 1 -- start start 2024-09-12T01:45:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.710+0000 7f6debfac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.710+0000 7f6debfac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 0x7f6de4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6debfac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6de41082a0 0x7f6de419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6debfac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6de41120a0 con 0x7f6de4102d30 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6debfac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4111f20 con 0x7f6de41082a0 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6debfac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4112220 con 0x7f6de41036e0 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33570/0 (socket says 172.21.15.122:33570) 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 -- 172.21.15.122:0/158224290 learned_addr learned my addr 172.21.15.122:0/158224290 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6dea549640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6de41082a0 0x7f6de419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9547640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 0x7f6de4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:34.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 -- 172.21.15.122:0/158224290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 msgr2=0x7f6de4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 0x7f6de4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 -- 172.21.15.122:0/158224290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6de41082a0 msgr2=0x7f6de419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6de41082a0 0x7f6de419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 -- 172.21.15.122:0/158224290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de419d570 con 0x7f6de4102d30 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9547640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 0x7f6de4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6de9d48640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de41985c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009230 tx=0x7f6de0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de003f030 con 0x7f6de4102d30 2024-09-12T01:45:34.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de419d800 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de41a5140 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de0003ad0 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.711+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0035070 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.712+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de4103110 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.713+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6de0042030 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.714+0000 7f6ddaffd640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd00a2c10 0x7f6dd00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.714+0000 7f6de9547640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd00a2c10 0x7f6dd00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.714+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6de00efe00 con 0x7f6de4102d30 2024-09-12T01:45:34.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.715+0000 7f6de9547640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd00a2c10 0x7f6dd00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6de40fee70 tx=0x7f6dd4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:34.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.717+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6de0049030 con 0x7f6de4102d30 2024-09-12T01:45:34.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.888+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6de410a2b0 con 0x7f6de4102d30 2024-09-12T01:45:34.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.888+0000 7f6ddaffd640 1 -- 172.21.15.122:0/158224290 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6de00bcc00 con 0x7f6de4102d30 2024-09-12T01:45:34.889 INFO:teuthology.orchestra.run.smithi122.stdout:158913790008 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.890+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd00a2c10 msgr2=0x7f6dd00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.890+0000 7f6debfac640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd00a2c10 0x7f6dd00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6de40fee70 tx=0x7f6dd4005d30 comp rx=0 tx=0).stop 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 msgr2=0x7f6de41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de41985c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009230 tx=0x7f6de0004270 comp rx=0 tx=0).stop 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 shutdown_connections 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd00a2c10 0x7f6dd00a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6de41082a0 0x7f6de419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6de41036e0 0x7f6de4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 --2- 172.21.15.122:0/158224290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6de4102d30 0x7f6de41985c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 >> 172.21.15.122:0/158224290 conn(0x7f6de40fc7d0 msgr2=0x7f6de4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 shutdown_connections 2024-09-12T01:45:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.891+0000 7f6debfac640 1 -- 172.21.15.122:0/158224290 wait complete. 2024-09-12T01:45:34.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 -- 172.21.15.122:0/1781589710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84106a80 msgr2=0x7efc84106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1781589710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84106a80 0x7efc84106e60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc74009930 tx=0x7efc7402a420 comp rx=0 tx=0).stop 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 -- 172.21.15.122:0/1781589710 shutdown_connections 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1781589710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc84101420 0x7efc8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1781589710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc84100a80 0x7efc84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1781589710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84106a80 0x7efc84106e60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 -- 172.21.15.122:0/1781589710 >> 172.21.15.122:0/1781589710 conn(0x7efc840fc7a0 msgr2=0x7efc840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 -- 172.21.15.122:0/1781589710 shutdown_connections 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 -- 172.21.15.122:0/1781589710 wait complete. 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.953+0000 7efc88b5f640 1 Processor -- start 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 -- start start 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 0x7efc841992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc84101420 0x7efc84199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc84106a80 0x7efc841a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc84110e80 con 0x7efc84100a80 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc84110d00 con 0x7efc84106a80 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc84111000 con 0x7efc84101420 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 0x7efc841992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 0x7efc841992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33596/0 (socket says 172.21.15.122:33596) 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 -- 172.21.15.122:0/1142992304 learned_addr learned my addr 172.21.15.122:0/1142992304 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 -- 172.21.15.122:0/1142992304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc84101420 msgr2=0x7efc84199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc84101420 0x7efc84199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 -- 172.21.15.122:0/1142992304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc84106a80 msgr2=0x7efc841a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc84106a80 0x7efc841a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 -- 172.21.15.122:0/1142992304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc841a48b0 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc8259c640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 0x7efc841992f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc74009a50 tx=0x7efc74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7403f030 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc74003ce0 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc841a4a80 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74035070 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.954+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc841a4f30 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.956+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc84100ee0 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.956+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efc74042030 con 0x7efc84100a80 2024-09-12T01:45:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.957+0000 7efc7b7fe640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc700a2c90 0x7efc700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.957+0000 7efc81d9b640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc700a2c90 0x7efc700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.957+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efc740ef7a0 con 0x7efc84100a80 2024-09-12T01:45:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.958+0000 7efc81d9b640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc700a2c90 0x7efc700a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc8406b2e0 tx=0x7efc6c005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:34.961+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc740bc5a0 con 0x7efc84100a80 2024-09-12T01:45:35.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.131+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efc8402d7c0 con 0x7efc84100a80 2024-09-12T01:45:35.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.131+0000 7efc7b7fe640 1 -- 172.21.15.122:0/1142992304 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc740bbea0 con 0x7efc84100a80 2024-09-12T01:45:35.131 INFO:teuthology.orchestra.run.smithi122.stdout:107374182465 2024-09-12T01:45:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc700a2c90 msgr2=0x7efc700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc700a2c90 0x7efc700a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc8406b2e0 tx=0x7efc6c005bf0 comp rx=0 tx=0).stop 2024-09-12T01:45:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 msgr2=0x7efc841992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 0x7efc841992f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc74009a50 tx=0x7efc74004270 comp rx=0 tx=0).stop 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 shutdown_connections 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc700a2c90 0x7efc700a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc84106a80 0x7efc841a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc84101420 0x7efc84199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 --2- 172.21.15.122:0/1142992304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc84100a80 0x7efc841992f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.133+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 >> 172.21.15.122:0/1142992304 conn(0x7efc840fc7a0 msgr2=0x7efc840fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.134+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 shutdown_connections 2024-09-12T01:45:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:35.134+0000 7efc88b5f640 1 -- 172.21.15.122:0/1142992304 wait complete. 2024-09-12T01:45:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:35 smithi151 ceph-mon[35229]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:35 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/158224290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T01:45:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:35 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1142992304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T01:45:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:35 smithi183 ceph-mon[35127]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:35.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:35 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/158224290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T01:45:35.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:35 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1142992304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T01:45:35.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:35 smithi122 ceph-mon[30925]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:35 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/158224290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T01:45:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:35 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1142992304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T01:45:36.038 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444100 got 73014444103 for osd.2 2024-09-12T01:45:36.039 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:36.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 -- 172.21.15.122:0/1528367610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50101de0 msgr2=0x7efc501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:36.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 --2- 172.21.15.122:0/1528367610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50101de0 0x7efc501021c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efc40009900 tx=0x7efc4002a410 comp rx=0 tx=0).stop 2024-09-12T01:45:36.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 -- 172.21.15.122:0/1528367610 shutdown_connections 2024-09-12T01:45:36.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 --2- 172.21.15.122:0/1528367610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50106f10 0x7efc5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 --2- 172.21.15.122:0/1528367610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50102790 0x7efc501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 --2- 172.21.15.122:0/1528367610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50101de0 0x7efc501021c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 -- 172.21.15.122:0/1528367610 >> 172.21.15.122:0/1528367610 conn(0x7efc500fd8c0 msgr2=0x7efc500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 -- 172.21.15.122:0/1528367610 shutdown_connections 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.335+0000 7efc56109640 1 -- 172.21.15.122:0/1528367610 wait complete. 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 Processor -- start 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 -- start start 2024-09-12T01:45:36.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50101de0 0x7efc501985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50102790 0x7efc50198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 0x7efc5019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc50112010 con 0x7efc50106f10 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc50111e90 con 0x7efc50102790 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc56109640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc50112190 con 0x7efc50101de0 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc4ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 0x7efc5019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:36.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc4ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 0x7efc5019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33620/0 (socket says 172.21.15.122:33620) 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.336+0000 7efc4ffff640 1 -- 172.21.15.122:0/3406027160 learned_addr learned my addr 172.21.15.122:0/3406027160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4effd640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50102790 0x7efc50198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4f7fe640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50101de0 0x7efc501985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4ffff640 1 -- 172.21.15.122:0/3406027160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50101de0 msgr2=0x7efc501985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4ffff640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50101de0 0x7efc501985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4ffff640 1 -- 172.21.15.122:0/3406027160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50102790 msgr2=0x7efc50198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:36.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4ffff640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50102790 0x7efc50198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4ffff640 1 -- 172.21.15.122:0/3406027160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5019d560 con 0x7efc50106f10 2024-09-12T01:45:36.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4ffff640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 0x7efc5019ce80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc4800b970 tx=0x7efc4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:36.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48019070 con 0x7efc50106f10 2024-09-12T01:45:36.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5019d850 con 0x7efc50106f10 2024-09-12T01:45:36.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc48014440 con 0x7efc50106f10 2024-09-12T01:45:36.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480075d0 con 0x7efc50106f10 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.337+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a5130 con 0x7efc50106f10 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.338+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50102240 con 0x7efc50106f10 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.339+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efc4801c070 con 0x7efc50106f10 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.340+0000 7efc4cff9640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc380a2c10 0x7efc380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.340+0000 7efc4f7fe640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc380a2c10 0x7efc380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.340+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efc480ceb10 con 0x7efc50106f10 2024-09-12T01:45:36.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.340+0000 7efc4f7fe640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc380a2c10 0x7efc380a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc400096c0 tx=0x7efc40000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:36.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.343+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc4809ba20 con 0x7efc50106f10 2024-09-12T01:45:36.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.514+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7efc50109b60 con 0x7efc50106f10 2024-09-12T01:45:36.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.514+0000 7efc4cff9640 1 -- 172.21.15.122:0/3406027160 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc4809b320 con 0x7efc50106f10 2024-09-12T01:45:36.515 INFO:teuthology.orchestra.run.smithi122.stdout:240518168615 2024-09-12T01:45:36.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc380a2c10 msgr2=0x7efc380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:36.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc380a2c10 0x7efc380a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc400096c0 tx=0x7efc40000bc0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 msgr2=0x7efc5019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 0x7efc5019ce80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc4800b970 tx=0x7efc4800bd70 comp rx=0 tx=0).stop 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 shutdown_connections 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc380a2c10 0x7efc380a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc50106f10 0x7efc5019ce80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50102790 0x7efc50198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 --2- 172.21.15.122:0/3406027160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50101de0 0x7efc501985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 >> 172.21.15.122:0/3406027160 conn(0x7efc500fd8c0 msgr2=0x7efc500fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:36.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 shutdown_connections 2024-09-12T01:45:36.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:36.516+0000 7efc56109640 1 -- 172.21.15.122:0/3406027160 wait complete. 2024-09-12T01:45:36.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:36 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3406027160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T01:45:36.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:36 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3406027160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T01:45:36.973 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:36 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3406027160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T01:45:36.999 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182465 for osd.3 2024-09-12T01:45:36.999 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:37.395 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790008 for osd.5 2024-09-12T01:45:37.396 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:37.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:37 smithi122 ceph-mon[30925]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:37.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:37 smithi151 ceph-mon[35229]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:37 smithi183 ceph-mon[35127]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:38.081 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168615 for osd.8 2024-09-12T01:45:38.081 DEBUG:teuthology.parallel:result is None 2024-09-12T01:45:38.081 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T01:45:38.081 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:45:38.981 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:39.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.438+0000 7f6397fcb640 1 -- 172.21.15.122:0/2507451567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390106f10 msgr2=0x7f639010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.438+0000 7f6397fcb640 1 --2- 172.21.15.122:0/2507451567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390106f10 0x7f639010ea90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6388009ca0 tx=0x7f638802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.438+0000 7f6397fcb640 1 -- 172.21.15.122:0/2507451567 shutdown_connections 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.438+0000 7f6397fcb640 1 --2- 172.21.15.122:0/2507451567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390106f10 0x7f639010ea90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 --2- 172.21.15.122:0/2507451567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6390102790 0x7f63901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 --2- 172.21.15.122:0/2507451567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6390101de0 0x7f63901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- 172.21.15.122:0/2507451567 >> 172.21.15.122:0/2507451567 conn(0x7f63900fd8c0 msgr2=0x7f63900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- 172.21.15.122:0/2507451567 shutdown_connections 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- 172.21.15.122:0/2507451567 wait complete. 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 Processor -- start 2024-09-12T01:45:39.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- start start 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6390101de0 0x7f6390198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 0x7f6390198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6390106f10 0x7f639019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6390111f90 con 0x7f6390102790 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6390111e10 con 0x7f6390101de0 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.439+0000 7f6397fcb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6390112110 con 0x7f6390106f10 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 0x7f6390198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 0x7f6390198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33640/0 (socket says 172.21.15.122:33640) 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 -- 172.21.15.122:0/1321610245 learned_addr learned my addr 172.21.15.122:0/1321610245 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6396568640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6390106f10 0x7f639019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395d67640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6390101de0 0x7f6390198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 -- 172.21.15.122:0/1321610245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6390106f10 msgr2=0x7f639019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6390106f10 0x7f639019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 -- 172.21.15.122:0/1321610245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6390101de0 msgr2=0x7f6390198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6390101de0 0x7f6390198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 -- 172.21.15.122:0/1321610245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639019d500 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6395566640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 0x7f6390198a90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f638000b4a0 tx=0x7f638000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380014070 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6380010af0 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639019d7f0 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380017d10 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.440+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63901a50d0 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.441+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63901021c0 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.442+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6380017070 con 0x7f6390102790 2024-09-12T01:45:39.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.443+0000 7f6386ffd640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f637c0a2c10 0x7f637c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:39.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.443+0000 7f6395d67640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f637c0a2c10 0x7f637c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:39.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.443+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f63800ce3e0 con 0x7f6390102790 2024-09-12T01:45:39.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.444+0000 7f6395d67640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f637c0a2c10 0x7f637c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f638c0058c0 tx=0x7f638c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:39.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.448+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f638009aca0 con 0x7f6390102790 2024-09-12T01:45:39.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.644+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f639010a180 con 0x7f637c0a2c10 2024-09-12T01:45:39.644 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:39 smithi122 ceph-mon[30925]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:39.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.651+0000 7f6386ffd640 1 -- 172.21.15.122:0/1321610245 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258950 (secure 0 0 0) 0x7f639010a180 con 0x7f637c0a2c10 2024-09-12T01:45:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:39.655 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:45:39.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.654+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f637c0a2c10 msgr2=0x7f637c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:39.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.654+0000 7f6397fcb640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f637c0a2c10 0x7f637c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f638c0058c0 tx=0x7f638c00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.654+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 msgr2=0x7f6390198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:39.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.654+0000 7f6397fcb640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 0x7f6390198a90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f638000b4a0 tx=0x7f638000b960 comp rx=0 tx=0).stop 2024-09-12T01:45:39.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.654+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 shutdown_connections 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.654+0000 7f6397fcb640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f637c0a2c10 0x7f637c0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.655+0000 7f6397fcb640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6390106f10 0x7f639019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.655+0000 7f6397fcb640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6390102790 0x7f6390198a90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.655+0000 7f6397fcb640 1 --2- 172.21.15.122:0/1321610245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6390101de0 0x7f6390198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.655+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 >> 172.21.15.122:0/1321610245 conn(0x7f63900fd8c0 msgr2=0x7f63900fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.655+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 shutdown_connections 2024-09-12T01:45:39.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:39.655+0000 7f6397fcb640 1 -- 172.21.15.122:0/1321610245 wait complete. 2024-09-12T01:45:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:39 smithi151 ceph-mon[35229]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:39 smithi183 ceph-mon[35127]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:40.379 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-12T01:45:38.459672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656357,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":445,"num_read_kb":337,"num_write":345,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360296,"kb_used_data":11040,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581528,"statfs":{"total":1151940427776,"available":1151571484672,"internally_reserved":0,"allocated":11304960,"data_stored":7215627,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002733"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818499+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:44:45.818499+0000","last_peered":"2024-09-12T01:44:45.818499+0000","last_clean":"2024-09-12T01:44:45.818499+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:44:45.818499+0000","last_undegraded":"2024-09-12T01:44:45.818499+0000","last_fullsized":"2024-09-12T01:44:45.818499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076814+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:44:46.076814+0000","last_peered":"2024-09-12T01:44:46.076814+0000","last_clean":"2024-09-12T01:44:46.076814+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:44:46.076814+0000","last_undegraded":"2024-09-12T01:44:46.076814+0000","last_fullsized":"2024-09-12T01:44:46.076814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076808+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:44:46.076808+0000","last_peered":"2024-09-12T01:44:46.076808+0000","last_clean":"2024-09-12T01:44:46.076808+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:44:46.076808+0000","last_undegraded":"2024-09-12T01:44:46.076808+0000","last_fullsized":"2024-09-12T01:44:46.076808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815947+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:44:45.815947+0000","last_peered":"2024-09-12T01:44:45.815947+0000","last_clean":"2024-09-12T01:44:45.815947+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:44:45.815947+0000","last_undegraded":"2024-09-12T01:44:45.815947+0000","last_fullsized":"2024-09-12T01:44:45.815947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819617+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:44:45.819617+0000","last_peered":"2024-09-12T01:44:45.819617+0000","last_clean":"2024-09-12T01:44:45.819617+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:44:45.819617+0000","last_undegraded":"2024-09-12T01:44:45.819617+0000","last_fullsized":"2024-09-12T01:44:45.819617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818302+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:44:45.818302+0000","last_peered":"2024-09-12T01:44:45.818302+0000","last_clean":"2024-09-12T01:44:45.818302+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:44:45.818302+0000","last_undegraded":"2024-09-12T01:44:45.818302+0000","last_fullsized":"2024-09-12T01:44:45.818302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077967+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:44:46.077967+0000","last_peered":"2024-09-12T01:44:46.077967+0000","last_clean":"2024-09-12T01:44:46.077967+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:44:46.077967+0000","last_undegraded":"2024-09-12T01:44:46.077967+0000","last_fullsized":"2024-09-12T01:44:46.077967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821645+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:44:45.821645+0000","last_peered":"2024-09-12T01:44:45.821645+0000","last_clean":"2024-09-12T01:44:45.821645+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:44:45.821645+0000","last_undegraded":"2024-09-12T01:44:45.821645+0000","last_fullsized":"2024-09-12T01:44:45.821645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821329+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:44:45.821329+0000","last_peered":"2024-09-12T01:44:45.821329+0000","last_clean":"2024-09-12T01:44:45.821329+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:44:45.821329+0000","last_undegraded":"2024-09-12T01:44:45.821329+0000","last_fullsized":"2024-09-12T01:44:45.821329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824934+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:44:45.824934+0000","last_peered":"2024-09-12T01:44:45.824934+0000","last_clean":"2024-09-12T01:44:45.824934+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:44:45.824934+0000","last_undegraded":"2024-09-12T01:44:45.824934+0000","last_fullsized":"2024-09-12T01:44:45.824934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818038+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:44:45.818038+0000","last_peered":"2024-09-12T01:44:45.818038+0000","last_clean":"2024-09-12T01:44:45.818038+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:44:45.818038+0000","last_undegraded":"2024-09-12T01:44:45.818038+0000","last_fullsized":"2024-09-12T01:44:45.818038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822055+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:44:45.822055+0000","last_peered":"2024-09-12T01:44:45.822055+0000","last_clean":"2024-09-12T01:44:45.822055+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:44:45.822055+0000","last_undegraded":"2024-09-12T01:44:45.822055+0000","last_fullsized":"2024-09-12T01:44:45.822055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077339+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:44:46.077339+0000","last_peered":"2024-09-12T01:44:46.077339+0000","last_clean":"2024-09-12T01:44:46.077339+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:44:46.077339+0000","last_undegraded":"2024-09-12T01:44:46.077339+0000","last_fullsized":"2024-09-12T01:44:46.077339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076657+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:44:46.076657+0000","last_peered":"2024-09-12T01:44:46.076657+0000","last_clean":"2024-09-12T01:44:46.076657+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:44:46.076657+0000","last_undegraded":"2024-09-12T01:44:46.076657+0000","last_fullsized":"2024-09-12T01:44:46.076657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076627+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:44:46.076627+0000","last_peered":"2024-09-12T01:44:46.076627+0000","last_clean":"2024-09-12T01:44:46.076627+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:44:46.076627+0000","last_undegraded":"2024-09-12T01:44:46.076627+0000","last_fullsized":"2024-09-12T01:44:46.076627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820478+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:44:45.820478+0000","last_peered":"2024-09-12T01:44:45.820478+0000","last_clean":"2024-09-12T01:44:45.820478+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:44:45.820478+0000","last_undegraded":"2024-09-12T01:44:45.820478+0000","last_fullsized":"2024-09-12T01:44:45.820478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818598+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:44:45.818598+0000","last_peered":"2024-09-12T01:44:45.818598+0000","last_clean":"2024-09-12T01:44:45.818598+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:44:45.818598+0000","last_undegraded":"2024-09-12T01:44:45.818598+0000","last_fullsized":"2024-09-12T01:44:45.818598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816391+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:44:45.816391+0000","last_peered":"2024-09-12T01:44:45.816391+0000","last_clean":"2024-09-12T01:44:45.816391+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:44:45.816391+0000","last_undegraded":"2024-09-12T01:44:45.816391+0000","last_fullsized":"2024-09-12T01:44:45.816391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821743+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:44:45.821743+0000","last_peered":"2024-09-12T01:44:45.821743+0000","last_clean":"2024-09-12T01:44:45.821743+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:44:45.821743+0000","last_undegraded":"2024-09-12T01:44:45.821743+0000","last_fullsized":"2024-09-12T01:44:45.821743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818574+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:44:45.818574+0000","last_peered":"2024-09-12T01:44:45.818574+0000","last_clean":"2024-09-12T01:44:45.818574+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:44:45.818574+0000","last_undegraded":"2024-09-12T01:44:45.818574+0000","last_fullsized":"2024-09-12T01:44:45.818574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077436+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:44:46.077436+0000","last_peered":"2024-09-12T01:44:46.077436+0000","last_clean":"2024-09-12T01:44:46.077436+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:44:46.077436+0000","last_undegraded":"2024-09-12T01:44:46.077436+0000","last_fullsized":"2024-09-12T01:44:46.077436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820692+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:44:45.820692+0000","last_peered":"2024-09-12T01:44:45.820692+0000","last_clean":"2024-09-12T01:44:45.820692+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:44:45.820692+0000","last_undegraded":"2024-09-12T01:44:45.820692+0000","last_fullsized":"2024-09-12T01:44:45.820692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821444+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:44:45.821444+0000","last_peered":"2024-09-12T01:44:45.821444+0000","last_clean":"2024-09-12T01:44:45.821444+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:44:45.821444+0000","last_undegraded":"2024-09-12T01:44:45.821444+0000","last_fullsized":"2024-09-12T01:44:45.821444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824121+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:44:45.824121+0000","last_peered":"2024-09-12T01:44:45.824121+0000","last_clean":"2024-09-12T01:44:45.824121+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:44:45.824121+0000","last_undegraded":"2024-09-12T01:44:45.824121+0000","last_fullsized":"2024-09-12T01:44:45.824121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821120+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:44:45.821120+0000","last_peered":"2024-09-12T01:44:45.821120+0000","last_clean":"2024-09-12T01:44:45.821120+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:44:45.821120+0000","last_undegraded":"2024-09-12T01:44:45.821120+0000","last_fullsized":"2024-09-12T01:44:45.821120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822863+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:44:45.822863+0000","last_peered":"2024-09-12T01:44:45.822863+0000","last_clean":"2024-09-12T01:44:45.822863+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:44:45.822863+0000","last_undegraded":"2024-09-12T01:44:45.822863+0000","last_fullsized":"2024-09-12T01:44:45.822863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817853+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:44:45.817853+0000","last_peered":"2024-09-12T01:44:45.817853+0000","last_clean":"2024-09-12T01:44:45.817853+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:44:45.817853+0000","last_undegraded":"2024-09-12T01:44:45.817853+0000","last_fullsized":"2024-09-12T01:44:45.817853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077201+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:44:46.077201+0000","last_peered":"2024-09-12T01:44:46.077201+0000","last_clean":"2024-09-12T01:44:46.077201+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:44:46.077201+0000","last_undegraded":"2024-09-12T01:44:46.077201+0000","last_fullsized":"2024-09-12T01:44:46.077201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076827+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:44:46.076827+0000","last_peered":"2024-09-12T01:44:46.076827+0000","last_clean":"2024-09-12T01:44:46.076827+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:44:46.076827+0000","last_undegraded":"2024-09-12T01:44:46.076827+0000","last_fullsized":"2024-09-12T01:44:46.076827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820905+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:44:45.820905+0000","last_peered":"2024-09-12T01:44:45.820905+0000","last_clean":"2024-09-12T01:44:45.820905+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:44:45.820905+0000","last_undegraded":"2024-09-12T01:44:45.820905+0000","last_fullsized":"2024-09-12T01:44:45.820905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821482+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:44:45.821482+0000","last_peered":"2024-09-12T01:44:45.821482+0000","last_clean":"2024-09-12T01:44:45.821482+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:44:45.821482+0000","last_undegraded":"2024-09-12T01:44:45.821482+0000","last_fullsized":"2024-09-12T01:44:45.821482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817474+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:44:45.817474+0000","last_peered":"2024-09-12T01:44:45.817474+0000","last_clean":"2024-09-12T01:44:45.817474+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:44:45.817474+0000","last_undegraded":"2024-09-12T01:44:45.817474+0000","last_fullsized":"2024-09-12T01:44:45.817474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819263+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:44:45.819263+0000","last_peered":"2024-09-12T01:44:45.819263+0000","last_clean":"2024-09-12T01:44:45.819263+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:44:45.819263+0000","last_undegraded":"2024-09-12T01:44:45.819263+0000","last_fullsized":"2024-09-12T01:44:45.819263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:31.904961+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:45:31.904961+0000","last_peered":"2024-09-12T01:45:31.904961+0000","last_clean":"2024-09-12T01:45:31.904961+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:45:31.904961+0000","last_undegraded":"2024-09-12T01:45:31.904961+0000","last_fullsized":"2024-09-12T01:45:31.904961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815476+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:44:45.815476+0000","last_peered":"2024-09-12T01:44:45.815476+0000","last_clean":"2024-09-12T01:44:45.815476+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:44:45.815476+0000","last_undegraded":"2024-09-12T01:44:45.815476+0000","last_fullsized":"2024-09-12T01:44:45.815476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821664+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:44:45.821664+0000","last_peered":"2024-09-12T01:44:45.821664+0000","last_clean":"2024-09-12T01:44:45.821664+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:44:45.821664+0000","last_undegraded":"2024-09-12T01:44:45.821664+0000","last_fullsized":"2024-09-12T01:44:45.821664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816283+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:44:45.816283+0000","last_peered":"2024-09-12T01:44:45.816283+0000","last_clean":"2024-09-12T01:44:45.816283+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:44:45.816283+0000","last_undegraded":"2024-09-12T01:44:45.816283+0000","last_fullsized":"2024-09-12T01:44:45.816283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:33.906863+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:45:33.906863+0000","last_peered":"2024-09-12T01:45:33.906863+0000","last_clean":"2024-09-12T01:45:33.906863+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:45:33.906863+0000","last_undegraded":"2024-09-12T01:45:33.906863+0000","last_fullsized":"2024-09-12T01:45:33.906863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821400+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:44:45.821400+0000","last_peered":"2024-09-12T01:44:45.821400+0000","last_clean":"2024-09-12T01:44:45.821400+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:44:45.821400+0000","last_undegraded":"2024-09-12T01:44:45.821400+0000","last_fullsized":"2024-09-12T01:44:45.821400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077017+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:44:46.077017+0000","last_peered":"2024-09-12T01:44:46.077017+0000","last_clean":"2024-09-12T01:44:46.077017+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:44:46.077017+0000","last_undegraded":"2024-09-12T01:44:46.077017+0000","last_fullsized":"2024-09-12T01:44:46.077017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820850+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:44:45.820850+0000","last_peered":"2024-09-12T01:44:45.820850+0000","last_clean":"2024-09-12T01:44:45.820850+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:44:45.820850+0000","last_undegraded":"2024-09-12T01:44:45.820850+0000","last_fullsized":"2024-09-12T01:44:45.820850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":157,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:34.908189+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:45:34.908189+0000","last_peered":"2024-09-12T01:45:34.908189+0000","last_clean":"2024-09-12T01:45:34.908189+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:45:34.908189+0000","last_undegraded":"2024-09-12T01:45:34.908189+0000","last_fullsized":"2024-09-12T01:45:34.908189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":106,"num_read_kb":113,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820311+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:44:45.820311+0000","last_peered":"2024-09-12T01:44:45.820311+0000","last_clean":"2024-09-12T01:44:45.820311+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:44:45.820311+0000","last_undegraded":"2024-09-12T01:44:45.820311+0000","last_fullsized":"2024-09-12T01:44:45.820311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077080+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:44:46.077080+0000","last_peered":"2024-09-12T01:44:46.077080+0000","last_clean":"2024-09-12T01:44:46.077080+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:44:46.077080+0000","last_undegraded":"2024-09-12T01:44:46.077080+0000","last_fullsized":"2024-09-12T01:44:46.077080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817662+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:44:45.817662+0000","last_peered":"2024-09-12T01:44:45.817662+0000","last_clean":"2024-09-12T01:44:45.817662+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:44:45.817662+0000","last_undegraded":"2024-09-12T01:44:45.817662+0000","last_fullsized":"2024-09-12T01:44:45.817662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:35.909837+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:45:35.909837+0000","last_peered":"2024-09-12T01:45:35.909837+0000","last_clean":"2024-09-12T01:45:35.909837+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:45:35.909837+0000","last_undegraded":"2024-09-12T01:45:35.909837+0000","last_fullsized":"2024-09-12T01:45:35.909837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817679+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:44:45.817679+0000","last_peered":"2024-09-12T01:44:45.817679+0000","last_clean":"2024-09-12T01:44:45.817679+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:44:45.817679+0000","last_undegraded":"2024-09-12T01:44:45.817679+0000","last_fullsized":"2024-09-12T01:44:45.817679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822535+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:44:45.822535+0000","last_peered":"2024-09-12T01:44:45.822535+0000","last_clean":"2024-09-12T01:44:45.822535+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:44:45.822535+0000","last_undegraded":"2024-09-12T01:44:45.822535+0000","last_fullsized":"2024-09-12T01:44:45.822535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818060+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:44:45.818060+0000","last_peered":"2024-09-12T01:44:45.818060+0000","last_clean":"2024-09-12T01:44:45.818060+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:44:45.818060+0000","last_undegraded":"2024-09-12T01:44:45.818060+0000","last_fullsized":"2024-09-12T01:44:45.818060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818393+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:44:45.818393+0000","last_peered":"2024-09-12T01:44:45.818393+0000","last_clean":"2024-09-12T01:44:45.818393+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:44:45.818393+0000","last_undegraded":"2024-09-12T01:44:45.818393+0000","last_fullsized":"2024-09-12T01:44:45.818393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820335+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:44:45.820335+0000","last_peered":"2024-09-12T01:44:45.820335+0000","last_clean":"2024-09-12T01:44:45.820335+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:44:45.820335+0000","last_undegraded":"2024-09-12T01:44:45.820335+0000","last_fullsized":"2024-09-12T01:44:45.820335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818057+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:44:45.818057+0000","last_peered":"2024-09-12T01:44:45.818057+0000","last_clean":"2024-09-12T01:44:45.818057+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:44:45.818057+0000","last_undegraded":"2024-09-12T01:44:45.818057+0000","last_fullsized":"2024-09-12T01:44:45.818057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818530+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818530+0000","last_peered":"2024-09-12T01:44:45.818530+0000","last_clean":"2024-09-12T01:44:45.818530+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818530+0000","last_undegraded":"2024-09-12T01:44:45.818530+0000","last_fullsized":"2024-09-12T01:44:45.818530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821278+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:44:45.821278+0000","last_peered":"2024-09-12T01:44:45.821278+0000","last_clean":"2024-09-12T01:44:45.821278+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:44:45.821278+0000","last_undegraded":"2024-09-12T01:44:45.821278+0000","last_fullsized":"2024-09-12T01:44:45.821278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077069+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:44:46.077069+0000","last_peered":"2024-09-12T01:44:46.077069+0000","last_clean":"2024-09-12T01:44:46.077069+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:44:46.077069+0000","last_undegraded":"2024-09-12T01:44:46.077069+0000","last_fullsized":"2024-09-12T01:44:46.077069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:31.904674+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:45:31.904674+0000","last_peered":"2024-09-12T01:45:31.904674+0000","last_clean":"2024-09-12T01:45:31.904674+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:45:31.904674+0000","last_undegraded":"2024-09-12T01:45:31.904674+0000","last_fullsized":"2024-09-12T01:45:31.904674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824468+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:44:45.824468+0000","last_peered":"2024-09-12T01:44:45.824468+0000","last_clean":"2024-09-12T01:44:45.824468+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:44:45.824468+0000","last_undegraded":"2024-09-12T01:44:45.824468+0000","last_fullsized":"2024-09-12T01:44:45.824468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818919+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:44:45.818919+0000","last_peered":"2024-09-12T01:44:45.818919+0000","last_clean":"2024-09-12T01:44:45.818919+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:44:45.818919+0000","last_undegraded":"2024-09-12T01:44:45.818919+0000","last_fullsized":"2024-09-12T01:44:45.818919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819406+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:44:45.819406+0000","last_peered":"2024-09-12T01:44:45.819406+0000","last_clean":"2024-09-12T01:44:45.819406+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:44:45.819406+0000","last_undegraded":"2024-09-12T01:44:45.819406+0000","last_fullsized":"2024-09-12T01:44:45.819406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077325+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:44:46.077325+0000","last_peered":"2024-09-12T01:44:46.077325+0000","last_clean":"2024-09-12T01:44:46.077325+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:44:46.077325+0000","last_undegraded":"2024-09-12T01:44:46.077325+0000","last_fullsized":"2024-09-12T01:44:46.077325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'10","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:00.781231+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:45:00.781231+0000","last_peered":"2024-09-12T01:45:00.781231+0000","last_clean":"2024-09-12T01:45:00.781231+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:45:00.781231+0000","last_undegraded":"2024-09-12T01:45:00.781231+0000","last_fullsized":"2024-09-12T01:45:00.781231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821213+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:44:45.821213+0000","last_peered":"2024-09-12T01:44:45.821213+0000","last_clean":"2024-09-12T01:44:45.821213+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:44:45.821213+0000","last_undegraded":"2024-09-12T01:44:45.821213+0000","last_fullsized":"2024-09-12T01:44:45.821213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076938+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:44:46.076938+0000","last_peered":"2024-09-12T01:44:46.076938+0000","last_clean":"2024-09-12T01:44:46.076938+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:44:46.076938+0000","last_undegraded":"2024-09-12T01:44:46.076938+0000","last_fullsized":"2024-09-12T01:44:46.076938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819237+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:44:45.819237+0000","last_peered":"2024-09-12T01:44:45.819237+0000","last_clean":"2024-09-12T01:44:45.819237+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:44:45.819237+0000","last_undegraded":"2024-09-12T01:44:45.819237+0000","last_fullsized":"2024-09-12T01:44:45.819237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817496+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:44:45.817496+0000","last_peered":"2024-09-12T01:44:45.817496+0000","last_clean":"2024-09-12T01:44:45.817496+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:44:45.817496+0000","last_undegraded":"2024-09-12T01:44:45.817496+0000","last_fullsized":"2024-09-12T01:44:45.817496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821064+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:44:45.821064+0000","last_peered":"2024-09-12T01:44:45.821064+0000","last_clean":"2024-09-12T01:44:45.821064+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:44:45.821064+0000","last_undegraded":"2024-09-12T01:44:45.821064+0000","last_fullsized":"2024-09-12T01:44:45.821064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818032+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:44:45.818032+0000","last_peered":"2024-09-12T01:44:45.818032+0000","last_clean":"2024-09-12T01:44:45.818032+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:44:45.818032+0000","last_undegraded":"2024-09-12T01:44:45.818032+0000","last_fullsized":"2024-09-12T01:44:45.818032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816211+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:44:45.816211+0000","last_peered":"2024-09-12T01:44:45.816211+0000","last_clean":"2024-09-12T01:44:45.816211+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:44:45.816211+0000","last_undegraded":"2024-09-12T01:44:45.816211+0000","last_fullsized":"2024-09-12T01:44:45.816211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820739+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:44:45.820739+0000","last_peered":"2024-09-12T01:44:45.820739+0000","last_clean":"2024-09-12T01:44:45.820739+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:44:45.820739+0000","last_undegraded":"2024-09-12T01:44:45.820739+0000","last_fullsized":"2024-09-12T01:44:45.820739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076981+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:44:46.076981+0000","last_peered":"2024-09-12T01:44:46.076981+0000","last_clean":"2024-09-12T01:44:46.076981+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:44:46.076981+0000","last_undegraded":"2024-09-12T01:44:46.076981+0000","last_fullsized":"2024-09-12T01:44:46.076981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820929+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:44:45.820929+0000","last_peered":"2024-09-12T01:44:45.820929+0000","last_clean":"2024-09-12T01:44:45.820929+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:44:45.820929+0000","last_undegraded":"2024-09-12T01:44:45.820929+0000","last_fullsized":"2024-09-12T01:44:45.820929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65755,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":319,"num_read_kb":228,"num_write":162,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1091421,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029883,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438250,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":500281,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":498965,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:45:40.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:45:40.638 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:45:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:45:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:40 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:45:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:40 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:41.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:45:41.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:45:41.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:40 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:45:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.140+0000 7fa1947be640 1 -- 172.21.15.122:0/3983186679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c102ef0 msgr2=0x7fa18c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.140+0000 7fa1947be640 1 --2- 172.21.15.122:0/3983186679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c102ef0 0x7fa18c10da40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa184009ca0 tx=0x7fa18402a950 comp rx=0 tx=0).stop 2024-09-12T01:45:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- 172.21.15.122:0/3983186679 shutdown_connections 2024-09-12T01:45:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 --2- 172.21.15.122:0/3983186679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c102ef0 0x7fa18c10da40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 --2- 172.21.15.122:0/3983186679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102570 0x7fa18c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 --2- 172.21.15.122:0/3983186679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa18c100e20 0x7fa18c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- 172.21.15.122:0/3983186679 >> 172.21.15.122:0/3983186679 conn(0x7fa18c0fc7d0 msgr2=0x7fa18c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- 172.21.15.122:0/3983186679 shutdown_connections 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- 172.21.15.122:0/3983186679 wait complete. 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 Processor -- start 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- start start 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 0x7fa18c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa18c102570 0x7fa18c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102ef0 0x7fa18c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c110fc0 con 0x7fa18c100e20 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c110e40 con 0x7fa18c102ef0 2024-09-12T01:45:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.141+0000 7fa1947be640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c111140 con 0x7fa18c102570 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 0x7fa18c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 0x7fa18c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50696/0 (socket says 172.21.15.122:50696) 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 -- 172.21.15.122:0/1892997234 learned_addr learned my addr 172.21.15.122:0/1892997234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa191d59640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa18c102570 0x7fa18c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa192d5b640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102ef0 0x7fa18c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 -- 172.21.15.122:0/1892997234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa18c102570 msgr2=0x7fa18c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa18c102570 0x7fa18c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 -- 172.21.15.122:0/1892997234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102ef0 msgr2=0x7fa18c1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102ef0 0x7fa18c1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 -- 172.21.15.122:0/1892997234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c1a4930 con 0x7fa18c100e20 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa192d5b640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102ef0 0x7fa18c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:41.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa19255a640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 0x7fa18c199370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d770 tx=0x7fa17c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c017070 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c009820 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004cd0 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c1a4b00 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.142+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c1a4fc0 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.143+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c101200 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.145+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa17c0099d0 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.145+0000 7fa1837fe640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa1780a2c10 0x7fa1780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.145+0000 7fa191d59640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa1780a2c10 0x7fa1780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.145+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa17c0ceff0 con 0x7fa18c100e20 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.146+0000 7fa191d59640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa1780a2c10 0x7fa1780a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa18c06aac0 tx=0x7fa188009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:41.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.148+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa17c09bdf0 con 0x7fa18c100e20 2024-09-12T01:45:41.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.312+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa18c106f50 con 0x7fa1780a2c10 2024-09-12T01:45:41.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.318+0000 7fa1837fe640 1 -- 172.21.15.122:0/1892997234 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258950 (secure 0 0 0) 0x7fa18c106f50 con 0x7fa1780a2c10 2024-09-12T01:45:41.325 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:41.326 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:45:41.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa1780a2c10 msgr2=0x7fa1780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:41.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa1780a2c10 0x7fa1780a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa18c06aac0 tx=0x7fa188009110 comp rx=0 tx=0).stop 2024-09-12T01:45:41.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 msgr2=0x7fa18c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:41.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 0x7fa18c199370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d770 tx=0x7fa17c00db70 comp rx=0 tx=0).stop 2024-09-12T01:45:41.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 shutdown_connections 2024-09-12T01:45:41.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa1780a2c10 0x7fa1780a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa18c102ef0 0x7fa18c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa18c102570 0x7fa18c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 --2- 172.21.15.122:0/1892997234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18c100e20 0x7fa18c199370 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:41.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 >> 172.21.15.122:0/1892997234 conn(0x7fa18c0fc7d0 msgr2=0x7fa18c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:41.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 shutdown_connections 2024-09-12T01:45:41.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:41.321+0000 7fa1947be640 1 -- 172.21.15.122:0/1892997234 wait complete. 2024-09-12T01:45:41.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:41 smithi122 ceph-mon[30925]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:42.008 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-12T01:45:40.460234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656357,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":455,"num_read_kb":342,"num_write":345,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360296,"kb_used_data":11040,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581528,"statfs":{"total":1151940427776,"available":1151571484672,"internally_reserved":0,"allocated":11304960,"data_stored":7215627,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818499+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:44:45.818499+0000","last_peered":"2024-09-12T01:44:45.818499+0000","last_clean":"2024-09-12T01:44:45.818499+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:44:45.818499+0000","last_undegraded":"2024-09-12T01:44:45.818499+0000","last_fullsized":"2024-09-12T01:44:45.818499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076814+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:44:46.076814+0000","last_peered":"2024-09-12T01:44:46.076814+0000","last_clean":"2024-09-12T01:44:46.076814+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:44:46.076814+0000","last_undegraded":"2024-09-12T01:44:46.076814+0000","last_fullsized":"2024-09-12T01:44:46.076814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076808+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:44:46.076808+0000","last_peered":"2024-09-12T01:44:46.076808+0000","last_clean":"2024-09-12T01:44:46.076808+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:44:46.076808+0000","last_undegraded":"2024-09-12T01:44:46.076808+0000","last_fullsized":"2024-09-12T01:44:46.076808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815947+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:44:45.815947+0000","last_peered":"2024-09-12T01:44:45.815947+0000","last_clean":"2024-09-12T01:44:45.815947+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:44:45.815947+0000","last_undegraded":"2024-09-12T01:44:45.815947+0000","last_fullsized":"2024-09-12T01:44:45.815947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819617+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:44:45.819617+0000","last_peered":"2024-09-12T01:44:45.819617+0000","last_clean":"2024-09-12T01:44:45.819617+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:44:45.819617+0000","last_undegraded":"2024-09-12T01:44:45.819617+0000","last_fullsized":"2024-09-12T01:44:45.819617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818302+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:44:45.818302+0000","last_peered":"2024-09-12T01:44:45.818302+0000","last_clean":"2024-09-12T01:44:45.818302+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:44:45.818302+0000","last_undegraded":"2024-09-12T01:44:45.818302+0000","last_fullsized":"2024-09-12T01:44:45.818302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077967+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:44:46.077967+0000","last_peered":"2024-09-12T01:44:46.077967+0000","last_clean":"2024-09-12T01:44:46.077967+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:44:46.077967+0000","last_undegraded":"2024-09-12T01:44:46.077967+0000","last_fullsized":"2024-09-12T01:44:46.077967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821645+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:44:45.821645+0000","last_peered":"2024-09-12T01:44:45.821645+0000","last_clean":"2024-09-12T01:44:45.821645+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:44:45.821645+0000","last_undegraded":"2024-09-12T01:44:45.821645+0000","last_fullsized":"2024-09-12T01:44:45.821645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821329+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:44:45.821329+0000","last_peered":"2024-09-12T01:44:45.821329+0000","last_clean":"2024-09-12T01:44:45.821329+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:44:45.821329+0000","last_undegraded":"2024-09-12T01:44:45.821329+0000","last_fullsized":"2024-09-12T01:44:45.821329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824934+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:44:45.824934+0000","last_peered":"2024-09-12T01:44:45.824934+0000","last_clean":"2024-09-12T01:44:45.824934+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:44:45.824934+0000","last_undegraded":"2024-09-12T01:44:45.824934+0000","last_fullsized":"2024-09-12T01:44:45.824934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818038+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:44:45.818038+0000","last_peered":"2024-09-12T01:44:45.818038+0000","last_clean":"2024-09-12T01:44:45.818038+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:44:45.818038+0000","last_undegraded":"2024-09-12T01:44:45.818038+0000","last_fullsized":"2024-09-12T01:44:45.818038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822055+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:44:45.822055+0000","last_peered":"2024-09-12T01:44:45.822055+0000","last_clean":"2024-09-12T01:44:45.822055+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:44:45.822055+0000","last_undegraded":"2024-09-12T01:44:45.822055+0000","last_fullsized":"2024-09-12T01:44:45.822055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077339+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:44:46.077339+0000","last_peered":"2024-09-12T01:44:46.077339+0000","last_clean":"2024-09-12T01:44:46.077339+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:44:46.077339+0000","last_undegraded":"2024-09-12T01:44:46.077339+0000","last_fullsized":"2024-09-12T01:44:46.077339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076657+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:44:46.076657+0000","last_peered":"2024-09-12T01:44:46.076657+0000","last_clean":"2024-09-12T01:44:46.076657+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:44:46.076657+0000","last_undegraded":"2024-09-12T01:44:46.076657+0000","last_fullsized":"2024-09-12T01:44:46.076657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076627+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:44:46.076627+0000","last_peered":"2024-09-12T01:44:46.076627+0000","last_clean":"2024-09-12T01:44:46.076627+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:44:46.076627+0000","last_undegraded":"2024-09-12T01:44:46.076627+0000","last_fullsized":"2024-09-12T01:44:46.076627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820478+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:44:45.820478+0000","last_peered":"2024-09-12T01:44:45.820478+0000","last_clean":"2024-09-12T01:44:45.820478+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:44:45.820478+0000","last_undegraded":"2024-09-12T01:44:45.820478+0000","last_fullsized":"2024-09-12T01:44:45.820478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818598+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:44:45.818598+0000","last_peered":"2024-09-12T01:44:45.818598+0000","last_clean":"2024-09-12T01:44:45.818598+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:44:45.818598+0000","last_undegraded":"2024-09-12T01:44:45.818598+0000","last_fullsized":"2024-09-12T01:44:45.818598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816391+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:44:45.816391+0000","last_peered":"2024-09-12T01:44:45.816391+0000","last_clean":"2024-09-12T01:44:45.816391+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:44:45.816391+0000","last_undegraded":"2024-09-12T01:44:45.816391+0000","last_fullsized":"2024-09-12T01:44:45.816391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821743+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:44:45.821743+0000","last_peered":"2024-09-12T01:44:45.821743+0000","last_clean":"2024-09-12T01:44:45.821743+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:44:45.821743+0000","last_undegraded":"2024-09-12T01:44:45.821743+0000","last_fullsized":"2024-09-12T01:44:45.821743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818574+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:44:45.818574+0000","last_peered":"2024-09-12T01:44:45.818574+0000","last_clean":"2024-09-12T01:44:45.818574+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:44:45.818574+0000","last_undegraded":"2024-09-12T01:44:45.818574+0000","last_fullsized":"2024-09-12T01:44:45.818574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077436+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:44:46.077436+0000","last_peered":"2024-09-12T01:44:46.077436+0000","last_clean":"2024-09-12T01:44:46.077436+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:44:46.077436+0000","last_undegraded":"2024-09-12T01:44:46.077436+0000","last_fullsized":"2024-09-12T01:44:46.077436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820692+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:44:45.820692+0000","last_peered":"2024-09-12T01:44:45.820692+0000","last_clean":"2024-09-12T01:44:45.820692+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:44:45.820692+0000","last_undegraded":"2024-09-12T01:44:45.820692+0000","last_fullsized":"2024-09-12T01:44:45.820692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821444+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:44:45.821444+0000","last_peered":"2024-09-12T01:44:45.821444+0000","last_clean":"2024-09-12T01:44:45.821444+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:44:45.821444+0000","last_undegraded":"2024-09-12T01:44:45.821444+0000","last_fullsized":"2024-09-12T01:44:45.821444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824121+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:44:45.824121+0000","last_peered":"2024-09-12T01:44:45.824121+0000","last_clean":"2024-09-12T01:44:45.824121+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:44:45.824121+0000","last_undegraded":"2024-09-12T01:44:45.824121+0000","last_fullsized":"2024-09-12T01:44:45.824121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821120+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:44:45.821120+0000","last_peered":"2024-09-12T01:44:45.821120+0000","last_clean":"2024-09-12T01:44:45.821120+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:44:45.821120+0000","last_undegraded":"2024-09-12T01:44:45.821120+0000","last_fullsized":"2024-09-12T01:44:45.821120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822863+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:44:45.822863+0000","last_peered":"2024-09-12T01:44:45.822863+0000","last_clean":"2024-09-12T01:44:45.822863+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:44:45.822863+0000","last_undegraded":"2024-09-12T01:44:45.822863+0000","last_fullsized":"2024-09-12T01:44:45.822863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817853+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:44:45.817853+0000","last_peered":"2024-09-12T01:44:45.817853+0000","last_clean":"2024-09-12T01:44:45.817853+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:44:45.817853+0000","last_undegraded":"2024-09-12T01:44:45.817853+0000","last_fullsized":"2024-09-12T01:44:45.817853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077201+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:44:46.077201+0000","last_peered":"2024-09-12T01:44:46.077201+0000","last_clean":"2024-09-12T01:44:46.077201+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:44:46.077201+0000","last_undegraded":"2024-09-12T01:44:46.077201+0000","last_fullsized":"2024-09-12T01:44:46.077201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076827+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:44:46.076827+0000","last_peered":"2024-09-12T01:44:46.076827+0000","last_clean":"2024-09-12T01:44:46.076827+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:44:46.076827+0000","last_undegraded":"2024-09-12T01:44:46.076827+0000","last_fullsized":"2024-09-12T01:44:46.076827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820905+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:44:45.820905+0000","last_peered":"2024-09-12T01:44:45.820905+0000","last_clean":"2024-09-12T01:44:45.820905+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:44:45.820905+0000","last_undegraded":"2024-09-12T01:44:45.820905+0000","last_fullsized":"2024-09-12T01:44:45.820905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821482+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:44:45.821482+0000","last_peered":"2024-09-12T01:44:45.821482+0000","last_clean":"2024-09-12T01:44:45.821482+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:44:45.821482+0000","last_undegraded":"2024-09-12T01:44:45.821482+0000","last_fullsized":"2024-09-12T01:44:45.821482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817474+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:44:45.817474+0000","last_peered":"2024-09-12T01:44:45.817474+0000","last_clean":"2024-09-12T01:44:45.817474+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:44:45.817474+0000","last_undegraded":"2024-09-12T01:44:45.817474+0000","last_fullsized":"2024-09-12T01:44:45.817474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819263+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:44:45.819263+0000","last_peered":"2024-09-12T01:44:45.819263+0000","last_clean":"2024-09-12T01:44:45.819263+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:44:45.819263+0000","last_undegraded":"2024-09-12T01:44:45.819263+0000","last_fullsized":"2024-09-12T01:44:45.819263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:36.911725+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:45:36.911725+0000","last_peered":"2024-09-12T01:45:36.911725+0000","last_clean":"2024-09-12T01:45:36.911725+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:45:36.911725+0000","last_undegraded":"2024-09-12T01:45:36.911725+0000","last_fullsized":"2024-09-12T01:45:36.911725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815476+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:44:45.815476+0000","last_peered":"2024-09-12T01:44:45.815476+0000","last_clean":"2024-09-12T01:44:45.815476+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:44:45.815476+0000","last_undegraded":"2024-09-12T01:44:45.815476+0000","last_fullsized":"2024-09-12T01:44:45.815476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821664+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:44:45.821664+0000","last_peered":"2024-09-12T01:44:45.821664+0000","last_clean":"2024-09-12T01:44:45.821664+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:44:45.821664+0000","last_undegraded":"2024-09-12T01:44:45.821664+0000","last_fullsized":"2024-09-12T01:44:45.821664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816283+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:44:45.816283+0000","last_peered":"2024-09-12T01:44:45.816283+0000","last_clean":"2024-09-12T01:44:45.816283+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:44:45.816283+0000","last_undegraded":"2024-09-12T01:44:45.816283+0000","last_fullsized":"2024-09-12T01:44:45.816283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:33.906863+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:45:33.906863+0000","last_peered":"2024-09-12T01:45:33.906863+0000","last_clean":"2024-09-12T01:45:33.906863+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:45:33.906863+0000","last_undegraded":"2024-09-12T01:45:33.906863+0000","last_fullsized":"2024-09-12T01:45:33.906863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821400+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:44:45.821400+0000","last_peered":"2024-09-12T01:44:45.821400+0000","last_clean":"2024-09-12T01:44:45.821400+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:44:45.821400+0000","last_undegraded":"2024-09-12T01:44:45.821400+0000","last_fullsized":"2024-09-12T01:44:45.821400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077017+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:44:46.077017+0000","last_peered":"2024-09-12T01:44:46.077017+0000","last_clean":"2024-09-12T01:44:46.077017+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:44:46.077017+0000","last_undegraded":"2024-09-12T01:44:46.077017+0000","last_fullsized":"2024-09-12T01:44:46.077017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820850+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:44:45.820850+0000","last_peered":"2024-09-12T01:44:45.820850+0000","last_clean":"2024-09-12T01:44:45.820850+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:44:45.820850+0000","last_undegraded":"2024-09-12T01:44:45.820850+0000","last_fullsized":"2024-09-12T01:44:45.820850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":157,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:34.908189+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:45:34.908189+0000","last_peered":"2024-09-12T01:45:34.908189+0000","last_clean":"2024-09-12T01:45:34.908189+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:45:34.908189+0000","last_undegraded":"2024-09-12T01:45:34.908189+0000","last_fullsized":"2024-09-12T01:45:34.908189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":106,"num_read_kb":113,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820311+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:44:45.820311+0000","last_peered":"2024-09-12T01:44:45.820311+0000","last_clean":"2024-09-12T01:44:45.820311+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:44:45.820311+0000","last_undegraded":"2024-09-12T01:44:45.820311+0000","last_fullsized":"2024-09-12T01:44:45.820311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077080+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:44:46.077080+0000","last_peered":"2024-09-12T01:44:46.077080+0000","last_clean":"2024-09-12T01:44:46.077080+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:44:46.077080+0000","last_undegraded":"2024-09-12T01:44:46.077080+0000","last_fullsized":"2024-09-12T01:44:46.077080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817662+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:44:45.817662+0000","last_peered":"2024-09-12T01:44:45.817662+0000","last_clean":"2024-09-12T01:44:45.817662+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:44:45.817662+0000","last_undegraded":"2024-09-12T01:44:45.817662+0000","last_fullsized":"2024-09-12T01:44:45.817662+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:35.909837+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:45:35.909837+0000","last_peered":"2024-09-12T01:45:35.909837+0000","last_clean":"2024-09-12T01:45:35.909837+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:45:35.909837+0000","last_undegraded":"2024-09-12T01:45:35.909837+0000","last_fullsized":"2024-09-12T01:45:35.909837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817679+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:44:45.817679+0000","last_peered":"2024-09-12T01:44:45.817679+0000","last_clean":"2024-09-12T01:44:45.817679+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:44:45.817679+0000","last_undegraded":"2024-09-12T01:44:45.817679+0000","last_fullsized":"2024-09-12T01:44:45.817679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.822535+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:44:45.822535+0000","last_peered":"2024-09-12T01:44:45.822535+0000","last_clean":"2024-09-12T01:44:45.822535+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:44:45.822535+0000","last_undegraded":"2024-09-12T01:44:45.822535+0000","last_fullsized":"2024-09-12T01:44:45.822535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818060+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:44:45.818060+0000","last_peered":"2024-09-12T01:44:45.818060+0000","last_clean":"2024-09-12T01:44:45.818060+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:44:45.818060+0000","last_undegraded":"2024-09-12T01:44:45.818060+0000","last_fullsized":"2024-09-12T01:44:45.818060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818393+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:44:45.818393+0000","last_peered":"2024-09-12T01:44:45.818393+0000","last_clean":"2024-09-12T01:44:45.818393+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:44:45.818393+0000","last_undegraded":"2024-09-12T01:44:45.818393+0000","last_fullsized":"2024-09-12T01:44:45.818393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820335+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:44:45.820335+0000","last_peered":"2024-09-12T01:44:45.820335+0000","last_clean":"2024-09-12T01:44:45.820335+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:44:45.820335+0000","last_undegraded":"2024-09-12T01:44:45.820335+0000","last_fullsized":"2024-09-12T01:44:45.820335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818057+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:44:45.818057+0000","last_peered":"2024-09-12T01:44:45.818057+0000","last_clean":"2024-09-12T01:44:45.818057+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:44:45.818057+0000","last_undegraded":"2024-09-12T01:44:45.818057+0000","last_fullsized":"2024-09-12T01:44:45.818057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818530+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818530+0000","last_peered":"2024-09-12T01:44:45.818530+0000","last_clean":"2024-09-12T01:44:45.818530+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818530+0000","last_undegraded":"2024-09-12T01:44:45.818530+0000","last_fullsized":"2024-09-12T01:44:45.818530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821278+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:44:45.821278+0000","last_peered":"2024-09-12T01:44:45.821278+0000","last_clean":"2024-09-12T01:44:45.821278+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:44:45.821278+0000","last_undegraded":"2024-09-12T01:44:45.821278+0000","last_fullsized":"2024-09-12T01:44:45.821278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077069+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:44:46.077069+0000","last_peered":"2024-09-12T01:44:46.077069+0000","last_clean":"2024-09-12T01:44:46.077069+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:44:46.077069+0000","last_undegraded":"2024-09-12T01:44:46.077069+0000","last_fullsized":"2024-09-12T01:44:46.077069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:36.910963+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:45:36.910963+0000","last_peered":"2024-09-12T01:45:36.910963+0000","last_clean":"2024-09-12T01:45:36.910963+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:45:36.910963+0000","last_undegraded":"2024-09-12T01:45:36.910963+0000","last_fullsized":"2024-09-12T01:45:36.910963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.824468+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:44:45.824468+0000","last_peered":"2024-09-12T01:44:45.824468+0000","last_clean":"2024-09-12T01:44:45.824468+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:44:45.824468+0000","last_undegraded":"2024-09-12T01:44:45.824468+0000","last_fullsized":"2024-09-12T01:44:45.824468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818919+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:44:45.818919+0000","last_peered":"2024-09-12T01:44:45.818919+0000","last_clean":"2024-09-12T01:44:45.818919+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:44:45.818919+0000","last_undegraded":"2024-09-12T01:44:45.818919+0000","last_fullsized":"2024-09-12T01:44:45.818919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819406+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:44:45.819406+0000","last_peered":"2024-09-12T01:44:45.819406+0000","last_clean":"2024-09-12T01:44:45.819406+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:44:45.819406+0000","last_undegraded":"2024-09-12T01:44:45.819406+0000","last_fullsized":"2024-09-12T01:44:45.819406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077325+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:44:46.077325+0000","last_peered":"2024-09-12T01:44:46.077325+0000","last_clean":"2024-09-12T01:44:46.077325+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:44:46.077325+0000","last_undegraded":"2024-09-12T01:44:46.077325+0000","last_fullsized":"2024-09-12T01:44:46.077325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'10","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:45:00.781231+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:45:00.781231+0000","last_peered":"2024-09-12T01:45:00.781231+0000","last_clean":"2024-09-12T01:45:00.781231+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:45:00.781231+0000","last_undegraded":"2024-09-12T01:45:00.781231+0000","last_fullsized":"2024-09-12T01:45:00.781231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821213+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:44:45.821213+0000","last_peered":"2024-09-12T01:44:45.821213+0000","last_clean":"2024-09-12T01:44:45.821213+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:44:45.821213+0000","last_undegraded":"2024-09-12T01:44:45.821213+0000","last_fullsized":"2024-09-12T01:44:45.821213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076938+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:44:46.076938+0000","last_peered":"2024-09-12T01:44:46.076938+0000","last_clean":"2024-09-12T01:44:46.076938+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:44:46.076938+0000","last_undegraded":"2024-09-12T01:44:46.076938+0000","last_fullsized":"2024-09-12T01:44:46.076938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819237+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:44:45.819237+0000","last_peered":"2024-09-12T01:44:45.819237+0000","last_clean":"2024-09-12T01:44:45.819237+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:44:45.819237+0000","last_undegraded":"2024-09-12T01:44:45.819237+0000","last_fullsized":"2024-09-12T01:44:45.819237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.817496+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:44:45.817496+0000","last_peered":"2024-09-12T01:44:45.817496+0000","last_clean":"2024-09-12T01:44:45.817496+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:44:45.817496+0000","last_undegraded":"2024-09-12T01:44:45.817496+0000","last_fullsized":"2024-09-12T01:44:45.817496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821064+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:44:45.821064+0000","last_peered":"2024-09-12T01:44:45.821064+0000","last_clean":"2024-09-12T01:44:45.821064+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:44:45.821064+0000","last_undegraded":"2024-09-12T01:44:45.821064+0000","last_fullsized":"2024-09-12T01:44:45.821064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818032+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:44:45.818032+0000","last_peered":"2024-09-12T01:44:45.818032+0000","last_clean":"2024-09-12T01:44:45.818032+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:44:45.818032+0000","last_undegraded":"2024-09-12T01:44:45.818032+0000","last_fullsized":"2024-09-12T01:44:45.818032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816211+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:44:45.816211+0000","last_peered":"2024-09-12T01:44:45.816211+0000","last_clean":"2024-09-12T01:44:45.816211+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:44:45.816211+0000","last_undegraded":"2024-09-12T01:44:45.816211+0000","last_fullsized":"2024-09-12T01:44:45.816211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820739+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:44:45.820739+0000","last_peered":"2024-09-12T01:44:45.820739+0000","last_clean":"2024-09-12T01:44:45.820739+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:44:45.820739+0000","last_undegraded":"2024-09-12T01:44:45.820739+0000","last_fullsized":"2024-09-12T01:44:45.820739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.076981+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:44:46.076981+0000","last_peered":"2024-09-12T01:44:46.076981+0000","last_clean":"2024-09-12T01:44:46.076981+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:44:46.076981+0000","last_undegraded":"2024-09-12T01:44:46.076981+0000","last_fullsized":"2024-09-12T01:44:46.076981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820929+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:44:45.820929+0000","last_peered":"2024-09-12T01:44:45.820929+0000","last_clean":"2024-09-12T01:44:45.820929+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:44:45.820929+0000","last_undegraded":"2024-09-12T01:44:45.820929+0000","last_fullsized":"2024-09-12T01:44:45.820929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65755,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":329,"num_read_kb":233,"num_write":162,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1091421,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029883,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438250,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":500281,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":498965,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:45:42.012 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T01:45:42.012 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T01:45:42.013 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T01:45:42.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T01:45:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:41 smithi151 ceph-mon[35229]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:42.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:41 smithi183 ceph-mon[35127]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:45:42.258 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:42.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 -- 172.21.15.122:0/2542420150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874100a90 msgr2=0x7f2874100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:42.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 --2- 172.21.15.122:0/2542420150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874100a90 0x7f2874100ef0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f285c009930 tx=0x7f285c02a420 comp rx=0 tx=0).stop 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 -- 172.21.15.122:0/2542420150 shutdown_connections 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 --2- 172.21.15.122:0/2542420150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2874101430 0x7f287410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 --2- 172.21.15.122:0/2542420150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874100a90 0x7f2874100ef0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 --2- 172.21.15.122:0/2542420150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2874106a90 0x7f2874106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 -- 172.21.15.122:0/2542420150 >> 172.21.15.122:0/2542420150 conn(0x7f28740fc7d0 msgr2=0x7f28740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.753+0000 7f287a215640 1 -- 172.21.15.122:0/2542420150 shutdown_connections 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 -- 172.21.15.122:0/2542420150 wait complete. 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 Processor -- start 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 -- start start 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2874100a90 0x7f28741992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 0x7f2874199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:42.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2874106a90 0x7f28741a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2874110e90 con 0x7f2874101430 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2874110d10 con 0x7f2874106a90 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.754+0000 7f287a215640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2874111010 con 0x7f2874100a90 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 0x7f2874199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 0x7f2874199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50716/0 (socket says 172.21.15.122:50716) 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 -- 172.21.15.122:0/2757876169 learned_addr learned my addr 172.21.15.122:0/2757876169 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f2873fff640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2874106a90 0x7f28741a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f28737fe640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2874100a90 0x7f28741992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 -- 172.21.15.122:0/2757876169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2874100a90 msgr2=0x7f28741992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2874100a90 0x7f28741992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 -- 172.21.15.122:0/2757876169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2874106a90 msgr2=0x7f28741a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:42.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2874106a90 0x7f28741a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 -- 172.21.15.122:0/2757876169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28741a4890 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f286affd640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 0x7f2874199810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f285c009230 tx=0x7f285c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c03f030 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285c003ad0 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28741a4a60 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c035070 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.755+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28741a4f10 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.756+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2874100ef0 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.758+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f285c042030 con 0x7f2874101430 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.758+0000 7f28717fa640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28600a2c90 0x7f28600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:42.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.758+0000 7f28737fe640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28600a2c90 0x7f28600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:42.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.759+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f285c0ef590 con 0x7f2874101430 2024-09-12T01:45:42.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.759+0000 7f28737fe640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28600a2c90 0x7f28600a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f286400bb60 tx=0x7f286400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:42.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.762+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f285c0bbd40 con 0x7f2874101430 2024-09-12T01:45:42.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.963+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2874109020 con 0x7f2874101430 2024-09-12T01:45:42.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:42 smithi122 ceph-mon[30925]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:42.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.963+0000 7f28717fa640 1 -- 172.21.15.122:0/2757876169 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f285c0c0040 con 0x7f2874101430 2024-09-12T01:45:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:42.964 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T01:45:42.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28600a2c90 msgr2=0x7f28600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:42.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28600a2c90 0x7f28600a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f286400bb60 tx=0x7f286400b390 comp rx=0 tx=0).stop 2024-09-12T01:45:42.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 msgr2=0x7f2874199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:42.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 0x7f2874199810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f285c009230 tx=0x7f285c004270 comp rx=0 tx=0).stop 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 shutdown_connections 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f28600a2c90 0x7f28600a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2874106a90 0x7f28741a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2874101430 0x7f2874199810 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 --2- 172.21.15.122:0/2757876169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2874100a90 0x7f28741992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 >> 172.21.15.122:0/2757876169 conn(0x7f28740fc7d0 msgr2=0x7f28740fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 shutdown_connections 2024-09-12T01:45:42.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:42.965+0000 7f287a215640 1 -- 172.21.15.122:0/2757876169 wait complete. 2024-09-12T01:45:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:42 smithi151 ceph-mon[35229]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:42 smithi183 ceph-mon[35127]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:43.593 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T01:45:43.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:45:43.840 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:43.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:43 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2757876169' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T01:45:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:43 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2757876169' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T01:45:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:43 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2757876169' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 -- 172.21.15.122:0/90133728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 msgr2=0x7f0dfc103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 --2- 172.21.15.122:0/90133728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc103170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009930 tx=0x7f0dec02a420 comp rx=0 tx=0).stop 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 -- 172.21.15.122:0/90133728 shutdown_connections 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 --2- 172.21.15.122:0/90133728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc108290 0x7f0dfc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 --2- 172.21.15.122:0/90133728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc103740 0x7f0dfc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 --2- 172.21.15.122:0/90133728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc103170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 -- 172.21.15.122:0/90133728 >> 172.21.15.122:0/90133728 conn(0x7f0dfc0fc7d0 msgr2=0x7f0dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 -- 172.21.15.122:0/90133728 shutdown_connections 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.307+0000 7f0e0270c640 1 -- 172.21.15.122:0/90133728 wait complete. 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 Processor -- start 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 -- start start 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc103740 0x7f0dfc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc108290 0x7f0dfc19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc1120d0 con 0x7f0dfc102d90 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc111f50 con 0x7f0dfc108290 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0e0270c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112250 con 0x7f0dfc103740 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50738/0 (socket says 172.21.15.122:50738) 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.308+0000 7f0dfbfff640 1 -- 172.21.15.122:0/16331565 learned_addr learned my addr 172.21.15.122:0/16331565 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0e00ca9640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc108290 0x7f0dfc19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0df3fff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc103740 0x7f0dfc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0dfbfff640 1 -- 172.21.15.122:0/16331565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc103740 msgr2=0x7f0dfc198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0dfbfff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc103740 0x7f0dfc198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0dfbfff640 1 -- 172.21.15.122:0/16331565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc108290 msgr2=0x7f0dfc19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0dfbfff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc108290 0x7f0dfc19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0dfbfff640 1 -- 172.21.15.122:0/16331565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc19d580 con 0x7f0dfc102d90 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0df3fff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc103740 0x7f0dfc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0dfbfff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc1985d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009230 tx=0x7f0dec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec03f030 con 0x7f0dfc102d90 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dec003ad0 con 0x7f0dfc102d90 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc19d810 con 0x7f0dfc102d90 2024-09-12T01:45:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec035070 con 0x7f0dfc102d90 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.309+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc100920 con 0x7f0dfc102d90 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.310+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc06a7f0 con 0x7f0dfc102d90 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.311+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0dec042030 con 0x7f0dfc102d90 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.312+0000 7f0df9ffb640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0de80a2c10 0x7f0de80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.312+0000 7f0df3fff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0de80a2c10 0x7f0de80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.312+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0dec0efb00 con 0x7f0dfc102d90 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.312+0000 7f0df3fff640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0de80a2c10 0x7f0de80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0de40058c0 tx=0x7f0de4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.315+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0dec0bc900 con 0x7f0dfc102d90 2024-09-12T01:45:44.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.525+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dfc199930 con 0x7f0dfc102d90 2024-09-12T01:45:44.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.525+0000 7f0df9ffb640 1 -- 172.21.15.122:0/16331565 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f0dec0bc200 con 0x7f0dfc102d90 2024-09-12T01:45:44.526 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:44.526 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"btime":"2024-09-12T01:44:47:831953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:44:47.439571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:45:44.526 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:45:44.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0de80a2c10 msgr2=0x7f0de80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0de80a2c10 0x7f0de80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0de40058c0 tx=0x7f0de4005850 comp rx=0 tx=0).stop 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 msgr2=0x7f0dfc1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc1985d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009230 tx=0x7f0dec004270 comp rx=0 tx=0).stop 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 shutdown_connections 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0de80a2c10 0x7f0de80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.527+0000 7f0e0270c640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dfc108290 0x7f0dfc19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.528+0000 7f0e0270c640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dfc103740 0x7f0dfc198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.528+0000 7f0e0270c640 1 --2- 172.21.15.122:0/16331565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dfc102d90 0x7f0dfc1985d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.528+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 >> 172.21.15.122:0/16331565 conn(0x7f0dfc0fc7d0 msgr2=0x7f0dfc105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:44.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.528+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 shutdown_connections 2024-09-12T01:45:44.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:44.528+0000 7f0e0270c640 1 -- 172.21.15.122:0/16331565 wait complete. 2024-09-12T01:45:44.840 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:44 smithi122 ceph-mon[30925]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:45:44.840 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:44 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/16331565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:44 smithi151 ceph-mon[35229]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:45:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:44 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/16331565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:45.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:45:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:44 smithi183 ceph-mon[35127]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:45:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:44 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/16331565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:45.436 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 -- 172.21.15.122:0/3382567633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 msgr2=0x7f1c5010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 --2- 172.21.15.122:0/3382567633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5010eaa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009900 tx=0x7f1c4802a410 comp rx=0 tx=0).stop 2024-09-12T01:45:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 -- 172.21.15.122:0/3382567633 shutdown_connections 2024-09-12T01:45:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 --2- 172.21.15.122:0/3382567633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5010eaa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 --2- 172.21.15.122:0/3382567633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c501027a0 0x7f1c501067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 --2- 172.21.15.122:0/3382567633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c50101df0 0x7f1c501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 -- 172.21.15.122:0/3382567633 >> 172.21.15.122:0/3382567633 conn(0x7f1c500fd8f0 msgr2=0x7f1c500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 -- 172.21.15.122:0/3382567633 shutdown_connections 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 -- 172.21.15.122:0/3382567633 wait complete. 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.833+0000 7f1c55fce640 1 Processor -- start 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 -- start start 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50101df0 0x7f1c501983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c501027a0 0x7f1c50198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50111f80 con 0x7f1c50106f20 2024-09-12T01:45:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50111e00 con 0x7f1c50101df0 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c55fce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112100 con 0x7f1c501027a0 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50760/0 (socket says 172.21.15.122:50760) 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 -- 172.21.15.122:0/2388414223 learned_addr learned my addr 172.21.15.122:0/2388414223 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4effd640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c501027a0 0x7f1c50198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4f7fe640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50101df0 0x7f1c501983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 -- 172.21.15.122:0/2388414223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c501027a0 msgr2=0x7f1c50198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c501027a0 0x7f1c50198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 -- 172.21.15.122:0/2388414223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50101df0 msgr2=0x7f1c501983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50101df0 0x7f1c501983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 -- 172.21.15.122:0/2388414223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5019d370 con 0x7f1c50106f20 2024-09-12T01:45:45.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4f7fe640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50101df0 0x7f1c501983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4effd640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c501027a0 0x7f1c50198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4ffff640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5019cc90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c48005ab0 tx=0x7f1c480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.834+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4803f030 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.835+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c48003ce0 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.835+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5019d600 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.835+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48035070 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.835+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501a4f40 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.836+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c501021d0 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.837+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1c48042030 con 0x7f1c50106f20 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.837+0000 7f1c4cff9640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c380a2c10 0x7f1c380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.838+0000 7f1c4f7fe640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c380a2c10 0x7f1c380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.838+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1c480efa90 con 0x7f1c50106f20 2024-09-12T01:45:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.838+0000 7f1c4f7fe640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c380a2c10 0x7f1c380a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005fd0 tx=0x7f1c400093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:45.841+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c480bc890 con 0x7f1c50106f20 2024-09-12T01:45:46.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.042+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c5010a100 con 0x7f1c50106f20 2024-09-12T01:45:46.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.042+0000 7f1c4cff9640 1 -- 172.21.15.122:0/2388414223 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f1c480bc190 con 0x7f1c50106f20 2024-09-12T01:45:46.044 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:46.044 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"btime":"2024-09-12T01:44:47:831953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:44:47.439571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:45:46.044 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:45:46.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.044+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c380a2c10 msgr2=0x7f1c380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.044+0000 7f1c55fce640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c380a2c10 0x7f1c380a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005fd0 tx=0x7f1c400093e0 comp rx=0 tx=0).stop 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.044+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 msgr2=0x7f1c5019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.044+0000 7f1c55fce640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5019cc90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c48005ab0 tx=0x7f1c480042f0 comp rx=0 tx=0).stop 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 shutdown_connections 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c380a2c10 0x7f1c380a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c50106f20 0x7f1c5019cc90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c501027a0 0x7f1c50198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 --2- 172.21.15.122:0/2388414223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50101df0 0x7f1c501983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:46.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 >> 172.21.15.122:0/2388414223 conn(0x7f1c500fd8f0 msgr2=0x7f1c500fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:46.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 shutdown_connections 2024-09-12T01:45:46.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:46.045+0000 7f1c55fce640 1 -- 172.21.15.122:0/2388414223 wait complete. 2024-09-12T01:45:46.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:46 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2388414223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:46.365 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:46 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2388414223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:46.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:46 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2388414223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:46.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:45:46.978 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:47.354 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:47 smithi122 ceph-mon[30925]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:47.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:47 smithi151 ceph-mon[35229]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:47.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- 172.21.15.122:0/2079752245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 msgr2=0x7f3948103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:47.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- 172.21.15.122:0/2079752245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948103180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3940009900 tx=0x7f394002a480 comp rx=0 tx=0).stop 2024-09-12T01:45:47.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- 172.21.15.122:0/2079752245 shutdown_connections 2024-09-12T01:45:47.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- 172.21.15.122:0/2079752245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f394810c180 0x7f394810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- 172.21.15.122:0/2079752245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39481036c0 0x7f394810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- 172.21.15.122:0/2079752245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948103180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- 172.21.15.122:0/2079752245 >> 172.21.15.122:0/2079752245 conn(0x7f39480fc7a0 msgr2=0x7f39480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- 172.21.15.122:0/2079752245 shutdown_connections 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- 172.21.15.122:0/2079752245 wait complete. 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 Processor -- start 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- start start 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39481036c0 0x7f3948100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f394810c180 0x7f3948101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111f60 con 0x7f3948102da0 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111de0 con 0x7f394810c180 2024-09-12T01:45:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.433+0000 7f394f21f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39481120e0 con 0x7f39481036c0 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50778/0 (socket says 172.21.15.122:50778) 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 -- 172.21.15.122:0/3461025534 learned_addr learned my addr 172.21.15.122:0/3461025534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394d7bc640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f394810c180 0x7f3948101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f3947fff640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39481036c0 0x7f3948100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 -- 172.21.15.122:0/3461025534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39481036c0 msgr2=0x7f3948100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39481036c0 0x7f3948100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 -- 172.21.15.122:0/3461025534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f394810c180 msgr2=0x7f3948101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f394810c180 0x7f3948101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 -- 172.21.15.122:0/3461025534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39481a69a0 con 0x7f3948102da0 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f3947fff640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39481036c0 0x7f3948100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:47.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394cfbb640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948100a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3940005ab0 tx=0x7f39400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394003f030 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3940003ce0 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940035070 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.434+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39481a6c30 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.435+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a7160 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.435+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3948103180 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.437+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3940042030 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.437+0000 7f3945ffb640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f39340a2c90 0x7f39340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.437+0000 7f3947fff640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f39340a2c90 0x7f39340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.437+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f39400efda0 con 0x7f3948102da0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.438+0000 7f3947fff640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f39340a2c90 0x7f39340a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3948102490 tx=0x7f39380082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:47.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.440+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39400bcb60 con 0x7f3948102da0 2024-09-12T01:45:47.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:47 smithi183 ceph-mon[35127]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:47.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.609+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3948107220 con 0x7f3948102da0 2024-09-12T01:45:47.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.611+0000 7f3945ffb640 1 -- 172.21.15.122:0/3461025534 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7f39400bc460 con 0x7f3948102da0 2024-09-12T01:45:47.612 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:47.612 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:45:47.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f39340a2c90 msgr2=0x7f39340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:47.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f39340a2c90 0x7f39340a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3948102490 tx=0x7f39380082f0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 msgr2=0x7f3948100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948100a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3940005ab0 tx=0x7f39400042f0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 shutdown_connections 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f39340a2c90 0x7f39340a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f394810c180 0x7f3948101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39481036c0 0x7f3948100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 --2- 172.21.15.122:0/3461025534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948102da0 0x7f3948100a00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.613+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 >> 172.21.15.122:0/3461025534 conn(0x7f39480fc7a0 msgr2=0x7f39480fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.614+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 shutdown_connections 2024-09-12T01:45:47.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:47.614+0000 7f394f21f640 1 -- 172.21.15.122:0/3461025534 wait complete. 2024-09-12T01:45:48.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:45:48.354 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:48 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3461025534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:48.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:48 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3461025534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:48.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:48 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3461025534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:45:48.577 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 -- 172.21.15.122:0/2960967932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184102d90 msgr2=0x7fc184103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 --2- 172.21.15.122:0/2960967932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184102d90 0x7fc184103170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc174009930 tx=0x7fc17402a420 comp rx=0 tx=0).stop 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 -- 172.21.15.122:0/2960967932 shutdown_connections 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 --2- 172.21.15.122:0/2960967932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc184108290 0x7fc18410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 --2- 172.21.15.122:0/2960967932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184103740 0x7fc184107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 --2- 172.21.15.122:0/2960967932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184102d90 0x7fc184103170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 -- 172.21.15.122:0/2960967932 >> 172.21.15.122:0/2960967932 conn(0x7fc1840fc7d0 msgr2=0x7fc1840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 -- 172.21.15.122:0/2960967932 shutdown_connections 2024-09-12T01:45:49.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 -- 172.21.15.122:0/2960967932 wait complete. 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.065+0000 7fc189533640 1 Processor -- start 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 -- start start 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184102d90 0x7fc1841985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc184103740 0x7fc184198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 0x7fc18419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc184112090 con 0x7fc184108290 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc184111f10 con 0x7fc184102d90 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc189533640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc184112210 con 0x7fc184103740 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 0x7fc18419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 0x7fc18419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50804/0 (socket says 172.21.15.122:50804) 2024-09-12T01:45:49.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 -- 172.21.15.122:0/4201256902 learned_addr learned my addr 172.21.15.122:0/4201256902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1827fc640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc184103740 0x7fc184198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc182ffd640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184102d90 0x7fc1841985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 -- 172.21.15.122:0/4201256902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc184103740 msgr2=0x7fc184198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc184103740 0x7fc184198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 -- 172.21.15.122:0/4201256902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184102d90 msgr2=0x7fc1841985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184102d90 0x7fc1841985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc1837fe640 1 -- 172.21.15.122:0/4201256902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18419d580 con 0x7fc184108290 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.066+0000 7fc182ffd640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184102d90 0x7fc1841985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.067+0000 7fc1837fe640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 0x7fc18419cea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00cad0 tx=0x7fc17c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.067+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00ae20 con 0x7fc184108290 2024-09-12T01:45:49.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.067+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c005020 con 0x7fc184108290 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.067+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18419d870 con 0x7fc184108290 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.067+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00f6f0 con 0x7fc184108290 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.067+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841a5150 con 0x7fc184108290 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.068+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18406a7f0 con 0x7fc184108290 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.069+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc17c0051d0 con 0x7fc184108290 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.069+0000 7fc16bfff640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.070+0000 7fc182ffd640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:49.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.070+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc17c0cf240 con 0x7fc184108290 2024-09-12T01:45:49.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.070+0000 7fc182ffd640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc174000c00 tx=0x7fc174005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:49.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.073+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc17c014030 con 0x7fc184108290 2024-09-12T01:45:49.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:49 smithi122 ceph-mon[30925]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.278+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc18410a2a0 con 0x7fc184108290 2024-09-12T01:45:49.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.280+0000 7fc16bfff640 1 -- 172.21.15.122:0/4201256902 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7fc17c0afe20 con 0x7fc184108290 2024-09-12T01:45:49.281 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:45:49.281 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"btime":"2024-09-12T01:44:47:831953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:44:47.439571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:45:49.282 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:45:49.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc16c0a2c10 msgr2=0x7fc16c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:49.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc174000c00 tx=0x7fc174005a20 comp rx=0 tx=0).stop 2024-09-12T01:45:49.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 msgr2=0x7fc18419cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 0x7fc18419cea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00cad0 tx=0x7fc17c00cf90 comp rx=0 tx=0).stop 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 shutdown_connections 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc184108290 0x7fc18419cea0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc184103740 0x7fc184198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 --2- 172.21.15.122:0/4201256902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184102d90 0x7fc1841985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 >> 172.21.15.122:0/4201256902 conn(0x7fc1840fc7d0 msgr2=0x7fc184105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 shutdown_connections 2024-09-12T01:45:49.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:49.281+0000 7fc189533640 1 -- 172.21.15.122:0/4201256902 wait complete. 2024-09-12T01:45:49.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:49 smithi151 ceph-mon[35229]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:49.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:49 smithi183 ceph-mon[35127]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:49.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:44:36.280693+0000', 'modified': '2024-09-12T01:44:47.439571+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24436, 'mds_2': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/3975453954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 3975453954}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 3975453954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/2236567181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 2236567181}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 2236567181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1190566748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1190566748}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1190566748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.151:6837/3984907767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3984907767}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3984907767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6837/3258783936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3258783936}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3258783936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.151:6839/1184770501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 1184770501}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 1184770501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24428, 24436, 24452]} 2024-09-12T01:45:49.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T01:45:49.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T01:45:50.181 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:50.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:50 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4201256902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:50.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:50 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4201256902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:50.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:50 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4201256902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.678+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3423269143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 msgr2=0x7fe618101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.678+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3423269143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe618101200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe608009900 tx=0x7fe60802a480 comp rx=0 tx=0).stop 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3423269143 shutdown_connections 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3423269143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102ef0 0x7fe61810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3423269143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe618102570 0x7fe6181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3423269143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe618101200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3423269143 >> 172.21.15.122:0/3423269143 conn(0x7fe6180fc7d0 msgr2=0x7fe6180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3423269143 shutdown_connections 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3423269143 wait complete. 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 Processor -- start 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- start start 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe6181993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102570 0x7fe618199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe618102ef0 0x7fe6181a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe618110ff0 con 0x7fe618100e20 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe618110e70 con 0x7fe618102570 2024-09-12T01:45:50.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61ee2f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111170 con 0x7fe618102ef0 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61cbcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe6181993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.679+0000 7fe61cbcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe6181993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40384/0 (socket says 172.21.15.122:40384) 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 -- 172.21.15.122:0/3103482660 learned_addr learned my addr 172.21.15.122:0/3103482660 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe618102ef0 0x7fe6181a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe617fff640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102570 0x7fe618199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 -- 172.21.15.122:0/3103482660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe618102ef0 msgr2=0x7fe6181a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe618102ef0 0x7fe6181a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 -- 172.21.15.122:0/3103482660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102570 msgr2=0x7fe618199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102570 0x7fe618199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 -- 172.21.15.122:0/3103482660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6181a4980 con 0x7fe618100e20 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe617fff640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102570 0x7fe618199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe6181993c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe608006010 tx=0x7fe608003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe615ffb640 1 -- 172.21.15.122:0/3103482660 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60803f070 con 0x7fe618100e20 2024-09-12T01:45:50.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe615ffb640 1 -- 172.21.15.122:0/3103482660 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe608003e30 con 0x7fe618100e20 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6181a4b50 con 0x7fe618100e20 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe615ffb640 1 -- 172.21.15.122:0/3103482660 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608035030 con 0x7fe618100e20 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.680+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a5000 con 0x7fe618100e20 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe615ffb640 1 -- 172.21.15.122:0/3103482660 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe608042070 con 0x7fe618100e20 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 Processor -- start 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 -- start start 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe615ffb640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6000a2c10 0x7fe6000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757ee9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ef8fc0 0x563757efb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563757e50090 con 0x563757ee9890 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563757df7a70 con 0x563757eea1b0 2024-09-12T01:45:50.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe621208740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563757e4c050 con 0x563757ef8fc0 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe61cbcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757ee9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.683+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6000a2c10 0x7fe6000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757ee9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40388/0 (socket says 172.21.15.122:40388) 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 -- 172.21.15.122:0/979136525 learned_addr learned my addr 172.21.15.122:0/979136525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe615ffb640 1 -- 172.21.15.122:0/3103482660 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe6080efc20 con 0x7fe618100e20 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe617fff640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ef8fc0 0x563757efb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 -- 172.21.15.122:0/979136525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ef8fc0 msgr2=0x563757efb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ef8fc0 0x563757efb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 -- 172.21.15.122:0/979136525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 msgr2=0x563757ef8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 -- 172.21.15.122:0/979136525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563757e04130 con 0x563757ee9890 2024-09-12T01:45:50.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe617fff640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ef8fc0 0x563757efb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6000a2c10 0x7fe6000a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe61000adf0 tx=0x7fe61000f790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757ee9c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b8cf0 tx=0x7fe608087440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca88d863124c95db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe5feffd640 1 -- 172.21.15.122:0/979136525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080bd450 con 0x563757ee9890 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe5feffd640 1 -- 172.21.15.122:0/979136525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6080c5e50 con 0x563757ee9890 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.684+0000 7fe5feffd640 1 -- 172.21.15.122:0/979136525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080c51e0 con 0x563757ee9890 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 -- 172.21.15.122:0/979136525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 msgr2=0x563757ee9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757ee9c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b8cf0 tx=0x7fe608087440 comp rx=0 tx=0).stop 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 -- 172.21.15.122:0/979136525 shutdown_connections 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ef8fc0 0x563757efb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 --2- 172.21.15.122:0/979136525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757ee9c70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 -- 172.21.15.122:0/979136525 >> 172.21.15.122:0/979136525 conn(0x563757ee5f60 msgr2=0x563757ee6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 -- 172.21.15.122:0/979136525 shutdown_connections 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 -- 172.21.15.122:0/979136525 wait complete. 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 -- start start 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.685+0000 7fe621208740 1 Processor -- start 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 0x563757ef6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563757efb950 con 0x563757ef8fc0 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563757e0af20 con 0x563757eea1b0 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563757efbad0 con 0x563757ee9890 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe621208740 10 client.0 _ll_register_callbacks cb 0x563757dd4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:45:50.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 0x563757ef6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 0x563757ef6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40402/0 (socket says 172.21.15.122:40402) 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 -- 172.21.15.122:0/1337197317 learned_addr learned my addr 172.21.15.122:0/1337197317 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe617fff640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 -- 172.21.15.122:0/1337197317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 msgr2=0x563757ef8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 -- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 msgr2=0x563757ef5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 -- 172.21.15.122:0/1337197317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563757bc12b0 con 0x563757ef8fc0 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe617fff640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:45:50.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 0x563757ef6100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe610010e00 tx=0x7fe610004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5abc072c75b18c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.686+0000 7fe5f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1337197317 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610022070 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1337197317 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe610004d20 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6080c5e50 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1337197317 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe610018070 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 10 client.14973 fetch_fsmap learned FSMap version 25 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563757f07200 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1337197317 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6080c5e50 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1337197317 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe610004f60 con 0x563757ef8fc0 2024-09-12T01:45:50.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 10 client.14973 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 10 client.14973 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 20 client.14973 populate_metadata read hostname 'smithi122' 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f098b0 0x563757f0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 4 client.14973 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563757e5d600 con 0x563757f098b0 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.687+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f098b0 0x563757f0ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.688+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f098b0 0x563757f0ad20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eac693947aef6962 server_cookie=51164215d441b622 in_seq=0 out_seq=0 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.688+0000 7fe5f7fff640 10 client.14973 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1337197317 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563757e5d600 con 0x563757f098b0 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe5f7fff640 10 client.14973 handle_command_reply: tid=0 2024-09-12T01:45:50.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 client.14973 shutdown 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 msgr2=0x563757ef6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 0x563757ef6100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe610010e00 tx=0x7fe610004ab0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 shutdown_connections 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f098b0 0x563757f0ad20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ef8fc0 0x563757ef6100 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757eea1b0 0x563757ef5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 --2- 172.21.15.122:0/1337197317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe5f7fff640 0 client.14973 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:45:50.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 >> 172.21.15.122:0/1337197317 conn(0x563757ee5f60 msgr2=0x563757eebcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:50.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 shutdown_connections 2024-09-12T01:45:50.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 1 -- 172.21.15.122:0/1337197317 wait complete. 2024-09-12T01:45:50.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.689+0000 7fe621208740 20 client.14973 trim_cache size 0 max 16384 2024-09-12T01:45:50.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 Processor -- start 2024-09-12T01:45:50.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 -- start start 2024-09-12T01:45:50.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 0x563757ef4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757ef49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563757f09060 con 0x563757eea1b0 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563757f09730 con 0x563757ef8fc0 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe621208740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563757f09e00 con 0x563757ee9890 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe617fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 0x563757ef4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe617fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 0x563757ef4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40414/0 (socket says 172.21.15.122:40414) 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.707+0000 7fe617fff640 1 -- 172.21.15.122:0/2135271733 learned_addr learned my addr 172.21.15.122:0/2135271733 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757ef49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe617fff640 1 -- 172.21.15.122:0/2135271733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 msgr2=0x563757ef5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe617fff640 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe617fff640 1 -- 172.21.15.122:0/2135271733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 msgr2=0x563757ef49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe617fff640 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757ef49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe617fff640 1 -- 172.21.15.122:0/2135271733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563757f0a2b0 con 0x563757eea1b0 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe617fff640 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 0x563757ef4490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00ea10 tx=0x7fe60c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fdf98a461d1055 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe5fe7fc640 1 -- 172.21.15.122:0/2135271733 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c01a020 con 0x563757eea1b0 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe5fe7fc640 1 -- 172.21.15.122:0/2135271733 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c016480 con 0x563757eea1b0 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.708+0000 7fe5fe7fc640 1 -- 172.21.15.122:0/2135271733 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c014700 con 0x563757eea1b0 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- 172.21.15.122:0/2135271733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 msgr2=0x563757ef4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 0x563757ef4490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00ea10 tx=0x7fe60c00eed0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- 172.21.15.122:0/2135271733 shutdown_connections 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757ef49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757eea1b0 0x563757ef4490 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- 172.21.15.122:0/2135271733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757ee9890 0x563757ef5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- 172.21.15.122:0/2135271733 >> 172.21.15.122:0/2135271733 conn(0x563757ee5f60 msgr2=0x563757ee6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- 172.21.15.122:0/2135271733 shutdown_connections 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- 172.21.15.122:0/2135271733 wait complete. 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- start start 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 Processor -- start 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757f131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757eea1b0 0x563757f13710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757f0c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563757f0aba0 con 0x563757ee9890 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563757f0a4d0 con 0x563757ef8fc0 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563757f0b270 con 0x563757eea1b0 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.709+0000 7fe621208740 10 client.0 _ll_register_callbacks cb 0x7fe61000ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757f131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757f131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40426/0 (socket says 172.21.15.122:40426) 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 -- 172.21.15.122:0/1069822044 learned_addr learned my addr 172.21.15.122:0/1069822044 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe617fff640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757eea1b0 0x563757f13710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61d3cc640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757f0c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 -- 172.21.15.122:0/1069822044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757eea1b0 msgr2=0x563757f13710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757eea1b0 0x563757f13710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 -- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 msgr2=0x563757f0c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757f0c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 -- 172.21.15.122:0/1069822044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563757f0cd30 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe61cbcb640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757f131d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b8780 tx=0x7fe60802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d8772f75911079 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe5f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1069822044 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608049040 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1069822044 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe60803b460 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60c016480 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1069822044 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6080bca20 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe621208740 10 client.14979 fetch_fsmap learned FSMap version 25 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563757f11780 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.710+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1069822044 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c016480 con 0x563757ee9890 2024-09-12T01:45:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1069822044 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe6080b8990 con 0x563757ee9890 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe621208740 10 client.14979 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe621208740 10 client.14979 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe621208740 20 client.14979 populate_metadata read hostname 'smithi122' 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe621208740 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f784a0 0x563757f771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe621208740 4 client.14979 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "status"}) -- 0x563757e57b10 con 0x563757f784a0 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe617fff640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f784a0 0x563757f771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe617fff640 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f784a0 0x563757f771b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3682b01921e46d70 server_cookie=ecb333351f04b140 in_seq=0 out_seq=0 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.711+0000 7fe5f7fff640 10 client.14979 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe5f7fff640 1 -- 172.21.15.122:0/1069822044 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563757e57b10 con 0x563757f784a0 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe5f7fff640 10 client.14979 handle_command_reply: tid=0 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 client.14979 shutdown 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 msgr2=0x563757f131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757f131d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe6080b8780 tx=0x7fe60802a480 comp rx=0 tx=0).stop 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 shutdown_connections 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x563757f784a0 0x563757f771b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe5f7fff640 0 client.14979 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563757ef8fc0 0x563757f0c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563757eea1b0 0x563757f13710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 --2- 172.21.15.122:0/1069822044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563757ee9890 0x563757f131d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 >> 172.21.15.122:0/1069822044 conn(0x563757ee5f60 msgr2=0x563757ee6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 shutdown_connections 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 1 -- 172.21.15.122:0/1069822044 wait complete. 2024-09-12T01:45:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.712+0000 7fe621208740 20 client.14979 trim_cache size 0 max 16384 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24428, 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 65.901705672000006, 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 25, 2024-09-12T01:45:50.723 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:45:50.724 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:45:50.724 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 96.033646969000003 2024-09-12T01:45:50.724 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:45:50.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6000a2c10 msgr2=0x7fe6000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6000a2c10 0x7fe6000a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe61000adf0 tx=0x7fe61000f790 comp rx=0 tx=0).stop 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 msgr2=0x7fe6181993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe6181993c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe608006010 tx=0x7fe608003c10 comp rx=0 tx=0).stop 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 shutdown_connections 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe6000a2c10 0x7fe6000a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe618102ef0 0x7fe6181a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe618102570 0x7fe618199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 --2- 172.21.15.122:0/3103482660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe618100e20 0x7fe6181993c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 >> 172.21.15.122:0/3103482660 conn(0x7fe6180fc7d0 msgr2=0x7fe6180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 shutdown_connections 2024-09-12T01:45:50.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:50.713+0000 7fe61ee2f640 1 -- 172.21.15.122:0/3103482660 wait complete. 2024-09-12T01:45:51.421 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:51 smithi122 ceph-mon[30925]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:51.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T01:45:51.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:51 smithi183 ceph-mon[35127]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:51.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:51 smithi151 ceph-mon[35229]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:45:51.692 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:52.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3026668968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc101de0 msgr2=0x7f78dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3026668968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc101de0 0x7f78dc1021c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009900 tx=0x7f78d002a410 comp rx=0 tx=0).stop 2024-09-12T01:45:52.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3026668968 shutdown_connections 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3026668968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc106f10 0x7f78dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3026668968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 0x7f78dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3026668968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc101de0 0x7f78dc1021c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3026668968 >> 172.21.15.122:0/3026668968 conn(0x7f78dc0fd8c0 msgr2=0x7f78dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3026668968 shutdown_connections 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.356+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3026668968 wait complete. 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 Processor -- start 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 -- start start 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc112010 con 0x7f78dc106f10 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc111e90 con 0x7f78dc102790 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e2e1d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc112190 con 0x7f78dc101de0 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40454/0 (socket says 172.21.15.122:40454) 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 -- 172.21.15.122:0/3647514153 learned_addr learned my addr 172.21.15.122:0/3647514153 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 -- 172.21.15.122:0/3647514153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc101de0 msgr2=0x7f78dc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78dbfff640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 -- 172.21.15.122:0/3647514153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 msgr2=0x7f78dc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e13ba640 1 -- 172.21.15.122:0/3647514153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc19d560 con 0x7f78dc106f10 2024-09-12T01:45:52.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.357+0000 7f78dbfff640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.358+0000 7f78e13ba640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78d400b970 tx=0x7f78d400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.358+0000 7f78d9ffb640 1 -- 172.21.15.122:0/3647514153 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4019070 con 0x7f78dc106f10 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.358+0000 7f78d9ffb640 1 -- 172.21.15.122:0/3647514153 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d400cb40 con 0x7f78dc106f10 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.358+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc19d850 con 0x7f78dc106f10 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.358+0000 7f78d9ffb640 1 -- 172.21.15.122:0/3647514153 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4007650 con 0x7f78dc106f10 2024-09-12T01:45:52.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.358+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc1a5130 con 0x7f78dc106f10 2024-09-12T01:45:52.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.360+0000 7f78d9ffb640 1 -- 172.21.15.122:0/3647514153 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f78d401c070 con 0x7f78dc106f10 2024-09-12T01:45:52.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.361+0000 7f78d9ffb640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f78cc0a2c90 0x7f78cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.361+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f78cc0a2c90 0x7f78cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.361+0000 7f78d9ffb640 1 -- 172.21.15.122:0/3647514153 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f78d40ce6a0 con 0x7f78dc106f10 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.361+0000 7f78e51f6740 1 Processor -- start 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.361+0000 7f78e51f6740 1 -- start start 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x558734027db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x558734032670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x558734034fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e51f6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558733f8e090 con 0x5587340282f0 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f78cc0a2c90 0x7f78cc0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000c00 tx=0x7f78d0032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e51f6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558733f35a70 con 0x5587340279d0 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e51f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558733f8a050 con 0x558734032bb0 2024-09-12T01:45:52.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x558734032670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x558734032670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40462/0 (socket says 172.21.15.122:40462) 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 -- 172.21.15.122:0/1915644719 learned_addr learned my addr 172.21.15.122:0/1915644719 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e13ba640 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x558734034fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 -- 172.21.15.122:0/1915644719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 msgr2=0x558734034fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x558734027db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x558734034fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 -- 172.21.15.122:0/1915644719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 msgr2=0x558734027db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x558734027db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 -- 172.21.15.122:0/1915644719 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558733f42130 con 0x5587340282f0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78dbfff640 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x558734032670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f78c800a9d0 tx=0x7f78c800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa21ab13884fe52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.362+0000 7f78c6ffd640 1 -- 172.21.15.122:0/1915644719 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8015070 con 0x5587340282f0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78c6ffd640 1 -- 172.21.15.122:0/1915644719 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c800cc60 con 0x5587340282f0 2024-09-12T01:45:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78c6ffd640 1 -- 172.21.15.122:0/1915644719 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c80105b0 con 0x5587340282f0 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 -- 172.21.15.122:0/1915644719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 msgr2=0x558734032670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x558734032670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f78c800a9d0 tx=0x7f78c800ae90 comp rx=0 tx=0).stop 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 -- 172.21.15.122:0/1915644719 shutdown_connections 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x558734034fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x558734032670 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1915644719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x558734027db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 -- 172.21.15.122:0/1915644719 >> 172.21.15.122:0/1915644719 conn(0x558734023f60 msgr2=0x558734024340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 -- 172.21.15.122:0/1915644719 shutdown_connections 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.363+0000 7f78e51f6740 1 -- 172.21.15.122:0/1915644719 wait complete. 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 -- start start 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 Processor -- start 2024-09-12T01:45:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587340279d0 0x5587340323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558734032bb0 0x55873402fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558734035780 con 0x5587340282f0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558733f48f20 con 0x558734032bb0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558734035900 con 0x5587340279d0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e51f6740 10 client.0 _ll_register_callbacks cb 0x558733f12030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40474/0 (socket says 172.21.15.122:40474) 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78dbfff640 1 -- 172.21.15.122:0/3864504941 learned_addr learned my addr 172.21.15.122:0/3864504941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587340279d0 0x5587340323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.364+0000 7f78e13ba640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558734032bb0 0x55873402fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78dbfff640 1 -- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587340279d0 msgr2=0x5587340323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78dbfff640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587340279d0 0x5587340323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78dbfff640 1 -- 172.21.15.122:0/3864504941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558734032bb0 msgr2=0x55873402fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78dbfff640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558734032bb0 0x55873402fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78dbfff640 1 -- 172.21.15.122:0/3864504941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558733cff2b0 con 0x5587340282f0 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78dbfff640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402f750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f78c8008010 tx=0x7f78c8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa71e78eb01237 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78bbfff640 1 -- 172.21.15.122:0/3864504941 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8020070 con 0x5587340282f0 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78bbfff640 1 -- 172.21.15.122:0/3864504941 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f78c801a450 con 0x5587340282f0 2024-09-12T01:45:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78c800cc60 con 0x5587340282f0 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78bbfff640 1 -- 172.21.15.122:0/3864504941 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78c8015040 con 0x5587340282f0 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78e51f6740 10 client.14991 fetch_fsmap learned FSMap version 25 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558734041040 con 0x5587340282f0 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78bbfff640 1 -- 172.21.15.122:0/3864504941 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c800cc60 con 0x5587340282f0 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.365+0000 7f78bbfff640 1 -- 172.21.15.122:0/3864504941 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78c802d050 con 0x5587340282f0 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e51f6740 10 client.14991 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e51f6740 10 client.14991 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e51f6740 20 client.14991 populate_metadata read hostname 'smithi122' 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e51f6740 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558734043260 0x5587340446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e51f6740 4 client.14991 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558733f9b600 con 0x558734043260 2024-09-12T01:45:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558734043260 0x5587340446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78bbfff640 10 client.14991 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.366+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558734043260 0x5587340446d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26d82449550c833e server_cookie=5e1ab5e25ecce219 in_seq=0 out_seq=0 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78bbfff640 1 -- 172.21.15.122:0/3864504941 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558733f9b600 con 0x558734043260 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78bbfff640 10 client.14991 handle_command_reply: tid=0 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 client.14991 shutdown 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 msgr2=0x55873402f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402f750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f78c8008010 tx=0x7f78c8011ba0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 shutdown_connections 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558734043260 0x5587340446d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558734032bb0 0x55873402fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.367+0000 7f78e51f6740 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402f750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.368+0000 7f78e51f6740 1 --2- 172.21.15.122:0/3864504941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587340279d0 0x5587340323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.368+0000 7f78bbfff640 0 client.14991 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:45:52.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.368+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 >> 172.21.15.122:0/3864504941 conn(0x558734023f60 msgr2=0x558734029c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:52.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.368+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 shutdown_connections 2024-09-12T01:45:52.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.368+0000 7f78e51f6740 1 -- 172.21.15.122:0/3864504941 wait complete. 2024-09-12T01:45:52.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.368+0000 7f78e51f6740 20 client.14991 trim_cache size 0 max 16384 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 Processor -- start 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 -- start start 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x5587340446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x55873402e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558734042a10 con 0x5587340282f0 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587340430e0 con 0x5587340279d0 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78e51f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587340437b0 con 0x558734032bb0 2024-09-12T01:45:52.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40478/0 (socket says 172.21.15.122:40478) 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.385+0000 7f78dbfff640 1 -- 172.21.15.122:0/1196990708 learned_addr learned my addr 172.21.15.122:0/1196990708 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78dbfff640 1 -- 172.21.15.122:0/1196990708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 msgr2=0x55873402e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78dbfff640 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x55873402e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78dbfff640 1 -- 172.21.15.122:0/1196990708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 msgr2=0x5587340446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78dbfff640 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x5587340446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78dbfff640 1 -- 172.21.15.122:0/1196990708 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558734028120 con 0x5587340282f0 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78dbfff640 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402def0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f78c800aed0 tx=0x7f78c8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af6a93323e5000 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78c67fc640 1 -- 172.21.15.122:0/1196990708 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8020040 con 0x5587340282f0 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78c67fc640 1 -- 172.21.15.122:0/1196990708 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c8033020 con 0x5587340282f0 2024-09-12T01:45:52.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78c67fc640 1 -- 172.21.15.122:0/1196990708 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8000e60 con 0x5587340282f0 2024-09-12T01:45:52.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78e51f6740 1 -- 172.21.15.122:0/1196990708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 msgr2=0x55873402def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.386+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402def0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f78c800aed0 tx=0x7f78c8011f90 comp rx=0 tx=0).stop 2024-09-12T01:45:52.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- 172.21.15.122:0/1196990708 shutdown_connections 2024-09-12T01:45:52.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x55873402e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873402def0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 --2- 172.21.15.122:0/1196990708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x5587340446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- 172.21.15.122:0/1196990708 >> 172.21.15.122:0/1196990708 conn(0x558734023f60 msgr2=0x55873402adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- 172.21.15.122:0/1196990708 shutdown_connections 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- 172.21.15.122:0/1196990708 wait complete. 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- start start 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 Processor -- start 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x55873404c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873404c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x5587340a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558734044550 con 0x5587340282f0 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558734043e80 con 0x5587340279d0 2024-09-12T01:45:52.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558734044c20 con 0x558734032bb0 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78e51f6740 10 client.0 _ll_register_callbacks cb 0x7f78c800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.387+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873404c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873404c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40482/0 (socket says 172.21.15.122:40482) 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 -- 172.21.15.122:0/9290931 learned_addr learned my addr 172.21.15.122:0/9290931 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x55873404c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78e13ba640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x5587340a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 -- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 msgr2=0x5587340a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x5587340a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 -- 172.21.15.122:0/9290931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 msgr2=0x55873404c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x55873404c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 -- 172.21.15.122:0/9290931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587340aa000 con 0x5587340282f0 2024-09-12T01:45:52.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78e13ba640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x5587340a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78dbfff640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873404c870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f78c8004700 tx=0x7f78c800a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb8992d03c11cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78bbfff640 1 -- 172.21.15.122:0/9290931 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8020070 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78bbfff640 1 -- 172.21.15.122:0/9290931 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f78c801a660 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78c8033020 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78bbfff640 1 -- 172.21.15.122:0/9290931 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78c802ed30 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78e51f6740 10 client.14997 fetch_fsmap learned FSMap version 25 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587340b1420 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.388+0000 7f78bbfff640 1 -- 172.21.15.122:0/9290931 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c8033020 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78bbfff640 1 -- 172.21.15.122:0/9290931 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78c8041050 con 0x5587340282f0 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 10 client.14997 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:45:52.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 10 client.14997 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 20 client.14997 populate_metadata read hostname 'smithi122' 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5587340b5c00 0x5587340b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 4 client.14997 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "status"}) -- 0x558733f95b10 con 0x5587340b5c00 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5587340b5c00 0x5587340b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78bbfff640 10 client.14997 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e0bb9640 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5587340b5c00 0x5587340b4910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e200ba3fb3880ed0 server_cookie=344c465ea60f6bc2 in_seq=0 out_seq=0 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78bbfff640 1 -- 172.21.15.122:0/9290931 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558733f95b10 con 0x5587340b5c00 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78bbfff640 10 client.14997 handle_command_reply: tid=0 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.389+0000 7f78e51f6740 1 client.14997 shutdown 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 msgr2=0x55873404c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873404c870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f78c8004700 tx=0x7f78c800a940 comp rx=0 tx=0).stop 2024-09-12T01:45:52.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 shutdown_connections 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5587340b5c00 0x5587340b4910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558734032bb0 0x5587340a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587340282f0 0x55873404c870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 --2- 172.21.15.122:0/9290931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587340279d0 0x55873404c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 >> 172.21.15.122:0/9290931 conn(0x558734023f60 msgr2=0x5587340227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78bbfff640 0 client.14997 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 shutdown_connections 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 1 -- 172.21.15.122:0/9290931 wait complete. 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e51f6740 20 client.14997 trim_cache size 0 max 16384 2024-09-12T01:45:52.396 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24436, 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 68.590640113000006, 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 25, 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 100.485495032 2024-09-12T01:45:52.397 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:45:52.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f78cc0a2c90 msgr2=0x7f78cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f78cc0a2c90 0x7f78cc0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000c00 tx=0x7f78d0032d50 comp rx=0 tx=0).stop 2024-09-12T01:45:52.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 msgr2=0x7f78dc19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:52.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.390+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78d400b970 tx=0x7f78d400be30 comp rx=0 tx=0).stop 2024-09-12T01:45:52.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 shutdown_connections 2024-09-12T01:45:52.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f78cc0a2c90 0x7f78cc0a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc106f10 0x7f78dc19ce80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78dc102790 0x7f78dc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 --2- 172.21.15.122:0/3647514153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78dc101de0 0x7f78dc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:52.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 >> 172.21.15.122:0/3647514153 conn(0x7f78dc0fd8c0 msgr2=0x7f78dc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:52.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 shutdown_connections 2024-09-12T01:45:52.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:52.391+0000 7f78e2e1d640 1 -- 172.21.15.122:0/3647514153 wait complete. 2024-09-12T01:45:53.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T01:45:53.386 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:53.781 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:53 smithi122 ceph-mon[30925]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:53.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:53 smithi151 ceph-mon[35229]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:53.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 -- 172.21.15.122:0/3338314165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044102de0 msgr2=0x7f50441031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 --2- 172.21.15.122:0/3338314165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044102de0 0x7f50441031c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f503402a480 comp rx=0 tx=0).stop 2024-09-12T01:45:53.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 -- 172.21.15.122:0/3338314165 shutdown_connections 2024-09-12T01:45:53.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 --2- 172.21.15.122:0/3338314165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504410c180 0x7f504410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 --2- 172.21.15.122:0/3338314165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5044103700 0x7f504410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 --2- 172.21.15.122:0/3338314165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044102de0 0x7f50441031c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 -- 172.21.15.122:0/3338314165 >> 172.21.15.122:0/3338314165 conn(0x7f50440fc7a0 msgr2=0x7f50440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 -- 172.21.15.122:0/3338314165 shutdown_connections 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.861+0000 7f5049038640 1 -- 172.21.15.122:0/3338314165 wait complete. 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 Processor -- start 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 -- start start 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044102de0 0x7f5044198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 0x7f50441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f504410c180 0x7f504419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5044111fb0 con 0x7f5044103700 2024-09-12T01:45:53.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5044111e30 con 0x7f5044102de0 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f5049038640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5044112130 con 0x7f504410c180 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.862+0000 7f504259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 0x7f50441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 0x7f50441989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40490/0 (socket says 172.21.15.122:40490) 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 -- 172.21.15.122:0/2075011124 learned_addr learned my addr 172.21.15.122:0/2075011124 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f5042d9d640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044102de0 0x7f5044198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504359e640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f504410c180 0x7f504419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 -- 172.21.15.122:0/2075011124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f504410c180 msgr2=0x7f504419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f504410c180 0x7f504419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 -- 172.21.15.122:0/2075011124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044102de0 msgr2=0x7f5044198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044102de0 0x7f5044198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 -- 172.21.15.122:0/2075011124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504419d420 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f504259c640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 0x7f50441989b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f503800b4a0 tx=0x7f503800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f502bfff640 1 -- 172.21.15.122:0/2075011124 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038014070 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504419d710 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f502bfff640 1 -- 172.21.15.122:0/2075011124 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5038010af0 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f502bfff640 1 -- 172.21.15.122:0/2075011124 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038017d10 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.863+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50441a4ff0 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f502bfff640 1 -- 172.21.15.122:0/2075011124 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5038017070 con 0x7f5044103700 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 Processor -- start 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 -- start start 2024-09-12T01:45:53.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a38b0 0x55ed720a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720a41d0 0x55ed720b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 0x55ed720b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ed7200a090 con 0x55ed720b2fc0 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f502bfff640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f502c0a2c10 0x7f502c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed71fb1a70 con 0x55ed720a38b0 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504b411740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed72006050 con 0x55ed720a41d0 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 0x55ed720b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504359e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a38b0 0x55ed720a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504359e640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f502c0a2c10 0x7f502c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 0x55ed720b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40500/0 (socket says 172.21.15.122:40500) 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f504259c640 1 -- 172.21.15.122:0/3433080824 learned_addr learned my addr 172.21.15.122:0/3433080824 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.866+0000 7f5042d9d640 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720a41d0 0x55ed720b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f502bfff640 1 -- 172.21.15.122:0/2075011124 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f50380ce450 con 0x7f5044103700 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504259c640 1 -- 172.21.15.122:0/3433080824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720a41d0 msgr2=0x55ed720b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504259c640 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720a41d0 0x55ed720b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504259c640 1 -- 172.21.15.122:0/3433080824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a38b0 msgr2=0x55ed720a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504259c640 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a38b0 0x55ed720a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504259c640 1 -- 172.21.15.122:0/3433080824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed71fbe130 con 0x55ed720b2fc0 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504359e640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f502c0a2c10 0x7f502c0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f503c005fd0 tx=0x7f503c009970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:53.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f504259c640 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 0x55ed720b53b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5038097120 tx=0x7f50380047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151afe846a0bb71a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f50297fa640 1 -- 172.21.15.122:0/3433080824 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503809f010 con 0x55ed720b2fc0 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f50297fa640 1 -- 172.21.15.122:0/3433080824 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380a38d0 con 0x55ed720b2fc0 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.867+0000 7f50297fa640 1 -- 172.21.15.122:0/3433080824 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038010ca0 con 0x55ed720b2fc0 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- 172.21.15.122:0/3433080824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 msgr2=0x55ed720b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 0x55ed720b53b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5038097120 tx=0x7f50380047d0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- 172.21.15.122:0/3433080824 shutdown_connections 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720b2fc0 0x55ed720b53b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720a41d0 0x55ed720b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- 172.21.15.122:0/3433080824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a38b0 0x55ed720a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- 172.21.15.122:0/3433080824 >> 172.21.15.122:0/3433080824 conn(0x55ed7209ff60 msgr2=0x55ed720a0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- 172.21.15.122:0/3433080824 shutdown_connections 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- 172.21.15.122:0/3433080824 wait complete. 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- start start 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 Processor -- start 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ed720b5b90 con 0x55ed720a38b0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed71fc4f20 con 0x55ed720a41d0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed720b5d10 con 0x55ed720b2fc0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f504b411740 10 client.0 _ll_register_callbacks cb 0x55ed71f8e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f5042d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f5042d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40506/0 (socket says 172.21.15.122:40506) 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.868+0000 7f5042d9d640 1 -- 172.21.15.122:0/4022868651 learned_addr learned my addr 172.21.15.122:0/4022868651 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f504259c640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f504359e640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5042d9d640 1 -- 172.21.15.122:0/4022868651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 msgr2=0x55ed720b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5042d9d640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5042d9d640 1 -- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 msgr2=0x55ed720afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5042d9d640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5042d9d640 1 -- 172.21.15.122:0/4022868651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed71d7b2b0 con 0x55ed720a38b0 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f504359e640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5042d9d640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720b28a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f5034031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b27874793dca74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5026ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5026ff5640 1 -- 172.21.15.122:0/4022868651 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034035040 con 0x55ed720a38b0 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5026ff5640 1 -- 172.21.15.122:0/4022868651 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5034004240 con 0x55ed720a38b0 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50380a38d0 con 0x55ed720a38b0 2024-09-12T01:45:53.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5026ff5640 1 -- 172.21.15.122:0/4022868651 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5034004440 con 0x55ed720a38b0 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f504b411740 10 client.15009 fetch_fsmap learned FSMap version 25 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed720c1540 con 0x55ed720a38b0 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.869+0000 7f5026ff5640 1 -- 172.21.15.122:0/4022868651 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380a38d0 con 0x55ed720a38b0 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f5026ff5640 1 -- 172.21.15.122:0/4022868651 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50340035f0 con 0x55ed720a38b0 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504b411740 10 client.15009 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504b411740 10 client.15009 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504b411740 20 client.15009 populate_metadata read hostname 'smithi122' 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504b411740 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed720c3ba0 0x55ed720c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504b411740 4 client.15009 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:45:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed72017600 con 0x55ed720c3ba0 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504259c640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed720c3ba0 0x55ed720c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f504259c640 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed720c3ba0 0x55ed720c5010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819122e7b8d2dddd server_cookie=49a9733700f1c50e in_seq=0 out_seq=0 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.870+0000 7f5026ff5640 10 client.15009 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f5026ff5640 1 -- 172.21.15.122:0/4022868651 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed72017600 con 0x55ed720c3ba0 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f5026ff5640 10 client.15009 handle_command_reply: tid=0 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 client.15009 shutdown 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 msgr2=0x55ed720b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720b28a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f5034031920 comp rx=0 tx=0).stop 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 shutdown_connections 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed720c3ba0 0x55ed720c5010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 --2- 172.21.15.122:0/4022868651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720b28a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f5026ff5640 0 client.15009 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 >> 172.21.15.122:0/4022868651 conn(0x55ed7209ff60 msgr2=0x55ed720a5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 shutdown_connections 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 1 -- 172.21.15.122:0/4022868651 wait complete. 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.871+0000 7f504b411740 20 client.15009 trim_cache size 0 max 16384 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 Processor -- start 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 -- start start 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ed720c3350 con 0x55ed720a38b0 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed720c3a20 con 0x55ed720a41d0 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504b411740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed720c40f0 con 0x55ed720b2fc0 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40522/0 (socket says 172.21.15.122:40522) 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 -- 172.21.15.122:0/1147230808 learned_addr learned my addr 172.21.15.122:0/1147230808 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f504359e640 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 -- 172.21.15.122:0/1147230808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 msgr2=0x55ed720ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 -- 172.21.15.122:0/1147230808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 msgr2=0x55ed720ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.889+0000 7f5042d9d640 1 -- 172.21.15.122:0/1147230808 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed720b0f70 con 0x55ed720a38b0 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f5042d9d640 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720ab130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5034000c00 tx=0x7f50340038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934f7b9c413d337 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f5028ff9640 1 -- 172.21.15.122:0/1147230808 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034035070 con 0x55ed720a38b0 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f5028ff9640 1 -- 172.21.15.122:0/1147230808 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5034003c30 con 0x55ed720a38b0 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f5028ff9640 1 -- 172.21.15.122:0/1147230808 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034004ff0 con 0x55ed720a38b0 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 -- 172.21.15.122:0/1147230808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 msgr2=0x55ed720ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720ab130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5034000c00 tx=0x7f50340038b0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 -- 172.21.15.122:0/1147230808 shutdown_connections 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 --2- 172.21.15.122:0/1147230808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720ab130 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 -- 172.21.15.122:0/1147230808 >> 172.21.15.122:0/1147230808 conn(0x55ed7209ff60 msgr2=0x55ed720aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 -- 172.21.15.122:0/1147230808 shutdown_connections 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.890+0000 7f504b411740 1 -- 172.21.15.122:0/1147230808 wait complete. 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 -- start start 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 Processor -- start 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ed720c4e90 con 0x55ed720a38b0 2024-09-12T01:45:53.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed720c47c0 con 0x55ed720a41d0 2024-09-12T01:45:53.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed720c5560 con 0x55ed720b2fc0 2024-09-12T01:45:53.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f504b411740 10 client.0 _ll_register_callbacks cb 0x7f50340315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:45:53.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40530/0 (socket says 172.21.15.122:40530) 2024-09-12T01:45:53.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 -- 172.21.15.122:0/2242171228 learned_addr learned my addr 172.21.15.122:0/2242171228 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 -- 172.21.15.122:0/2242171228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 msgr2=0x55ed720c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 -- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 msgr2=0x55ed720cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.891+0000 7f5042d9d640 1 -- 172.21.15.122:0/2242171228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed720c7410 con 0x55ed720a38b0 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5042d9d640 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720cd6b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5034031bf0 tx=0x7f5034003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b601b35f7e1ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5026ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5026ff5640 1 -- 172.21.15.122:0/2242171228 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503405f060 con 0x55ed720a38b0 2024-09-12T01:45:53.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5034003c30 con 0x55ed720a38b0 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5026ff5640 1 -- 172.21.15.122:0/2242171228 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5034005050 con 0x55ed720a38b0 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5026ff5640 1 -- 172.21.15.122:0/2242171228 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5034035030 con 0x55ed720a38b0 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 10 client.15015 fetch_fsmap learned FSMap version 25 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed720cbe80 con 0x55ed720a38b0 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5026ff5640 1 -- 172.21.15.122:0/2242171228 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5034003c30 con 0x55ed720a38b0 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f5026ff5640 1 -- 172.21.15.122:0/2242171228 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5034064070 con 0x55ed720a38b0 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 10 client.15015 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 10 client.15015 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 20 client.15015 populate_metadata read hostname 'smithi122' 2024-09-12T01:45:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed72132820 0x55ed72131530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 4 client.15015 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.892+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "status"}) -- 0x55ed72011b10 con 0x55ed72132820 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504259c640 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed72132820 0x55ed72131530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f5026ff5640 10 client.15015 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504259c640 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed72132820 0x55ed72131530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6043d8348c7c74 server_cookie=ff74c16f96c0128c in_seq=0 out_seq=0 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f5026ff5640 1 -- 172.21.15.122:0/2242171228 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ed72011b10 con 0x55ed72132820 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f5026ff5640 10 client.15015 handle_command_reply: tid=0 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 client.15015 shutdown 2024-09-12T01:45:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 msgr2=0x55ed720cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720cd6b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5034031bf0 tx=0x7f5034003de0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 shutdown_connections 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed72132820 0x55ed72131530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed720b2fc0 0x55ed720c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed720a41d0 0x55ed720cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f5026ff5640 0 client.15015 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 --2- 172.21.15.122:0/2242171228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed720a38b0 0x55ed720cd6b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 >> 172.21.15.122:0/2242171228 conn(0x55ed7209ff60 msgr2=0x55ed720ab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 shutdown_connections 2024-09-12T01:45:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 1 -- 172.21.15.122:0/2242171228 wait complete. 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.893+0000 7f504b411740 20 client.15015 trim_cache size 0 max 16384 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24452, 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:45:53.916 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 77.124381413999998, 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 25, 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 87.719882663999996 2024-09-12T01:45:53.917 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f502c0a2c10 msgr2=0x7f502c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f502c0a2c10 0x7f502c0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f503c005fd0 tx=0x7f503c009970 comp rx=0 tx=0).stop 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 msgr2=0x7f50441989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 0x7f50441989b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f503800b4a0 tx=0x7f503800b960 comp rx=0 tx=0).stop 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 shutdown_connections 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f502c0a2c10 0x7f502c0a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f504410c180 0x7f504419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5044103700 0x7f50441989b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 --2- 172.21.15.122:0/2075011124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5044102de0 0x7f5044198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 >> 172.21.15.122:0/2075011124 conn(0x7f50440fc7a0 msgr2=0x7f50440fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 shutdown_connections 2024-09-12T01:45:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:53.894+0000 7f5049038640 1 -- 172.21.15.122:0/2075011124 wait complete. 2024-09-12T01:45:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:53 smithi183 ceph-mon[35127]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:54.639 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T01:45:54.640 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T01:45:54.651 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T01:45:54.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T01:45:54.894 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:55.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 -- 172.21.15.122:0/2270731039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 msgr2=0x7ff45810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 --2- 172.21.15.122:0/2270731039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff45810a050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff450009ca0 tx=0x7ff45002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 -- 172.21.15.122:0/2270731039 shutdown_connections 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 --2- 172.21.15.122:0/2270731039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff45810a050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 --2- 172.21.15.122:0/2270731039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458102e00 0x7ff458103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 --2- 172.21.15.122:0/2270731039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff458101c00 0x7ff458102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 -- 172.21.15.122:0/2270731039 >> 172.21.15.122:0/2270731039 conn(0x7ff4580fd390 msgr2=0x7ff4580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 -- 172.21.15.122:0/2270731039 shutdown_connections 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 -- 172.21.15.122:0/2270731039 wait complete. 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.346+0000 7ff45d927640 1 Processor -- start 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 -- start start 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff458101c00 0x7ff458075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:55.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458102e00 0x7ff4580745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff458074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff45810d860 con 0x7ff4581037c0 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff45810d6e0 con 0x7ff458102e00 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff45d927640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff45810d9e0 con 0x7ff458101c00 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff458074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff458074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40546/0 (socket says 172.21.15.122:40546) 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 -- 172.21.15.122:0/2212335552 learned_addr learned my addr 172.21.15.122:0/2212335552 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff456ffd640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff458101c00 0x7ff458075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4567fc640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458102e00 0x7ff4580745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 -- 172.21.15.122:0/2212335552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff458101c00 msgr2=0x7ff458075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:55.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff458101c00 0x7ff458075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 -- 172.21.15.122:0/2212335552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458102e00 msgr2=0x7ff4580745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458102e00 0x7ff4580745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.347+0000 7ff4577fe640 1 -- 172.21.15.122:0/2212335552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4580753e0 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.348+0000 7ff4577fe640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff458074b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff450005e30 tx=0x7ff450031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.348+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450003c30 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.348+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff450003dd0 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.348+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4581a1f00 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.348+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45002fde0 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.348+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4581a23b0 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.349+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff458108f30 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.350+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff450004050 con 0x7ff4581037c0 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.351+0000 7ff45c925640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4440a2c10 0x7ff4440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.351+0000 7ff456ffd640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4440a2c10 0x7ff4440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:55.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.351+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff4500f0470 con 0x7ff4581037c0 2024-09-12T01:45:55.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.351+0000 7ff456ffd640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4440a2c10 0x7ff4440a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff448005fd0 tx=0x7ff44800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:55.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.355+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4500bd380 con 0x7ff4581037c0 2024-09-12T01:45:55.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.538+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff458062630 con 0x7ff4440a2c10 2024-09-12T01:45:55.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.539+0000 7ff45c925640 1 -- 172.21.15.122:0/2212335552 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff458062630 con 0x7ff4440a2c10 2024-09-12T01:45:55.539 INFO:teuthology.orchestra.run.smithi122.stdout:Backend: cephadm 2024-09-12T01:45:55.539 INFO:teuthology.orchestra.run.smithi122.stdout:Available: Yes 2024-09-12T01:45:55.539 INFO:teuthology.orchestra.run.smithi122.stdout:Paused: No 2024-09-12T01:45:55.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4440a2c10 msgr2=0x7ff4440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:55.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4440a2c10 0x7ff4440a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff448005fd0 tx=0x7ff44800b410 comp rx=0 tx=0).stop 2024-09-12T01:45:55.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 msgr2=0x7ff458074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff458074b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff450005e30 tx=0x7ff450031ee0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 shutdown_connections 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4440a2c10 0x7ff4440a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4581037c0 0x7ff458074b20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458102e00 0x7ff4580745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 --2- 172.21.15.122:0/2212335552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff458101c00 0x7ff458075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 >> 172.21.15.122:0/2212335552 conn(0x7ff4580fd390 msgr2=0x7ff458106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 shutdown_connections 2024-09-12T01:45:55.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:55.541+0000 7ff45d927640 1 -- 172.21.15.122:0/2212335552 wait complete. 2024-09-12T01:45:55.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:55 smithi151 ceph-mon[35229]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:45:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:55 smithi183 ceph-mon[35127]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:45:55.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:55 smithi122 ceph-mon[30925]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:45:56.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T01:45:56.463 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:56.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:56 smithi122 ceph-mon[30925]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:45:56.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:56 smithi151 ceph-mon[35229]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 -- 172.21.15.122:0/2148938174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c103870 msgr2=0x7f968c10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 --2- 172.21.15.122:0/2148938174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c103870 0x7f968c10a100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9684009cd0 tx=0x7f968402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 -- 172.21.15.122:0/2148938174 shutdown_connections 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 --2- 172.21.15.122:0/2148938174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c103870 0x7f968c10a100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 --2- 172.21.15.122:0/2148938174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f968c102eb0 0x7f968c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 --2- 172.21.15.122:0/2148938174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f968c0ffed0 0x7f968c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 -- 172.21.15.122:0/2148938174 >> 172.21.15.122:0/2148938174 conn(0x7f968c0fba40 msgr2=0x7f968c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 -- 172.21.15.122:0/2148938174 shutdown_connections 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.899+0000 7f9692565640 1 -- 172.21.15.122:0/2148938174 wait complete. 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 Processor -- start 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 -- start start 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 0x7f968c102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f968c102eb0 0x7f968c100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f968c103870 0x7f968c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10d890 con 0x7f968c0ffed0 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10d710 con 0x7f968c102eb0 2024-09-12T01:45:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9692565640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10da10 con 0x7f968c103870 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 0x7f968c102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 0x7f968c102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40568/0 (socket says 172.21.15.122:40568) 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 -- 172.21.15.122:0/3932613212 learned_addr learned my addr 172.21.15.122:0/3932613212 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f9690b02640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f968c103870 0x7f968c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968b7fe640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f968c102eb0 0x7f968c100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 -- 172.21.15.122:0/3932613212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f968c103870 msgr2=0x7f968c1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f968c103870 0x7f968c1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 -- 172.21.15.122:0/3932613212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f968c102eb0 msgr2=0x7f968c100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f968c102eb0 0x7f968c100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.900+0000 7f968bfff640 1 -- 172.21.15.122:0/3932613212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c101c60 con 0x7f968c0ffed0 2024-09-12T01:45:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.901+0000 7f968bfff640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 0x7f968c102810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f967c00c970 tx=0x7f967c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.901+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c004220 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.901+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f967c0043c0 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.901+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00f450 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.901+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c1a1fb0 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.901+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1a2410 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.902+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c109080 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.903+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f967c0079b0 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.904+0000 7f96897fa640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96740a2c10 0x7f96740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.904+0000 7f968b7fe640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96740a2c10 0x7f96740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.904+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f967c0cf3a0 con 0x7f968c0ffed0 2024-09-12T01:45:56.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.904+0000 7f968b7fe640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96740a2c10 0x7f96740a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9680005e10 tx=0x7f9680005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:56.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:56.907+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f967c014030 con 0x7f968c0ffed0 2024-09-12T01:45:56.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:56 smithi183 ceph-mon[35127]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:45:57.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.081+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f968c062630 con 0x7f96740a2c10 2024-09-12T01:45:57.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.094+0000 7f96897fa640 1 -- 172.21.15.122:0/3932613212 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f968c062630 con 0x7f96740a2c10 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.a smithi122 running (93s) 17s ago 93s 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 b2eb61ef28b7 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.b smithi151 running (91s) 24s ago 90s 28.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 f3305815c64c 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.c smithi183 running (105s) 24s ago 105s 20.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 9902db5727df 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.d smithi122 running (113s) 17s ago 113s 13.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 4525fa5dcf64 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.e smithi151 running (110s) 24s ago 110s 19.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 ac2e85ffdfb7 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.f smithi183 running (108s) 24s ago 107s 17.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 00a7be5dffbc 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.g smithi122 running (97s) 17s ago 96s 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 4aa22800bb34 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.h smithi151 running (102s) 24s ago 102s 18.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 7c09818ca910 2024-09-12T01:45:57.094 INFO:teuthology.orchestra.run.smithi122.stdout:mds.i smithi183 running (100s) 24s ago 99s 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 5fb738928dc8 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mds.j smithi122 running (116s) 17s ago 116s 13.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 285a0d556f53 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mds.k smithi151 running (2m) 24s ago 119s 13.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 fbb294fe9528 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mds.l smithi183 running (2m) 24s ago 2m 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 669df5a4698d 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.x smithi122 *:9283,8765 running (8m) 17s ago 8m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 bdeedc2eecd2 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.y smithi151 *:8443,8765 running (7m) 24s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 c7ca2074172f 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.z smithi183 *:8443,8765 running (7m) 24s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 673b42d8bd2e 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mon.a smithi122 running (8m) 17s ago 9m 51.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 2745149ee039 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mon.b smithi151 running (7m) 24s ago 7m 39.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 088859bd539a 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:mon.c smithi183 running (7m) 24s ago 7m 40.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9a051b6e9f29 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 smithi122 running (6m) 17s ago 6m 45.2M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 0e96fe802064 2024-09-12T01:45:57.095 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 smithi151 running (6m) 24s ago 6m 59.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fcb5a81c9cc5 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 smithi183 running (6m) 24s ago 6m 58.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fba22aa1be47 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 smithi122 running (5m) 17s ago 5m 56.0M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 07d05a42b41c 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 smithi151 running (5m) 24s ago 5m 53.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 56c5070ccc6f 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 smithi183 running (4m) 24s ago 4m 54.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 53a54b31a17b 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 smithi122 running (4m) 17s ago 4m 41.1M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 a7b336e634da 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 smithi151 running (4m) 24s ago 4m 44.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 eb3ba75362b5 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.8 smithi183 running (3m) 24s ago 3m 45.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 908fcc2c8ea4 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.9 smithi122 running (3m) 17s ago 3m 40.5M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 8d551e800f33 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.10 smithi151 running (2m) 24s ago 2m 55.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 dc321a7eec3b 2024-09-12T01:45:57.096 INFO:teuthology.orchestra.run.smithi122.stdout:osd.11 smithi183 running (2m) 24s ago 2m 41.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 0bb907b9b9be 2024-09-12T01:45:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96740a2c10 msgr2=0x7f96740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96740a2c10 0x7f96740a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9680005e10 tx=0x7f9680005da0 comp rx=0 tx=0).stop 2024-09-12T01:45:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 msgr2=0x7f968c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 0x7f968c102810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f967c00c970 tx=0x7f967c00ce30 comp rx=0 tx=0).stop 2024-09-12T01:45:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 shutdown_connections 2024-09-12T01:45:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96740a2c10 0x7f96740a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:57.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f968c103870 0x7f968c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:57.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f968c102eb0 0x7f968c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:57.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 --2- 172.21.15.122:0/3932613212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f968c0ffed0 0x7f968c102810 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:57.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 >> 172.21.15.122:0/3932613212 conn(0x7f968c0fba40 msgr2=0x7f968c106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:57.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 shutdown_connections 2024-09-12T01:45:57.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:57.096+0000 7f9692565640 1 -- 172.21.15.122:0/3932613212 wait complete. 2024-09-12T01:45:57.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:57 smithi122 ceph-mon[30925]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:57.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T01:45:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:57 smithi151 ceph-mon[35229]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:57 smithi183 ceph-mon[35127]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:45:58.058 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 -- 172.21.15.122:0/3899795118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a80761a0 msgr2=0x7f47a8074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 --2- 172.21.15.122:0/3899795118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a80761a0 0x7f47a8074600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4798009900 tx=0x7f479802a410 comp rx=0 tx=0).stop 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 -- 172.21.15.122:0/3899795118 shutdown_connections 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 --2- 172.21.15.122:0/3899795118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 0x7f47a810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 --2- 172.21.15.122:0/3899795118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47a8074b40 0x7f47a8074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 --2- 172.21.15.122:0/3899795118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a80761a0 0x7f47a8074600 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.544+0000 7f47ad653640 1 -- 172.21.15.122:0/3899795118 >> 172.21.15.122:0/3899795118 conn(0x7f47a80fd3c0 msgr2=0x7f47a80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.545+0000 7f47ad653640 1 -- 172.21.15.122:0/3899795118 shutdown_connections 2024-09-12T01:45:58.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.545+0000 7f47ad653640 1 -- 172.21.15.122:0/3899795118 wait complete. 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.545+0000 7f47ad653640 1 Processor -- start 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.545+0000 7f47ad653640 1 -- start start 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 0x7f47a819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47a80761a0 0x7f47a819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 0x7f47a81a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810fa40 con 0x7f47a8074b40 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810f8c0 con 0x7f47a8109a60 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810fbc0 con 0x7f47a80761a0 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 0x7f47a819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 0x7f47a819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40588/0 (socket says 172.21.15.122:40588) 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 -- 172.21.15.122:0/3309964810 learned_addr learned my addr 172.21.15.122:0/3309964810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f479e7fc640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47a80761a0 0x7f47a819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:58.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a77fe640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 0x7f47a81a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 -- 172.21.15.122:0/3309964810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47a80761a0 msgr2=0x7f47a819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47a80761a0 0x7f47a819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 -- 172.21.15.122:0/3309964810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 msgr2=0x7f47a81a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 0x7f47a81a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 -- 172.21.15.122:0/3309964810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81a1cf0 con 0x7f47a8074b40 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a77fe640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 0x7f47a81a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a6ffd640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 0x7f47a819a0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4798009230 tx=0x7f4798004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479803f030 con 0x7f47a8074b40 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798003ce0 con 0x7f47a8074b40 2024-09-12T01:45:58.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81a1f80 con 0x7f47a8074b40 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798035070 con 0x7f47a8074b40 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.546+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a2380 con 0x7f47a8074b40 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.548+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a8108870 con 0x7f47a8074b40 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.549+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4798042030 con 0x7f47a8074b40 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.549+0000 7f47a4ff9640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47940a2c40 0x7f47940a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.549+0000 7f479e7fc640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47940a2c40 0x7f47940a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.549+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f47980ef8d0 con 0x7f47a8074b40 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.550+0000 7f479e7fc640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47940a2c40 0x7f47940a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4790005d00 tx=0x7f4790005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:45:58.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.553+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47980bc690 con 0x7f47a8074b40 2024-09-12T01:45:58.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.730+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f47a8062630 con 0x7f47940a2c40 2024-09-12T01:45:58.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:58 smithi122 ceph-mon[30925]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:45:58.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.736+0000 7f47a4ff9640 1 -- 172.21.15.122:0/3309964810 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f47a8062630 con 0x7f47940a2c40 2024-09-12T01:45:58.736 INFO:teuthology.orchestra.run.smithi122.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T01:45:58.736 INFO:teuthology.orchestra.run.smithi122.stdout:mds.all 12/12 25s ago 2m smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T01:45:58.736 INFO:teuthology.orchestra.run.smithi122.stdout:mgr 3/3 25s ago 7m smithi122=x;smithi151=y;smithi183=z;count:3 2024-09-12T01:45:58.736 INFO:teuthology.orchestra.run.smithi122.stdout:mon 3/3 25s ago 7m smithi122:172.21.15.122=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-09-12T01:45:58.736 INFO:teuthology.orchestra.run.smithi122.stdout:osd 12 25s ago - 2024-09-12T01:45:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47940a2c40 msgr2=0x7f47940a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47940a2c40 0x7f47940a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4790005d00 tx=0x7f4790005c50 comp rx=0 tx=0).stop 2024-09-12T01:45:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 msgr2=0x7f47a819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:45:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 0x7f47a819a0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4798009230 tx=0x7f4798004360 comp rx=0 tx=0).stop 2024-09-12T01:45:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 shutdown_connections 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47940a2c40 0x7f47940a5100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47a8109a60 0x7f47a81a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47a80761a0 0x7f47a819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 --2- 172.21.15.122:0/3309964810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47a8074b40 0x7f47a819a0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 >> 172.21.15.122:0/3309964810 conn(0x7f47a80fd3c0 msgr2=0x7f47a8105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 shutdown_connections 2024-09-12T01:45:58.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:45:58.738+0000 7f47ad653640 1 -- 172.21.15.122:0/3309964810 wait complete. 2024-09-12T01:45:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:58 smithi151 ceph-mon[35229]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:45:58.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:58 smithi183 ceph-mon[35127]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:45:59.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T01:45:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:45:59.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:45:59 smithi122 ceph-mon[30925]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:45:59 smithi151 ceph-mon[35229]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:45:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:45:59 smithi183 ceph-mon[35127]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:46:00.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 -- 172.21.15.122:0/3547419549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac102510 msgr2=0x7f9fac108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:00.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/3547419549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac102510 0x7f9fac108da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009ca0 tx=0x7f9fa402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 -- 172.21.15.122:0/3547419549 shutdown_connections 2024-09-12T01:46:00.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/3547419549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac102510 0x7f9fac108da0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/3547419549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 0x7f9fac101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/3547419549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fac100950 0x7f9fac100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 -- 172.21.15.122:0/3547419549 >> 172.21.15.122:0/3547419549 conn(0x7f9fac0fc0e0 msgr2=0x7f9fac0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 -- 172.21.15.122:0/3547419549 shutdown_connections 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.154+0000 7f9fb1da2640 1 -- 172.21.15.122:0/3547419549 wait complete. 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 Processor -- start 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 -- start start 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 0x7f9fac06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 0x7f9fac19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fac102510 0x7f9fac19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10c5d0 con 0x7f9fac100950 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10c450 con 0x7f9fac101b50 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb1da2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10c750 con 0x7f9fac102510 2024-09-12T01:46:00.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb0da0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 0x7f9fac06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb0da0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 0x7f9fac06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40606/0 (socket says 172.21.15.122:40606) 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb0da0640 1 -- 172.21.15.122:0/2562848455 learned_addr learned my addr 172.21.15.122:0/2562848455 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fb15a1640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fac102510 0x7f9fac19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.155+0000 7f9fabfff640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 0x7f9fac19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb0da0640 1 -- 172.21.15.122:0/2562848455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fac102510 msgr2=0x7f9fac19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb0da0640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fac102510 0x7f9fac19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb0da0640 1 -- 172.21.15.122:0/2562848455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 msgr2=0x7f9fac19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb0da0640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 0x7f9fac19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb0da0640 1 -- 172.21.15.122:0/2562848455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac19fa10 con 0x7f9fac100950 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fabfff640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 0x7f9fac19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb0da0640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 0x7f9fac06b2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c970 tx=0x7f9fa000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0004220 con 0x7f9fac100950 2024-09-12T01:46:00.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac19fca0 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fa00043c0 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0004a00 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.156+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1a0190 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.157+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac107d00 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.158+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9fa00079b0 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.159+0000 7f9fa9ffb640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9f9c0a2c10 0x7f9f9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.159+0000 7f9fabfff640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9f9c0a2c10 0x7f9f9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.159+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9fa00cef70 con 0x7f9fac100950 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.159+0000 7f9fabfff640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9f9c0a2c10 0x7f9f9c0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fac06b870 tx=0x7f9f98005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:00.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.163+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fa0014030 con 0x7f9fac100950 2024-09-12T01:46:00.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.334+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9fac062630 con 0x7f9f9c0a2c10 2024-09-12T01:46:00.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.335+0000 7f9fa9ffb640 1 -- 172.21.15.122:0/2562848455 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9fac062630 con 0x7f9f9c0a2c10 2024-09-12T01:46:00.335 INFO:teuthology.orchestra.run.smithi122.stdout:HOST ADDR LABELS STATUS 2024-09-12T01:46:00.335 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 172.21.15.122 2024-09-12T01:46:00.335 INFO:teuthology.orchestra.run.smithi122.stdout:smithi151 172.21.15.151 2024-09-12T01:46:00.336 INFO:teuthology.orchestra.run.smithi122.stdout:smithi183 172.21.15.183 2024-09-12T01:46:00.336 INFO:teuthology.orchestra.run.smithi122.stdout:3 hosts in cluster 2024-09-12T01:46:00.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.337+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9f9c0a2c10 msgr2=0x7f9f9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.337+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9f9c0a2c10 0x7f9f9c0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fac06b870 tx=0x7f9f98005ec0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.337+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 msgr2=0x7f9fac06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.337+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 0x7f9fac06b2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c970 tx=0x7f9fa000ce30 comp rx=0 tx=0).stop 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.337+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 shutdown_connections 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.337+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9f9c0a2c10 0x7f9f9c0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.338+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fac102510 0x7f9fac19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.338+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fac101b50 0x7f9fac19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.338+0000 7f9fb1da2640 1 --2- 172.21.15.122:0/2562848455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fac100950 0x7f9fac06b2f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.338+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 >> 172.21.15.122:0/2562848455 conn(0x7f9fac0fc0e0 msgr2=0x7f9fac104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.338+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 shutdown_connections 2024-09-12T01:46:00.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:00.338+0000 7f9fb1da2640 1 -- 172.21.15.122:0/2562848455 wait complete. 2024-09-12T01:46:00.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:00 smithi122 ceph-mon[30925]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:00 smithi151 ceph-mon[35229]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:00 smithi183 ceph-mon[35127]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:01.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T01:46:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:01.726 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:01 smithi122 ceph-mon[30925]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:01.726 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:01 smithi122 ceph-mon[30925]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:46:01.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 -- 172.21.15.122:0/2450963550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a04101d10 msgr2=0x7f8a04102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:01.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/2450963550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a04101d10 0x7f8a04102190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009900 tx=0x7f89f002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:01.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 -- 172.21.15.122:0/2450963550 shutdown_connections 2024-09-12T01:46:01.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/2450963550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a041026d0 0x7f8a04108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/2450963550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a04101d10 0x7f8a04102190 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/2450963550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04100b10 0x7f8a04100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 -- 172.21.15.122:0/2450963550 >> 172.21.15.122:0/2450963550 conn(0x7f8a040fc2a0 msgr2=0x7f8a040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 -- 172.21.15.122:0/2450963550 shutdown_connections 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.808+0000 7f8a0acb5640 1 -- 172.21.15.122:0/2450963550 wait complete. 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 Processor -- start 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 -- start start 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a04100b10 0x7f8a04197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04101d10 0x7f8a04198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 0x7f8a0419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410c810 con 0x7f8a041026d0 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410c690 con 0x7f8a04101d10 2024-09-12T01:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a0acb5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410c990 con 0x7f8a04100b10 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 0x7f8a0419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 0x7f8a0419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49140/0 (socket says 172.21.15.122:49140) 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 -- 172.21.15.122:0/708624969 learned_addr learned my addr 172.21.15.122:0/708624969 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a03fff640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04101d10 0x7f8a04198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a08a51640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a04100b10 0x7f8a04197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 -- 172.21.15.122:0/708624969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a04100b10 msgr2=0x7f8a04197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a04100b10 0x7f8a04197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 -- 172.21.15.122:0/708624969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04101d10 msgr2=0x7f8a04198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04101d10 0x7f8a04198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.809+0000 7f8a09252640 1 -- 172.21.15.122:0/708624969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0419fbf0 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.810+0000 7f8a09252640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 0x7f8a0419f580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b970 tx=0x7f89fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.810+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc019070 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.810+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89fc00cb40 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.810+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0419fee0 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.810+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc007650 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.810+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a0410 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.811+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a041a06a0 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.812+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f89fc01c070 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.812+0000 7f8a01ffb640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89f40a2c40 0x7f89f40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.813+0000 7f8a08a51640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89f40a2c40 0x7f89f40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.813+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89fc0ce6c0 con 0x7f8a041026d0 2024-09-12T01:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.813+0000 7f8a08a51640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89f40a2c40 0x7f89f40a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f89f80058c0 tx=0x7f89f800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:01.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.817+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89fc09b5d0 con 0x7f8a041026d0 2024-09-12T01:46:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:01 smithi151 ceph-mon[35229]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:01 smithi151 ceph-mon[35229]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:46:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:01 smithi183 ceph-mon[35127]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:01 smithi183 ceph-mon[35127]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:46:01.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.989+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8a041a0870 con 0x7f89f40a2c40 2024-09-12T01:46:01.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.991+0000 7f8a01ffb640 1 -- 172.21.15.122:0/708624969 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8a041a0870 con 0x7f89f40a2c40 2024-09-12T01:46:01.991 INFO:teuthology.orchestra.run.smithi122.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T01:46:01.991 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T01:46:01.991 INFO:teuthology.orchestra.run.smithi122.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T01:46:01.991 INFO:teuthology.orchestra.run.smithi122.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89f40a2c40 msgr2=0x7f89f40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89f40a2c40 0x7f89f40a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f89f80058c0 tx=0x7f89f800a4c0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 msgr2=0x7f8a0419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 0x7f8a0419f580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b970 tx=0x7f89fc00be30 comp rx=0 tx=0).stop 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 shutdown_connections 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89f40a2c40 0x7f89f40a5100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a041026d0 0x7f8a0419f580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04101d10 0x7f8a04198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 --2- 172.21.15.122:0/708624969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a04100b10 0x7f8a04197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:01.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 >> 172.21.15.122:0/708624969 conn(0x7f8a040fc2a0 msgr2=0x7f8a0406a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:01.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 shutdown_connections 2024-09-12T01:46:01.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:01.993+0000 7f8a0acb5640 1 -- 172.21.15.122:0/708624969 wait complete. 2024-09-12T01:46:02.692 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T01:46:02.700 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T01:46:02.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T01:46:02.947 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:03.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1570420936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 msgr2=0x7fd7dc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1570420936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc102190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009900 tx=0x7fd7d002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1570420936 shutdown_connections 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1570420936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 0x7fd7dc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1570420936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc102190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1570420936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 0x7fd7dc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1570420936 >> 172.21.15.122:0/1570420936 conn(0x7fd7dc0fc2a0 msgr2=0x7fd7dc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1570420936 shutdown_connections 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.433+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1570420936 wait complete. 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 Processor -- start 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 -- start start 2024-09-12T01:46:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 0x7fd7dc196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 0x7fd7dc19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc10c810 con 0x7fd7dc101d10 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc10c690 con 0x7fd7dc1026d0 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7e14ca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc10c990 con 0x7fd7dc100b10 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49152/0 (socket says 172.21.15.122:49152) 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 -- 172.21.15.122:0/1133136407 learned_addr learned my addr 172.21.15.122:0/1133136407 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7daffd640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 0x7fd7dc196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7db7fe640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 0x7fd7dc19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 -- 172.21.15.122:0/1133136407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 msgr2=0x7fd7dc196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 0x7fd7dc196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 -- 172.21.15.122:0/1133136407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 msgr2=0x7fd7dc19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 0x7fd7dc19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.434+0000 7fd7da7fc640 1 -- 172.21.15.122:0/1133136407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc19eac0 con 0x7fd7dc101d10 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7db7fe640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 0x7fd7dc19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7daffd640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 0x7fd7dc196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7da7fc640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc1973d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009230 tx=0x7fd7d00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0046070 con 0x7fd7dc101d10 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc06a790 con 0x7fd7dc101d10 2024-09-12T01:46:03.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d0003ce0 con 0x7fd7dc101d10 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0035050 con 0x7fd7dc101d10 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.435+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc06acf0 con 0x7fd7dc101d10 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.436+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7dc100f10 con 0x7fd7dc101d10 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.437+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd7d0049070 con 0x7fd7dc101d10 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.437+0000 7fd7c3fff640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7c40a2c40 0x7fd7c40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.438+0000 7fd7daffd640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7c40a2c40 0x7fd7c40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.438+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd7d00f1010 con 0x7fd7dc101d10 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.438+0000 7fd7daffd640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7c40a2c40 0x7fd7c40a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0058c0 tx=0x7fd7cc005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.442+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7d007e1b0 con 0x7fd7dc101d10 2024-09-12T01:46:03.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.644+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd7dc107b40 con 0x7fd7dc101d10 2024-09-12T01:46:03.645 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:03 smithi122 ceph-mon[30925]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:03.645 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:03 smithi122 ceph-mon[30925]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:03.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.645+0000 7fd7c3fff640 1 -- 172.21.15.122:0/1133136407 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3098 (secure 0 0 0) 0x7fd7d00bde10 con 0x7fd7dc101d10 2024-09-12T01:46:03.645 INFO:teuthology.orchestra.run.smithi122.stdout:e25 2024-09-12T01:46:03.645 INFO:teuthology.orchestra.run.smithi122.stdout:btime 2024-09-12T01:44:47:831953+0000 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:legacy client fscid: 1 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:Filesystem 'cephfs' (1) 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:fs_name cephfs 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 25 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-09-12T01:44:36.280693+0000 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-09-12T01:44:47.439571+0000 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:tableserver 0 2024-09-12T01:46:03.646 INFO:teuthology.orchestra.run.smithi122.stdout:root 0 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:session_timeout 300 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:session_autoclose 300 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:max_file_size 1099511627776 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:max_xattr_size 65536 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:required_client_features {} 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:last_failure 0 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:last_failure_osd_epoch 0 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:max_mds 3 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:in 0,1,2 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:up {0=24452,1=24436,2=24428} 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:failed 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:damaged 2024-09-12T01:46:03.647 INFO:teuthology.orchestra.run.smithi122.stdout:stopped 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:data_pools [3] 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:metadata_pool 2 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:inline_data disabled 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:balancer 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:bal_rank_mask -1 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:standby_count_wanted 0 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:qdb_cluster leader: 24452 members: 24428,24436,24452 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b{0:24452} state up:active seq 4 addr [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.f{0:24424} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/1190566748,v1:172.21.15.183:6835/1190566748] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c{1:24436} state up:active seq 9 addr [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.e{1:24407} state up:standby-replay seq 1 addr [v2:172.21.15.151:6834/2236567181,v1:172.21.15.151:6835/2236567181] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.648 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.h{2:24428} state up:active seq 9 addr [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.l{2:24400} state up:standby-replay seq 1 addr [v2:172.21.15.183:6832/3975453954,v1:172.21.15.183:6833/3975453954] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:Standby daemons: 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.122:6834/3513109229,v1:172.21.15.122:6835/3513109229] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.122:6836/6361172,v1:172.21.15.122:6837/6361172] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.i{-1:14598} state up:standby seq 1 addr [v2:172.21.15.183:6838/3259334666,v1:172.21.15.183:6839/3259334666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.g{-1:14604} state up:standby seq 1 addr [v2:172.21.15.122:6838/2056437657,v1:172.21.15.122:6839/2056437657] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a{-1:14616} state up:standby seq 1 addr [v2:172.21.15.122:6840/1896979097,v1:172.21.15.122:6841/1896979097] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.649 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.k{-1:24389} state up:standby seq 1 addr [v2:172.21.15.151:6832/3451432768,v1:172.21.15.151:6833/3451432768] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T01:46:03.650 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:46:03.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7c40a2c40 msgr2=0x7fd7c40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7c40a2c40 0x7fd7c40a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0058c0 tx=0x7fd7cc005ec0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 msgr2=0x7fd7dc1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc1973d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009230 tx=0x7fd7d00043b0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 shutdown_connections 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7c40a2c40 0x7fd7c40a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7dc1026d0 0x7fd7dc19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7dc101d10 0x7fd7dc1973d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 --2- 172.21.15.122:0/1133136407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7dc100b10 0x7fd7dc196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 >> 172.21.15.122:0/1133136407 conn(0x7fd7dc0fc2a0 msgr2=0x7fd7dc105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 shutdown_connections 2024-09-12T01:46:03.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:03.647+0000 7fd7e14ca640 1 -- 172.21.15.122:0/1133136407 wait complete. 2024-09-12T01:46:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:03 smithi151 ceph-mon[35229]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:03 smithi151 ceph-mon[35229]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:03 smithi183 ceph-mon[35127]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:03 smithi183 ceph-mon[35127]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:04.329 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T01:46:04.573 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:04.694 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:04 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1133136407' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T01:46:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:04 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1133136407' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T01:46:04.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:04 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1133136407' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T01:46:05.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3251382662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 msgr2=0x7ff698102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3251382662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698102000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c02a410 comp rx=0 tx=0).stop 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3251382662 shutdown_connections 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3251382662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6981037c0 0x7ff69810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3251382662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698102e00 0x7ff698103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3251382662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698102000 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3251382662 >> 172.21.15.122:0/3251382662 conn(0x7ff6980fd390 msgr2=0x7ff6980ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3251382662 shutdown_connections 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.052+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3251382662 wait complete. 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 Processor -- start 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 -- start start 2024-09-12T01:46:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff698102e00 0x7ff6980745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6981037c0 0x7ff698074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff69810d8e0 con 0x7ff698101c00 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff69810d760 con 0x7ff6981037c0 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69fcc7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff69810da60 con 0x7ff698102e00 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69da63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69da63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49182/0 (socket says 172.21.15.122:49182) 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69da63640 1 -- 172.21.15.122:0/3887309760 learned_addr learned my addr 172.21.15.122:0/3887309760 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.053+0000 7ff69e264640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6981037c0 0x7ff698074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69d262640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff698102e00 0x7ff6980745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69da63640 1 -- 172.21.15.122:0/3887309760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff698102e00 msgr2=0x7ff6980745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69da63640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff698102e00 0x7ff6980745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69da63640 1 -- 172.21.15.122:0/3887309760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6981037c0 msgr2=0x7ff698074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69da63640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6981037c0 0x7ff698074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69da63640 1 -- 172.21.15.122:0/3887309760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6980753e0 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69d262640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff698102e00 0x7ff6980745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69da63640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698075f90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff68c031610 tx=0x7ff68c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c03f030 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68c003ce0 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6981a2090 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c035070 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.054+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6981a2610 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.055+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff698109020 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.057+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff68c042030 con 0x7ff698101c00 2024-09-12T01:46:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.057+0000 7ff68affd640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6840a2c90 0x7ff6840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:05.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.057+0000 7ff69d262640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6840a2c90 0x7ff6840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:05.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.057+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff68c0efa70 con 0x7ff698101c00 2024-09-12T01:46:05.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.058+0000 7ff69d262640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6840a2c90 0x7ff6840a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff698075d10 tx=0x7ff694005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:05.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.061+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff68c0bc870 con 0x7ff698101c00 2024-09-12T01:46:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.230+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff69802d7c0 con 0x7ff698101c00 2024-09-12T01:46:05.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.230+0000 7ff68affd640 1 -- 172.21.15.122:0/3887309760 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5321 (secure 0 0 0) 0x7ff68c0bc170 con 0x7ff698101c00 2024-09-12T01:46:05.230 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 83 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:fsid 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-09-12T01:36:52.283857+0000 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-09-12T01:44:45.811494+0000 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:crush_version 26 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:full_ratio 0.95 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:backfillfull_ratio 0.9 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:nearfull_ratio 0.85 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:require_min_compat_client luminous 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:min_compat_client luminous 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:require_osd_release squid 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:stretch_mode_enabled false 2024-09-12T01:46:05.231 INFO:teuthology.orchestra.run.smithi122.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:max_osd 12 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6802/1924566162,v1:172.21.15.122:6803/1924566162] [v2:172.21.15.122:6804/1924566162,v1:172.21.15.122:6805/1924566162] exists,up f5f05d0c-f7d1-4a68-8a67-6daf86c09856 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/148046823,v1:172.21.15.151:6801/148046823] [v2:172.21.15.151:6802/148046823,v1:172.21.15.151:6803/148046823] exists,up 474588fe-b8aa-4c63-a179-9fb1a91609c6 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 up in weight 1 up_from 17 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/3556820790,v1:172.21.15.183:6801/3556820790] [v2:172.21.15.183:6802/3556820790,v1:172.21.15.183:6803/3556820790] exists,up f1bc9ea7-5717-4f77-a9a1-dee0231402da 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6810/533436685,v1:172.21.15.122:6811/533436685] [v2:172.21.15.122:6812/533436685,v1:172.21.15.122:6813/533436685] exists,up a02adc8a-32be-42b2-9b14-d6f23ac36887 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/190394298,v1:172.21.15.151:6809/190394298] [v2:172.21.15.151:6810/190394298,v1:172.21.15.151:6811/190394298] exists,up 79ee3483-5327-4ea0-ab19-e2614d5aec71 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/685154842,v1:172.21.15.183:6809/685154842] [v2:172.21.15.183:6810/685154842,v1:172.21.15.183:6811/685154842] exists,up f02377d1-5979-48eb-945c-65d7675d8283 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6818/913950299,v1:172.21.15.122:6819/913950299] [v2:172.21.15.122:6820/913950299,v1:172.21.15.122:6821/913950299] exists,up 3c4782f3-b6a1-45f7-83b4-e109d523dffc 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/1163877446,v1:172.21.15.151:6817/1163877446] [v2:172.21.15.151:6818/1163877446,v1:172.21.15.151:6819/1163877446] exists,up 142f0be6-d6a8-49d4-a4a1-462d5aece8d1 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.8 up in weight 1 up_from 56 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/830128598,v1:172.21.15.183:6817/830128598] [v2:172.21.15.183:6818/830128598,v1:172.21.15.183:6819/830128598] exists,up 0da6dee8-5cbe-43a4-8be6-b4c58619d9a5 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6826/163584006,v1:172.21.15.122:6827/163584006] [v2:172.21.15.122:6828/163584006,v1:172.21.15.122:6829/163584006] exists,up dc2fd685-4e31-49ab-b2ed-8244461230a6 2024-09-12T01:46:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:osd.10 up in weight 1 up_from 68 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/1544038944,v1:172.21.15.151:6825/1544038944] [v2:172.21.15.151:6826/1544038944,v1:172.21.15.151:6827/1544038944] exists,up d577aa12-49ce-464c-b1b7-18e5b0765883 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/2785958349,v1:172.21.15.183:6825/2785958349] [v2:172.21.15.183:6826/2785958349,v1:172.21.15.183:6827/2785958349] exists,up 07eb6a15-ca7e-4169-8e16-6f47e7046f9e 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.13 [2,11] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.1c [8,11] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.0 [4,7] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.c [3,9] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.11 [4,8] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.28 [5,8] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.34 [4,9] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.36 [4,9] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.39 [5,6] 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6800/3688521366 expires 2024-09-13T01:37:44.367349+0000 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/3407930740 expires 2024-09-13T01:37:44.367349+0000 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6801/3688521366 expires 2024-09-13T01:37:44.367349+0000 2024-09-12T01:46:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/1117597805 expires 2024-09-13T01:37:44.367349+0000 2024-09-12T01:46:05.234 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/2717375533 expires 2024-09-13T01:37:44.367349+0000 2024-09-12T01:46:05.234 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/3107764322 expires 2024-09-13T01:37:21.796085+0000 2024-09-12T01:46:05.234 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6801/1950823209 expires 2024-09-13T01:37:21.796085+0000 2024-09-12T01:46:05.234 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6800/1950823209 expires 2024-09-13T01:37:21.796085+0000 2024-09-12T01:46:05.234 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/1058528799 expires 2024-09-13T01:37:21.796085+0000 2024-09-12T01:46:05.234 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/472737560 expires 2024-09-13T01:37:21.796085+0000 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6840a2c90 msgr2=0x7ff6840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6840a2c90 0x7ff6840a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff698075d10 tx=0x7ff694005d30 comp rx=0 tx=0).stop 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 msgr2=0x7ff698075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698075f90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff68c031610 tx=0x7ff68c0043b0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 shutdown_connections 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff6840a2c90 0x7ff6840a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6981037c0 0x7ff698074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff698102e00 0x7ff6980745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 --2- 172.21.15.122:0/3887309760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff698101c00 0x7ff698075f90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 >> 172.21.15.122:0/3887309760 conn(0x7ff6980fd390 msgr2=0x7ff6981061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 shutdown_connections 2024-09-12T01:46:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:05.233+0000 7ff69fcc7640 1 -- 172.21.15.122:0/3887309760 wait complete. 2024-09-12T01:46:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:05 smithi151 ceph-mon[35229]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:46:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:05 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3887309760' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T01:46:05.911 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:05 smithi122 ceph-mon[30925]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:46:05.911 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:05 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3887309760' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T01:46:05.938 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T01:46:05.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:05 smithi183 ceph-mon[35127]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:46:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:05 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3887309760' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T01:46:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.628+0000 7f743bda7640 1 -- 172.21.15.122:0/1449591377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101e20 msgr2=0x7f74341056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 --2- 172.21.15.122:0/1449591377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101e20 0x7f74341056b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7424009900 tx=0x7f742402a410 comp rx=0 tx=0).stop 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 -- 172.21.15.122:0/1449591377 shutdown_connections 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 --2- 172.21.15.122:0/1449591377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7434105de0 0x7f743410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 --2- 172.21.15.122:0/1449591377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101e20 0x7f74341056b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 --2- 172.21.15.122:0/1449591377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434101470 0x7f7434101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 -- 172.21.15.122:0/1449591377 >> 172.21.15.122:0/1449591377 conn(0x7f7434077230 msgr2=0x7f74340fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 -- 172.21.15.122:0/1449591377 shutdown_connections 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 -- 172.21.15.122:0/1449591377 wait complete. 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 Processor -- start 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.629+0000 7f743bda7640 1 -- start start 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 0x7f743419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7434101e20 0x7f743419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434105de0 0x7f743419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7434110e80 con 0x7f7434101470 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7434110d00 con 0x7f7434101e20 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7434111000 con 0x7f7434105de0 2024-09-12T01:46:06.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 0x7f743419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 0x7f743419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49202/0 (socket says 172.21.15.122:49202) 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 -- 172.21.15.122:0/2582157780 learned_addr learned my addr 172.21.15.122:0/2582157780 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743a344640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434105de0 0x7f743419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439342640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7434101e20 0x7f743419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 -- 172.21.15.122:0/2582157780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434105de0 msgr2=0x7f743419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434105de0 0x7f743419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 -- 172.21.15.122:0/2582157780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7434101e20 msgr2=0x7f743419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7434101e20 0x7f743419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 -- 172.21.15.122:0/2582157780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743419f650 con 0x7f7434101470 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f7439b43640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 0x7f743419a6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f743000d770 tx=0x7f743000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430017070 con 0x7f7434101470 2024-09-12T01:46:06.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7430005500 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743419f940 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430004e00 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.630+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74341a7220 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.631+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7434101850 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.632+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7430004fa0 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.633+0000 7f742affd640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74200a2c90 0x7f74200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.633+0000 7f7439342640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74200a2c90 0x7f74200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.633+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f74300ceba0 con 0x7f7434101470 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.634+0000 7f7439342640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74200a2c90 0x7f74200a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7434105b50 tx=0x7f7424005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:06.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.637+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f743009b9a0 con 0x7f7434101470 2024-09-12T01:46:06.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.839+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74340fdd40 con 0x7f7434101470 2024-09-12T01:46:06.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.840+0000 7f742affd640 1 -- 172.21.15.122:0/2582157780 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f743009b2a0 con 0x7f7434101470 2024-09-12T01:46:06.840 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:06.840 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"btime":"2024-09-12T01:44:47:831953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:44:47.439571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:46:06.840 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74200a2c90 msgr2=0x7f74200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74200a2c90 0x7f74200a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7434105b50 tx=0x7f7424005a20 comp rx=0 tx=0).stop 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 msgr2=0x7f743419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 0x7f743419a6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f743000d770 tx=0x7f743000dc30 comp rx=0 tx=0).stop 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 shutdown_connections 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f74200a2c90 0x7f74200a5150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434105de0 0x7f743419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7434101e20 0x7f743419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 --2- 172.21.15.122:0/2582157780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7434101470 0x7f743419a6a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:06.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 >> 172.21.15.122:0/2582157780 conn(0x7f7434077230 msgr2=0x7f7434100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:06.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 shutdown_connections 2024-09-12T01:46:06.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:06.842+0000 7f743bda7640 1 -- 172.21.15.122:0/2582157780 wait complete. 2024-09-12T01:46:07.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:07.800 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:07 smithi151 ceph-mon[35229]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:07 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2582157780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:07.921 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:07 smithi122 ceph-mon[30925]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:07.921 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:07 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2582157780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:07 smithi183 ceph-mon[35127]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:07 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2582157780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 -- 172.21.15.122:0/3051388430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 msgr2=0x7f77081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 --2- 172.21.15.122:0/3051388430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081021c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7704009900 tx=0x7f770402a410 comp rx=0 tx=0).stop 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 -- 172.21.15.122:0/3051388430 shutdown_connections 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 --2- 172.21.15.122:0/3051388430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708106f10 0x7f770810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 --2- 172.21.15.122:0/3051388430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708102790 0x7f77081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 --2- 172.21.15.122:0/3051388430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081021c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 -- 172.21.15.122:0/3051388430 >> 172.21.15.122:0/3051388430 conn(0x7f77080fd8c0 msgr2=0x7f77080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 -- 172.21.15.122:0/3051388430 shutdown_connections 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.263+0000 7f77100b8640 1 -- 172.21.15.122:0/3051388430 wait complete. 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 Processor -- start 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 -- start start 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f770819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111f90 con 0x7f7708101de0 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111e10 con 0x7f7708106f10 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f77100b8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112110 con 0x7f7708102790 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49234/0 (socket says 172.21.15.122:49234) 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 -- 172.21.15.122:0/604218262 learned_addr learned my addr 172.21.15.122:0/604218262 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770d653640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 -- 172.21.15.122:0/604218262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 msgr2=0x7f7708198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 -- 172.21.15.122:0/604218262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 msgr2=0x7f770819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f770819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 -- 172.21.15.122:0/604218262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770819d480 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.264+0000 7f770de54640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081984d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7704005ab0 tx=0x7f77040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.265+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770403f030 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.265+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704003ce0 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.265+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704035070 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.265+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770819d710 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.265+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081a5050 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.266+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77081021c0 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.267+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7704042030 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.267+0000 7f76feffd640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c90 0x7f76f40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.267+0000 7f770d653640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c90 0x7f76f40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.267+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f77040efb00 con 0x7f7708101de0 2024-09-12T01:46:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.268+0000 7f770d653640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c90 0x7f76f40a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f77080fdd50 tx=0x7f76f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:08.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.271+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77040bc900 con 0x7f7708101de0 2024-09-12T01:46:08.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.472+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f770810a150 con 0x7f7708101de0 2024-09-12T01:46:08.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.473+0000 7f76feffd640 1 -- 172.21.15.122:0/604218262 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f77040bc200 con 0x7f7708101de0 2024-09-12T01:46:08.473 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:08.473 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":25,"btime":"2024-09-12T01:44:47:831953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:44:47.439571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:46:08.474 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 25 2024-09-12T01:46:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c90 msgr2=0x7f76f40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c90 0x7f76f40a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f77080fdd50 tx=0x7f76f8008040 comp rx=0 tx=0).stop 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 msgr2=0x7f77081984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081984d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7704005ab0 tx=0x7f77040042f0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 shutdown_connections 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c90 0x7f76f40a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f770819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 --2- 172.21.15.122:0/604218262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081984d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 >> 172.21.15.122:0/604218262 conn(0x7f77080fd8c0 msgr2=0x7f77080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.475+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 shutdown_connections 2024-09-12T01:46:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:08.476+0000 7f77100b8640 1 -- 172.21.15.122:0/604218262 wait complete. 2024-09-12T01:46:08.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:08 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/604218262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:08 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/604218262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:08.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:08 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/604218262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:09.159 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:46:09.403 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:09 smithi122 ceph-mon[30925]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:46:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:09 smithi151 ceph-mon[35229]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 -- 172.21.15.122:0/2622867311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04106f10 msgr2=0x7fcc0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2622867311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04106f10 0x7fcc0410ea90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009ca0 tx=0x7fcbfc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 -- 172.21.15.122:0/2622867311 shutdown_connections 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2622867311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04106f10 0x7fcc0410ea90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2622867311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc04102790 0x7fcc041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2622867311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc04101de0 0x7fcc041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 -- 172.21.15.122:0/2622867311 >> 172.21.15.122:0/2622867311 conn(0x7fcc040fd8c0 msgr2=0x7fcc040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 -- 172.21.15.122:0/2622867311 shutdown_connections 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.901+0000 7fcc08b32640 1 -- 172.21.15.122:0/2622867311 wait complete. 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 Processor -- start 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 -- start start 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 0x7fcc0419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc04102790 0x7fcc0419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc04106f10 0x7fcc0419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04111f90 con 0x7fcc04101de0 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04111e10 con 0x7fcc04106f10 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc08b32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04112110 con 0x7fcc04102790 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc0259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 0x7fcc0419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc0259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 0x7fcc0419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49258/0 (socket says 172.21.15.122:49258) 2024-09-12T01:46:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc0259c640 1 -- 172.21.15.122:0/2869380587 learned_addr learned my addr 172.21.15.122:0/2869380587 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc02d9d640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc04106f10 0x7fcc0419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.902+0000 7fcc01d9b640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc04102790 0x7fcc0419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc0259c640 1 -- 172.21.15.122:0/2869380587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc04102790 msgr2=0x7fcc0419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc0259c640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc04102790 0x7fcc0419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc0259c640 1 -- 172.21.15.122:0/2869380587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc04106f10 msgr2=0x7fcc0419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc0259c640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc04106f10 0x7fcc0419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc0259c640 1 -- 172.21.15.122:0/2869380587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0419f610 con 0x7fcc04101de0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc0259c640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 0x7fcc0419a660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c890 tx=0x7fcbf400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4004220 con 0x7fcc04101de0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0419f900 con 0x7fcc04101de0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf40043c0 con 0x7fcc04101de0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc041a71e0 con 0x7fcc04101de0 2024-09-12T01:46:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.903+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf401d060 con 0x7fcc04101de0 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.904+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc041021c0 con 0x7fcc04101de0 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.905+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcbf40079b0 con 0x7fcc04101de0 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.906+0000 7fcbf37fe640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbec0a2c10 0x7fcbec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.906+0000 7fcc01d9b640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbec0a2c10 0x7fcbec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.906+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcbf40cf410 con 0x7fcc04101de0 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.906+0000 7fcc01d9b640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbec0a2c10 0x7fcbec0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc04076480 tx=0x7fcbf8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:09.909+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbf4014030 con 0x7fcc04101de0 2024-09-12T01:46:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:09 smithi183 ceph-mon[35127]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T01:46:10.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.078+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc0419b9c0 con 0x7fcc04101de0 2024-09-12T01:46:10.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.080+0000 7fcbf37fe640 1 -- 172.21.15.122:0/2869380587 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7fcc0419b9c0 con 0x7fcc04101de0 2024-09-12T01:46:10.080 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:10.081 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:46:10.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbec0a2c10 msgr2=0x7fcbec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:10.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbec0a2c10 0x7fcbec0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc04076480 tx=0x7fcbf8008040 comp rx=0 tx=0).stop 2024-09-12T01:46:10.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 msgr2=0x7fcc0419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:10.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 0x7fcc0419a660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c890 tx=0x7fcbf400cd50 comp rx=0 tx=0).stop 2024-09-12T01:46:10.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 shutdown_connections 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbec0a2c10 0x7fcbec0a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc04106f10 0x7fcc0419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc04102790 0x7fcc0419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 --2- 172.21.15.122:0/2869380587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc04101de0 0x7fcc0419a660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.082+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 >> 172.21.15.122:0/2869380587 conn(0x7fcc040fd8c0 msgr2=0x7fcc040feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.083+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 shutdown_connections 2024-09-12T01:46:10.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:10.083+0000 7fcc08b32640 1 -- 172.21.15.122:0/2869380587 wait complete. 2024-09-12T01:46:10.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:10 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2869380587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:10.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:44:36.280693+0000', 'modified': '2024-09-12T01:44:47.439571+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24436, 'mds_2': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/3975453954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 3975453954}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 3975453954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/2236567181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 2236567181}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 2236567181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1190566748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1190566748}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1190566748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.151:6837/3984907767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3984907767}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3984907767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6837/3258783936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3258783936}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3258783936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.151:6839/1184770501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 1184770501}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 1184770501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24428, 24436, 24452]} 2024-09-12T01:46:10.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T01:46:10.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T01:46:10.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:10 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2869380587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:10.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:10 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2869380587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:11.064 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:11.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/4020937272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 msgr2=0x7ffa54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/4020937272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa54100ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009930 tx=0x7ffa4002a420 comp rx=0 tx=0).stop 2024-09-12T01:46:11.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/4020937272 shutdown_connections 2024-09-12T01:46:11.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/4020937272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54101420 0x7ffa5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/4020937272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa54100ee0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/4020937272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 0x7ffa54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/4020937272 >> 172.21.15.122:0/4020937272 conn(0x7ffa540fc7a0 msgr2=0x7ffa540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/4020937272 shutdown_connections 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/4020937272 wait complete. 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 Processor -- start 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.544+0000 7ffa5ac1c640 1 -- start start 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa541985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54101420 0x7ffa54198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 0x7ffa5419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54112020 con 0x7ffa54100a80 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54111ea0 con 0x7ffa54106a80 2024-09-12T01:46:11.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541121a0 con 0x7ffa54101420 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa541985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa541985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55956/0 (socket says 172.21.15.122:55956) 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 -- 172.21.15.122:0/3583639100 learned_addr learned my addr 172.21.15.122:0/3583639100 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa591b9640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 0x7ffa5419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 -- 172.21.15.122:0/3583639100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54101420 msgr2=0x7ffa54198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa53fff640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54101420 0x7ffa54198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54101420 0x7ffa54198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 -- 172.21.15.122:0/3583639100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 msgr2=0x7ffa5419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 0x7ffa5419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 -- 172.21.15.122:0/3583639100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5419d550 con 0x7ffa54100a80 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa591b9640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 0x7ffa5419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa589b8640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa541985a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b4a0 tx=0x7ffa4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:11.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa51ffb640 1 -- 172.21.15.122:0/3583639100 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48014070 con 0x7ffa54100a80 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa51ffb640 1 -- 172.21.15.122:0/3583639100 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa48010af0 con 0x7ffa54100a80 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5419d840 con 0x7ffa54100a80 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.545+0000 7ffa51ffb640 1 -- 172.21.15.122:0/3583639100 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48017d10 con 0x7ffa54100a80 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.546+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541a5120 con 0x7ffa54100a80 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.548+0000 7ffa51ffb640 1 -- 172.21.15.122:0/3583639100 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffa48017070 con 0x7ffa54100a80 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.548+0000 7ffa5cff5740 1 Processor -- start 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.548+0000 7ffa5cff5740 1 -- start start 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f09db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa51ffb640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa440a2c90 0x7ffa440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f16fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7e70090 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7e17a70 con 0x55bdf7f099d0 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7e6c050 con 0x55bdf7f14bb0 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f14670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55962/0 (socket says 172.21.15.122:55962) 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 -- 172.21.15.122:0/2302897178 learned_addr learned my addr 172.21.15.122:0/2302897178 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa591b9640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa440a2c90 0x7ffa440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa51ffb640 1 -- 172.21.15.122:0/3583639100 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffa480ce4a0 con 0x7ffa54100a80 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa589b8640 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f09db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 -- 172.21.15.122:0/2302897178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 msgr2=0x55bdf7f16fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f16fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 -- 172.21.15.122:0/2302897178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 msgr2=0x55bdf7f09db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f09db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 -- 172.21.15.122:0/2302897178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf7e24130 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa591b9640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa440a2c90 0x7ffa440a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00c470 tx=0x7ffa4c0099d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.549+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f14670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa40031550 tx=0x7ffa40032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe40c36822e392c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa3effd640 1 -- 172.21.15.122:0/2302897178 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4003f070 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa3effd640 1 -- 172.21.15.122:0/2302897178 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40032e20 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa3effd640 1 -- 172.21.15.122:0/2302897178 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40035070 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2302897178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 msgr2=0x55bdf7f14670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f14670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa40031550 tx=0x7ffa40032cb0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2302897178 shutdown_connections 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f16fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f14670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2302897178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f09db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2302897178 >> 172.21.15.122:0/2302897178 conn(0x55bdf7f05f60 msgr2=0x55bdf7f06340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2302897178 shutdown_connections 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.550+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2302897178 wait complete. 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 -- start start 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 Processor -- start 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 0x55bdf7f14350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f17780 con 0x55bdf7f099d0 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7e2af20 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f17900 con 0x55bdf7f14bb0 2024-09-12T01:46:11.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa5cff5740 10 client.0 _ll_register_callbacks cb 0x55bdf7df4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24428, 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 86.766652055999998, 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 25, 2024-09-12T01:46:11.592 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 116.89859333699999 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa589b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 0x55bdf7f14350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa589b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 0x55bdf7f14350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55966/0 (socket says 172.21.15.122:55966) 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa589b8640 1 -- 172.21.15.122:0/1307107938 learned_addr learned my addr 172.21.15.122:0/1307107938 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.551+0000 7ffa53fff640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa591b9640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa589b8640 1 -- 172.21.15.122:0/1307107938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 msgr2=0x55bdf7f11be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa589b8640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f11be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa589b8640 1 -- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f0a2f0 msgr2=0x55bdf7f116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa589b8640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa589b8640 1 -- 172.21.15.122:0/1307107938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf7be12b0 con 0x55bdf7f099d0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa53fff640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa589b8640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 0x55bdf7f14350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b470 tx=0x7ffa48009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7f1a94875b818f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa33fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa33fff640 1 -- 172.21.15.122:0/1307107938 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4801e070 con 0x55bdf7f099d0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa33fff640 1 -- 172.21.15.122:0/1307107938 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffa480972e0 con 0x55bdf7f099d0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa40032e20 con 0x55bdf7f099d0 2024-09-12T01:46:11.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa33fff640 1 -- 172.21.15.122:0/1307107938 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa480b8050 con 0x55bdf7f099d0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 10 client.15087 fetch_fsmap learned FSMap version 25 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdf7f22f90 con 0x55bdf7f099d0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa33fff640 1 -- 172.21.15.122:0/1307107938 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40032e20 con 0x55bdf7f099d0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa33fff640 1 -- 172.21.15.122:0/1307107938 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa480ce100 con 0x55bdf7f099d0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 10 client.15087 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 10 client.15087 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 20 client.15087 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f255a0 0x55bdf7f26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 4 client.15087 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.552+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf7e7d600 con 0x55bdf7f255a0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.553+0000 7ffa53fff640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f255a0 0x55bdf7f26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.553+0000 7ffa53fff640 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f255a0 0x55bdf7f26a10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=384fd2841fcd64dd server_cookie=bfd48d8937e2608 in_seq=0 out_seq=0 2024-09-12T01:46:11.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.553+0000 7ffa33fff640 10 client.15087 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa33fff640 1 -- 172.21.15.122:0/1307107938 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdf7e7d600 con 0x55bdf7f255a0 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa33fff640 10 client.15087 handle_command_reply: tid=0 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 client.15087 shutdown 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 msgr2=0x55bdf7f14350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 0x55bdf7f14350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b470 tx=0x7ffa48009390 comp rx=0 tx=0).stop 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 shutdown_connections 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f255a0 0x55bdf7f26a10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/1307107938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f099d0 0x55bdf7f14350 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa33fff640 0 client.15087 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:11.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 >> 172.21.15.122:0/1307107938 conn(0x55bdf7f05f60 msgr2=0x55bdf7f0bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 shutdown_connections 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 1 -- 172.21.15.122:0/1307107938 wait complete. 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.554+0000 7ffa5cff5740 20 client.15087 trim_cache size 0 max 16384 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 Processor -- start 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 -- start start 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f0e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f0ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f24d50 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f25420 con 0x55bdf7f099d0 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f25af0 con 0x55bdf7f14bb0 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f0e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f0e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55972/0 (socket says 172.21.15.122:55972) 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa53fff640 1 -- 172.21.15.122:0/2367694328 learned_addr learned my addr 172.21.15.122:0/2367694328 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa589b8640 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.572+0000 7ffa591b9640 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f0ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa53fff640 1 -- 172.21.15.122:0/2367694328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 msgr2=0x55bdf7f0ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f0ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa53fff640 1 -- 172.21.15.122:0/2367694328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 msgr2=0x55bdf7f26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa53fff640 1 -- 172.21.15.122:0/2367694328 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf7f0a120 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa591b9640 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f0e900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa40031550 tx=0x7ffa40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f769978099805 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa3e7fc640 1 -- 172.21.15.122:0/2367694328 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40035040 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa3e7fc640 1 -- 172.21.15.122:0/2367694328 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40052020 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.573+0000 7ffa3e7fc640 1 -- 172.21.15.122:0/2367694328 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4003f040 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2367694328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 msgr2=0x55bdf7f0e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f0e900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffa40031550 tx=0x7ffa40000bc0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2367694328 shutdown_connections 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f0e900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2367694328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2367694328 >> 172.21.15.122:0/2367694328 conn(0x55bdf7f05f60 msgr2=0x55bdf7f0ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2367694328 shutdown_connections 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2367694328 wait complete. 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 -- start start 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 Processor -- start 2024-09-12T01:46:11.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f28170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.574+0000 7ffa5cff5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f8dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f26890 con 0x55bdf7f14bb0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f261c0 con 0x55bdf7f099d0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdf7f26f60 con 0x55bdf7f0a2f0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 10 client.0 _ll_register_callbacks cb 0x7ffa48009250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f8dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f8dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55980/0 (socket says 172.21.15.122:55980) 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 -- 172.21.15.122:0/2649378183 learned_addr learned my addr 172.21.15.122:0/2649378183 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa53fff640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa589b8640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f28170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 -- 172.21.15.122:0/2649378183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f0a2f0 msgr2=0x55bdf7f286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 -- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 msgr2=0x55bdf7f28170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f28170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 -- 172.21.15.122:0/2649378183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf7f8e1e0 con 0x55bdf7f14bb0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa591b9640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f8dca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00cd00 tx=0x7ffa4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fd8a9cc139cb4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa33fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa33fff640 1 -- 172.21.15.122:0/2649378183 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c029060 con 0x55bdf7f14bb0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa33fff640 1 -- 172.21.15.122:0/2649378183 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffa4c024680 con 0x55bdf7f14bb0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa40052020 con 0x55bdf7f14bb0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa33fff640 1 -- 172.21.15.122:0/2649378183 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c024880 con 0x55bdf7f14bb0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 10 client.15093 fetch_fsmap learned FSMap version 25 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.575+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdf7f8f4f0 con 0x55bdf7f14bb0 2024-09-12T01:46:11.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa33fff640 1 -- 172.21.15.122:0/2649378183 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40052020 con 0x55bdf7f14bb0 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa33fff640 1 -- 172.21.15.122:0/2649378183 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa4c02e070 con 0x55bdf7f14bb0 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa5cff5740 10 client.15093 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa5cff5740 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa5cff5740 20 client.15093 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f94140 0x55bdf7f92e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa5cff5740 4 client.15093 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "status"}) -- 0x55bdf7e77b10 con 0x55bdf7f94140 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa589b8640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f94140 0x55bdf7f92e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa589b8640 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f94140 0x55bdf7f92e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fbcd7fd2f1e3e85 server_cookie=92d2b6ddc47ed852 in_seq=0 out_seq=0 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.576+0000 7ffa33fff640 10 client.15093 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa33fff640 1 -- 172.21.15.122:0/2649378183 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bdf7e77b10 con 0x55bdf7f94140 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa33fff640 10 client.15093 handle_command_reply: tid=0 2024-09-12T01:46:11.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 client.15093 shutdown 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 msgr2=0x55bdf7f8dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f8dca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00cd00 tx=0x7ffa4c013000 comp rx=0 tx=0).stop 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 shutdown_connections 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x55bdf7f94140 0x55bdf7f92e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bdf7f14bb0 0x55bdf7f8dca0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf7f0a2f0 0x55bdf7f286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa33fff640 0 client.15093 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 --2- 172.21.15.122:0/2649378183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf7f099d0 0x55bdf7f28170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 >> 172.21.15.122:0/2649378183 conn(0x55bdf7f05f60 msgr2=0x55bdf7f12d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 shutdown_connections 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 1 -- 172.21.15.122:0/2649378183 wait complete. 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.577+0000 7ffa5cff5740 20 client.15093 trim_cache size 0 max 16384 2024-09-12T01:46:11.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa440a2c90 msgr2=0x7ffa440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa440a2c90 0x7ffa440a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00c470 tx=0x7ffa4c0099d0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 msgr2=0x7ffa541985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa541985a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b4a0 tx=0x7ffa4800b960 comp rx=0 tx=0).stop 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 shutdown_connections 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa440a2c90 0x7ffa440a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa54106a80 0x7ffa5419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa54101420 0x7ffa54198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 --2- 172.21.15.122:0/3583639100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa54100a80 0x7ffa541985a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 >> 172.21.15.122:0/3583639100 conn(0x7ffa540fc7a0 msgr2=0x7ffa540fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 shutdown_connections 2024-09-12T01:46:11.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:11.578+0000 7ffa5ac1c640 1 -- 172.21.15.122:0/3583639100 wait complete. 2024-09-12T01:46:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:11 smithi122 ceph-mon[30925]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:46:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:11 smithi151 ceph-mon[35229]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:46:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:11 smithi183 ceph-mon[35127]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T01:46:12.272 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T01:46:12.516 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.987+0000 7f0abb0b4640 1 -- 172.21.15.122:0/498659284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 msgr2=0x7f0ab41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.987+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/498659284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41029b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009900 tx=0x7f0a9c02a480 comp rx=0 tx=0).stop 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.987+0000 7f0abb0b4640 1 -- 172.21.15.122:0/498659284 shutdown_connections 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.987+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/498659284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ab4102ef0 0x7f0ab410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.987+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/498659284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41029b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.987+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/498659284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4100e20 0x7f0ab4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- 172.21.15.122:0/498659284 >> 172.21.15.122:0/498659284 conn(0x7f0ab40fc7d0 msgr2=0x7f0ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- 172.21.15.122:0/498659284 shutdown_connections 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- 172.21.15.122:0/498659284 wait complete. 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 Processor -- start 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- start start 2024-09-12T01:46:13.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ab4100e20 0x7f0ab41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4102ef0 0x7f0ab41a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4110fc0 con 0x7f0ab4102570 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4110e40 con 0x7f0ab4100e20 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0abb0b4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4111140 con 0x7f0ab4102ef0 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56002/0 (socket says 172.21.15.122:56002) 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.988+0000 7f0ab3fff640 1 -- 172.21.15.122:0/899221521 learned_addr learned my addr 172.21.15.122:0/899221521 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab9651640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4102ef0 0x7f0ab41a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ab4100e20 0x7f0ab41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab3fff640 1 -- 172.21.15.122:0/899221521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4102ef0 msgr2=0x7f0ab41a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4102ef0 0x7f0ab41a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab3fff640 1 -- 172.21.15.122:0/899221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ab4100e20 msgr2=0x7f0ab41993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ab4100e20 0x7f0ab41993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab3fff640 1 -- 172.21.15.122:0/899221521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab41a4970 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab9651640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4102ef0 0x7f0ab41a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41998f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006010 tx=0x7f0a9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab1ffb640 1 -- 172.21.15.122:0/899221521 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c03f070 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab1ffb640 1 -- 172.21.15.122:0/899221521 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c031c30 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0ab1ffb640 1 -- 172.21.15.122:0/899221521 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c035030 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab41a4b40 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.989+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab41a4f90 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.991+0000 7f0ab1ffb640 1 -- 172.21.15.122:0/899221521 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0a9c042070 con 0x7f0ab4102570 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 Processor -- start 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 -- start start 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a452cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a461a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a461fc0 0x558a4a4643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a4a3b9090 con 0x558a4a4531f0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a4a360a70 con 0x558a4a4528d0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0abd48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a4a3b5050 con 0x558a4a461fc0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab1ffb640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0aa00a2c10 0x7f0aa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a461a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a461a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56008/0 (socket says 172.21.15.122:56008) 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 -- 172.21.15.122:0/1931525450 learned_addr learned my addr 172.21.15.122:0/1931525450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab9651640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0aa00a2c10 0x7f0aa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a461fc0 0x558a4a4643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab9651640 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a452cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab1ffb640 1 -- 172.21.15.122:0/899221521 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0a9c0efae0 con 0x7f0ab4102570 2024-09-12T01:46:13.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 -- 172.21.15.122:0/1931525450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a461fc0 msgr2=0x558a4a4643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a461fc0 0x558a4a4643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 -- 172.21.15.122:0/1931525450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 msgr2=0x558a4a452cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a452cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.992+0000 7f0ab8e50640 1 -- 172.21.15.122:0/1931525450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4a36d130 con 0x558a4a4531f0 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0ab9651640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0aa00a2c10 0x7f0aa00a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0aac006fd0 tx=0x7f0aac0146c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a461a80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e70 tx=0x7f0aa400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4b250861aedc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0aaaffd640 1 -- 172.21.15.122:0/1931525450 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4003b70 con 0x558a4a4531f0 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0aaaffd640 1 -- 172.21.15.122:0/1931525450 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa4003d10 con 0x558a4a4531f0 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0aaaffd640 1 -- 172.21.15.122:0/1931525450 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40118c0 con 0x558a4a4531f0 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0abd48d740 1 -- 172.21.15.122:0/1931525450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 msgr2=0x558a4a461a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0abd48d740 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a461a80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e70 tx=0x7f0aa400f730 comp rx=0 tx=0).stop 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0abd48d740 1 -- 172.21.15.122:0/1931525450 shutdown_connections 2024-09-12T01:46:13.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.993+0000 7f0abd48d740 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a461fc0 0x558a4a4643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a461a80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 --2- 172.21.15.122:0/1931525450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a452cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- 172.21.15.122:0/1931525450 >> 172.21.15.122:0/1931525450 conn(0x558a4a44ef60 msgr2=0x558a4a44f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- 172.21.15.122:0/1931525450 shutdown_connections 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- 172.21.15.122:0/1931525450 wait complete. 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- start start 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 Processor -- start 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4528d0 0x558a4a461890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a45ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a461fc0 0x558a4a45f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a4a464bf0 con 0x558a4a4531f0 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a4a373f20 con 0x558a4a461fc0 2024-09-12T01:46:13.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a4a464d70 con 0x558a4a4528d0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0abd48d740 10 client.0 _ll_register_callbacks cb 0x558a4a33d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a45ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a45ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56018/0 (socket says 172.21.15.122:56018) 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.994+0000 7f0ab3fff640 1 -- 172.21.15.122:0/2139513113 learned_addr learned my addr 172.21.15.122:0/2139513113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4528d0 0x558a4a461890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab9651640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a461fc0 0x558a4a45f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab3fff640 1 -- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4528d0 msgr2=0x558a4a461890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4528d0 0x558a4a461890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab3fff640 1 -- 172.21.15.122:0/2139513113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a461fc0 msgr2=0x558a4a45f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a461fc0 0x558a4a45f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab3fff640 1 -- 172.21.15.122:0/2139513113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4a12a2b0 con 0x558a4a4531f0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a45ebe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005940 tx=0x7f0a9c08c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02ccc5d398a61f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:13.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0a93fff640 1 -- 172.21.15.122:0/2139513113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c049040 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0a93fff640 1 -- 172.21.15.122:0/2139513113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0a9c0b8850 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aa4003d10 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0a93fff640 1 -- 172.21.15.122:0/2139513113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a9c031de0 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0abd48d740 10 client.15105 fetch_fsmap learned FSMap version 25 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a4a470580 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0a93fff640 1 -- 172.21.15.122:0/2139513113 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa4003d10 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.995+0000 7f0a93fff640 1 -- 172.21.15.122:0/2139513113 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0a9c0c0150 con 0x558a4a4531f0 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0abd48d740 10 client.15105 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0abd48d740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0abd48d740 20 client.15105 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a472c40 0x558a4a4740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0abd48d740 4 client.15105 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a4a3c6600 con 0x558a4a472c40 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a472c40 0x558a4a4740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0a93fff640 10 client.15105 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.996+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a472c40 0x558a4a4740b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdc71c353f0057d2 server_cookie=59b8916461c1d72e in_seq=0 out_seq=0 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0a93fff640 1 -- 172.21.15.122:0/2139513113 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a4a3c6600 con 0x558a4a472c40 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0a93fff640 10 client.15105 handle_command_reply: tid=0 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 client.15105 shutdown 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 msgr2=0x558a4a45ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a45ebe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005940 tx=0x7f0a9c08c150 comp rx=0 tx=0).stop 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 shutdown_connections 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a472c40 0x558a4a4740b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a461fc0 0x558a4a45f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a4531f0 0x558a4a45ebe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2139513113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4528d0 0x558a4a461890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 >> 172.21.15.122:0/2139513113 conn(0x558a4a44ef60 msgr2=0x558a4a454c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:13.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0a93fff640 0 client.15105 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:13.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 shutdown_connections 2024-09-12T01:46:13.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 1 -- 172.21.15.122:0/2139513113 wait complete. 2024-09-12T01:46:13.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:12.997+0000 7f0abd48d740 20 client.15105 trim_cache size 0 max 16384 2024-09-12T01:46:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 Processor -- start 2024-09-12T01:46:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 -- start start 2024-09-12T01:46:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a472570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a45e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a4a4723f0 con 0x558a4a461fc0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a4a472ac0 con 0x558a4a4528d0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0abd48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a4a473190 con 0x558a4a4531f0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a45e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a45e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56022/0 (socket says 172.21.15.122:56022) 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 -- 172.21.15.122:0/3339834442 learned_addr learned my addr 172.21.15.122:0/3339834442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 -- 172.21.15.122:0/3339834442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 msgr2=0x558a4a4740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 -- 172.21.15.122:0/3339834442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 msgr2=0x558a4a472570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a472570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.015+0000 7f0ab9651640 1 -- 172.21.15.122:0/3339834442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4a453020 con 0x558a4a461fc0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0ab9651640 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a45e4a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00afa0 tx=0x7f0aac009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f793522b73a5d329 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0aaa7fc640 1 -- 172.21.15.122:0/3339834442 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0223f0 con 0x558a4a461fc0 2024-09-12T01:46:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0aaa7fc640 1 -- 172.21.15.122:0/3339834442 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aac027070 con 0x558a4a461fc0 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0aaa7fc640 1 -- 172.21.15.122:0/3339834442 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac016890 con 0x558a4a461fc0 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 -- 172.21.15.122:0/3339834442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 msgr2=0x558a4a45e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a45e4a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00afa0 tx=0x7f0aac009000 comp rx=0 tx=0).stop 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 -- 172.21.15.122:0/3339834442 shutdown_connections 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a45e4a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 --2- 172.21.15.122:0/3339834442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a472570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.016+0000 7f0abd48d740 1 -- 172.21.15.122:0/3339834442 >> 172.21.15.122:0/3339834442 conn(0x558a4a44ef60 msgr2=0x558a4a44f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 -- 172.21.15.122:0/3339834442 shutdown_connections 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 -- 172.21.15.122:0/3339834442 wait complete. 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 -- start start 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 Processor -- start 2024-09-12T01:46:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a475370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a475df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a4a473f30 con 0x558a4a461fc0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a4a473860 con 0x558a4a4528d0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a4a474600 con 0x558a4a4531f0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0abd48d740 10 client.0 _ll_register_callbacks cb 0x7f0a9c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0ab9651640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a475df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0ab9651640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a475df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56024/0 (socket says 172.21.15.122:56024) 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0ab9651640 1 -- 172.21.15.122:0/2248425055 learned_addr learned my addr 172.21.15.122:0/2248425055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.017+0000 7f0ab3fff640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a475370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab9651640 1 -- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 msgr2=0x558a4a4758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab9651640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab9651640 1 -- 172.21.15.122:0/2248425055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 msgr2=0x558a4a475370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab9651640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a475370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab9651640 1 -- 172.21.15.122:0/2248425055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4a4db5e0 con 0x558a4a461fc0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0ab9651640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a475df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0aac004810 tx=0x7f0aac022af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e716c53d738ad29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0a93fff640 1 -- 172.21.15.122:0/2248425055 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac02f020 con 0x558a4a461fc0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0a93fff640 1 -- 172.21.15.122:0/2248425055 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0aac016a20 con 0x558a4a461fc0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac027070 con 0x558a4a461fc0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0a93fff640 1 -- 172.21.15.122:0/2248425055 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0aac016c20 con 0x558a4a461fc0 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0abd48d740 10 client.15111 fetch_fsmap learned FSMap version 25 2024-09-12T01:46:13.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a4a4dc8e0 con 0x558a4a461fc0 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.018+0000 7f0a93fff640 1 -- 172.21.15.122:0/2248425055 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aac027070 con 0x558a4a461fc0 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0a93fff640 1 -- 172.21.15.122:0/2248425055 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0aac015040 con 0x558a4a461fc0 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0abd48d740 10 client.15111 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0abd48d740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0abd48d740 20 client.15111 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a4e11b0 0x558a4a4dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0abd48d740 4 client.15111 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "status"}) -- 0x558a4a3c0b10 con 0x558a4a4e11b0 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a4e11b0 0x558a4a4dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0a93fff640 10 client.15111 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0ab8e50640 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a4e11b0 0x558a4a4dfec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=713db6d5d0681843 server_cookie=3334c9f6509c3c94 in_seq=0 out_seq=0 2024-09-12T01:46:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0a93fff640 1 -- 172.21.15.122:0/2248425055 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558a4a3c0b10 con 0x558a4a4e11b0 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.019+0000 7f0a93fff640 10 client.15111 handle_command_reply: tid=0 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 client.15111 shutdown 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 msgr2=0x558a4a475df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a475df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0aac004810 tx=0x7f0aac022af0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 shutdown_connections 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x558a4a4e11b0 0x558a4a4dfec0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a4a461fc0 0x558a4a475df0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a4a4531f0 0x558a4a4758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0a93fff640 0 client.15111 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 --2- 172.21.15.122:0/2248425055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4a4528d0 0x558a4a475370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 >> 172.21.15.122:0/2248425055 conn(0x558a4a44ef60 msgr2=0x558a4a459a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 shutdown_connections 2024-09-12T01:46:13.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 1 -- 172.21.15.122:0/2248425055 wait complete. 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abd48d740 20 client.15111 trim_cache size 0 max 16384 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24436, 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:46:13.030 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 89.220668677999996, 2024-09-12T01:46:13.031 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 25, 2024-09-12T01:46:13.031 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:46:13.031 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:46:13.031 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 121.115523534 2024-09-12T01:46:13.031 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0aa00a2c10 msgr2=0x7f0aa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0aa00a2c10 0x7f0aa00a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0aac006fd0 tx=0x7f0aac0146c0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 msgr2=0x7f0ab41998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.020+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41998f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006010 tx=0x7f0a9c003c10 comp rx=0 tx=0).stop 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 shutdown_connections 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0aa00a2c10 0x7f0aa00a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ab4102ef0 0x7f0ab41a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ab4102570 0x7f0ab41998f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 --2- 172.21.15.122:0/899221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ab4100e20 0x7f0ab41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 >> 172.21.15.122:0/899221521 conn(0x7f0ab40fc7d0 msgr2=0x7f0ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:13.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 shutdown_connections 2024-09-12T01:46:13.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:13.021+0000 7f0abb0b4640 1 -- 172.21.15.122:0/899221521 wait complete. 2024-09-12T01:46:13.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:13 smithi122 ceph-mon[30925]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:13.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T01:46:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:13 smithi151 ceph-mon[35229]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:13 smithi183 ceph-mon[35127]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T01:46:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:14.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.545+0000 7f1846746640 1 -- 172.21.15.122:0/2406172322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 msgr2=0x7f184010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.545+0000 7f1846746640 1 --2- 172.21.15.122:0/2406172322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f184010d930 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1838009cd0 tx=0x7f183802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 -- 172.21.15.122:0/2406172322 shutdown_connections 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 --2- 172.21.15.122:0/2406172322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f184010d930 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 --2- 172.21.15.122:0/2406172322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840100a60 0x7f1840100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 --2- 172.21.15.122:0/2406172322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1840106a60 0x7f1840106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 -- 172.21.15.122:0/2406172322 >> 172.21.15.122:0/2406172322 conn(0x7f18400fc780 msgr2=0x7f18400feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 -- 172.21.15.122:0/2406172322 shutdown_connections 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 -- 172.21.15.122:0/2406172322 wait complete. 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 Processor -- start 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.546+0000 7f1846746640 1 -- start start 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1840100a60 0x7f1840198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f1840198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840106a60 0x7f184019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1840110e60 con 0x7f1840101400 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1840110ce0 con 0x7f1840106a60 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1840110fe0 con 0x7f1840100a60 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f1840198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f1840198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56054/0 (socket says 172.21.15.122:56054) 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 -- 172.21.15.122:0/2336353354 learned_addr learned my addr 172.21.15.122:0/2336353354 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183ffff640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1840100a60 0x7f1840198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1844ce3640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840106a60 0x7f184019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 -- 172.21.15.122:0/2336353354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1840100a60 msgr2=0x7f1840198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1840100a60 0x7f1840198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 -- 172.21.15.122:0/2336353354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840106a60 msgr2=0x7f184019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840106a60 0x7f184019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 -- 172.21.15.122:0/2336353354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184019d3d0 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1844ce3640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840106a60 0x7f184019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183f7fe640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f1840198960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f183400d950 tx=0x7f183400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183d7fa640 1 -- 172.21.15.122:0/2336353354 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834017070 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183d7fa640 1 -- 172.21.15.122:0/2336353354 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1834009c30 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184019d6c0 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f183d7fa640 1 -- 172.21.15.122:0/2336353354 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183400f6b0 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.547+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18401a4fa0 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.549+0000 7f183d7fa640 1 -- 172.21.15.122:0/2336353354 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f18340109d0 con 0x7f1840101400 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 Processor -- start 2024-09-12T01:46:14.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 -- start start 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147759d0 0x55ac14775db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac14780670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac14782fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac146dc090 con 0x55ac147759d0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac14683a70 con 0x55ac14780bb0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1848b1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac146d8050 con 0x55ac147762f0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f183d7fa640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18280a7020 0x7f18280a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1844ce3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147759d0 0x55ac14775db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1844ce3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147759d0 0x55ac14775db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56066/0 (socket says 172.21.15.122:56066) 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1844ce3640 1 -- 172.21.15.122:0/2969934401 learned_addr learned my addr 172.21.15.122:0/2969934401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f183ffff640 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac14780670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f1844ce3640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18280a7020 0x7f18280a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f183f7fe640 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac14782fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.550+0000 7f183d7fa640 1 -- 172.21.15.122:0/2336353354 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f18340cee60 con 0x7f1840101400 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f183ffff640 1 -- 172.21.15.122:0/2969934401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 msgr2=0x55ac14782fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f183ffff640 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac14782fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f183ffff640 1 -- 172.21.15.122:0/2969934401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147759d0 msgr2=0x55ac14775db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f183ffff640 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147759d0 0x55ac14775db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f183ffff640 1 -- 172.21.15.122:0/2969934401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac14690130 con 0x55ac147762f0 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f1844ce3640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18280a7020 0x7f18280a94e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1838033040 tx=0x7f18380093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f183ffff640 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac14780670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18300045b0 tx=0x7f183000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2379c5e31177044e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f18267fc640 1 -- 172.21.15.122:0/2969934401 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830003bd0 con 0x55ac147762f0 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f18267fc640 1 -- 172.21.15.122:0/2969934401 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830015070 con 0x55ac147762f0 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.551+0000 7f18267fc640 1 -- 172.21.15.122:0/2969934401 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830004d00 con 0x55ac147762f0 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 -- 172.21.15.122:0/2969934401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 msgr2=0x55ac14780670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac14780670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18300045b0 tx=0x7f183000bb20 comp rx=0 tx=0).stop 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 -- 172.21.15.122:0/2969934401 shutdown_connections 2024-09-12T01:46:14.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac14782fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac14780670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 --2- 172.21.15.122:0/2969934401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147759d0 0x55ac14775db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 -- 172.21.15.122:0/2969934401 >> 172.21.15.122:0/2969934401 conn(0x55ac14771f60 msgr2=0x55ac14772340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 -- 172.21.15.122:0/2969934401 shutdown_connections 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 -- 172.21.15.122:0/2969934401 wait complete. 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 -- start start 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.552+0000 7f1848b1f740 1 Processor -- start 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac14780430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac1477d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 0x55ac1477dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac14783780 con 0x55ac14780bb0 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac14696f20 con 0x55ac147759d0 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac14783900 con 0x55ac147762f0 2024-09-12T01:46:14.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 10 client.0 _ll_register_callbacks cb 0x55ac14660030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 0x55ac1477dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 0x55ac1477dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56076/0 (socket says 172.21.15.122:56076) 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 -- 172.21.15.122:0/1356432664 learned_addr learned my addr 172.21.15.122:0/1356432664 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f183f7fe640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac1477d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f183ffff640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac14780430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 -- 172.21.15.122:0/1356432664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 msgr2=0x55ac1477d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac1477d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 -- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 msgr2=0x55ac14780430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac14780430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 -- 172.21.15.122:0/1356432664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1444d2b0 con 0x55ac14780bb0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1844ce3640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 0x55ac1477dcc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1838031e90 tx=0x7f1838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d695ab60d9a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f181f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f181f7fe640 1 -- 172.21.15.122:0/1356432664 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183804f070 con 0x55ac14780bb0 2024-09-12T01:46:14.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1830015070 con 0x55ac14780bb0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.553+0000 7f181f7fe640 1 -- 172.21.15.122:0/1356432664 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1838004250 con 0x55ac14780bb0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f181f7fe640 1 -- 172.21.15.122:0/1356432664 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1838004450 con 0x55ac14780bb0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 10 client.15123 fetch_fsmap learned FSMap version 25 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac1478f0e0 con 0x55ac14780bb0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f181f7fe640 1 -- 172.21.15.122:0/1356432664 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830015070 con 0x55ac14780bb0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f181f7fe640 1 -- 172.21.15.122:0/1356432664 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1838030070 con 0x55ac14780bb0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 10 client.15123 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 20 client.15123 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147916c0 0x55ac14792b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 4 client.15123 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac146e9600 con 0x55ac147916c0 2024-09-12T01:46:14.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.554+0000 7f183ffff640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147916c0 0x55ac14792b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f181f7fe640 10 client.15123 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f183ffff640 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147916c0 0x55ac14792b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c34bbfd03830fc server_cookie=d33a86bc01e08f80 in_seq=0 out_seq=0 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f181f7fe640 1 -- 172.21.15.122:0/1356432664 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac146e9600 con 0x55ac147916c0 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f181f7fe640 10 client.15123 handle_command_reply: tid=0 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f1848b1f740 1 client.15123 shutdown 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 msgr2=0x55ac1477dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.555+0000 7f1848b1f740 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 0x55ac1477dcc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1838031e90 tx=0x7f1838004a40 comp rx=0 tx=0).stop 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 shutdown_connections 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147916c0 0x55ac14792b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac14780bb0 0x55ac1477dcc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147762f0 0x55ac1477d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 --2- 172.21.15.122:0/1356432664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac14780430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 >> 172.21.15.122:0/1356432664 conn(0x55ac14771f60 msgr2=0x55ac14777c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f181f7fe640 0 client.15123 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 shutdown_connections 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 1 -- 172.21.15.122:0/1356432664 wait complete. 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.556+0000 7f1848b1f740 20 client.15123 trim_cache size 0 max 16384 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 Processor -- start 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 -- start start 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac147916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac1477d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac14780bb0 0x55ac1477dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac14790e70 con 0x55ac147762f0 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac14791540 con 0x55ac147759d0 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.573+0000 7f1848b1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac14791c10 con 0x55ac14780bb0 2024-09-12T01:46:14.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac1477d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac1477d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56080/0 (socket says 172.21.15.122:56080) 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183f7fe640 1 -- 172.21.15.122:0/148981020 learned_addr learned my addr 172.21.15.122:0/148981020 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac147916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f1844ce3640 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac14780bb0 0x55ac1477dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 -- 172.21.15.122:0/148981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac14780bb0 msgr2=0x55ac1477dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac14780bb0 0x55ac1477dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 -- 172.21.15.122:0/148981020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 msgr2=0x55ac1477d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac1477d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 -- 172.21.15.122:0/148981020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac14771d90 con 0x55ac147759d0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183f7fe640 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac1477d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.574+0000 7f183ffff640 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac147916c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1830006010 tx=0x7f183000b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ad5d6ba4ec52ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.575+0000 7f1825ffb640 1 -- 172.21.15.122:0/148981020 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830003dd0 con 0x55ac147759d0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.575+0000 7f1825ffb640 1 -- 172.21.15.122:0/148981020 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830015070 con 0x55ac147759d0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.575+0000 7f1825ffb640 1 -- 172.21.15.122:0/148981020 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300116a0 con 0x55ac147759d0 2024-09-12T01:46:14.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.575+0000 7f1848b1f740 1 -- 172.21.15.122:0/148981020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 msgr2=0x55ac147916c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.575+0000 7f1848b1f740 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac147916c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1830006010 tx=0x7f183000b5b0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- 172.21.15.122:0/148981020 shutdown_connections 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac14780bb0 0x55ac1477dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac1477d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 --2- 172.21.15.122:0/148981020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac147759d0 0x55ac147916c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- 172.21.15.122:0/148981020 >> 172.21.15.122:0/148981020 conn(0x55ac14771f60 msgr2=0x55ac1477ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- 172.21.15.122:0/148981020 shutdown_connections 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- 172.21.15.122:0/148981020 wait complete. 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- start start 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 Processor -- start 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 0x55ac14793c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac14794180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac147946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac147929b0 con 0x55ac147762f0 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac147922e0 con 0x55ac14780bb0 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.576+0000 7f1848b1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac14793080 con 0x55ac147759d0 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f1848b1f740 10 client.0 _ll_register_callbacks cb 0x7f18380098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 0x55ac14793c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 0x55ac14793c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:46538/0 (socket says 172.21.15.122:46538) 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 -- 172.21.15.122:0/84976324 learned_addr learned my addr 172.21.15.122:0/84976324 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f1844ce3640 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac147946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 -- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 msgr2=0x55ac147946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac147946c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 -- 172.21.15.122:0/84976324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 msgr2=0x55ac14794180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac14794180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 -- 172.21.15.122:0/84976324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac147f9e40 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f183ffff640 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 0x55ac14793c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18300039c0 tx=0x7f183000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30af1e39af4bf3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.577+0000 7f181f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.584+0000 7f183d7fa640 1 -- 172.21.15.122:0/2336353354 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1834097bd0 con 0x7f1840101400 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.585+0000 7f181f7fe640 1 -- 172.21.15.122:0/84976324 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300041b0 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.585+0000 7f181f7fe640 1 -- 172.21.15.122:0/84976324 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1830003de0 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.585+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1830015070 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f181f7fe640 1 -- 172.21.15.122:0/84976324 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f183002f440 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 10 client.24706 fetch_fsmap learned FSMap version 25 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac147fb120 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f181f7fe640 1 -- 172.21.15.122:0/84976324 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830015070 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f181f7fe640 1 -- 172.21.15.122:0/84976324 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f183002d020 con 0x55ac147759d0 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 10 client.24706 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T01:46:14.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 10 client.24706 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 20 client.24706 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147ff580 0x55ac147fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24452, 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:46:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 97.818405523999999, 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 25, 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 108.413908554 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:14.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 4 client.24706 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "status"}) -- 0x55ac146e3b10 con 0x55ac147ff580 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.586+0000 7f183f7fe640 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147ff580 0x55ac147fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f181f7fe640 10 client.24706 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f183f7fe640 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147ff580 0x55ac147fe290 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d4238ff588c1d1 server_cookie=f2b7e58cc3a34cdd in_seq=0 out_seq=0 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f181f7fe640 1 -- 172.21.15.122:0/84976324 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ac146e3b10 con 0x55ac147ff580 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f181f7fe640 10 client.24706 handle_command_reply: tid=0 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 client.24706 shutdown 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 msgr2=0x55ac14793c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 0x55ac14793c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18300039c0 tx=0x7f183000be50 comp rx=0 tx=0).stop 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 shutdown_connections 2024-09-12T01:46:14.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac147ff580 0x55ac147fe290 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac14780bb0 0x55ac147946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac147762f0 0x55ac14794180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 --2- 172.21.15.122:0/84976324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac147759d0 0x55ac14793c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f181f7fe640 0 client.24706 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 >> 172.21.15.122:0/84976324 conn(0x55ac14771f60 msgr2=0x55ac1477d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 shutdown_connections 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.587+0000 7f1848b1f740 1 -- 172.21.15.122:0/84976324 wait complete. 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1848b1f740 20 client.24706 trim_cache size 0 max 16384 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18280a7020 msgr2=0x7f18280a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18280a7020 0x7f18280a94e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1838033040 tx=0x7f18380093f0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 msgr2=0x7f1840198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f1840198960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f183400d950 tx=0x7f183400de10 comp rx=0 tx=0).stop 2024-09-12T01:46:14.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 shutdown_connections 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18280a7020 0x7f18280a94e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1840106a60 0x7f184019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1840101400 0x7f1840198960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 --2- 172.21.15.122:0/2336353354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1840100a60 0x7f1840198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 >> 172.21.15.122:0/2336353354 conn(0x7f18400fc780 msgr2=0x7f18401048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 shutdown_connections 2024-09-12T01:46:14.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:14.588+0000 7f1846746640 1 -- 172.21.15.122:0/2336353354 wait complete. 2024-09-12T01:46:15.263 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T01:46:15.273 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T01:46:15.273 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T01:46:15.274 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T01:46:15.274 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T01:46:15.303 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T01:46:15.303 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T01:46:15.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:15.303 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-12T01:46:15.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:15.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-12T01:46:15.390 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-09-12T01:46:15.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:46:15.391 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T01:46:15.391 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T01:46:15.392 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:15.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T01:46:15.688 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:15.811 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:15 smithi122 ceph-mon[30925]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:46:15.811 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:15 smithi122 ceph-mon[30925]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T01:46:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:15 smithi151 ceph-mon[35229]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:46:15.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:15 smithi151 ceph-mon[35229]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T01:46:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:15 smithi183 ceph-mon[35127]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T01:46:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:15 smithi183 ceph-mon[35127]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T01:46:16.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 -- 172.21.15.122:0/3711789601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 msgr2=0x7fe66010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:16.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 --2- 172.21.15.122:0/3711789601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe66010da40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe658009ca0 tx=0x7fe65802a950 comp rx=0 tx=0).stop 2024-09-12T01:46:16.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 -- 172.21.15.122:0/3711789601 shutdown_connections 2024-09-12T01:46:16.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 --2- 172.21.15.122:0/3711789601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe66010da40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 --2- 172.21.15.122:0/3711789601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe660102570 0x7fe6601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 --2- 172.21.15.122:0/3711789601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe660100e20 0x7fe660101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 -- 172.21.15.122:0/3711789601 >> 172.21.15.122:0/3711789601 conn(0x7fe6600fc7d0 msgr2=0x7fe6600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 -- 172.21.15.122:0/3711789601 shutdown_connections 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.171+0000 7fe667f0d640 1 -- 172.21.15.122:0/3711789601 wait complete. 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 Processor -- start 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 -- start start 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe660100e20 0x7fe660199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe660102570 0x7fe6601998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe6601a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe660110fa0 con 0x7fe660102ef0 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe660110e20 con 0x7fe660102570 2024-09-12T01:46:16.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe667f0d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe660111120 con 0x7fe660100e20 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe6601a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe6601a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56096/0 (socket says 172.21.15.122:56096) 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 -- 172.21.15.122:0/229419422 learned_addr learned my addr 172.21.15.122:0/229419422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe665ca9640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe660100e20 0x7fe660199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 -- 172.21.15.122:0/229419422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe660100e20 msgr2=0x7fe660199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6654a8640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe660102570 0x7fe6601998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe660100e20 0x7fe660199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 -- 172.21.15.122:0/229419422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe660102570 msgr2=0x7fe6601998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe660102570 0x7fe6601998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.172+0000 7fe6664aa640 1 -- 172.21.15.122:0/229419422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6601a4930 con 0x7fe660102ef0 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.173+0000 7fe6664aa640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe6601a43f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe65802ae70 tx=0x7fe65802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.173+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658034040 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.173+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658030920 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.173+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658045480 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.173+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6601a4b00 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.173+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6601a4f60 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.174+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe660101200 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.175+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe658037020 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.176+0000 7fe656ffd640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe64c0a2c00 0x7fe64c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.176+0000 7fe665ca9640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe64c0a2c00 0x7fe64c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.176+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe6580eff50 con 0x7fe660102ef0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.176+0000 7fe665ca9640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe64c0a2c00 0x7fe64c0a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00a810 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.180+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6580bceb0 con 0x7fe660102ef0 2024-09-12T01:46:16.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.390+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe6601a5190 con 0x7fe660102ef0 2024-09-12T01:46:16.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.391+0000 7fe656ffd640 1 -- 172.21.15.122:0/229419422 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fe6580bc7b0 con 0x7fe660102ef0 2024-09-12T01:46:16.392 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T01:46:16.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe64c0a2c00 msgr2=0x7fe64c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:16.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe64c0a2c00 0x7fe64c0a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00a810 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2024-09-12T01:46:16.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 msgr2=0x7fe6601a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:16.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe6601a43f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe65802ae70 tx=0x7fe65802f5d0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 shutdown_connections 2024-09-12T01:46:16.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe64c0a2c00 0x7fe64c0a50c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102ef0 0x7fe6601a43f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe660102570 0x7fe6601998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 --2- 172.21.15.122:0/229419422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe660100e20 0x7fe660199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:16.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 >> 172.21.15.122:0/229419422 conn(0x7fe6600fc7d0 msgr2=0x7fe660106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:16.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 shutdown_connections 2024-09-12T01:46:16.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:16.393+0000 7fe667f0d640 1 -- 172.21.15.122:0/229419422 wait complete. 2024-09-12T01:46:16.702 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:16 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/229419422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:46:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:16 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/229419422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:46:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/229419422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T01:46:17.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T01:46:17.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T01:46:17.049 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:46:17.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:17.049 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:46:17.066 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42209sec preferred_lft 42209sec 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link noprefixroute 2024-09-12T01:46:17.067 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-12T01:46:17.068 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T01:46:17.068 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-12T01:46:17.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:46:17.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:17.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:17.276 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T01:46:17.300 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-12T01:46:17.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:17.302 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-12T01:46:17.377 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-12T01:46:17.377 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T01:46:17.377 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T01:46:17.378 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T01:46:17.378 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T01:46:17.378 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:46:17.378 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-12T01:46:17.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T01:46:17.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T01:46:17.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T01:46:17.378 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-12T01:46:17.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:46:17.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:17.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:17.670 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-12T01:46:17.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:17.686 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-09-12T01:46:17.742 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:46:17.742 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-12T01:46:17.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:46:17.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T01:46:17.742 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-12T01:46:17.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:46:17.837 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:17 smithi122 ceph-mon[30925]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:46:17.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:17.859 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T01:46:17.859 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:46:17.859 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-12T01:46:17.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T01:46:17.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T01:46:17.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T01:46:17.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T01:46:17.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T01:46:17.860 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-12T01:46:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:17 smithi151 ceph-mon[35229]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:46:17.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:46:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:17 smithi183 ceph-mon[35127]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:46:17.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:17.982 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T01:46:17.982 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-12T01:46:17.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-12T01:46:17.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T01:46:17.982 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-12T01:46:18.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:46:18.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:18.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:46:18.058 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:18.073 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T01:46:18.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T01:46:18.074 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:18.128 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T01:46:18.128 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T01:46:18.129 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff 2024-09-12T01:46:18.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:18.129 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T01:46:18.231 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T01:46:18.232 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T01:46:18.232 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-12T01:46:18.361 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T01:46:18.361 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T01:46:18.361 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-12T01:46:18.361 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.151:3300,172.21.15.183:3300" 2024-09-12T01:46:18.361 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:3300,172.21.15.151:3300,172.21.15.183:3300:/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff 2024-09-12T01:46:18.361 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=5731f69a-70a7-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T01:46:18.362 INFO:teuthology.orchestra.run.smithi122.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T01:46:18.362 INFO:teuthology.orchestra.run.smithi122.stdout: You just mounted a file system that supports labels which does not 2024-09-12T01:46:18.362 INFO:teuthology.orchestra.run.smithi122.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T01:46:18.362 INFO:teuthology.orchestra.run.smithi122.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T01:46:18.362 INFO:teuthology.orchestra.run.smithi122.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T01:46:18.364 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T01:46:18.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:46:18.365 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:18.391 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T01:46:18.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:46:18.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:18.489 INFO:teuthology.orchestra.run.smithi122.stdout:client15135 2024-09-12T01:46:18.489 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:46:18.489 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:46:18.506 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T01:46:18.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:46:18.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:18.603 INFO:teuthology.orchestra.run.smithi122.stdout:client15135 2024-09-12T01:46:18.603 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T01:46:18.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T01:46:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:18.659 INFO:teuthology.orchestra.run.smithi122.stdout:client15135 2024-09-12T01:46:18.659 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5731f69a-70a7-11ef-bcea-c7b262605968.client15135/status') 2024-09-12T01:46:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T01:46:18.717 INFO:teuthology.orchestra.run.smithi122.stdout:instance: client.15135 (3)192.168.0.1:0/1150577743 2024-09-12T01:46:18.717 INFO:teuthology.orchestra.run.smithi122.stdout:blocklisted: false 2024-09-12T01:46:18.718 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2024-09-12T01:46:18.718 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2024-09-12T01:46:18.718 INFO:teuthology.orchestra.run.smithi122.stderr:70 bytes copied, 2.2244e-05 s, 3.1 MB/s 2024-09-12T01:46:18.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T01:46:18.725 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T01:46:18.735 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T01:46:18.735 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T01:46:18.735 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T01:46:18.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 -- 172.21.15.122:0/996163759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c101c30 msgr2=0x7f250c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 --2- 172.21.15.122:0/996163759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c101c30 0x7f250c1020b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2500009900 tx=0x7f250002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 -- 172.21.15.122:0/996163759 shutdown_connections 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 --2- 172.21.15.122:0/996163759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c1025f0 0x7f250c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 --2- 172.21.15.122:0/996163759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c101c30 0x7f250c1020b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 --2- 172.21.15.122:0/996163759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c100a40 0x7f250c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 -- 172.21.15.122:0/996163759 >> 172.21.15.122:0/996163759 conn(0x7f250c0fc1e0 msgr2=0x7f250c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 -- 172.21.15.122:0/996163759 shutdown_connections 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.836+0000 7f2512781640 1 -- 172.21.15.122:0/996163759 wait complete. 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 Processor -- start 2024-09-12T01:46:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 -- start start 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c100a40 0x7f250c196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c101c30 0x7f250c1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 0x7f250c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f250c10c6c0 con 0x7f250c1025f0 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f250c10c540 con 0x7f250c100a40 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2512781640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f250c10c840 con 0x7f250c101c30 2024-09-12T01:46:18.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2510d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 0x7f250c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2510d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 0x7f250c19e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56120/0 (socket says 172.21.15.122:56120) 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f2510d1e640 1 -- 172.21.15.122:0/3365106431 learned_addr learned my addr 172.21.15.122:0/3365106431 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f250b7fe640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c101c30 0x7f250c1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.837+0000 7f250bfff640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c100a40 0x7f250c196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2510d1e640 1 -- 172.21.15.122:0/3365106431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c101c30 msgr2=0x7f250c1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2510d1e640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c101c30 0x7f250c1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2510d1e640 1 -- 172.21.15.122:0/3365106431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c100a40 msgr2=0x7f250c196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2510d1e640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c100a40 0x7f250c196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2510d1e640 1 -- 172.21.15.122:0/3365106431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c19e8f0 con 0x7f250c1025f0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2510d1e640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 0x7f250c19e250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f250400eaa0 tx=0x7f250400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250400cdb0 con 0x7f250c1025f0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2504005020 con 0x7f250c1025f0 2024-09-12T01:46:18.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25040116b0 con 0x7f250c1025f0 2024-09-12T01:46:18.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c19ebe0 con 0x7f250c1025f0 2024-09-12T01:46:18.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.838+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c19f100 con 0x7f250c1025f0 2024-09-12T01:46:18.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.839+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c107db0 con 0x7f250c1025f0 2024-09-12T01:46:18.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.840+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2504005560 con 0x7f250c1025f0 2024-09-12T01:46:18.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.841+0000 7f25097fa640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24f40a2c30 0x7f24f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:18.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.841+0000 7f250bfff640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24f40a2c30 0x7f24f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:18.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.841+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f25040cf290 con 0x7f250c1025f0 2024-09-12T01:46:18.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.842+0000 7f250bfff640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24f40a2c30 0x7f24f40a50f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f250c197b80 tx=0x7f24fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:18.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:18.845+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2504016030 con 0x7f250c1025f0 2024-09-12T01:46:19.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.049+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f250c062e90 con 0x7f250c1025f0 2024-09-12T01:46:19.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.587+0000 7f25097fa640 1 -- 172.21.15.122:0/3365106431 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f250409c1f0 con 0x7f250c1025f0 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.589+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24f40a2c30 msgr2=0x7f24f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.589+0000 7f2512781640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24f40a2c30 0x7f24f40a50f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f250c197b80 tx=0x7f24fc005f90 comp rx=0 tx=0).stop 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.589+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 msgr2=0x7f250c19e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.589+0000 7f2512781640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 0x7f250c19e250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f250400eaa0 tx=0x7f250400ef60 comp rx=0 tx=0).stop 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 shutdown_connections 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24f40a2c30 0x7f24f40a50f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f250c1025f0 0x7f250c19e250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c101c30 0x7f250c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 --2- 172.21.15.122:0/3365106431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250c100a40 0x7f250c196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 >> 172.21.15.122:0/3365106431 conn(0x7f250c0fc1e0 msgr2=0x7f250c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 shutdown_connections 2024-09-12T01:46:19.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.590+0000 7f2512781640 1 -- 172.21.15.122:0/3365106431 wait complete. 2024-09-12T01:46:19.601 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T01:46:19.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 -- 172.21.15.122:0/895946726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0101c40 msgr2=0x7f4bd01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/895946726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0101c40 0x7f4bd01020c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009900 tx=0x7f4bc402a410 comp rx=0 tx=0).stop 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 -- 172.21.15.122:0/895946726 shutdown_connections 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/895946726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd0102600 0x7f4bd0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/895946726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0101c40 0x7f4bd01020c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/895946726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd0100a50 0x7f4bd0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 -- 172.21.15.122:0/895946726 >> 172.21.15.122:0/895946726 conn(0x7f4bd00fc1b0 msgr2=0x7f4bd00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 -- 172.21.15.122:0/895946726 shutdown_connections 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.704+0000 7f4bd5f03640 1 -- 172.21.15.122:0/895946726 wait complete. 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 Processor -- start 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 -- start start 2024-09-12T01:46:19.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd0100a50 0x7f4bd0196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd0101c40 0x7f4bd01971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 0x7f4bd019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd010c6d0 con 0x7f4bd0102600 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd010c550 con 0x7f4bd0100a50 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bd5f03640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd010c850 con 0x7f4bd0101c40 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 0x7f4bd019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 0x7f4bd019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56144/0 (socket says 172.21.15.122:56144) 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 -- 172.21.15.122:0/1896001084 learned_addr learned my addr 172.21.15.122:0/1896001084 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bceffd640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd0101c40 0x7f4bd01971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:19.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 -- 172.21.15.122:0/1896001084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd0101c40 msgr2=0x7f4bd01971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcf7fe640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd0100a50 0x7f4bd0196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd0101c40 0x7f4bd01971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 -- 172.21.15.122:0/1896001084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd0100a50 msgr2=0x7f4bd0196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd0100a50 0x7f4bd0196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 -- 172.21.15.122:0/1896001084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd019e8d0 con 0x7f4bd0102600 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.705+0000 7f4bcffff640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 0x7f4bd019e230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800ebb0 tx=0x7f4bc8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.706+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8007ca0 con 0x7f4bd0102600 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.706+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc8007e40 con 0x7f4bd0102600 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.706+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80058a0 con 0x7f4bd0102600 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.706+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd006a300 con 0x7f4bd0102600 2024-09-12T01:46:19.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.706+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd006a850 con 0x7f4bd0102600 2024-09-12T01:46:19.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.707+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd0100e50 con 0x7f4bd0102600 2024-09-12T01:46:19.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.708+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4bc8004a40 con 0x7f4bd0102600 2024-09-12T01:46:19.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.708+0000 7f4bccff9640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4bb80a2c30 0x7f4bb80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:19.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.709+0000 7f4bcf7fe640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4bb80a2c30 0x7f4bb80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:19.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.709+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4bc80d06b0 con 0x7f4bd0102600 2024-09-12T01:46:19.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.709+0000 7f4bcf7fe640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4bb80a2c30 0x7f4bb80a50f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0197b60 tx=0x7f4bc0005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:19.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.713+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bc809d610 con 0x7f4bd0102600 2024-09-12T01:46:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:19 smithi151 ceph-mon[35229]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T01:46:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:19 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3365106431' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T01:46:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.908+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f4bd006aab0 con 0x7f4bb80a2c30 2024-09-12T01:46:19.913 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-mon[30925]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T01:46:19.914 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3365106431' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T01:46:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.926+0000 7f4bccff9640 1 -- 172.21.15.122:0/1896001084 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f4bd006aab0 con 0x7f4bb80a2c30 2024-09-12T01:46:19.926 INFO:teuthology.orchestra.run.smithi122.stdout:{} 2024-09-12T01:46:19.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4bb80a2c30 msgr2=0x7f4bb80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4bb80a2c30 0x7f4bb80a50f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0197b60 tx=0x7f4bc0005c80 comp rx=0 tx=0).stop 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 msgr2=0x7f4bd019e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 0x7f4bd019e230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800ebb0 tx=0x7f4bc8007590 comp rx=0 tx=0).stop 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 shutdown_connections 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4bb80a2c30 0x7f4bb80a50f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bd0102600 0x7f4bd019e230 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.928+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd0101c40 0x7f4bd01971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.929+0000 7f4bd5f03640 1 --2- 172.21.15.122:0/1896001084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bd0100a50 0x7f4bd0196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.929+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 >> 172.21.15.122:0/1896001084 conn(0x7f4bd00fc1b0 msgr2=0x7f4bd0104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.929+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 shutdown_connections 2024-09-12T01:46:19.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:19.929+0000 7f4bd5f03640 1 -- 172.21.15.122:0/1896001084 wait complete. 2024-09-12T01:46:19.939 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T01:46:19.950 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T01:46:20.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:46:20.018 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:19 smithi183 ceph-mon[35127]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T01:46:20.018 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:19 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3365106431' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T01:46:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 -- 172.21.15.122:0/2834490194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118101c20 msgr2=0x7f61181020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 --2- 172.21.15.122:0/2834490194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118101c20 0x7f61181020a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6108009ca0 tx=0x7f610802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 -- 172.21.15.122:0/2834490194 shutdown_connections 2024-09-12T01:46:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 --2- 172.21.15.122:0/2834490194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61181025e0 0x7f6118108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 --2- 172.21.15.122:0/2834490194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118101c20 0x7f61181020a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 --2- 172.21.15.122:0/2834490194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6118100a30 0x7f6118100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 -- 172.21.15.122:0/2834490194 >> 172.21.15.122:0/2834490194 conn(0x7f61180fc1b0 msgr2=0x7f61180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 -- 172.21.15.122:0/2834490194 shutdown_connections 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.117+0000 7f611f03c640 1 -- 172.21.15.122:0/2834490194 wait complete. 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 Processor -- start 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 -- start start 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 0x7f6118197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6118101c20 0x7f6118198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61181025e0 0x7f611819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f611810d800 con 0x7f6118100a30 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f611810d680 con 0x7f61181025e0 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611f03c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f611810d980 con 0x7f6118101c20 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 0x7f6118197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 0x7f6118197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56178/0 (socket says 172.21.15.122:56178) 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 -- 172.21.15.122:0/2065777062 learned_addr learned my addr 172.21.15.122:0/2065777062 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f6117fff640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6118101c20 0x7f6118198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 -- 172.21.15.122:0/2065777062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6118101c20 msgr2=0x7f6118198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611d5d9640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61181025e0 0x7f611819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6118101c20 0x7f6118198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 -- 172.21.15.122:0/2065777062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61181025e0 msgr2=0x7f611819f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61181025e0 0x7f611819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.118+0000 7f611cdd8640 1 -- 172.21.15.122:0/2065777062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611819fb00 con 0x7f6118100a30 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.119+0000 7f611cdd8640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 0x7f6118197ee0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f611000d770 tx=0x7f611000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.119+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110017070 con 0x7f6118100a30 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.119+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6110009c30 con 0x7f6118100a30 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.119+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611819fdf0 con 0x7f6118100a30 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.119+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110003dd0 con 0x7f6118100a30 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.119+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61181a0260 con 0x7f6118100a30 2024-09-12T01:46:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.120+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6118107e70 con 0x7f6118100a30 2024-09-12T01:46:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.121+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6110003f70 con 0x7f6118100a30 2024-09-12T01:46:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.122+0000 7f6115ffb640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f61040a2c10 0x7f61040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.122+0000 7f6117fff640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f61040a2c10 0x7f61040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.122+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f61100cea10 con 0x7f6118100a30 2024-09-12T01:46:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.122+0000 7f6117fff640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f61040a2c10 0x7f61040a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6108000f80 tx=0x7f6108009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:20.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.126+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f611009b970 con 0x7f6118100a30 2024-09-12T01:46:20.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:46:19.913+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:46:20.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:46:19.913+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:46:20.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:46:19.913+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:46:20.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:46:19.913+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:46:20.239 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 01:46:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T01:46:19.913+0000 7fda5098d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T01:46:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.294+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6118062ef0 con 0x7f6118100a30 2024-09-12T01:46:20.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.295+0000 7f6115ffb640 1 -- 172.21.15.122:0/2065777062 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7f6110009de0 con 0x7f6118100a30 2024-09-12T01:46:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:20.296 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.297+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f61040a2c10 msgr2=0x7f61040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.297+0000 7f611f03c640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f61040a2c10 0x7f61040a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6108000f80 tx=0x7f6108009360 comp rx=0 tx=0).stop 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.297+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 msgr2=0x7f6118197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.297+0000 7f611f03c640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 0x7f6118197ee0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f611000d770 tx=0x7f611000dc30 comp rx=0 tx=0).stop 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 shutdown_connections 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f61040a2c10 0x7f61040a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61181025e0 0x7f611819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6118101c20 0x7f6118198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 --2- 172.21.15.122:0/2065777062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6118100a30 0x7f6118197ee0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 >> 172.21.15.122:0/2065777062 conn(0x7f61180fc1b0 msgr2=0x7f6118105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:20.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 shutdown_connections 2024-09-12T01:46:20.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.298+0000 7f611f03c640 1 -- 172.21.15.122:0/2065777062 wait complete. 2024-09-12T01:46:20.309 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:39:58.671405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T01:44:32.683657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T01:44:34.505051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T01:46:20.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T01:46:20.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 -- 172.21.15.122:0/472181414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 msgr2=0x7f32081072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 --2- 172.21.15.122:0/472181414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081072d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3204009900 tx=0x7f320402a410 comp rx=0 tx=0).stop 2024-09-12T01:46:20.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 -- 172.21.15.122:0/472181414 shutdown_connections 2024-09-12T01:46:20.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 --2- 172.21.15.122:0/472181414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3208107810 0x7f3208109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 --2- 172.21.15.122:0/472181414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081072d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 --2- 172.21.15.122:0/472181414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32081025b0 0x7f32081049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 -- 172.21.15.122:0/472181414 >> 172.21.15.122:0/472181414 conn(0x7f32080fc1b0 msgr2=0x7f32080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 -- 172.21.15.122:0/472181414 shutdown_connections 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.411+0000 7f320f6ac640 1 -- 172.21.15.122:0/472181414 wait complete. 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 Processor -- start 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 -- start start 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32081025b0 0x7f3208197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3208107810 0x7f320819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f320810d7c0 con 0x7f3208104ee0 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f320810d640 con 0x7f32081025b0 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320f6ac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f320810d940 con 0x7f3208107810 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39414/0 (socket says 172.21.15.122:39414) 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 -- 172.21.15.122:0/1989309538 learned_addr learned my addr 172.21.15.122:0/1989309538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320d448640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32081025b0 0x7f3208197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 -- 172.21.15.122:0/1989309538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3208107810 msgr2=0x7f320819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3208107810 0x7f320819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 -- 172.21.15.122:0/1989309538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32081025b0 msgr2=0x7f3208197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32081025b0 0x7f3208197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.412+0000 7f320cc47640 1 -- 172.21.15.122:0/1989309538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320819fb90 con 0x7f3208104ee0 2024-09-12T01:46:20.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.413+0000 7f320cc47640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081984b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f3204031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:20.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.413+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320403f030 con 0x7f3208104ee0 2024-09-12T01:46:20.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.413+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320819fe20 con 0x7f3208104ee0 2024-09-12T01:46:20.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.413+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081a0280 con 0x7f3208104ee0 2024-09-12T01:46:20.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.413+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32040044a0 con 0x7f3208104ee0 2024-09-12T01:46:20.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.413+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204049060 con 0x7f3208104ee0 2024-09-12T01:46:20.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.414+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec005290 con 0x7f3208104ee0 2024-09-12T01:46:20.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.415+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3204042030 con 0x7f3208104ee0 2024-09-12T01:46:20.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.415+0000 7f31fa7fc640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b90 0x7f31f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.415+0000 7f320d448640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b90 0x7f31f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.415+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f32040ef220 con 0x7f3208104ee0 2024-09-12T01:46:20.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.416+0000 7f320d448640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b90 0x7f31f40a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0058c0 tx=0x7f31fc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:20.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.420+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32040bc180 con 0x7f3208104ee0 2024-09-12T01:46:20.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.585+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31ec005d90 con 0x7f3208104ee0 2024-09-12T01:46:20.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.586+0000 7f31fa7fc640 1 -- 172.21.15.122:0/1989309538 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f32040bba80 con 0x7f3208104ee0 2024-09-12T01:46:20.586 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-12T01:46:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b90 msgr2=0x7f31f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b90 0x7f31f40a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0058c0 tx=0x7f31fc00a4c0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 msgr2=0x7f32081984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081984b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3204006010 tx=0x7f3204031b60 comp rx=0 tx=0).stop 2024-09-12T01:46:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 shutdown_connections 2024-09-12T01:46:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f31f40a2b90 0x7f31f40a5050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3208107810 0x7f320819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3208104ee0 0x7f32081984b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 --2- 172.21.15.122:0/1989309538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32081025b0 0x7f3208197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 >> 172.21.15.122:0/1989309538 conn(0x7f32080fc1b0 msgr2=0x7f3208103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:20.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 shutdown_connections 2024-09-12T01:46:20.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.588+0000 7f320f6ac640 1 -- 172.21.15.122:0/1989309538 wait complete. 2024-09-12T01:46:20.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T01:46:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 -- 172.21.15.122:0/2810368050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef041024b0 msgr2=0x7fef04108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 --2- 172.21.15.122:0/2810368050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef041024b0 0x7fef04108d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feefc009ca0 tx=0x7feefc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 -- 172.21.15.122:0/2810368050 shutdown_connections 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 --2- 172.21.15.122:0/2810368050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef041024b0 0x7fef04108d40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 --2- 172.21.15.122:0/2810368050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef04101af0 0x7fef04101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 --2- 172.21.15.122:0/2810368050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef04100900 0x7fef04100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 -- 172.21.15.122:0/2810368050 >> 172.21.15.122:0/2810368050 conn(0x7fef040fc0a0 msgr2=0x7fef040fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 -- 172.21.15.122:0/2810368050 shutdown_connections 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.705+0000 7fef0999f640 1 -- 172.21.15.122:0/2810368050 wait complete. 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 Processor -- start 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 -- start start 2024-09-12T01:46:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 0x7fef04197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef04101af0 0x7fef041983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef041024b0 0x7fef0419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410c680 con 0x7fef04100900 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410c500 con 0x7fef04101af0 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0999f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410c800 con 0x7fef041024b0 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 0x7fef04197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 0x7fef04197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39440/0 (socket says 172.21.15.122:39440) 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 -- 172.21.15.122:0/1851205372 learned_addr learned my addr 172.21.15.122:0/1851205372 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 -- 172.21.15.122:0/1851205372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef041024b0 msgr2=0x7fef0419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef03fff640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef04101af0 0x7fef041983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0919e640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef041024b0 0x7fef0419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef041024b0 0x7fef0419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 -- 172.21.15.122:0/1851205372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef04101af0 msgr2=0x7fef041983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef04101af0 0x7fef041983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 -- 172.21.15.122:0/1851205372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0419fac0 con 0x7fef04100900 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef0899d640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 0x7fef04197ea0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feef800b450 tx=0x7feef800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.706+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8015020 con 0x7fef04100900 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.707+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef8011d70 con 0x7fef04100900 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.707+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0419fdb0 con 0x7fef04100900 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.707+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef800f340 con 0x7fef04100900 2024-09-12T01:46:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.707+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a02a0 con 0x7fef04100900 2024-09-12T01:46:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.708+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef041a0530 con 0x7fef04100900 2024-09-12T01:46:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.709+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7feef8018020 con 0x7fef04100900 2024-09-12T01:46:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.710+0000 7fef01ffb640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feef40a2c20 0x7feef40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.710+0000 7fef03fff640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feef40a2c20 0x7feef40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.710+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7feef80cebd0 con 0x7fef04100900 2024-09-12T01:46:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.710+0000 7fef03fff640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feef40a2c20 0x7feef40a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fef041993b0 tx=0x7feef0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.714+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feef801f020 con 0x7fef04100900 2024-09-12T01:46:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3365106431' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T01:46:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:20 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:46:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2065777062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1989309538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:46:20.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.880+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef041a0700 con 0x7fef04100900 2024-09-12T01:46:20.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.880+0000 7fef01ffb640 1 -- 172.21.15.122:0/1851205372 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7feef800d040 con 0x7fef04100900 2024-09-12T01:46:20.880 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feef40a2c20 msgr2=0x7feef40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feef40a2c20 0x7feef40a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fef041993b0 tx=0x7feef0008040 comp rx=0 tx=0).stop 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 msgr2=0x7fef04197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 0x7fef04197ea0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feef800b450 tx=0x7feef800b910 comp rx=0 tx=0).stop 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 shutdown_connections 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7feef40a2c20 0x7feef40a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef041024b0 0x7fef0419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef04101af0 0x7fef041983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 --2- 172.21.15.122:0/1851205372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef04100900 0x7fef04197ea0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 >> 172.21.15.122:0/1851205372 conn(0x7fef040fc0a0 msgr2=0x7fef04069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:20.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 shutdown_connections 2024-09-12T01:46:20.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.882+0000 7fef0999f640 1 -- 172.21.15.122:0/1851205372 wait complete. 2024-09-12T01:46:20.892 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3365106431' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T01:46:20.892 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:20 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:46:20.892 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2065777062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:20.892 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1989309538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:46:20.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T01:46:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3365106431' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T01:46:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:20 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:46:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2065777062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:20.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1989309538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:46:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 -- 172.21.15.122:0/1971680444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d70101c00 msgr2=0x7f6d70102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 --2- 172.21.15.122:0/1971680444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d70101c00 0x7f6d70102080 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009900 tx=0x7f6d6002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 -- 172.21.15.122:0/1971680444 shutdown_connections 2024-09-12T01:46:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 --2- 172.21.15.122:0/1971680444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d701025c0 0x7f6d70108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 --2- 172.21.15.122:0/1971680444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d70101c00 0x7f6d70102080 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 --2- 172.21.15.122:0/1971680444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 0x7f6d70100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 -- 172.21.15.122:0/1971680444 >> 172.21.15.122:0/1971680444 conn(0x7f6d700fc1b0 msgr2=0x7f6d700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 -- 172.21.15.122:0/1971680444 shutdown_connections 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.996+0000 7f6d783c4640 1 -- 172.21.15.122:0/1971680444 wait complete. 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 Processor -- start 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 -- start start 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 0x7f6d70197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d70101c00 0x7f6d70198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 0x7f6d7019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010d7e0 con 0x7f6d701025c0 2024-09-12T01:46:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010d660 con 0x7f6d70101c00 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d783c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010d960 con 0x7f6d70100a10 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 0x7f6d7019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 0x7f6d7019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39464/0 (socket says 172.21.15.122:39464) 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 -- 172.21.15.122:0/3383986826 learned_addr learned my addr 172.21.15.122:0/3383986826 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d7595f640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d70101c00 0x7f6d70198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76160640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 0x7f6d70197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 -- 172.21.15.122:0/3383986826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 msgr2=0x7f6d70197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 0x7f6d70197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 -- 172.21.15.122:0/3383986826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d70101c00 msgr2=0x7f6d70198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d70101c00 0x7f6d70198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 -- 172.21.15.122:0/3383986826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7019fa50 con 0x7f6d701025c0 2024-09-12T01:46:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76160640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 0x7f6d70197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d7595f640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d70101c00 0x7f6d70198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.997+0000 7f6d76961640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 0x7f6d7019f3e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800eaa0 tx=0x7f6d6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.998+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6800cdb0 con 0x7f6d701025c0 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.998+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7019fd40 con 0x7f6d701025c0 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.998+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d68005020 con 0x7f6d701025c0 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.998+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d701a0200 con 0x7f6d701025c0 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.998+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6801f060 con 0x7f6d701025c0 2024-09-12T01:46:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:20.999+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d70107ea0 con 0x7f6d701025c0 2024-09-12T01:46:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.000+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6d68005560 con 0x7f6d701025c0 2024-09-12T01:46:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.000+0000 7f6d677fe640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6d5c0a2c20 0x7f6d5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.001+0000 7f6d76160640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6d5c0a2c20 0x7f6d5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.001+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6d680cf830 con 0x7f6d701025c0 2024-09-12T01:46:21.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.001+0000 7f6d76160640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6d5c0a2c20 0x7f6d5c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0058c0 tx=0x7f6d6c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.005+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d68016030 con 0x7f6d701025c0 2024-09-12T01:46:21.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.168+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d70062df0 con 0x7f6d701025c0 2024-09-12T01:46:21.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.168+0000 7f6d677fe640 1 -- 172.21.15.122:0/3383986826 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f6d6809c630 con 0x7f6d701025c0 2024-09-12T01:46:21.168 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-09-12T01:46:21.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6d5c0a2c20 msgr2=0x7f6d5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6d5c0a2c20 0x7f6d5c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0058c0 tx=0x7f6d6c005850 comp rx=0 tx=0).stop 2024-09-12T01:46:21.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 msgr2=0x7f6d7019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 0x7f6d7019f3e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800eaa0 tx=0x7f6d6800ef60 comp rx=0 tx=0).stop 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 shutdown_connections 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6d5c0a2c20 0x7f6d5c0a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d701025c0 0x7f6d7019f3e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d70101c00 0x7f6d70198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 --2- 172.21.15.122:0/3383986826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70100a10 0x7f6d70197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 >> 172.21.15.122:0/3383986826 conn(0x7f6d700fc1b0 msgr2=0x7f6d70105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 shutdown_connections 2024-09-12T01:46:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.170+0000 7f6d783c4640 1 -- 172.21.15.122:0/3383986826 wait complete. 2024-09-12T01:46:21.181 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T01:46:21.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:21.427 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:21 smithi122 ceph-mon[30925]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:21 smithi122 ceph-mon[30925]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T01:46:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:21 smithi122 ceph-mon[30925]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T01:46:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:21 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1851205372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:46:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:21 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3383986826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:46:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:21 smithi151 ceph-mon[35229]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:21 smithi151 ceph-mon[35229]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T01:46:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:21 smithi151 ceph-mon[35229]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T01:46:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:21 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1851205372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:46:21.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:21 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3383986826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:46:21.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 -- 172.21.15.122:0/2465057207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d8100480 msgr2=0x7f81d8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 --2- 172.21.15.122:0/2465057207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d8100480 0x7f81d8100860 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009900 tx=0x7f81c802a410 comp rx=0 tx=0).stop 2024-09-12T01:46:21.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 -- 172.21.15.122:0/2465057207 shutdown_connections 2024-09-12T01:46:21.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 --2- 172.21.15.122:0/2465057207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81d81055b0 0x7f81d810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 --2- 172.21.15.122:0/2465057207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81d8100e30 0x7f81d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 --2- 172.21.15.122:0/2465057207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d8100480 0x7f81d8100860 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 -- 172.21.15.122:0/2465057207 >> 172.21.15.122:0/2465057207 conn(0x7f81d8077210 msgr2=0x7f81d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 -- 172.21.15.122:0/2465057207 shutdown_connections 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.902+0000 7f81dc88d640 1 -- 172.21.15.122:0/2465057207 wait complete. 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 Processor -- start 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 -- start start 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81d8100480 0x7f81d81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81d8100e30 0x7f81d8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 0x7f81d819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8110e50 con 0x7f81d81055b0 2024-09-12T01:46:21.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8110cd0 con 0x7f81d8100e30 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81dc88d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8110fd0 con 0x7f81d8100480 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 0x7f81d819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 0x7f81d819bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39476/0 (socket says 172.21.15.122:39476) 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 -- 172.21.15.122:0/3817380781 learned_addr learned my addr 172.21.15.122:0/3817380781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81cffff640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81d8100e30 0x7f81d8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d659c640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81d8100480 0x7f81d81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 -- 172.21.15.122:0/3817380781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81d8100480 msgr2=0x7f81d81972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81d8100480 0x7f81d81972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 -- 172.21.15.122:0/3817380781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81d8100e30 msgr2=0x7f81d8197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81d8100e30 0x7f81d8197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 -- 172.21.15.122:0/3817380781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d819c2a0 con 0x7f81d81055b0 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.903+0000 7f81d6d9d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 0x7f81d819bbc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f81d000eaa0 tx=0x7f81d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:21.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.904+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0004d20 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.904+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d0005020 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.904+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00116d0 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.904+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d819c590 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.904+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d81a3e70 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.905+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d8100860 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.906+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f81d00051d0 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.907+0000 7f81cf7fe640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81c40a2c90 0x7f81c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.907+0000 7f81d659c640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81c40a2c90 0x7f81c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.907+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f81d00cf720 con 0x7f81d81055b0 2024-09-12T01:46:21.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.907+0000 7f81d659c640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81c40a2c90 0x7f81c40a5150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f81c80096c0 tx=0x7f81c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:21.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:21.910+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81d0016030 con 0x7f81d81055b0 2024-09-12T01:46:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:21 smithi183 ceph-mon[35127]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:46:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:21 smithi183 ceph-mon[35127]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T01:46:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:21 smithi183 ceph-mon[35127]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T01:46:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:21 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1851205372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:46:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:21 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3383986826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:46:22.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.113+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81d8198650 con 0x7f81d81055b0 2024-09-12T01:46:22.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.114+0000 7f81cf7fe640 1 -- 172.21.15.122:0/3817380781 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f81d8198650 con 0x7f81d81055b0 2024-09-12T01:46:22.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:22.115 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:46:22.115 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81c40a2c90 msgr2=0x7f81c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81c40a2c90 0x7f81c40a5150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f81c80096c0 tx=0x7f81c8000bc0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 msgr2=0x7f81d819bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 0x7f81d819bbc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f81d000eaa0 tx=0x7f81d000eea0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 shutdown_connections 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81c40a2c90 0x7f81c40a5150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81d81055b0 0x7f81d819bbc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81d8100e30 0x7f81d8197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 --2- 172.21.15.122:0/3817380781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81d8100480 0x7f81d81972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 >> 172.21.15.122:0/3817380781 conn(0x7f81d8077210 msgr2=0x7f81d8103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:22.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 shutdown_connections 2024-09-12T01:46:22.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.117+0000 7f81dc88d640 1 -- 172.21.15.122:0/3817380781 wait complete. 2024-09-12T01:46:22.850 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:22 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3817380781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:22 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3817380781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:22.881 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T01:46:22.881 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T01:46:22.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:46:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:22 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3817380781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:22.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 -- 172.21.15.122:0/4289891994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 msgr2=0x7f5690102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 --2- 172.21.15.122:0/4289891994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f5690102080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5684009900 tx=0x7f568402a410 comp rx=0 tx=0).stop 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 -- 172.21.15.122:0/4289891994 shutdown_connections 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 --2- 172.21.15.122:0/4289891994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56901025c0 0x7f5690108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 --2- 172.21.15.122:0/4289891994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f5690102080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 --2- 172.21.15.122:0/4289891994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690100a10 0x7f5690100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 -- 172.21.15.122:0/4289891994 >> 172.21.15.122:0/4289891994 conn(0x7f56900fc170 msgr2=0x7f56900fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 -- 172.21.15.122:0/4289891994 shutdown_connections 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.987+0000 7f5697f6d640 1 -- 172.21.15.122:0/4289891994 wait complete. 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 Processor -- start 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 -- start start 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5690100a10 0x7f5690069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:22.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f569019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56901025c0 0x7f569019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f569010c670 con 0x7f5690101c00 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f569010c4f0 con 0x7f56901025c0 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5697f6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f569010c7f0 con 0x7f5690100a10 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f569019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f569019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39500/0 (socket says 172.21.15.122:39500) 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 -- 172.21.15.122:0/517542877 learned_addr learned my addr 172.21.15.122:0/517542877 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695d09640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5690100a10 0x7f5690069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 -- 172.21.15.122:0/517542877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5690100a10 msgr2=0x7f5690069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5690100a10 0x7f5690069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 -- 172.21.15.122:0/517542877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56901025c0 msgr2=0x7f569019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56901025c0 0x7f569019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:22.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 -- 172.21.15.122:0/517542877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569019fad0 con 0x7f5690101c00 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695d09640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5690100a10 0x7f5690069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5695508640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f569019cba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5684009230 tx=0x7f5684004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.988+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568403f030 con 0x7f5690101c00 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.989+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684003ce0 con 0x7f5690101c00 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.989+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569019fd00 con 0x7f5690101c00 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.989+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684035070 con 0x7f5690101c00 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.989+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a01e0 con 0x7f5690101c00 2024-09-12T01:46:22.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.989+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5690107e00 con 0x7f5690101c00 2024-09-12T01:46:22.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.991+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5684042030 con 0x7f5690101c00 2024-09-12T01:46:22.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.991+0000 7f5682ffd640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f567c0a2ca0 0x7f567c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:22.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.991+0000 7f5695d09640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f567c0a2ca0 0x7f567c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:22.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.991+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f56840efbf0 con 0x7f5690101c00 2024-09-12T01:46:22.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.992+0000 7f5695d09640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f567c0a2ca0 0x7f567c0a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5690101a60 tx=0x7f568c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:22.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:22.995+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56840bc9f0 con 0x7f5690101c00 2024-09-12T01:46:23.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.160+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5690062630 con 0x7f567c0a2ca0 2024-09-12T01:46:23.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.166+0000 7f5682ffd640 1 -- 172.21.15.122:0/517542877 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259150 (secure 0 0 0) 0x7f5690062630 con 0x7f567c0a2ca0 2024-09-12T01:46:23.169 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f567c0a2ca0 msgr2=0x7f567c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f567c0a2ca0 0x7f567c0a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5690101a60 tx=0x7f568c005d30 comp rx=0 tx=0).stop 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 msgr2=0x7f569019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f569019cba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5684009230 tx=0x7f5684004360 comp rx=0 tx=0).stop 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 shutdown_connections 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f567c0a2ca0 0x7f567c0a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56901025c0 0x7f569019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5690101c00 0x7f569019cba0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 --2- 172.21.15.122:0/517542877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5690100a10 0x7f5690069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 >> 172.21.15.122:0/517542877 conn(0x7f56900fc170 msgr2=0x7f5690104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 shutdown_connections 2024-09-12T01:46:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.169+0000 7f5697f6d640 1 -- 172.21.15.122:0/517542877 wait complete. 2024-09-12T01:46:23.180 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-12T01:46:22.470281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662821,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":714,"num_read_kb":481,"num_write":356,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360304,"kb_used_data":11056,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581520,"statfs":{"total":1151940427776,"available":1151571476480,"internally_reserved":0,"allocated":11321344,"data_stored":7225699,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6464,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":49,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472514+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:46:14.472514+0000","last_peered":"2024-09-12T01:46:14.472514+0000","last_clean":"2024-09-12T01:46:14.472514+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:46:14.472514+0000","last_undegraded":"2024-09-12T01:46:14.472514+0000","last_fullsized":"2024-09-12T01:46:14.472514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.451157+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:46:14.451157+0000","last_peered":"2024-09-12T01:46:14.451157+0000","last_clean":"2024-09-12T01:46:14.451157+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:46:14.451157+0000","last_undegraded":"2024-09-12T01:46:14.451157+0000","last_fullsized":"2024-09-12T01:46:14.451157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462071+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:46:14.462071+0000","last_peered":"2024-09-12T01:46:14.462071+0000","last_clean":"2024-09-12T01:46:14.462071+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:46:14.462071+0000","last_undegraded":"2024-09-12T01:46:14.462071+0000","last_fullsized":"2024-09-12T01:46:14.462071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462225+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:46:14.462225+0000","last_peered":"2024-09-12T01:46:14.462225+0000","last_clean":"2024-09-12T01:46:14.462225+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:46:14.462225+0000","last_undegraded":"2024-09-12T01:46:14.462225+0000","last_fullsized":"2024-09-12T01:46:14.462225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456436+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:46:14.456436+0000","last_peered":"2024-09-12T01:46:14.456436+0000","last_clean":"2024-09-12T01:46:14.456436+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:46:14.456436+0000","last_undegraded":"2024-09-12T01:46:14.456436+0000","last_fullsized":"2024-09-12T01:46:14.456436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470354+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:46:14.470354+0000","last_peered":"2024-09-12T01:46:14.470354+0000","last_clean":"2024-09-12T01:46:14.470354+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:46:14.470354+0000","last_undegraded":"2024-09-12T01:46:14.470354+0000","last_fullsized":"2024-09-12T01:46:14.470354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468414+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:46:14.468414+0000","last_peered":"2024-09-12T01:46:14.468414+0000","last_clean":"2024-09-12T01:46:14.468414+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:46:14.468414+0000","last_undegraded":"2024-09-12T01:46:14.468414+0000","last_fullsized":"2024-09-12T01:46:14.468414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470863+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:46:14.470863+0000","last_peered":"2024-09-12T01:46:14.470863+0000","last_clean":"2024-09-12T01:46:14.470863+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:46:14.470863+0000","last_undegraded":"2024-09-12T01:46:14.470863+0000","last_fullsized":"2024-09-12T01:46:14.470863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467613+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:46:14.467613+0000","last_peered":"2024-09-12T01:46:14.467613+0000","last_clean":"2024-09-12T01:46:14.467613+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:46:14.467613+0000","last_undegraded":"2024-09-12T01:46:14.467613+0000","last_fullsized":"2024-09-12T01:46:14.467613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471418+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:46:14.471418+0000","last_peered":"2024-09-12T01:46:14.471418+0000","last_clean":"2024-09-12T01:46:14.471418+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:46:14.471418+0000","last_undegraded":"2024-09-12T01:46:14.471418+0000","last_fullsized":"2024-09-12T01:46:14.471418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473581+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:46:14.473581+0000","last_peered":"2024-09-12T01:46:14.473581+0000","last_clean":"2024-09-12T01:46:14.473581+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:46:14.473581+0000","last_undegraded":"2024-09-12T01:46:14.473581+0000","last_fullsized":"2024-09-12T01:46:14.473581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466586+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:46:14.466586+0000","last_peered":"2024-09-12T01:46:14.466586+0000","last_clean":"2024-09-12T01:46:14.466586+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:46:14.466586+0000","last_undegraded":"2024-09-12T01:46:14.466586+0000","last_fullsized":"2024-09-12T01:46:14.466586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470765+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:46:14.470765+0000","last_peered":"2024-09-12T01:46:14.470765+0000","last_clean":"2024-09-12T01:46:14.470765+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:46:14.470765+0000","last_undegraded":"2024-09-12T01:46:14.470765+0000","last_fullsized":"2024-09-12T01:46:14.470765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456811+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:46:14.456811+0000","last_peered":"2024-09-12T01:46:14.456811+0000","last_clean":"2024-09-12T01:46:14.456811+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:46:14.456811+0000","last_undegraded":"2024-09-12T01:46:14.456811+0000","last_fullsized":"2024-09-12T01:46:14.456811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462152+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:46:14.462152+0000","last_peered":"2024-09-12T01:46:14.462152+0000","last_clean":"2024-09-12T01:46:14.462152+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:46:14.462152+0000","last_undegraded":"2024-09-12T01:46:14.462152+0000","last_fullsized":"2024-09-12T01:46:14.462152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470248+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:46:14.470248+0000","last_peered":"2024-09-12T01:46:14.470248+0000","last_clean":"2024-09-12T01:46:14.470248+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:46:14.470248+0000","last_undegraded":"2024-09-12T01:46:14.470248+0000","last_fullsized":"2024-09-12T01:46:14.470248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472627+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:46:14.472627+0000","last_peered":"2024-09-12T01:46:14.472627+0000","last_clean":"2024-09-12T01:46:14.472627+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:46:14.472627+0000","last_undegraded":"2024-09-12T01:46:14.472627+0000","last_fullsized":"2024-09-12T01:46:14.472627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466306+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:46:14.466306+0000","last_peered":"2024-09-12T01:46:14.466306+0000","last_clean":"2024-09-12T01:46:14.466306+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:46:14.466306+0000","last_undegraded":"2024-09-12T01:46:14.466306+0000","last_fullsized":"2024-09-12T01:46:14.466306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457876+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:46:14.457876+0000","last_peered":"2024-09-12T01:46:14.457876+0000","last_clean":"2024-09-12T01:46:14.457876+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:46:14.457876+0000","last_undegraded":"2024-09-12T01:46:14.457876+0000","last_fullsized":"2024-09-12T01:46:14.457876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461219+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:46:14.461219+0000","last_peered":"2024-09-12T01:46:14.461219+0000","last_clean":"2024-09-12T01:46:14.461219+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:46:14.461219+0000","last_undegraded":"2024-09-12T01:46:14.461219+0000","last_fullsized":"2024-09-12T01:46:14.461219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469829+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:46:14.469829+0000","last_peered":"2024-09-12T01:46:14.469829+0000","last_clean":"2024-09-12T01:46:14.469829+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:46:14.469829+0000","last_undegraded":"2024-09-12T01:46:14.469829+0000","last_fullsized":"2024-09-12T01:46:14.469829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471133+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:46:14.471133+0000","last_peered":"2024-09-12T01:46:14.471133+0000","last_clean":"2024-09-12T01:46:14.471133+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:46:14.471133+0000","last_undegraded":"2024-09-12T01:46:14.471133+0000","last_fullsized":"2024-09-12T01:46:14.471133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460490+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:46:14.460490+0000","last_peered":"2024-09-12T01:46:14.460490+0000","last_clean":"2024-09-12T01:46:14.460490+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:46:14.460490+0000","last_undegraded":"2024-09-12T01:46:14.460490+0000","last_fullsized":"2024-09-12T01:46:14.460490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.455176+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:46:14.455176+0000","last_peered":"2024-09-12T01:46:14.455176+0000","last_clean":"2024-09-12T01:46:14.455176+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:46:14.455176+0000","last_undegraded":"2024-09-12T01:46:14.455176+0000","last_fullsized":"2024-09-12T01:46:14.455176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449237+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:46:14.449237+0000","last_peered":"2024-09-12T01:46:14.449237+0000","last_clean":"2024-09-12T01:46:14.449237+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:46:14.449237+0000","last_undegraded":"2024-09-12T01:46:14.449237+0000","last_fullsized":"2024-09-12T01:46:14.449237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459600+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:46:14.459600+0000","last_peered":"2024-09-12T01:46:14.459600+0000","last_clean":"2024-09-12T01:46:14.459600+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:46:14.459600+0000","last_undegraded":"2024-09-12T01:46:14.459600+0000","last_fullsized":"2024-09-12T01:46:14.459600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456999+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:46:14.456999+0000","last_peered":"2024-09-12T01:46:14.456999+0000","last_clean":"2024-09-12T01:46:14.456999+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:46:14.456999+0000","last_undegraded":"2024-09-12T01:46:14.456999+0000","last_fullsized":"2024-09-12T01:46:14.456999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.453782+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:46:14.453782+0000","last_peered":"2024-09-12T01:46:14.453782+0000","last_clean":"2024-09-12T01:46:14.453782+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:46:14.453782+0000","last_undegraded":"2024-09-12T01:46:14.453782+0000","last_fullsized":"2024-09-12T01:46:14.453782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457999+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:46:14.457999+0000","last_peered":"2024-09-12T01:46:14.457999+0000","last_clean":"2024-09-12T01:46:14.457999+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:46:14.457999+0000","last_undegraded":"2024-09-12T01:46:14.457999+0000","last_fullsized":"2024-09-12T01:46:14.457999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460598+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:46:14.460598+0000","last_peered":"2024-09-12T01:46:14.460598+0000","last_clean":"2024-09-12T01:46:14.460598+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:46:14.460598+0000","last_undegraded":"2024-09-12T01:46:14.460598+0000","last_fullsized":"2024-09-12T01:46:14.460598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463452+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:46:14.463452+0000","last_peered":"2024-09-12T01:46:14.463452+0000","last_clean":"2024-09-12T01:46:14.463452+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:46:14.463452+0000","last_undegraded":"2024-09-12T01:46:14.463452+0000","last_fullsized":"2024-09-12T01:46:14.463452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463643+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:46:14.463643+0000","last_peered":"2024-09-12T01:46:14.463643+0000","last_clean":"2024-09-12T01:46:14.463643+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:46:14.463643+0000","last_undegraded":"2024-09-12T01:46:14.463643+0000","last_fullsized":"2024-09-12T01:46:14.463643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457236+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:46:14.457236+0000","last_peered":"2024-09-12T01:46:14.457236+0000","last_clean":"2024-09-12T01:46:14.457236+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:46:14.457236+0000","last_undegraded":"2024-09-12T01:46:14.457236+0000","last_fullsized":"2024-09-12T01:46:14.457236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:16.966056+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:46:16.966056+0000","last_peered":"2024-09-12T01:46:16.966056+0000","last_clean":"2024-09-12T01:46:16.966056+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:46:16.966056+0000","last_undegraded":"2024-09-12T01:46:16.966056+0000","last_fullsized":"2024-09-12T01:46:16.966056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461733+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:46:14.461733+0000","last_peered":"2024-09-12T01:46:14.461733+0000","last_clean":"2024-09-12T01:46:14.461733+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:46:14.461733+0000","last_undegraded":"2024-09-12T01:46:14.461733+0000","last_fullsized":"2024-09-12T01:46:14.461733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.452061+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:46:14.452061+0000","last_peered":"2024-09-12T01:46:14.452061+0000","last_clean":"2024-09-12T01:46:14.452061+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:46:14.452061+0000","last_undegraded":"2024-09-12T01:46:14.452061+0000","last_fullsized":"2024-09-12T01:46:14.452061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466729+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:46:14.466729+0000","last_peered":"2024-09-12T01:46:14.466729+0000","last_clean":"2024-09-12T01:46:14.466729+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:46:14.466729+0000","last_undegraded":"2024-09-12T01:46:14.466729+0000","last_fullsized":"2024-09-12T01:46:14.466729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:18.968110+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:46:18.968110+0000","last_peered":"2024-09-12T01:46:18.968110+0000","last_clean":"2024-09-12T01:46:18.968110+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:46:18.968110+0000","last_undegraded":"2024-09-12T01:46:18.968110+0000","last_fullsized":"2024-09-12T01:46:18.968110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473143+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:46:14.473143+0000","last_peered":"2024-09-12T01:46:14.473143+0000","last_clean":"2024-09-12T01:46:14.473143+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:46:14.473143+0000","last_undegraded":"2024-09-12T01:46:14.473143+0000","last_fullsized":"2024-09-12T01:46:14.473143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460971+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:46:14.460971+0000","last_peered":"2024-09-12T01:46:14.460971+0000","last_clean":"2024-09-12T01:46:14.460971+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:46:14.460971+0000","last_undegraded":"2024-09-12T01:46:14.460971+0000","last_fullsized":"2024-09-12T01:46:14.460971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473209+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:46:14.473209+0000","last_peered":"2024-09-12T01:46:14.473209+0000","last_clean":"2024-09-12T01:46:14.473209+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:46:14.473209+0000","last_undegraded":"2024-09-12T01:46:14.473209+0000","last_fullsized":"2024-09-12T01:46:14.473209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":255,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:19.970676+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:46:19.970676+0000","last_peered":"2024-09-12T01:46:19.970676+0000","last_clean":"2024-09-12T01:46:19.970676+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:46:19.970676+0000","last_undegraded":"2024-09-12T01:46:19.970676+0000","last_fullsized":"2024-09-12T01:46:19.970676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68075,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":199,"num_read_kb":166,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457287+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:46:14.457287+0000","last_peered":"2024-09-12T01:46:14.457287+0000","last_clean":"2024-09-12T01:46:14.457287+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:46:14.457287+0000","last_undegraded":"2024-09-12T01:46:14.457287+0000","last_fullsized":"2024-09-12T01:46:14.457287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470461+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:46:14.470461+0000","last_peered":"2024-09-12T01:46:14.470461+0000","last_clean":"2024-09-12T01:46:14.470461+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:46:14.470461+0000","last_undegraded":"2024-09-12T01:46:14.470461+0000","last_fullsized":"2024-09-12T01:46:14.470461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472069+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:46:14.472069+0000","last_peered":"2024-09-12T01:46:14.472069+0000","last_clean":"2024-09-12T01:46:14.472069+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:46:14.472069+0000","last_undegraded":"2024-09-12T01:46:14.472069+0000","last_fullsized":"2024-09-12T01:46:14.472069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:15.965863+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:46:15.965863+0000","last_peered":"2024-09-12T01:46:15.965863+0000","last_clean":"2024-09-12T01:46:15.965863+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:46:15.965863+0000","last_undegraded":"2024-09-12T01:46:15.965863+0000","last_fullsized":"2024-09-12T01:46:15.965863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468634+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:46:14.468634+0000","last_peered":"2024-09-12T01:46:14.468634+0000","last_clean":"2024-09-12T01:46:14.468634+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:46:14.468634+0000","last_undegraded":"2024-09-12T01:46:14.468634+0000","last_fullsized":"2024-09-12T01:46:14.468634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469018+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:46:14.469018+0000","last_peered":"2024-09-12T01:46:14.469018+0000","last_clean":"2024-09-12T01:46:14.469018+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:46:14.469018+0000","last_undegraded":"2024-09-12T01:46:14.469018+0000","last_fullsized":"2024-09-12T01:46:14.469018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.458089+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:46:14.458089+0000","last_peered":"2024-09-12T01:46:14.458089+0000","last_clean":"2024-09-12T01:46:14.458089+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:46:14.458089+0000","last_undegraded":"2024-09-12T01:46:14.458089+0000","last_fullsized":"2024-09-12T01:46:14.458089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467837+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:46:14.467837+0000","last_peered":"2024-09-12T01:46:14.467837+0000","last_clean":"2024-09-12T01:46:14.467837+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:46:14.467837+0000","last_undegraded":"2024-09-12T01:46:14.467837+0000","last_fullsized":"2024-09-12T01:46:14.467837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465783+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:46:14.465783+0000","last_peered":"2024-09-12T01:46:14.465783+0000","last_clean":"2024-09-12T01:46:14.465783+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:46:14.465783+0000","last_undegraded":"2024-09-12T01:46:14.465783+0000","last_fullsized":"2024-09-12T01:46:14.465783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473700+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:46:14.473700+0000","last_peered":"2024-09-12T01:46:14.473700+0000","last_clean":"2024-09-12T01:46:14.473700+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:46:14.473700+0000","last_undegraded":"2024-09-12T01:46:14.473700+0000","last_fullsized":"2024-09-12T01:46:14.473700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818582+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818582+0000","last_peered":"2024-09-12T01:44:45.818582+0000","last_clean":"2024-09-12T01:44:45.818582+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818582+0000","last_undegraded":"2024-09-12T01:44:45.818582+0000","last_fullsized":"2024-09-12T01:44:45.818582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.464071+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:46:14.464071+0000","last_peered":"2024-09-12T01:46:14.464071+0000","last_clean":"2024-09-12T01:46:14.464071+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:46:14.464071+0000","last_undegraded":"2024-09-12T01:46:14.464071+0000","last_fullsized":"2024-09-12T01:46:14.464071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449016+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:46:14.449016+0000","last_peered":"2024-09-12T01:46:14.449016+0000","last_clean":"2024-09-12T01:46:14.449016+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:46:14.449016+0000","last_undegraded":"2024-09-12T01:46:14.449016+0000","last_fullsized":"2024-09-12T01:46:14.449016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:16.966739+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:46:16.966739+0000","last_peered":"2024-09-12T01:46:16.966739+0000","last_clean":"2024-09-12T01:46:16.966739+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:46:16.966739+0000","last_undegraded":"2024-09-12T01:46:16.966739+0000","last_fullsized":"2024-09-12T01:46:16.966739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468646+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:46:14.468646+0000","last_peered":"2024-09-12T01:46:14.468646+0000","last_clean":"2024-09-12T01:46:14.468646+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:46:14.468646+0000","last_undegraded":"2024-09-12T01:46:14.468646+0000","last_fullsized":"2024-09-12T01:46:14.468646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.450592+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:46:14.450592+0000","last_peered":"2024-09-12T01:46:14.450592+0000","last_clean":"2024-09-12T01:46:14.450592+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:46:14.450592+0000","last_undegraded":"2024-09-12T01:46:14.450592+0000","last_fullsized":"2024-09-12T01:46:14.450592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468594+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:46:14.468594+0000","last_peered":"2024-09-12T01:46:14.468594+0000","last_clean":"2024-09-12T01:46:14.468594+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:46:14.468594+0000","last_undegraded":"2024-09-12T01:46:14.468594+0000","last_fullsized":"2024-09-12T01:46:14.468594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449242+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:46:14.449242+0000","last_peered":"2024-09-12T01:46:14.449242+0000","last_clean":"2024-09-12T01:46:14.449242+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:46:14.449242+0000","last_undegraded":"2024-09-12T01:46:14.449242+0000","last_fullsized":"2024-09-12T01:46:14.449242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.792350+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:46:14.792350+0000","last_peered":"2024-09-12T01:46:14.792350+0000","last_clean":"2024-09-12T01:46:14.792350+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:46:14.792350+0000","last_undegraded":"2024-09-12T01:46:14.792350+0000","last_fullsized":"2024-09-12T01:46:14.792350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465155+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:46:14.465155+0000","last_peered":"2024-09-12T01:46:14.465155+0000","last_clean":"2024-09-12T01:46:14.465155+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:46:14.465155+0000","last_undegraded":"2024-09-12T01:46:14.465155+0000","last_fullsized":"2024-09-12T01:46:14.465155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457998+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:46:14.457998+0000","last_peered":"2024-09-12T01:46:14.457998+0000","last_clean":"2024-09-12T01:46:14.457998+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:46:14.457998+0000","last_undegraded":"2024-09-12T01:46:14.457998+0000","last_fullsized":"2024-09-12T01:46:14.457998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.440457+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:46:14.440457+0000","last_peered":"2024-09-12T01:46:14.440457+0000","last_clean":"2024-09-12T01:46:14.440457+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:46:14.440457+0000","last_undegraded":"2024-09-12T01:46:14.440457+0000","last_fullsized":"2024-09-12T01:46:14.440457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463449+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:46:14.463449+0000","last_peered":"2024-09-12T01:46:14.463449+0000","last_clean":"2024-09-12T01:46:14.463449+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:46:14.463449+0000","last_undegraded":"2024-09-12T01:46:14.463449+0000","last_fullsized":"2024-09-12T01:46:14.463449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470639+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:46:14.470639+0000","last_peered":"2024-09-12T01:46:14.470639+0000","last_clean":"2024-09-12T01:46:14.470639+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:46:14.470639+0000","last_undegraded":"2024-09-12T01:46:14.470639+0000","last_fullsized":"2024-09-12T01:46:14.470639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473199+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:46:14.473199+0000","last_peered":"2024-09-12T01:46:14.473199+0000","last_clean":"2024-09-12T01:46:14.473199+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:46:14.473199+0000","last_undegraded":"2024-09-12T01:46:14.473199+0000","last_fullsized":"2024-09-12T01:46:14.473199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469618+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:46:14.469618+0000","last_peered":"2024-09-12T01:46:14.469618+0000","last_clean":"2024-09-12T01:46:14.469618+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:46:14.469618+0000","last_undegraded":"2024-09-12T01:46:14.469618+0000","last_fullsized":"2024-09-12T01:46:14.469618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473089+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:46:14.473089+0000","last_peered":"2024-09-12T01:46:14.473089+0000","last_clean":"2024-09-12T01:46:14.473089+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:46:14.473089+0000","last_undegraded":"2024-09-12T01:46:14.473089+0000","last_fullsized":"2024-09-12T01:46:14.473089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.454649+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:46:14.454649+0000","last_peered":"2024-09-12T01:46:14.454649+0000","last_clean":"2024-09-12T01:46:14.454649+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:46:14.454649+0000","last_undegraded":"2024-09-12T01:46:14.454649+0000","last_fullsized":"2024-09-12T01:46:14.454649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459700+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:46:14.459700+0000","last_peered":"2024-09-12T01:46:14.459700+0000","last_clean":"2024-09-12T01:46:14.459700+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:46:14.459700+0000","last_undegraded":"2024-09-12T01:46:14.459700+0000","last_fullsized":"2024-09-12T01:46:14.459700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72219,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":588,"num_read_kb":372,"num_write":173,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1092119,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168625,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1030024,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438391,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":509124,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":499214,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:46:23.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:46:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 -- 172.21.15.122:0/1007587135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 msgr2=0x7efe10100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 --2- 172.21.15.122:0/1007587135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe10100e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009900 tx=0x7efe0c02a410 comp rx=0 tx=0).stop 2024-09-12T01:46:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 -- 172.21.15.122:0/1007587135 shutdown_connections 2024-09-12T01:46:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 --2- 172.21.15.122:0/1007587135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe101025e0 0x7efe10108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 --2- 172.21.15.122:0/1007587135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10101c20 0x7efe101020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 --2- 172.21.15.122:0/1007587135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe10100e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 -- 172.21.15.122:0/1007587135 >> 172.21.15.122:0/1007587135 conn(0x7efe100fc1b0 msgr2=0x7efe100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 -- 172.21.15.122:0/1007587135 shutdown_connections 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 -- 172.21.15.122:0/1007587135 wait complete. 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 Processor -- start 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.321+0000 7efe1840e640 1 -- start start 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe1010b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe10101c20 0x7efe101099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101025e0 0x7efe10109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010d800 con 0x7efe10100a30 2024-09-12T01:46:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010d680 con 0x7efe10101c20 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010d980 con 0x7efe101025e0 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe1010b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe1010b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39512/0 (socket says 172.21.15.122:39512) 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 -- 172.21.15.122:0/960280844 learned_addr learned my addr 172.21.15.122:0/960280844 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe169ab640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101025e0 0x7efe10109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 -- 172.21.15.122:0/960280844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101025e0 msgr2=0x7efe10109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101025e0 0x7efe10109f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 -- 172.21.15.122:0/960280844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe10101c20 msgr2=0x7efe101099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe10101c20 0x7efe101099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 -- 172.21.15.122:0/960280844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1010a7c0 con 0x7efe10100a30 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe169ab640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101025e0 0x7efe10109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe161aa640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe1010b3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe0c031610 tx=0x7efe0c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c03f030 con 0x7efe10100a30 2024-09-12T01:46:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c003ce0 con 0x7efe10100a30 2024-09-12T01:46:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe101a1f70 con 0x7efe10100a30 2024-09-12T01:46:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c035070 con 0x7efe10100a30 2024-09-12T01:46:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.322+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe101a23d0 con 0x7efe10100a30 2024-09-12T01:46:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.323+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe10107e70 con 0x7efe10100a30 2024-09-12T01:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.324+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7efe0c042030 con 0x7efe10100a30 2024-09-12T01:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.325+0000 7efe037fe640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efdfc0a2c20 0x7efdfc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.325+0000 7efe159a9640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efdfc0a2c20 0x7efdfc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.325+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efe0c0ef7b0 con 0x7efe10100a30 2024-09-12T01:46:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.326+0000 7efe159a9640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efdfc0a2c20 0x7efdfc0a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe04005fd0 tx=0x7efe04005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:23.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.329+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe0c0bc5b0 con 0x7efe10100a30 2024-09-12T01:46:23.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.492+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe10062630 con 0x7efdfc0a2c20 2024-09-12T01:46:23.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.499+0000 7efe037fe640 1 -- 172.21.15.122:0/960280844 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259150 (secure 0 0 0) 0x7efe10062630 con 0x7efdfc0a2c20 2024-09-12T01:46:23.501 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:23.503 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:46:23.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efdfc0a2c20 msgr2=0x7efdfc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efdfc0a2c20 0x7efdfc0a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe04005fd0 tx=0x7efe04005850 comp rx=0 tx=0).stop 2024-09-12T01:46:23.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 msgr2=0x7efe1010b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe1010b3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe0c031610 tx=0x7efe0c0043b0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 shutdown_connections 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efdfc0a2c20 0x7efdfc0a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe101025e0 0x7efe10109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe10101c20 0x7efe101099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 --2- 172.21.15.122:0/960280844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe10100a30 0x7efe1010b3b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 >> 172.21.15.122:0/960280844 conn(0x7efe100fc1b0 msgr2=0x7efe10105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 shutdown_connections 2024-09-12T01:46:23.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.502+0000 7efe1840e640 1 -- 172.21.15.122:0/960280844 wait complete. 2024-09-12T01:46:23.513 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-12T01:46:22.470281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662821,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":714,"num_read_kb":481,"num_write":356,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360304,"kb_used_data":11056,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581520,"statfs":{"total":1151940427776,"available":1151571476480,"internally_reserved":0,"allocated":11321344,"data_stored":7225699,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6464,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":49,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472514+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:46:14.472514+0000","last_peered":"2024-09-12T01:46:14.472514+0000","last_clean":"2024-09-12T01:46:14.472514+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:46:14.472514+0000","last_undegraded":"2024-09-12T01:46:14.472514+0000","last_fullsized":"2024-09-12T01:46:14.472514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.451157+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:46:14.451157+0000","last_peered":"2024-09-12T01:46:14.451157+0000","last_clean":"2024-09-12T01:46:14.451157+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:46:14.451157+0000","last_undegraded":"2024-09-12T01:46:14.451157+0000","last_fullsized":"2024-09-12T01:46:14.451157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462071+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:46:14.462071+0000","last_peered":"2024-09-12T01:46:14.462071+0000","last_clean":"2024-09-12T01:46:14.462071+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:46:14.462071+0000","last_undegraded":"2024-09-12T01:46:14.462071+0000","last_fullsized":"2024-09-12T01:46:14.462071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462225+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:46:14.462225+0000","last_peered":"2024-09-12T01:46:14.462225+0000","last_clean":"2024-09-12T01:46:14.462225+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:46:14.462225+0000","last_undegraded":"2024-09-12T01:46:14.462225+0000","last_fullsized":"2024-09-12T01:46:14.462225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456436+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:46:14.456436+0000","last_peered":"2024-09-12T01:46:14.456436+0000","last_clean":"2024-09-12T01:46:14.456436+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:46:14.456436+0000","last_undegraded":"2024-09-12T01:46:14.456436+0000","last_fullsized":"2024-09-12T01:46:14.456436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470354+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:46:14.470354+0000","last_peered":"2024-09-12T01:46:14.470354+0000","last_clean":"2024-09-12T01:46:14.470354+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:46:14.470354+0000","last_undegraded":"2024-09-12T01:46:14.470354+0000","last_fullsized":"2024-09-12T01:46:14.470354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468414+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:46:14.468414+0000","last_peered":"2024-09-12T01:46:14.468414+0000","last_clean":"2024-09-12T01:46:14.468414+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:46:14.468414+0000","last_undegraded":"2024-09-12T01:46:14.468414+0000","last_fullsized":"2024-09-12T01:46:14.468414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470863+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:46:14.470863+0000","last_peered":"2024-09-12T01:46:14.470863+0000","last_clean":"2024-09-12T01:46:14.470863+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:46:14.470863+0000","last_undegraded":"2024-09-12T01:46:14.470863+0000","last_fullsized":"2024-09-12T01:46:14.470863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467613+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:46:14.467613+0000","last_peered":"2024-09-12T01:46:14.467613+0000","last_clean":"2024-09-12T01:46:14.467613+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:46:14.467613+0000","last_undegraded":"2024-09-12T01:46:14.467613+0000","last_fullsized":"2024-09-12T01:46:14.467613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471418+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:46:14.471418+0000","last_peered":"2024-09-12T01:46:14.471418+0000","last_clean":"2024-09-12T01:46:14.471418+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:46:14.471418+0000","last_undegraded":"2024-09-12T01:46:14.471418+0000","last_fullsized":"2024-09-12T01:46:14.471418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473581+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:46:14.473581+0000","last_peered":"2024-09-12T01:46:14.473581+0000","last_clean":"2024-09-12T01:46:14.473581+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:46:14.473581+0000","last_undegraded":"2024-09-12T01:46:14.473581+0000","last_fullsized":"2024-09-12T01:46:14.473581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466586+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:46:14.466586+0000","last_peered":"2024-09-12T01:46:14.466586+0000","last_clean":"2024-09-12T01:46:14.466586+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:46:14.466586+0000","last_undegraded":"2024-09-12T01:46:14.466586+0000","last_fullsized":"2024-09-12T01:46:14.466586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470765+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:46:14.470765+0000","last_peered":"2024-09-12T01:46:14.470765+0000","last_clean":"2024-09-12T01:46:14.470765+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:46:14.470765+0000","last_undegraded":"2024-09-12T01:46:14.470765+0000","last_fullsized":"2024-09-12T01:46:14.470765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456811+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:46:14.456811+0000","last_peered":"2024-09-12T01:46:14.456811+0000","last_clean":"2024-09-12T01:46:14.456811+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:46:14.456811+0000","last_undegraded":"2024-09-12T01:46:14.456811+0000","last_fullsized":"2024-09-12T01:46:14.456811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462152+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:46:14.462152+0000","last_peered":"2024-09-12T01:46:14.462152+0000","last_clean":"2024-09-12T01:46:14.462152+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:46:14.462152+0000","last_undegraded":"2024-09-12T01:46:14.462152+0000","last_fullsized":"2024-09-12T01:46:14.462152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470248+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:46:14.470248+0000","last_peered":"2024-09-12T01:46:14.470248+0000","last_clean":"2024-09-12T01:46:14.470248+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:46:14.470248+0000","last_undegraded":"2024-09-12T01:46:14.470248+0000","last_fullsized":"2024-09-12T01:46:14.470248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472627+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:46:14.472627+0000","last_peered":"2024-09-12T01:46:14.472627+0000","last_clean":"2024-09-12T01:46:14.472627+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:46:14.472627+0000","last_undegraded":"2024-09-12T01:46:14.472627+0000","last_fullsized":"2024-09-12T01:46:14.472627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466306+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:46:14.466306+0000","last_peered":"2024-09-12T01:46:14.466306+0000","last_clean":"2024-09-12T01:46:14.466306+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:46:14.466306+0000","last_undegraded":"2024-09-12T01:46:14.466306+0000","last_fullsized":"2024-09-12T01:46:14.466306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457876+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:46:14.457876+0000","last_peered":"2024-09-12T01:46:14.457876+0000","last_clean":"2024-09-12T01:46:14.457876+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:46:14.457876+0000","last_undegraded":"2024-09-12T01:46:14.457876+0000","last_fullsized":"2024-09-12T01:46:14.457876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461219+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:46:14.461219+0000","last_peered":"2024-09-12T01:46:14.461219+0000","last_clean":"2024-09-12T01:46:14.461219+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:46:14.461219+0000","last_undegraded":"2024-09-12T01:46:14.461219+0000","last_fullsized":"2024-09-12T01:46:14.461219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469829+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:46:14.469829+0000","last_peered":"2024-09-12T01:46:14.469829+0000","last_clean":"2024-09-12T01:46:14.469829+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:46:14.469829+0000","last_undegraded":"2024-09-12T01:46:14.469829+0000","last_fullsized":"2024-09-12T01:46:14.469829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471133+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:46:14.471133+0000","last_peered":"2024-09-12T01:46:14.471133+0000","last_clean":"2024-09-12T01:46:14.471133+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:46:14.471133+0000","last_undegraded":"2024-09-12T01:46:14.471133+0000","last_fullsized":"2024-09-12T01:46:14.471133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460490+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:46:14.460490+0000","last_peered":"2024-09-12T01:46:14.460490+0000","last_clean":"2024-09-12T01:46:14.460490+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:46:14.460490+0000","last_undegraded":"2024-09-12T01:46:14.460490+0000","last_fullsized":"2024-09-12T01:46:14.460490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.455176+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:46:14.455176+0000","last_peered":"2024-09-12T01:46:14.455176+0000","last_clean":"2024-09-12T01:46:14.455176+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:46:14.455176+0000","last_undegraded":"2024-09-12T01:46:14.455176+0000","last_fullsized":"2024-09-12T01:46:14.455176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449237+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:46:14.449237+0000","last_peered":"2024-09-12T01:46:14.449237+0000","last_clean":"2024-09-12T01:46:14.449237+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:46:14.449237+0000","last_undegraded":"2024-09-12T01:46:14.449237+0000","last_fullsized":"2024-09-12T01:46:14.449237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459600+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:46:14.459600+0000","last_peered":"2024-09-12T01:46:14.459600+0000","last_clean":"2024-09-12T01:46:14.459600+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:46:14.459600+0000","last_undegraded":"2024-09-12T01:46:14.459600+0000","last_fullsized":"2024-09-12T01:46:14.459600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456999+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:46:14.456999+0000","last_peered":"2024-09-12T01:46:14.456999+0000","last_clean":"2024-09-12T01:46:14.456999+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:46:14.456999+0000","last_undegraded":"2024-09-12T01:46:14.456999+0000","last_fullsized":"2024-09-12T01:46:14.456999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.453782+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:46:14.453782+0000","last_peered":"2024-09-12T01:46:14.453782+0000","last_clean":"2024-09-12T01:46:14.453782+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:46:14.453782+0000","last_undegraded":"2024-09-12T01:46:14.453782+0000","last_fullsized":"2024-09-12T01:46:14.453782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457999+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:46:14.457999+0000","last_peered":"2024-09-12T01:46:14.457999+0000","last_clean":"2024-09-12T01:46:14.457999+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:46:14.457999+0000","last_undegraded":"2024-09-12T01:46:14.457999+0000","last_fullsized":"2024-09-12T01:46:14.457999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460598+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:46:14.460598+0000","last_peered":"2024-09-12T01:46:14.460598+0000","last_clean":"2024-09-12T01:46:14.460598+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:46:14.460598+0000","last_undegraded":"2024-09-12T01:46:14.460598+0000","last_fullsized":"2024-09-12T01:46:14.460598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463452+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:46:14.463452+0000","last_peered":"2024-09-12T01:46:14.463452+0000","last_clean":"2024-09-12T01:46:14.463452+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:46:14.463452+0000","last_undegraded":"2024-09-12T01:46:14.463452+0000","last_fullsized":"2024-09-12T01:46:14.463452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463643+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:46:14.463643+0000","last_peered":"2024-09-12T01:46:14.463643+0000","last_clean":"2024-09-12T01:46:14.463643+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:46:14.463643+0000","last_undegraded":"2024-09-12T01:46:14.463643+0000","last_fullsized":"2024-09-12T01:46:14.463643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457236+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:46:14.457236+0000","last_peered":"2024-09-12T01:46:14.457236+0000","last_clean":"2024-09-12T01:46:14.457236+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:46:14.457236+0000","last_undegraded":"2024-09-12T01:46:14.457236+0000","last_fullsized":"2024-09-12T01:46:14.457236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:16.966056+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:46:16.966056+0000","last_peered":"2024-09-12T01:46:16.966056+0000","last_clean":"2024-09-12T01:46:16.966056+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:46:16.966056+0000","last_undegraded":"2024-09-12T01:46:16.966056+0000","last_fullsized":"2024-09-12T01:46:16.966056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461733+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:46:14.461733+0000","last_peered":"2024-09-12T01:46:14.461733+0000","last_clean":"2024-09-12T01:46:14.461733+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:46:14.461733+0000","last_undegraded":"2024-09-12T01:46:14.461733+0000","last_fullsized":"2024-09-12T01:46:14.461733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.452061+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:46:14.452061+0000","last_peered":"2024-09-12T01:46:14.452061+0000","last_clean":"2024-09-12T01:46:14.452061+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:46:14.452061+0000","last_undegraded":"2024-09-12T01:46:14.452061+0000","last_fullsized":"2024-09-12T01:46:14.452061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466729+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:46:14.466729+0000","last_peered":"2024-09-12T01:46:14.466729+0000","last_clean":"2024-09-12T01:46:14.466729+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:46:14.466729+0000","last_undegraded":"2024-09-12T01:46:14.466729+0000","last_fullsized":"2024-09-12T01:46:14.466729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:18.968110+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:46:18.968110+0000","last_peered":"2024-09-12T01:46:18.968110+0000","last_clean":"2024-09-12T01:46:18.968110+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:46:18.968110+0000","last_undegraded":"2024-09-12T01:46:18.968110+0000","last_fullsized":"2024-09-12T01:46:18.968110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473143+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:46:14.473143+0000","last_peered":"2024-09-12T01:46:14.473143+0000","last_clean":"2024-09-12T01:46:14.473143+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:46:14.473143+0000","last_undegraded":"2024-09-12T01:46:14.473143+0000","last_fullsized":"2024-09-12T01:46:14.473143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460971+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:46:14.460971+0000","last_peered":"2024-09-12T01:46:14.460971+0000","last_clean":"2024-09-12T01:46:14.460971+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:46:14.460971+0000","last_undegraded":"2024-09-12T01:46:14.460971+0000","last_fullsized":"2024-09-12T01:46:14.460971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473209+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:46:14.473209+0000","last_peered":"2024-09-12T01:46:14.473209+0000","last_clean":"2024-09-12T01:46:14.473209+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:46:14.473209+0000","last_undegraded":"2024-09-12T01:46:14.473209+0000","last_fullsized":"2024-09-12T01:46:14.473209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":255,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:19.970676+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:46:19.970676+0000","last_peered":"2024-09-12T01:46:19.970676+0000","last_clean":"2024-09-12T01:46:19.970676+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:46:19.970676+0000","last_undegraded":"2024-09-12T01:46:19.970676+0000","last_fullsized":"2024-09-12T01:46:19.970676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68075,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":199,"num_read_kb":166,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457287+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:46:14.457287+0000","last_peered":"2024-09-12T01:46:14.457287+0000","last_clean":"2024-09-12T01:46:14.457287+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:46:14.457287+0000","last_undegraded":"2024-09-12T01:46:14.457287+0000","last_fullsized":"2024-09-12T01:46:14.457287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470461+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:46:14.470461+0000","last_peered":"2024-09-12T01:46:14.470461+0000","last_clean":"2024-09-12T01:46:14.470461+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:46:14.470461+0000","last_undegraded":"2024-09-12T01:46:14.470461+0000","last_fullsized":"2024-09-12T01:46:14.470461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472069+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:46:14.472069+0000","last_peered":"2024-09-12T01:46:14.472069+0000","last_clean":"2024-09-12T01:46:14.472069+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:46:14.472069+0000","last_undegraded":"2024-09-12T01:46:14.472069+0000","last_fullsized":"2024-09-12T01:46:14.472069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:15.965863+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:46:15.965863+0000","last_peered":"2024-09-12T01:46:15.965863+0000","last_clean":"2024-09-12T01:46:15.965863+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:46:15.965863+0000","last_undegraded":"2024-09-12T01:46:15.965863+0000","last_fullsized":"2024-09-12T01:46:15.965863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468634+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:46:14.468634+0000","last_peered":"2024-09-12T01:46:14.468634+0000","last_clean":"2024-09-12T01:46:14.468634+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:46:14.468634+0000","last_undegraded":"2024-09-12T01:46:14.468634+0000","last_fullsized":"2024-09-12T01:46:14.468634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469018+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:46:14.469018+0000","last_peered":"2024-09-12T01:46:14.469018+0000","last_clean":"2024-09-12T01:46:14.469018+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:46:14.469018+0000","last_undegraded":"2024-09-12T01:46:14.469018+0000","last_fullsized":"2024-09-12T01:46:14.469018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.458089+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:46:14.458089+0000","last_peered":"2024-09-12T01:46:14.458089+0000","last_clean":"2024-09-12T01:46:14.458089+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:46:14.458089+0000","last_undegraded":"2024-09-12T01:46:14.458089+0000","last_fullsized":"2024-09-12T01:46:14.458089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467837+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:46:14.467837+0000","last_peered":"2024-09-12T01:46:14.467837+0000","last_clean":"2024-09-12T01:46:14.467837+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:46:14.467837+0000","last_undegraded":"2024-09-12T01:46:14.467837+0000","last_fullsized":"2024-09-12T01:46:14.467837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465783+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:46:14.465783+0000","last_peered":"2024-09-12T01:46:14.465783+0000","last_clean":"2024-09-12T01:46:14.465783+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:46:14.465783+0000","last_undegraded":"2024-09-12T01:46:14.465783+0000","last_fullsized":"2024-09-12T01:46:14.465783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473700+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:46:14.473700+0000","last_peered":"2024-09-12T01:46:14.473700+0000","last_clean":"2024-09-12T01:46:14.473700+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:46:14.473700+0000","last_undegraded":"2024-09-12T01:46:14.473700+0000","last_fullsized":"2024-09-12T01:46:14.473700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818582+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818582+0000","last_peered":"2024-09-12T01:44:45.818582+0000","last_clean":"2024-09-12T01:44:45.818582+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818582+0000","last_undegraded":"2024-09-12T01:44:45.818582+0000","last_fullsized":"2024-09-12T01:44:45.818582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.464071+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:46:14.464071+0000","last_peered":"2024-09-12T01:46:14.464071+0000","last_clean":"2024-09-12T01:46:14.464071+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:46:14.464071+0000","last_undegraded":"2024-09-12T01:46:14.464071+0000","last_fullsized":"2024-09-12T01:46:14.464071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449016+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:46:14.449016+0000","last_peered":"2024-09-12T01:46:14.449016+0000","last_clean":"2024-09-12T01:46:14.449016+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:46:14.449016+0000","last_undegraded":"2024-09-12T01:46:14.449016+0000","last_fullsized":"2024-09-12T01:46:14.449016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:16.966739+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:46:16.966739+0000","last_peered":"2024-09-12T01:46:16.966739+0000","last_clean":"2024-09-12T01:46:16.966739+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:46:16.966739+0000","last_undegraded":"2024-09-12T01:46:16.966739+0000","last_fullsized":"2024-09-12T01:46:16.966739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468646+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:46:14.468646+0000","last_peered":"2024-09-12T01:46:14.468646+0000","last_clean":"2024-09-12T01:46:14.468646+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:46:14.468646+0000","last_undegraded":"2024-09-12T01:46:14.468646+0000","last_fullsized":"2024-09-12T01:46:14.468646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.450592+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:46:14.450592+0000","last_peered":"2024-09-12T01:46:14.450592+0000","last_clean":"2024-09-12T01:46:14.450592+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:46:14.450592+0000","last_undegraded":"2024-09-12T01:46:14.450592+0000","last_fullsized":"2024-09-12T01:46:14.450592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468594+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:46:14.468594+0000","last_peered":"2024-09-12T01:46:14.468594+0000","last_clean":"2024-09-12T01:46:14.468594+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:46:14.468594+0000","last_undegraded":"2024-09-12T01:46:14.468594+0000","last_fullsized":"2024-09-12T01:46:14.468594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449242+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:46:14.449242+0000","last_peered":"2024-09-12T01:46:14.449242+0000","last_clean":"2024-09-12T01:46:14.449242+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:46:14.449242+0000","last_undegraded":"2024-09-12T01:46:14.449242+0000","last_fullsized":"2024-09-12T01:46:14.449242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.792350+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:46:14.792350+0000","last_peered":"2024-09-12T01:46:14.792350+0000","last_clean":"2024-09-12T01:46:14.792350+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:46:14.792350+0000","last_undegraded":"2024-09-12T01:46:14.792350+0000","last_fullsized":"2024-09-12T01:46:14.792350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465155+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:46:14.465155+0000","last_peered":"2024-09-12T01:46:14.465155+0000","last_clean":"2024-09-12T01:46:14.465155+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:46:14.465155+0000","last_undegraded":"2024-09-12T01:46:14.465155+0000","last_fullsized":"2024-09-12T01:46:14.465155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457998+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:46:14.457998+0000","last_peered":"2024-09-12T01:46:14.457998+0000","last_clean":"2024-09-12T01:46:14.457998+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:46:14.457998+0000","last_undegraded":"2024-09-12T01:46:14.457998+0000","last_fullsized":"2024-09-12T01:46:14.457998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.440457+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:46:14.440457+0000","last_peered":"2024-09-12T01:46:14.440457+0000","last_clean":"2024-09-12T01:46:14.440457+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:46:14.440457+0000","last_undegraded":"2024-09-12T01:46:14.440457+0000","last_fullsized":"2024-09-12T01:46:14.440457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463449+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:46:14.463449+0000","last_peered":"2024-09-12T01:46:14.463449+0000","last_clean":"2024-09-12T01:46:14.463449+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:46:14.463449+0000","last_undegraded":"2024-09-12T01:46:14.463449+0000","last_fullsized":"2024-09-12T01:46:14.463449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470639+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:46:14.470639+0000","last_peered":"2024-09-12T01:46:14.470639+0000","last_clean":"2024-09-12T01:46:14.470639+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:46:14.470639+0000","last_undegraded":"2024-09-12T01:46:14.470639+0000","last_fullsized":"2024-09-12T01:46:14.470639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473199+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:46:14.473199+0000","last_peered":"2024-09-12T01:46:14.473199+0000","last_clean":"2024-09-12T01:46:14.473199+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:46:14.473199+0000","last_undegraded":"2024-09-12T01:46:14.473199+0000","last_fullsized":"2024-09-12T01:46:14.473199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469618+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:46:14.469618+0000","last_peered":"2024-09-12T01:46:14.469618+0000","last_clean":"2024-09-12T01:46:14.469618+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:46:14.469618+0000","last_undegraded":"2024-09-12T01:46:14.469618+0000","last_fullsized":"2024-09-12T01:46:14.469618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473089+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:46:14.473089+0000","last_peered":"2024-09-12T01:46:14.473089+0000","last_clean":"2024-09-12T01:46:14.473089+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:46:14.473089+0000","last_undegraded":"2024-09-12T01:46:14.473089+0000","last_fullsized":"2024-09-12T01:46:14.473089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.454649+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:46:14.454649+0000","last_peered":"2024-09-12T01:46:14.454649+0000","last_clean":"2024-09-12T01:46:14.454649+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:46:14.454649+0000","last_undegraded":"2024-09-12T01:46:14.454649+0000","last_fullsized":"2024-09-12T01:46:14.454649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459700+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:46:14.459700+0000","last_peered":"2024-09-12T01:46:14.459700+0000","last_clean":"2024-09-12T01:46:14.459700+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:46:14.459700+0000","last_undegraded":"2024-09-12T01:46:14.459700+0000","last_fullsized":"2024-09-12T01:46:14.459700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72219,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":588,"num_read_kb":372,"num_write":173,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1092119,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168625,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1030024,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438391,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":509124,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":499214,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:46:23.517 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T01:46:23.517 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T01:46:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- 172.21.15.122:0/3568839404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 msgr2=0x7fc434108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/3568839404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434108e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009ca0 tx=0x7fc42c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- 172.21.15.122:0/3568839404 shutdown_connections 2024-09-12T01:46:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/3568839404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434108e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/3568839404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 0x7fc4341020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/3568839404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc434100a30 0x7fc434100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- 172.21.15.122:0/3568839404 >> 172.21.15.122:0/3568839404 conn(0x7fc4340fc1b0 msgr2=0x7fc4340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- 172.21.15.122:0/3568839404 shutdown_connections 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- 172.21.15.122:0/3568839404 wait complete. 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 Processor -- start 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- start start 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc434100a30 0x7fc43410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 0x7fc4341099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410d880 con 0x7fc4341025e0 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410d700 con 0x7fc434100a30 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.654+0000 7fc43b0ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410da00 con 0x7fc434101c20 2024-09-12T01:46:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39520/0 (socket says 172.21.15.122:39520) 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 -- 172.21.15.122:0/1501797733 learned_addr learned my addr 172.21.15.122:0/1501797733 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc433fff640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 0x7fc4341099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc438e56640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc434100a30 0x7fc43410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 -- 172.21.15.122:0/1501797733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 msgr2=0x7fc4341099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 0x7fc4341099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 -- 172.21.15.122:0/1501797733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc434100a30 msgr2=0x7fc43410b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc434100a30 0x7fc43410b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 -- 172.21.15.122:0/1501797733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43410a7c0 con 0x7fc4341025e0 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc433fff640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 0x7fc4341099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc439657640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434109f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0316b0 tx=0x7fc42c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c003e20 con 0x7fc4341025e0 2024-09-12T01:46:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42c004520 con 0x7fc4341025e0 2024-09-12T01:46:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4341a1fc0 con 0x7fc4341025e0 2024-09-12T01:46:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c02fe60 con 0x7fc4341025e0 2024-09-12T01:46:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.655+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4341a2500 con 0x7fc4341025e0 2024-09-12T01:46:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.656+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc434107dd0 con 0x7fc4341025e0 2024-09-12T01:46:23.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.657+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc42c0046d0 con 0x7fc4341025e0 2024-09-12T01:46:23.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.658+0000 7fc431ffb640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4240a2c20 0x7fc4240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:23.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.658+0000 7fc438e56640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4240a2c20 0x7fc4240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:23.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.658+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc42c0f0a40 con 0x7fc4341025e0 2024-09-12T01:46:23.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.659+0000 7fc438e56640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4240a2c20 0x7fc4240a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc434101a80 tx=0x7fc428008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:23.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.662+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc42c0bd950 con 0x7fc4341025e0 2024-09-12T01:46:23.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.826+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc434062630 con 0x7fc4240a2c20 2024-09-12T01:46:23.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.833+0000 7fc431ffb640 1 -- 172.21.15.122:0/1501797733 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259150 (secure 0 0 0) 0x7fc434062630 con 0x7fc4240a2c20 2024-09-12T01:46:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4240a2c20 msgr2=0x7fc4240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4240a2c20 0x7fc4240a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc434101a80 tx=0x7fc428008040 comp rx=0 tx=0).stop 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 msgr2=0x7fc434109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434109f30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0316b0 tx=0x7fc42c0316e0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 shutdown_connections 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4240a2c20 0x7fc4240a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4341025e0 0x7fc434109f30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc434101c20 0x7fc4341099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 --2- 172.21.15.122:0/1501797733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc434100a30 0x7fc43410b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 >> 172.21.15.122:0/1501797733 conn(0x7fc4340fc1b0 msgr2=0x7fc434104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 shutdown_connections 2024-09-12T01:46:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:23.836+0000 7fc43b0ba640 1 -- 172.21.15.122:0/1501797733 wait complete. 2024-09-12T01:46:23.845 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:23 smithi122 ceph-mon[30925]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T01:46:23.846 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-12T01:46:22.470281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662821,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":714,"num_read_kb":481,"num_write":356,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360304,"kb_used_data":11056,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581520,"statfs":{"total":1151940427776,"available":1151571476480,"internally_reserved":0,"allocated":11321344,"data_stored":7225699,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6464,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":49,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472514+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:46:14.472514+0000","last_peered":"2024-09-12T01:46:14.472514+0000","last_clean":"2024-09-12T01:46:14.472514+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:46:14.472514+0000","last_undegraded":"2024-09-12T01:46:14.472514+0000","last_fullsized":"2024-09-12T01:46:14.472514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.451157+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:46:14.451157+0000","last_peered":"2024-09-12T01:46:14.451157+0000","last_clean":"2024-09-12T01:46:14.451157+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:46:14.451157+0000","last_undegraded":"2024-09-12T01:46:14.451157+0000","last_fullsized":"2024-09-12T01:46:14.451157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462071+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:46:14.462071+0000","last_peered":"2024-09-12T01:46:14.462071+0000","last_clean":"2024-09-12T01:46:14.462071+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:46:14.462071+0000","last_undegraded":"2024-09-12T01:46:14.462071+0000","last_fullsized":"2024-09-12T01:46:14.462071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462225+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:46:14.462225+0000","last_peered":"2024-09-12T01:46:14.462225+0000","last_clean":"2024-09-12T01:46:14.462225+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:46:14.462225+0000","last_undegraded":"2024-09-12T01:46:14.462225+0000","last_fullsized":"2024-09-12T01:46:14.462225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456436+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:46:14.456436+0000","last_peered":"2024-09-12T01:46:14.456436+0000","last_clean":"2024-09-12T01:46:14.456436+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:46:14.456436+0000","last_undegraded":"2024-09-12T01:46:14.456436+0000","last_fullsized":"2024-09-12T01:46:14.456436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470354+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:46:14.470354+0000","last_peered":"2024-09-12T01:46:14.470354+0000","last_clean":"2024-09-12T01:46:14.470354+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:46:14.470354+0000","last_undegraded":"2024-09-12T01:46:14.470354+0000","last_fullsized":"2024-09-12T01:46:14.470354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468414+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:46:14.468414+0000","last_peered":"2024-09-12T01:46:14.468414+0000","last_clean":"2024-09-12T01:46:14.468414+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:46:14.468414+0000","last_undegraded":"2024-09-12T01:46:14.468414+0000","last_fullsized":"2024-09-12T01:46:14.468414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470863+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:46:14.470863+0000","last_peered":"2024-09-12T01:46:14.470863+0000","last_clean":"2024-09-12T01:46:14.470863+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:46:14.470863+0000","last_undegraded":"2024-09-12T01:46:14.470863+0000","last_fullsized":"2024-09-12T01:46:14.470863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467613+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:46:14.467613+0000","last_peered":"2024-09-12T01:46:14.467613+0000","last_clean":"2024-09-12T01:46:14.467613+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:46:14.467613+0000","last_undegraded":"2024-09-12T01:46:14.467613+0000","last_fullsized":"2024-09-12T01:46:14.467613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471418+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:46:14.471418+0000","last_peered":"2024-09-12T01:46:14.471418+0000","last_clean":"2024-09-12T01:46:14.471418+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:46:14.471418+0000","last_undegraded":"2024-09-12T01:46:14.471418+0000","last_fullsized":"2024-09-12T01:46:14.471418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473581+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:46:14.473581+0000","last_peered":"2024-09-12T01:46:14.473581+0000","last_clean":"2024-09-12T01:46:14.473581+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:46:14.473581+0000","last_undegraded":"2024-09-12T01:46:14.473581+0000","last_fullsized":"2024-09-12T01:46:14.473581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466586+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:46:14.466586+0000","last_peered":"2024-09-12T01:46:14.466586+0000","last_clean":"2024-09-12T01:46:14.466586+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:46:14.466586+0000","last_undegraded":"2024-09-12T01:46:14.466586+0000","last_fullsized":"2024-09-12T01:46:14.466586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470765+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:46:14.470765+0000","last_peered":"2024-09-12T01:46:14.470765+0000","last_clean":"2024-09-12T01:46:14.470765+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:46:14.470765+0000","last_undegraded":"2024-09-12T01:46:14.470765+0000","last_fullsized":"2024-09-12T01:46:14.470765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456811+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:46:14.456811+0000","last_peered":"2024-09-12T01:46:14.456811+0000","last_clean":"2024-09-12T01:46:14.456811+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:46:14.456811+0000","last_undegraded":"2024-09-12T01:46:14.456811+0000","last_fullsized":"2024-09-12T01:46:14.456811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462152+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:46:14.462152+0000","last_peered":"2024-09-12T01:46:14.462152+0000","last_clean":"2024-09-12T01:46:14.462152+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:46:14.462152+0000","last_undegraded":"2024-09-12T01:46:14.462152+0000","last_fullsized":"2024-09-12T01:46:14.462152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470248+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:46:14.470248+0000","last_peered":"2024-09-12T01:46:14.470248+0000","last_clean":"2024-09-12T01:46:14.470248+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:46:14.470248+0000","last_undegraded":"2024-09-12T01:46:14.470248+0000","last_fullsized":"2024-09-12T01:46:14.470248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472627+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:46:14.472627+0000","last_peered":"2024-09-12T01:46:14.472627+0000","last_clean":"2024-09-12T01:46:14.472627+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:46:14.472627+0000","last_undegraded":"2024-09-12T01:46:14.472627+0000","last_fullsized":"2024-09-12T01:46:14.472627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466306+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:46:14.466306+0000","last_peered":"2024-09-12T01:46:14.466306+0000","last_clean":"2024-09-12T01:46:14.466306+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:46:14.466306+0000","last_undegraded":"2024-09-12T01:46:14.466306+0000","last_fullsized":"2024-09-12T01:46:14.466306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457876+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:46:14.457876+0000","last_peered":"2024-09-12T01:46:14.457876+0000","last_clean":"2024-09-12T01:46:14.457876+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:46:14.457876+0000","last_undegraded":"2024-09-12T01:46:14.457876+0000","last_fullsized":"2024-09-12T01:46:14.457876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461219+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:46:14.461219+0000","last_peered":"2024-09-12T01:46:14.461219+0000","last_clean":"2024-09-12T01:46:14.461219+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:46:14.461219+0000","last_undegraded":"2024-09-12T01:46:14.461219+0000","last_fullsized":"2024-09-12T01:46:14.461219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469829+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:46:14.469829+0000","last_peered":"2024-09-12T01:46:14.469829+0000","last_clean":"2024-09-12T01:46:14.469829+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:46:14.469829+0000","last_undegraded":"2024-09-12T01:46:14.469829+0000","last_fullsized":"2024-09-12T01:46:14.469829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471133+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:46:14.471133+0000","last_peered":"2024-09-12T01:46:14.471133+0000","last_clean":"2024-09-12T01:46:14.471133+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:46:14.471133+0000","last_undegraded":"2024-09-12T01:46:14.471133+0000","last_fullsized":"2024-09-12T01:46:14.471133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460490+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:46:14.460490+0000","last_peered":"2024-09-12T01:46:14.460490+0000","last_clean":"2024-09-12T01:46:14.460490+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:46:14.460490+0000","last_undegraded":"2024-09-12T01:46:14.460490+0000","last_fullsized":"2024-09-12T01:46:14.460490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.455176+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:46:14.455176+0000","last_peered":"2024-09-12T01:46:14.455176+0000","last_clean":"2024-09-12T01:46:14.455176+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:46:14.455176+0000","last_undegraded":"2024-09-12T01:46:14.455176+0000","last_fullsized":"2024-09-12T01:46:14.455176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449237+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:46:14.449237+0000","last_peered":"2024-09-12T01:46:14.449237+0000","last_clean":"2024-09-12T01:46:14.449237+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:46:14.449237+0000","last_undegraded":"2024-09-12T01:46:14.449237+0000","last_fullsized":"2024-09-12T01:46:14.449237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459600+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:46:14.459600+0000","last_peered":"2024-09-12T01:46:14.459600+0000","last_clean":"2024-09-12T01:46:14.459600+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:46:14.459600+0000","last_undegraded":"2024-09-12T01:46:14.459600+0000","last_fullsized":"2024-09-12T01:46:14.459600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456999+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:46:14.456999+0000","last_peered":"2024-09-12T01:46:14.456999+0000","last_clean":"2024-09-12T01:46:14.456999+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:46:14.456999+0000","last_undegraded":"2024-09-12T01:46:14.456999+0000","last_fullsized":"2024-09-12T01:46:14.456999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.453782+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:46:14.453782+0000","last_peered":"2024-09-12T01:46:14.453782+0000","last_clean":"2024-09-12T01:46:14.453782+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:46:14.453782+0000","last_undegraded":"2024-09-12T01:46:14.453782+0000","last_fullsized":"2024-09-12T01:46:14.453782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457999+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:46:14.457999+0000","last_peered":"2024-09-12T01:46:14.457999+0000","last_clean":"2024-09-12T01:46:14.457999+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:46:14.457999+0000","last_undegraded":"2024-09-12T01:46:14.457999+0000","last_fullsized":"2024-09-12T01:46:14.457999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460598+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:46:14.460598+0000","last_peered":"2024-09-12T01:46:14.460598+0000","last_clean":"2024-09-12T01:46:14.460598+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:46:14.460598+0000","last_undegraded":"2024-09-12T01:46:14.460598+0000","last_fullsized":"2024-09-12T01:46:14.460598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463452+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:46:14.463452+0000","last_peered":"2024-09-12T01:46:14.463452+0000","last_clean":"2024-09-12T01:46:14.463452+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:46:14.463452+0000","last_undegraded":"2024-09-12T01:46:14.463452+0000","last_fullsized":"2024-09-12T01:46:14.463452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463643+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:46:14.463643+0000","last_peered":"2024-09-12T01:46:14.463643+0000","last_clean":"2024-09-12T01:46:14.463643+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:46:14.463643+0000","last_undegraded":"2024-09-12T01:46:14.463643+0000","last_fullsized":"2024-09-12T01:46:14.463643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457236+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:46:14.457236+0000","last_peered":"2024-09-12T01:46:14.457236+0000","last_clean":"2024-09-12T01:46:14.457236+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:46:14.457236+0000","last_undegraded":"2024-09-12T01:46:14.457236+0000","last_fullsized":"2024-09-12T01:46:14.457236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":121,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:16.966056+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:46:16.966056+0000","last_peered":"2024-09-12T01:46:16.966056+0000","last_clean":"2024-09-12T01:46:16.966056+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:46:16.966056+0000","last_undegraded":"2024-09-12T01:46:16.966056+0000","last_fullsized":"2024-09-12T01:46:16.966056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461733+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:46:14.461733+0000","last_peered":"2024-09-12T01:46:14.461733+0000","last_clean":"2024-09-12T01:46:14.461733+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:46:14.461733+0000","last_undegraded":"2024-09-12T01:46:14.461733+0000","last_fullsized":"2024-09-12T01:46:14.461733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.452061+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:46:14.452061+0000","last_peered":"2024-09-12T01:46:14.452061+0000","last_clean":"2024-09-12T01:46:14.452061+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:46:14.452061+0000","last_undegraded":"2024-09-12T01:46:14.452061+0000","last_fullsized":"2024-09-12T01:46:14.452061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466729+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:46:14.466729+0000","last_peered":"2024-09-12T01:46:14.466729+0000","last_clean":"2024-09-12T01:46:14.466729+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:46:14.466729+0000","last_undegraded":"2024-09-12T01:46:14.466729+0000","last_fullsized":"2024-09-12T01:46:14.466729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:18.968110+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:46:18.968110+0000","last_peered":"2024-09-12T01:46:18.968110+0000","last_clean":"2024-09-12T01:46:18.968110+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:46:18.968110+0000","last_undegraded":"2024-09-12T01:46:18.968110+0000","last_fullsized":"2024-09-12T01:46:18.968110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473143+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:46:14.473143+0000","last_peered":"2024-09-12T01:46:14.473143+0000","last_clean":"2024-09-12T01:46:14.473143+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:46:14.473143+0000","last_undegraded":"2024-09-12T01:46:14.473143+0000","last_fullsized":"2024-09-12T01:46:14.473143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460971+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:46:14.460971+0000","last_peered":"2024-09-12T01:46:14.460971+0000","last_clean":"2024-09-12T01:46:14.460971+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:46:14.460971+0000","last_undegraded":"2024-09-12T01:46:14.460971+0000","last_fullsized":"2024-09-12T01:46:14.460971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473209+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:46:14.473209+0000","last_peered":"2024-09-12T01:46:14.473209+0000","last_clean":"2024-09-12T01:46:14.473209+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:46:14.473209+0000","last_undegraded":"2024-09-12T01:46:14.473209+0000","last_fullsized":"2024-09-12T01:46:14.473209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":255,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:19.970676+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:46:19.970676+0000","last_peered":"2024-09-12T01:46:19.970676+0000","last_clean":"2024-09-12T01:46:19.970676+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:46:19.970676+0000","last_undegraded":"2024-09-12T01:46:19.970676+0000","last_fullsized":"2024-09-12T01:46:19.970676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68075,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":199,"num_read_kb":166,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457287+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:46:14.457287+0000","last_peered":"2024-09-12T01:46:14.457287+0000","last_clean":"2024-09-12T01:46:14.457287+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:46:14.457287+0000","last_undegraded":"2024-09-12T01:46:14.457287+0000","last_fullsized":"2024-09-12T01:46:14.457287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470461+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:46:14.470461+0000","last_peered":"2024-09-12T01:46:14.470461+0000","last_clean":"2024-09-12T01:46:14.470461+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:46:14.470461+0000","last_undegraded":"2024-09-12T01:46:14.470461+0000","last_fullsized":"2024-09-12T01:46:14.470461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472069+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:46:14.472069+0000","last_peered":"2024-09-12T01:46:14.472069+0000","last_clean":"2024-09-12T01:46:14.472069+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:46:14.472069+0000","last_undegraded":"2024-09-12T01:46:14.472069+0000","last_fullsized":"2024-09-12T01:46:14.472069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:15.965863+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:46:15.965863+0000","last_peered":"2024-09-12T01:46:15.965863+0000","last_clean":"2024-09-12T01:46:15.965863+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:46:15.965863+0000","last_undegraded":"2024-09-12T01:46:15.965863+0000","last_fullsized":"2024-09-12T01:46:15.965863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468634+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:46:14.468634+0000","last_peered":"2024-09-12T01:46:14.468634+0000","last_clean":"2024-09-12T01:46:14.468634+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:46:14.468634+0000","last_undegraded":"2024-09-12T01:46:14.468634+0000","last_fullsized":"2024-09-12T01:46:14.468634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469018+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:46:14.469018+0000","last_peered":"2024-09-12T01:46:14.469018+0000","last_clean":"2024-09-12T01:46:14.469018+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:46:14.469018+0000","last_undegraded":"2024-09-12T01:46:14.469018+0000","last_fullsized":"2024-09-12T01:46:14.469018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.839177+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:46.839177+0000","last_peered":"2024-09-12T01:44:46.839177+0000","last_clean":"2024-09-12T01:44:46.839177+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:46.839177+0000","last_undegraded":"2024-09-12T01:44:46.839177+0000","last_fullsized":"2024-09-12T01:44:46.839177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.458089+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:46:14.458089+0000","last_peered":"2024-09-12T01:46:14.458089+0000","last_clean":"2024-09-12T01:46:14.458089+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:46:14.458089+0000","last_undegraded":"2024-09-12T01:46:14.458089+0000","last_fullsized":"2024-09-12T01:46:14.458089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467837+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:46:14.467837+0000","last_peered":"2024-09-12T01:46:14.467837+0000","last_clean":"2024-09-12T01:46:14.467837+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:46:14.467837+0000","last_undegraded":"2024-09-12T01:46:14.467837+0000","last_fullsized":"2024-09-12T01:46:14.467837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465783+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:46:14.465783+0000","last_peered":"2024-09-12T01:46:14.465783+0000","last_clean":"2024-09-12T01:46:14.465783+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:46:14.465783+0000","last_undegraded":"2024-09-12T01:46:14.465783+0000","last_fullsized":"2024-09-12T01:46:14.465783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473700+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:46:14.473700+0000","last_peered":"2024-09-12T01:46:14.473700+0000","last_clean":"2024-09-12T01:46:14.473700+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:46:14.473700+0000","last_undegraded":"2024-09-12T01:46:14.473700+0000","last_fullsized":"2024-09-12T01:46:14.473700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818582+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818582+0000","last_peered":"2024-09-12T01:44:45.818582+0000","last_clean":"2024-09-12T01:44:45.818582+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818582+0000","last_undegraded":"2024-09-12T01:44:45.818582+0000","last_fullsized":"2024-09-12T01:44:45.818582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.464071+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:46:14.464071+0000","last_peered":"2024-09-12T01:46:14.464071+0000","last_clean":"2024-09-12T01:46:14.464071+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:46:14.464071+0000","last_undegraded":"2024-09-12T01:46:14.464071+0000","last_fullsized":"2024-09-12T01:46:14.464071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449016+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:46:14.449016+0000","last_peered":"2024-09-12T01:46:14.449016+0000","last_clean":"2024-09-12T01:46:14.449016+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:46:14.449016+0000","last_undegraded":"2024-09-12T01:46:14.449016+0000","last_fullsized":"2024-09-12T01:46:14.449016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:16.966739+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:46:16.966739+0000","last_peered":"2024-09-12T01:46:16.966739+0000","last_clean":"2024-09-12T01:46:16.966739+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:46:16.966739+0000","last_undegraded":"2024-09-12T01:46:16.966739+0000","last_fullsized":"2024-09-12T01:46:16.966739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468646+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:46:14.468646+0000","last_peered":"2024-09-12T01:46:14.468646+0000","last_clean":"2024-09-12T01:46:14.468646+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:46:14.468646+0000","last_undegraded":"2024-09-12T01:46:14.468646+0000","last_fullsized":"2024-09-12T01:46:14.468646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.450592+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:46:14.450592+0000","last_peered":"2024-09-12T01:46:14.450592+0000","last_clean":"2024-09-12T01:46:14.450592+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:46:14.450592+0000","last_undegraded":"2024-09-12T01:46:14.450592+0000","last_fullsized":"2024-09-12T01:46:14.450592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468594+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:46:14.468594+0000","last_peered":"2024-09-12T01:46:14.468594+0000","last_clean":"2024-09-12T01:46:14.468594+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:46:14.468594+0000","last_undegraded":"2024-09-12T01:46:14.468594+0000","last_fullsized":"2024-09-12T01:46:14.468594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449242+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:46:14.449242+0000","last_peered":"2024-09-12T01:46:14.449242+0000","last_clean":"2024-09-12T01:46:14.449242+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:46:14.449242+0000","last_undegraded":"2024-09-12T01:46:14.449242+0000","last_fullsized":"2024-09-12T01:46:14.449242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.792350+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:46:14.792350+0000","last_peered":"2024-09-12T01:46:14.792350+0000","last_clean":"2024-09-12T01:46:14.792350+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:46:14.792350+0000","last_undegraded":"2024-09-12T01:46:14.792350+0000","last_fullsized":"2024-09-12T01:46:14.792350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465155+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:46:14.465155+0000","last_peered":"2024-09-12T01:46:14.465155+0000","last_clean":"2024-09-12T01:46:14.465155+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:46:14.465155+0000","last_undegraded":"2024-09-12T01:46:14.465155+0000","last_fullsized":"2024-09-12T01:46:14.465155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457998+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:46:14.457998+0000","last_peered":"2024-09-12T01:46:14.457998+0000","last_clean":"2024-09-12T01:46:14.457998+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:46:14.457998+0000","last_undegraded":"2024-09-12T01:46:14.457998+0000","last_fullsized":"2024-09-12T01:46:14.457998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.440457+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:46:14.440457+0000","last_peered":"2024-09-12T01:46:14.440457+0000","last_clean":"2024-09-12T01:46:14.440457+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:46:14.440457+0000","last_undegraded":"2024-09-12T01:46:14.440457+0000","last_fullsized":"2024-09-12T01:46:14.440457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463449+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:46:14.463449+0000","last_peered":"2024-09-12T01:46:14.463449+0000","last_clean":"2024-09-12T01:46:14.463449+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:46:14.463449+0000","last_undegraded":"2024-09-12T01:46:14.463449+0000","last_fullsized":"2024-09-12T01:46:14.463449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470639+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:46:14.470639+0000","last_peered":"2024-09-12T01:46:14.470639+0000","last_clean":"2024-09-12T01:46:14.470639+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:46:14.470639+0000","last_undegraded":"2024-09-12T01:46:14.470639+0000","last_fullsized":"2024-09-12T01:46:14.470639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473199+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:46:14.473199+0000","last_peered":"2024-09-12T01:46:14.473199+0000","last_clean":"2024-09-12T01:46:14.473199+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:46:14.473199+0000","last_undegraded":"2024-09-12T01:46:14.473199+0000","last_fullsized":"2024-09-12T01:46:14.473199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469618+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:46:14.469618+0000","last_peered":"2024-09-12T01:46:14.469618+0000","last_clean":"2024-09-12T01:46:14.469618+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:46:14.469618+0000","last_undegraded":"2024-09-12T01:46:14.469618+0000","last_fullsized":"2024-09-12T01:46:14.469618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473089+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:46:14.473089+0000","last_peered":"2024-09-12T01:46:14.473089+0000","last_clean":"2024-09-12T01:46:14.473089+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:46:14.473089+0000","last_undegraded":"2024-09-12T01:46:14.473089+0000","last_fullsized":"2024-09-12T01:46:14.473089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.454649+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:46:14.454649+0000","last_peered":"2024-09-12T01:46:14.454649+0000","last_clean":"2024-09-12T01:46:14.454649+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:46:14.454649+0000","last_undegraded":"2024-09-12T01:46:14.454649+0000","last_fullsized":"2024-09-12T01:46:14.454649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459700+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:46:14.459700+0000","last_peered":"2024-09-12T01:46:14.459700+0000","last_clean":"2024-09-12T01:46:14.459700+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:46:14.459700+0000","last_undegraded":"2024-09-12T01:46:14.459700+0000","last_fullsized":"2024-09-12T01:46:14.459700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72219,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":588,"num_read_kb":372,"num_write":173,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":213496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1092119,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168625,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1030024,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438391,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444113,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":509124,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":499214,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:46:23.851 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:23 smithi151 ceph-mon[35229]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T01:46:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:23 smithi183 ceph-mon[35127]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T01:46:24.128 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:24.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 -- 172.21.15.122:0/732363522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 msgr2=0x7f84ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/732363522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec1067e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009ca0 tx=0x7f84dc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 -- 172.21.15.122:0/732363522 shutdown_connections 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/732363522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec106f10 0x7f84ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/732363522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec1067e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/732363522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec101de0 0x7f84ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 -- 172.21.15.122:0/732363522 >> 172.21.15.122:0/732363522 conn(0x7f84ec0fd8c0 msgr2=0x7f84ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 -- 172.21.15.122:0/732363522 shutdown_connections 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 -- 172.21.15.122:0/732363522 wait complete. 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 Processor -- start 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.603+0000 7f84f2a8a640 1 -- start start 2024-09-12T01:46:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec101de0 0x7f84ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec106f10 0x7f84ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec112010 con 0x7f84ec102790 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec111e90 con 0x7f84ec106f10 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec112190 con 0x7f84ec101de0 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39534/0 (socket says 172.21.15.122:39534) 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 -- 172.21.15.122:0/1536443058 learned_addr learned my addr 172.21.15.122:0/1536443058 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f0826640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec101de0 0x7f84ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 -- 172.21.15.122:0/1536443058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec101de0 msgr2=0x7f84ec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec101de0 0x7f84ec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 -- 172.21.15.122:0/1536443058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec106f10 msgr2=0x7f84ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec106f10 0x7f84ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 -- 172.21.15.122:0/1536443058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec1a6a70 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84ebfff640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec074a30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097b0 tx=0x7f84dc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc02fba0 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84dc02fd40 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec1a6d00 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.604+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc03b930 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.605+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec1a71e0 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.606+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ec102240 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.607+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f84dc030cc0 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.607+0000 7f84e9ffb640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84d80a2c10 0x7f84d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.607+0000 7f84f0826640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84d80a2c10 0x7f84d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.607+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f84dc0f08a0 con 0x7f84ec102790 2024-09-12T01:46:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.608+0000 7f84f0826640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84d80a2c10 0x7f84d80a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009890 tx=0x7f84d4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.611+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84dc0bd770 con 0x7f84ec102790 2024-09-12T01:46:24.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.811+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84ec02d7c0 con 0x7f84ec102790 2024-09-12T01:46:24.811 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:24 smithi122 ceph-mon[30925]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:24.811 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:24 smithi122 ceph-mon[30925]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:24.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.812+0000 7f84e9ffb640 1 -- 172.21.15.122:0/1536443058 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f84dc0098b0 con 0x7f84ec102790 2024-09-12T01:46:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:24.812 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:46:24.813 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84d80a2c10 msgr2=0x7f84d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84d80a2c10 0x7f84d80a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009890 tx=0x7f84d4008040 comp rx=0 tx=0).stop 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 msgr2=0x7f84ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec074a30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097b0 tx=0x7f84dc031b10 comp rx=0 tx=0).stop 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 shutdown_connections 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84d80a2c10 0x7f84d80a50d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec106f10 0x7f84ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84ec102790 0x7f84ec074a30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 --2- 172.21.15.122:0/1536443058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84ec101de0 0x7f84ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 >> 172.21.15.122:0/1536443058 conn(0x7f84ec0fd8c0 msgr2=0x7f84ec0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:24.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 shutdown_connections 2024-09-12T01:46:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:24.814+0000 7f84f2a8a640 1 -- 172.21.15.122:0/1536443058 wait complete. 2024-09-12T01:46:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:24 smithi183 ceph-mon[35127]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:24 smithi183 ceph-mon[35127]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:24 smithi151 ceph-mon[35229]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:24 smithi151 ceph-mon[35229]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:25.632 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:46:25.883 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:25 smithi183 ceph-mon[35127]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:25 smithi183 ceph-mon[35127]: pgmap v333: 97 pgs: 97 active+clean; 648 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T01:46:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:25 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1536443058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:25.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:25 smithi122 ceph-mon[30925]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:25 smithi122 ceph-mon[30925]: pgmap v333: 97 pgs: 97 active+clean; 648 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T01:46:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:25 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1536443058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:25 smithi151 ceph-mon[35229]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:25 smithi151 ceph-mon[35229]: pgmap v333: 97 pgs: 97 active+clean; 648 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T01:46:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1536443058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:26.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.337+0000 7ff0e43b7640 1 -- 172.21.15.122:0/569036792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc101df0 msgr2=0x7ff0dc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:26.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.337+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/569036792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc101df0 0x7ff0dc1021d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009900 tx=0x7ff0d402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- 172.21.15.122:0/569036792 shutdown_connections 2024-09-12T01:46:26.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/569036792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0dc106f20 0x7ff0dc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/569036792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc1027a0 0x7ff0dc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/569036792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc101df0 0x7ff0dc1021d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- 172.21.15.122:0/569036792 >> 172.21.15.122:0/569036792 conn(0x7ff0dc0fd8b0 msgr2=0x7ff0dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- 172.21.15.122:0/569036792 shutdown_connections 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- 172.21.15.122:0/569036792 wait complete. 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 Processor -- start 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- start start 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 0x7ff0dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0dc1027a0 0x7ff0dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 0x7ff0dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc111f40 con 0x7ff0dc106f20 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc111dc0 con 0x7ff0dc101df0 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.338+0000 7ff0e43b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1120c0 con 0x7ff0dc1027a0 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 0x7ff0dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:26.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 0x7ff0dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 0x7ff0dc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:43588/0 (socket says 172.21.15.122:43588) 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2153640 1 -- 172.21.15.122:0/485134335 learned_addr learned my addr 172.21.15.122:0/485134335 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 0x7ff0dc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39562/0 (socket says 172.21.15.122:39562) 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e1952640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0dc1027a0 0x7ff0dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 -- 172.21.15.122:0/485134335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0dc1027a0 msgr2=0x7ff0dc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0dc1027a0 0x7ff0dc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 -- 172.21.15.122:0/485134335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 msgr2=0x7ff0dc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 0x7ff0dc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 -- 172.21.15.122:0/485134335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc1a67e0 con 0x7ff0dc106f20 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2153640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 0x7ff0dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e2954640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 0x7ff0dc074f70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00cab0 tx=0x7ff0cc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:26.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00ad60 con 0x7ff0dc106f20 2024-09-12T01:46:26.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc00f930 con 0x7ff0dc106f20 2024-09-12T01:46:26.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.339+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc1a6ad0 con 0x7ff0dc106f20 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.340+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc018a00 con 0x7ff0dc106f20 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.340+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc1a7010 con 0x7ff0dc106f20 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.341+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc1032d0 con 0x7ff0dc106f20 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.341+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7ff0cc00f450 con 0x7ff0dc106f20 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.342+0000 7ff0db7fe640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff0c40a2af0 0x7ff0c40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.342+0000 7ff0e2153640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff0c40a2af0 0x7ff0c40a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.342+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff0cc0cf710 con 0x7ff0dc106f20 2024-09-12T01:46:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.343+0000 7ff0e2153640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff0c40a2af0 0x7ff0c40a4fb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40096c0 tx=0x7ff0d4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.346+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0cc014030 con 0x7ff0dc106f20 2024-09-12T01:46:26.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.517+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0dc075a20 con 0x7ff0dc106f20 2024-09-12T01:46:26.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.518+0000 7ff0db7fe640 1 -- 172.21.15.122:0/485134335 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7ff0cc09c510 con 0x7ff0dc106f20 2024-09-12T01:46:26.520 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:26.520 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff0c40a2af0 msgr2=0x7ff0c40a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff0c40a2af0 0x7ff0c40a4fb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40096c0 tx=0x7ff0d4005a20 comp rx=0 tx=0).stop 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 msgr2=0x7ff0dc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 0x7ff0dc074f70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00cab0 tx=0x7ff0cc00ceb0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 shutdown_connections 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff0c40a2af0 0x7ff0c40a4fb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc106f20 0x7ff0dc074f70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0dc1027a0 0x7ff0dc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 --2- 172.21.15.122:0/485134335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc101df0 0x7ff0dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:26.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 >> 172.21.15.122:0/485134335 conn(0x7ff0dc0fd8b0 msgr2=0x7ff0dc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:26.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 shutdown_connections 2024-09-12T01:46:26.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:26.521+0000 7ff0e43b7640 1 -- 172.21.15.122:0/485134335 wait complete. 2024-09-12T01:46:26.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:26 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/485134335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:26.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:26 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/485134335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:26 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/485134335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:27.144 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T01:46:27.144 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T01:46:27.154 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T01:46:27.155 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T01:46:27.155 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T01:46:27.155 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:46:27.155 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:46:27.158 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T01:46:27.170 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T01:46:27.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:46:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:27.916 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:27 smithi122 ceph-mon[30925]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T01:46:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:27 smithi183 ceph-mon[35127]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T01:46:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:27 smithi151 ceph-mon[35229]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T01:46:28.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 -- 172.21.15.122:0/304333834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8101420 msgr2=0x7f46b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 --2- 172.21.15.122:0/304333834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8101420 0x7f46b810d950 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009cd0 tx=0x7f46b002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 -- 172.21.15.122:0/304333834 shutdown_connections 2024-09-12T01:46:28.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 --2- 172.21.15.122:0/304333834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8101420 0x7f46b810d950 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 --2- 172.21.15.122:0/304333834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b8100a80 0x7f46b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 --2- 172.21.15.122:0/304333834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8106a80 0x7f46b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 -- 172.21.15.122:0/304333834 >> 172.21.15.122:0/304333834 conn(0x7f46b80fc7a0 msgr2=0x7f46b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 -- 172.21.15.122:0/304333834 shutdown_connections 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.130+0000 7f46be554640 1 -- 172.21.15.122:0/304333834 wait complete. 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 Processor -- start 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 -- start start 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8100a80 0x7f46b8197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b8101420 0x7f46b81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 0x7f46b8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8110e80 con 0x7f46b8106a80 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8110d00 con 0x7f46b8101420 2024-09-12T01:46:28.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46be554640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8111000 con 0x7f46b8100a80 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46bcaf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 0x7f46b8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46bcaf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 0x7f46b8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39576/0 (socket says 172.21.15.122:39576) 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.131+0000 7f46bcaf1640 1 -- 172.21.15.122:0/297139886 learned_addr learned my addr 172.21.15.122:0/297139886 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46b7fff640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8100a80 0x7f46b8197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46b77fe640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b8101420 0x7f46b81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46bcaf1640 1 -- 172.21.15.122:0/297139886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8100a80 msgr2=0x7f46b8197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8100a80 0x7f46b8197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46bcaf1640 1 -- 172.21.15.122:0/297139886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b8101420 msgr2=0x7f46b81975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b8101420 0x7f46b81975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46bcaf1640 1 -- 172.21.15.122:0/297139886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b81a68f0 con 0x7f46b8106a80 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46b7fff640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8100a80 0x7f46b8197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 0x7f46b8197b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f46b0005cc0 tx=0x7f46b0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46b57fa640 1 -- 172.21.15.122:0/297139886 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0004040 con 0x7f46b8106a80 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46b57fa640 1 -- 172.21.15.122:0/297139886 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b00046c0 con 0x7f46b8106a80 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b81a6b80 con 0x7f46b8106a80 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46b57fa640 1 -- 172.21.15.122:0/297139886 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0030e00 con 0x7f46b8106a80 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.132+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81a70c0 con 0x7f46b8106a80 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.134+0000 7f46b57fa640 1 -- 172.21.15.122:0/297139886 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f46b00305a0 con 0x7f46b8106a80 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.134+0000 7f46c092d740 1 Processor -- start 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 -- start start 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405c9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46b57fa640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46a00a2c10 0x7f46a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405cac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561405c04280 con 0x561405ca9d40 2024-09-12T01:46:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561405babb10 con 0x561405c9f480 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46c092d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561405c00240 con 0x561405c9eb60 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405cac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46b77fe640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46a00a2c10 0x7f46a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405cac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39592/0 (socket says 172.21.15.122:39592) 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 -- 172.21.15.122:0/1712504485 learned_addr learned my addr 172.21.15.122:0/1712504485 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46b57fa640 1 -- 172.21.15.122:0/297139886 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f46b002e030 con 0x7f46b8106a80 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46b77fe640 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46b7fff640 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405c9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 -- 172.21.15.122:0/1712504485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 msgr2=0x561405c9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405c9ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 -- 172.21.15.122:0/1712504485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 msgr2=0x561405ca9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46bcaf1640 1 -- 172.21.15.122:0/1712504485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561405bb81f0 con 0x561405ca9d40 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.135+0000 7f46b77fe640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46a00a2c10 0x7f46a00a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46b81991e0 tx=0x7f46ac009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405cac130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f46b0003f30 tx=0x7f46b0038f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64fa6a5d5509ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f469e7fc640 1 -- 172.21.15.122:0/1712504485 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00bcad0 con 0x561405ca9d40 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f469e7fc640 1 -- 172.21.15.122:0/1712504485 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b0036040 con 0x561405ca9d40 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f469e7fc640 1 -- 172.21.15.122:0/1712504485 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0044da0 con 0x561405ca9d40 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 -- 172.21.15.122:0/1712504485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 msgr2=0x561405cac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405cac130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f46b0003f30 tx=0x7f46b0038f90 comp rx=0 tx=0).stop 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 -- 172.21.15.122:0/1712504485 shutdown_connections 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405cac130 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 --2- 172.21.15.122:0/1712504485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405c9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 -- 172.21.15.122:0/1712504485 >> 172.21.15.122:0/1712504485 conn(0x561405c9b0f0 msgr2=0x561405c9b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 -- 172.21.15.122:0/1712504485 shutdown_connections 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.136+0000 7f46c092d740 1 -- 172.21.15.122:0/1712504485 wait complete. 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 -- start start 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 Processor -- start 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405ca6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405ca7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561405cac6d0 con 0x561405ca9d40 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561405bbef10 con 0x561405c9f480 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561405cac850 con 0x561405c9eb60 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46c092d740 10 client.0 _ll_register_callbacks cb 0x561405b87a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.137+0000 7f46bcaf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405ca7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405ca7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39602/0 (socket says 172.21.15.122:39602) 2024-09-12T01:46:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 -- 172.21.15.122:0/796368201 learned_addr learned my addr 172.21.15.122:0/796368201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46b7fff640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405ca6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46b77fe640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 -- 172.21.15.122:0/796368201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 msgr2=0x561405ca6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405ca6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 -- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 msgr2=0x561405ca7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 -- 172.21.15.122:0/796368201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614059756a0 con 0x561405ca9d40 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46bcaf1640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405ca7680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f46b00c58b0 tx=0x7f46b00bd1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7534011d7d9c2b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46977fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46977fe640 1 -- 172.21.15.122:0/796368201 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b00c5de0 con 0x561405ca9d40 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46977fe640 1 -- 172.21.15.122:0/796368201 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f46b00ee560 con 0x561405ca9d40 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b0036040 con 0x561405ca9d40 2024-09-12T01:46:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46977fe640 1 -- 172.21.15.122:0/796368201 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46b00bd870 con 0x561405ca9d40 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46c092d740 10 client.15225 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561405cac6d0 con 0x561405ca9d40 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.138+0000 7f46977fe640 1 -- 172.21.15.122:0/796368201 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b0036040 con 0x561405ca9d40 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46977fe640 1 -- 172.21.15.122:0/796368201 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f46b004e070 con 0x561405ca9d40 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46c092d740 10 client.15225 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46c092d740 10 client.15225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46c092d740 20 client.15225 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46c092d740 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405cb9460 0x561405cbb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46c092d740 4 client.15225 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561405c117f0 con 0x561405cb9460 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46b7fff640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405cb9460 0x561405cbb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46977fe640 10 client.15225 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.139+0000 7f46b7fff640 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405cb9460 0x561405cbb870 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19f4e7655334301e server_cookie=55be5524ae97ea3f in_seq=0 out_seq=0 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46977fe640 1 -- 172.21.15.122:0/796368201 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561405c117f0 con 0x561405cb9460 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46977fe640 10 client.15225 handle_command_reply: tid=0 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 client.15225 shutdown 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 msgr2=0x561405ca7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405ca7680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f46b00c58b0 tx=0x7f46b00bd1e0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 shutdown_connections 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405cb9460 0x561405cbb870 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405ca9d40 0x561405ca7680 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405ca7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 --2- 172.21.15.122:0/796368201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405ca6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 >> 172.21.15.122:0/796368201 conn(0x561405c9b0f0 msgr2=0x561405ca0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.140+0000 7f46977fe640 0 client.15225 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:28.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.141+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 shutdown_connections 2024-09-12T01:46:28.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.141+0000 7f46c092d740 1 -- 172.21.15.122:0/796368201 wait complete. 2024-09-12T01:46:28.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.141+0000 7f46c092d740 20 client.15225 trim_cache size 0 max 16384 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.157+0000 7f46c092d740 1 Processor -- start 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.157+0000 7f46c092d740 1 -- start start 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46c092d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405cc1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46c092d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 0x561405cc1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46c092d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405ca9d40 0x561405ca05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46c092d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b1fa0 con 0x561405c9f480 2024-09-12T01:46:28.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46c092d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81af870 con 0x561405ca9d40 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46c092d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ad6a0 con 0x561405c9eb60 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 0x561405cc1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 0x561405cc1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39614/0 (socket says 172.21.15.122:39614) 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 -- 172.21.15.122:0/3272785425 learned_addr learned my addr 172.21.15.122:0/3272785425 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b7fff640 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405cc1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 -- 172.21.15.122:0/3272785425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 msgr2=0x561405cc1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405cc1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 -- 172.21.15.122:0/3272785425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405ca9d40 msgr2=0x561405ca05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405ca9d40 0x561405ca05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 -- 172.21.15.122:0/3272785425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561405c96830 con 0x561405c9f480 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f46b77fe640 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 0x561405cc1680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46ac013c60 tx=0x7f46ac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb71f667603eba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f469dffb640 1 -- 172.21.15.122:0/3272785425 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004220 con 0x561405c9f480 2024-09-12T01:46:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f469dffb640 1 -- 172.21.15.122:0/3272785425 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac0043c0 con 0x561405c9f480 2024-09-12T01:46:28.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.158+0000 7f469dffb640 1 -- 172.21.15.122:0/3272785425 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac021bb0 con 0x561405c9f480 2024-09-12T01:46:28.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 -- 172.21.15.122:0/3272785425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 msgr2=0x561405cc1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 0x561405cc1680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46ac013c60 tx=0x7f46ac01f000 comp rx=0 tx=0).stop 2024-09-12T01:46:28.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 -- 172.21.15.122:0/3272785425 shutdown_connections 2024-09-12T01:46:28.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405ca9d40 0x561405ca05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9f480 0x561405cc1680 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 --2- 172.21.15.122:0/3272785425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405c9eb60 0x561405cc1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 -- 172.21.15.122:0/3272785425 >> 172.21.15.122:0/3272785425 conn(0x561405c9b0f0 msgr2=0x561405c9a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 -- 172.21.15.122:0/3272785425 shutdown_connections 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 -- 172.21.15.122:0/3272785425 wait complete. 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.159+0000 7f46c092d740 1 -- start start 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 Processor -- start 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 0x561405cb1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405cad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405ca9d40 0x561405cad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81add30 con 0x561405c9eb60 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81af1a0 con 0x561405c9f480 2024-09-12T01:46:28.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ae400 con 0x561405ca9d40 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46c092d740 10 client.0 _ll_register_callbacks cb 0x7f46b00b8da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 0x561405cb1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 0x561405cb1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39624/0 (socket says 172.21.15.122:39624) 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 -- 172.21.15.122:0/2669529408 learned_addr learned my addr 172.21.15.122:0/2669529408 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 -- 172.21.15.122:0/2669529408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405ca9d40 msgr2=0x561405cad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405ca9d40 0x561405cad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 -- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 msgr2=0x561405cad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405cad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 -- 172.21.15.122:0/2669529408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561405cadc60 con 0x561405c9eb60 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46b7fff640 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 0x561405cb1eb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46a8014550 tx=0x7f46a8014a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85864a2a50c742a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.160+0000 7f46977fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:28.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46977fe640 1 -- 172.21.15.122:0/2669529408 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8005020 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46977fe640 1 -- 172.21.15.122:0/2669529408 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f46a80051c0 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac0043c0 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46977fe640 1 -- 172.21.15.122:0/2669529408 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46a80053c0 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 10 client.15231 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561405d23f00 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46977fe640 1 -- 172.21.15.122:0/2669529408 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac0043c0 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46977fe640 1 -- 172.21.15.122:0/2669529408 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f46a801e770 con 0x561405c9eb60 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 10 client.15231 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 10 client.15231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 20 client.15231 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405d246e0 0x561405d26b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 4 client.15231 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561405c0bd00 con 0x561405d246e0 2024-09-12T01:46:28.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.161+0000 7f46b77fe640 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405d246e0 0x561405d26b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:28.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.162+0000 7f46977fe640 10 client.15231 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:28.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.162+0000 7f46b77fe640 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405d246e0 0x561405d26b60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e460549dcf28ca6b server_cookie=f372bd3215ff540 in_seq=0 out_seq=0 2024-09-12T01:46:28.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46977fe640 1 -- 172.21.15.122:0/2669529408 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561405c0bd00 con 0x561405d246e0 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46977fe640 10 client.15231 handle_command_reply: tid=0 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 client.15231 shutdown 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 msgr2=0x561405cb1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 0x561405cb1eb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46a8014550 tx=0x7f46a8014a10 comp rx=0 tx=0).stop 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 shutdown_connections 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561405d246e0 0x561405d26b60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561405ca9d40 0x561405cad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561405c9f480 0x561405cad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 --2- 172.21.15.122:0/2669529408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561405c9eb60 0x561405cb1eb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46977fe640 0 client.15231 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 >> 172.21.15.122:0/2669529408 conn(0x561405c9b0f0 msgr2=0x561405cb4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:28.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.163+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 shutdown_connections 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46c092d740 1 -- 172.21.15.122:0/2669529408 wait complete. 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46c092d740 20 client.15231 trim_cache size 0 max 16384 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "d7526da0-2c47-4bcf-96e9-8166d6e87fb2", 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-09-12T01:46:28.169 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46a00a2c10 msgr2=0x7f46a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46a00a2c10 0x7f46a00a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46b81991e0 tx=0x7f46ac009f90 comp rx=0 tx=0).stop 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 msgr2=0x7f46b8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 0x7f46b8197b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f46b0005cc0 tx=0x7f46b0032f40 comp rx=0 tx=0).stop 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 shutdown_connections 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f46a00a2c10 0x7f46a00a50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46b8106a80 0x7f46b8197b00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b8101420 0x7f46b81975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 --2- 172.21.15.122:0/297139886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b8100a80 0x7f46b8197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 >> 172.21.15.122:0/297139886 conn(0x7f46b80fc7a0 msgr2=0x7f46b80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:28.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 shutdown_connections 2024-09-12T01:46:28.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:28.164+0000 7f46be554640 1 -- 172.21.15.122:0/297139886 wait complete. 2024-09-12T01:46:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:28 smithi183 ceph-mon[35127]: scrub queued for path: / 2024-09-12T01:46:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:28 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:46:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:28 smithi183 ceph-mon[35127]: scrub summary: active paths [/] 2024-09-12T01:46:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:28 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:46:28.964 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:28 smithi122 ceph-mon[30925]: scrub queued for path: / 2024-09-12T01:46:28.965 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:28 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:46:28.965 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:28 smithi122 ceph-mon[30925]: scrub summary: active paths [/] 2024-09-12T01:46:28.965 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:28 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:46:28.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:46:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:28 smithi151 ceph-mon[35229]: scrub queued for path: / 2024-09-12T01:46:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:28 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:46:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:28 smithi151 ceph-mon[35229]: scrub summary: active paths [/] 2024-09-12T01:46:29.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:28 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:46:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 -- 172.21.15.122:0/95396407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 msgr2=0x7f1a94106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 --2- 172.21.15.122:0/95396407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a94106e60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009cd0 tx=0x7f1a9002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 -- 172.21.15.122:0/95396407 shutdown_connections 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 --2- 172.21.15.122:0/95396407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94101420 0x7f1a9410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 --2- 172.21.15.122:0/95396407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a94100a80 0x7f1a94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 --2- 172.21.15.122:0/95396407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a94106e60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 -- 172.21.15.122:0/95396407 >> 172.21.15.122:0/95396407 conn(0x7f1a940fc7a0 msgr2=0x7f1a940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 -- 172.21.15.122:0/95396407 shutdown_connections 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.168+0000 7f1a9af32640 1 -- 172.21.15.122:0/95396407 wait complete. 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 Processor -- start 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 -- start start 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94100a80 0x7f1a941992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a94101420 0x7f1a94199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a941a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94110ea0 con 0x7f1a94106a80 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94110d20 con 0x7f1a94101420 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a9af32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94111020 con 0x7f1a94100a80 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a994cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a941a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a994cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a941a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39640/0 (socket says 172.21.15.122:39640) 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.169+0000 7f1a994cf640 1 -- 172.21.15.122:0/673991159 learned_addr learned my addr 172.21.15.122:0/673991159 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a8bfff640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a94101420 0x7f1a94199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a98cce640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94100a80 0x7f1a941992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a994cf640 1 -- 172.21.15.122:0/673991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94100a80 msgr2=0x7f1a941992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a994cf640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94100a80 0x7f1a941992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a994cf640 1 -- 172.21.15.122:0/673991159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a94101420 msgr2=0x7f1a94199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a994cf640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a94101420 0x7f1a94199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a994cf640 1 -- 172.21.15.122:0/673991159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a941970c0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a994cf640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a941a4350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009fd0 tx=0x7f1a8c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c011830 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8c016070 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a941973b0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00f2e0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.170+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941978a0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.171+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a941021c0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.172+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1a8c0119d0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.173+0000 7f1a89ffb640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a7c0a2c10 0x7f1a7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.173+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1a8c0ceee0 con 0x7f1a94106a80 2024-09-12T01:46:29.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.173+0000 7f1a98cce640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a7c0a2c10 0x7f1a7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.174+0000 7f1a98cce640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a7c0a2c10 0x7f1a7c0a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a90006010 tx=0x7f1a90009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.180+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a8c09bdf0 con 0x7f1a94106a80 2024-09-12T01:46:29.253 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:29.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.363+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a94062630 con 0x7f1a7c0a2c10 2024-09-12T01:46:29.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.369+0000 7f1a89ffb640 1 -- 172.21.15.122:0/673991159 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259067 (secure 0 0 0) 0x7f1a94062630 con 0x7f1a7c0a2c10 2024-09-12T01:46:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:29.374 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a7c0a2c10 msgr2=0x7f1a7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a7c0a2c10 0x7f1a7c0a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a90006010 tx=0x7f1a90009800 comp rx=0 tx=0).stop 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 msgr2=0x7f1a941a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a941a4350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009fd0 tx=0x7f1a8c00b8e0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 shutdown_connections 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a7c0a2c10 0x7f1a7c0a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a94106a80 0x7f1a941a4350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a94101420 0x7f1a94199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 --2- 172.21.15.122:0/673991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94100a80 0x7f1a941992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 >> 172.21.15.122:0/673991159 conn(0x7f1a940fc7a0 msgr2=0x7f1a940fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:29.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 shutdown_connections 2024-09-12T01:46:29.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.372+0000 7f1a9af32640 1 -- 172.21.15.122:0/673991159 wait complete. 2024-09-12T01:46:29.883 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:29 smithi122 ceph-mon[30925]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T01:46:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:29 smithi183 ceph-mon[35127]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T01:46:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 -- 172.21.15.122:0/2575477077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 msgr2=0x7f8104106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 --2- 172.21.15.122:0/2575477077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f8104106840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009900 tx=0x7f80f802a410 comp rx=0 tx=0).stop 2024-09-12T01:46:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 -- 172.21.15.122:0/2575477077 shutdown_connections 2024-09-12T01:46:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 --2- 172.21.15.122:0/2575477077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8104106f70 0x7f810410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 --2- 172.21.15.122:0/2575477077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f8104106840 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 --2- 172.21.15.122:0/2575477077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104100500 0x7f81041008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 -- 172.21.15.122:0/2575477077 >> 172.21.15.122:0/2575477077 conn(0x7f81040fc5c0 msgr2=0x7f81040fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 -- 172.21.15.122:0/2575477077 shutdown_connections 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.961+0000 7f8109818640 1 -- 172.21.15.122:0/2575477077 wait complete. 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 Processor -- start 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 -- start start 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8104100500 0x7f8104198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f81041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104106f70 0x7f810419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8104111f50 con 0x7f8104100eb0 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8104111dd0 con 0x7f8104100500 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f8109818640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81041120d0 con 0x7f8104106f70 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f81027fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f81041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f81027fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f81041989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39652/0 (socket says 172.21.15.122:39652) 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.962+0000 7f81027fc640 1 -- 172.21.15.122:0/1733841430 learned_addr learned my addr 172.21.15.122:0/1733841430 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81037fe640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104106f70 0x7f810419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f8102ffd640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8104100500 0x7f8104198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81027fc640 1 -- 172.21.15.122:0/1733841430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104106f70 msgr2=0x7f810419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81027fc640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104106f70 0x7f810419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81027fc640 1 -- 172.21.15.122:0/1733841430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8104100500 msgr2=0x7f8104198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81027fc640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8104100500 0x7f8104198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81027fc640 1 -- 172.21.15.122:0/1733841430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419d440 con 0x7f8104100eb0 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f81027fc640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f81041989d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005940 tx=0x7f80f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f8108816640 1 -- 172.21.15.122:0/1733841430 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f803f030 con 0x7f8104100eb0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419d6d0 con 0x7f8104100eb0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f8108816640 1 -- 172.21.15.122:0/1733841430 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f8003ce0 con 0x7f8104100eb0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f8108816640 1 -- 172.21.15.122:0/1733841430 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8035070 con 0x7f8104100eb0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.963+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a5010 con 0x7f8104100eb0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.965+0000 7f8108816640 1 -- 172.21.15.122:0/1733841430 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f80f8042030 con 0x7f8104100eb0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 Processor -- start 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 -- start start 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8108816640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566e4680 0x557f566e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f5663f2e0 con 0x557f566d94a0 2024-09-12T01:46:29.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557f565e6ce0 con 0x557f566d9dc0 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f810bbf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f5663b2a0 con 0x557f566e4680 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f81037fe640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566d9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39654/0 (socket says 172.21.15.122:39654) 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 -- 172.21.15.122:0/234308107 learned_addr learned my addr 172.21.15.122:0/234308107 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8108816640 1 -- 172.21.15.122:0/1733841430 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f80f80ef7d0 con 0x7f8104100eb0 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f81027fc640 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 -- 172.21.15.122:0/234308107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566e4680 msgr2=0x557f566e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566e4680 0x557f566e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 -- 172.21.15.122:0/234308107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 msgr2=0x557f566e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f8102ffd640 1 -- 172.21.15.122:0/234308107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f565cb440 con 0x557f566d94a0 2024-09-12T01:46:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.966+0000 7f81027fc640 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f81037fe640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0092f0 tx=0x7f80fc0097f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f8102ffd640 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566d9880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f80f400cbb0 tx=0x7f80f400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b5a0870fa6d66f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f80e9ffb640 1 -- 172.21.15.122:0/234308107 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4005220 con 0x557f566d94a0 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f80e9ffb640 1 -- 172.21.15.122:0/234308107 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4004c80 con 0x557f566d94a0 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f80e9ffb640 1 -- 172.21.15.122:0/234308107 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40107d0 con 0x557f566d94a0 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 -- 172.21.15.122:0/234308107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 msgr2=0x557f566d9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566d9880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f80f400cbb0 tx=0x7f80f400e720 comp rx=0 tx=0).stop 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 -- 172.21.15.122:0/234308107 shutdown_connections 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566e4680 0x557f566e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 --2- 172.21.15.122:0/234308107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566d9880 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 -- 172.21.15.122:0/234308107 >> 172.21.15.122:0/234308107 conn(0x557f566d5a30 msgr2=0x557f566d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 -- 172.21.15.122:0/234308107 shutdown_connections 2024-09-12T01:46:29.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.967+0000 7f810bbf1740 1 -- 172.21.15.122:0/234308107 wait complete. 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 -- start start 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 Processor -- start 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d94a0 0x557f566e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 0x557f566e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f566e7250 con 0x557f566e4680 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557f565fa050 con 0x557f566d9dc0 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f566e73d0 con 0x557f566d94a0 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f810bbf1740 10 client.0 _ll_register_callbacks cb 0x557f5663a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.968+0000 7f81037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 0x557f566e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 0x557f566e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39656/0 (socket says 172.21.15.122:39656) 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 -- 172.21.15.122:0/2371184469 learned_addr learned my addr 172.21.15.122:0/2371184469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f8102ffd640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d94a0 0x557f566e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81027fc640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 -- 172.21.15.122:0/2371184469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d94a0 msgr2=0x557f566e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d94a0 0x557f566e4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 -- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 msgr2=0x557f566e12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 -- 172.21.15.122:0/2371184469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f563b03c0 con 0x557f566e4680 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81027fc640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f81037fe640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 0x557f566e1830 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0092c0 tx=0x7f80fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afd763840c4dc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f80e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f80e2ffd640 1 -- 172.21.15.122:0/2371184469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00fcf0 con 0x557f566e4680 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f80e2ffd640 1 -- 172.21.15.122:0/2371184469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f80fc020450 con 0x557f566e4680 2024-09-12T01:46:29.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f4004c80 con 0x557f566e4680 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f80e2ffd640 1 -- 172.21.15.122:0/2371184469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f80fc020650 con 0x557f566e4680 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f810bbf1740 10 client.15249 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.969+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f566f2ca0 con 0x557f566e4680 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f80e2ffd640 1 -- 172.21.15.122:0/2371184469 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4004c80 con 0x557f566e4680 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f80e2ffd640 1 -- 172.21.15.122:0/2371184469 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f80fc00f040 con 0x557f566e4680 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f810bbf1740 10 client.15249 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f810bbf1740 10 client.15249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f810bbf1740 20 client.15249 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f566f4160 0x557f566f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f810bbf1740 4 client.15249 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f5664c850 con 0x557f566f4160 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.970+0000 7f8102ffd640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f566f4160 0x557f566f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.971+0000 7f80e2ffd640 10 client.15249 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.971+0000 7f8102ffd640 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f566f4160 0x557f566f65d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b4304dd2d0418c server_cookie=e6e5f68b6ba665ce in_seq=0 out_seq=0 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.971+0000 7f80e2ffd640 1 -- 172.21.15.122:0/2371184469 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f5664c850 con 0x557f566f4160 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.971+0000 7f80e2ffd640 10 client.15249 handle_command_reply: tid=0 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.971+0000 7f810bbf1740 1 client.15249 shutdown 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.971+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 msgr2=0x557f566e1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 0x557f566e1830 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0092c0 tx=0x7f80fc004ab0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 shutdown_connections 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f566f4160 0x557f566f65d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566e4680 0x557f566e1830 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566d9dc0 0x557f566e12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2371184469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d94a0 0x557f566e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f80e2ffd640 0 client.15249 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 >> 172.21.15.122:0/2371184469 conn(0x557f566d5a30 msgr2=0x557f566db760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:29.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 shutdown_connections 2024-09-12T01:46:29.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 1 -- 172.21.15.122:0/2371184469 wait complete. 2024-09-12T01:46:29.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.972+0000 7f810bbf1740 20 client.15249 trim_cache size 0 max 16384 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 Processor -- start 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 -- start start 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81041afed0 con 0x557f566d94a0 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ad7a0 con 0x557f566e4680 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f810bbf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ab5d0 con 0x557f566d9dc0 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f8102ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f8102ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566f04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39660/0 (socket says 172.21.15.122:39660) 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f8102ffd640 1 -- 172.21.15.122:0/2607905757 learned_addr learned my addr 172.21.15.122:0/2607905757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:29.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.989+0000 7f81037fe640 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f81027fc640 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f8102ffd640 1 -- 172.21.15.122:0/2607905757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 msgr2=0x557f566fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f8102ffd640 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f8102ffd640 1 -- 172.21.15.122:0/2607905757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 msgr2=0x557f566fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f8102ffd640 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f8102ffd640 1 -- 172.21.15.122:0/2607905757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f566d5860 con 0x557f566d94a0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f81027fc640 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f8102ffd640 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566f04b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80f4000c00 tx=0x7f80f400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234c1691a38ba210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f80e97fa640 1 -- 172.21.15.122:0/2607905757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4004480 con 0x557f566d94a0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f80e97fa640 1 -- 172.21.15.122:0/2607905757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f402c070 con 0x557f566d94a0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.990+0000 7f80e97fa640 1 -- 172.21.15.122:0/2607905757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40107b0 con 0x557f566d94a0 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 -- 172.21.15.122:0/2607905757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 msgr2=0x557f566f04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:29.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566f04b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80f4000c00 tx=0x7f80f400c4d0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 -- 172.21.15.122:0/2607905757 shutdown_connections 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 --2- 172.21.15.122:0/2607905757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f566d94a0 0x557f566f04b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 -- 172.21.15.122:0/2607905757 >> 172.21.15.122:0/2607905757 conn(0x557f566d5a30 msgr2=0x557f566dfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 -- 172.21.15.122:0/2607905757 shutdown_connections 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 -- 172.21.15.122:0/2607905757 wait complete. 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 -- start start 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.991+0000 7f810bbf1740 1 Processor -- start 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f56757e90 0x557f56758310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81041abc60 con 0x557f56757e90 2024-09-12T01:46:29.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ad0d0 con 0x557f566e4680 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ac330 con 0x557f566d9dc0 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f810bbf1740 10 client.0 _ll_register_callbacks cb 0x7f80fc009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f56757e90 0x557f56758310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f8102ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f8102ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566e8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:38004/0 (socket says 172.21.15.122:38004) 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f8102ffd640 1 -- 172.21.15.122:0/940999841 learned_addr learned my addr 172.21.15.122:0/940999841 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81027fc640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 -- 172.21.15.122:0/940999841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 msgr2=0x557f566e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 -- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 msgr2=0x557f566e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566e9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 -- 172.21.15.122:0/940999841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f56758880 con 0x557f56757e90 2024-09-12T01:46:29.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f8102ffd640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81027fc640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f81037fe640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f56757e90 0x557f56758310 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80fc029980 tx=0x7f80fc029e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5897bc76414e7f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f80e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.992+0000 7f80e2ffd640 1 -- 172.21.15.122:0/940999841 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00f730 con 0x557f56757e90 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f80e2ffd640 1 -- 172.21.15.122:0/940999841 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f80fc029650 con 0x557f56757e90 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f402c070 con 0x557f56757e90 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f80e2ffd640 1 -- 172.21.15.122:0/940999841 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f80fc00faf0 con 0x557f56757e90 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f5675fb60 con 0x557f56757e90 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f80e2ffd640 1 -- 172.21.15.122:0/940999841 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f402c070 con 0x557f56757e90 2024-09-12T01:46:29.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f80e2ffd640 1 -- 172.21.15.122:0/940999841 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f80fc03e4d0 con 0x557f56757e90 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 10 client.15255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 20 client.15255 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f56760300 0x557f56762720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 4 client.15255 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.993+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f56646d60 con 0x557f56760300 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f8102ffd640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f56760300 0x557f56762720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f80e2ffd640 10 client.15255 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:29.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f8102ffd640 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f56760300 0x557f56762720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d3ec446d2c1ef3 server_cookie=ab63a6958912ba3 in_seq=0 out_seq=0 2024-09-12T01:46:30.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f80e2ffd640 1 -- 172.21.15.122:0/940999841 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557f56646d60 con 0x557f56760300 2024-09-12T01:46:30.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f80e2ffd640 10 client.15255 handle_command_reply: tid=0 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 client.15255 shutdown 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f56757e90 msgr2=0x557f56758310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f56757e90 0x557f56758310 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80fc029980 tx=0x7f80fc029e20 comp rx=0 tx=0).stop 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 shutdown_connections 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f56760300 0x557f56762720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f56757e90 0x557f56758310 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f566e4680 0x557f566e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f80e2ffd640 0 client.15255 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 --2- 172.21.15.122:0/940999841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f566d9dc0 0x557f566e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.994+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 >> 172.21.15.122:0/940999841 conn(0x557f566d5a30 msgr2=0x557f566e2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 shutdown_connections 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f810bbf1740 1 -- 172.21.15.122:0/940999841 wait complete. 2024-09-12T01:46:30.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f810bbf1740 20 client.15255 trim_cache size 0 max 16384 2024-09-12T01:46:30.002 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:30.002 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:46:30.002 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-09-12T01:46:30.002 INFO:teuthology.orchestra.run.smithi122.stdout: "d7526da0-2c47-4bcf-96e9-8166d6e87fb2": { 2024-09-12T01:46:30.003 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-09-12T01:46:30.003 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "d7526da0-2c47-4bcf-96e9-8166d6e87fb2", 2024-09-12T01:46:30.003 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-09-12T01:46:30.003 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:46:30.003 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:46:30.003 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:30.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80ec0a2c10 msgr2=0x7f80ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0092f0 tx=0x7f80fc0097f0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 msgr2=0x7f81041989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f81041989d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005940 tx=0x7f80f80042f0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 shutdown_connections 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80ec0a2c10 0x7f80ec0a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104106f70 0x7f810419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8104100eb0 0x7f81041989d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 --2- 172.21.15.122:0/1733841430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8104100500 0x7f8104198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:30.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 >> 172.21.15.122:0/1733841430 conn(0x7f81040fc5c0 msgr2=0x7f81040fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:30.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 shutdown_connections 2024-09-12T01:46:30.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:29.995+0000 7f8109818640 1 -- 172.21.15.122:0/1733841430 wait complete. 2024-09-12T01:46:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:29 smithi151 ceph-mon[35229]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T01:46:30.788 INFO:tasks.cephfs.filesystem:scrub status for tag:d7526da0-2c47-4bcf-96e9-8166d6e87fb2 - {'path': '/', 'tag': 'd7526da0-2c47-4bcf-96e9-8166d6e87fb2', 'options': 'recursive,force'} 2024-09-12T01:46:30.939 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:30 smithi122 ceph-mon[30925]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:30.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:30 smithi183 ceph-mon[35127]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:30 smithi151 ceph-mon[35229]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:31.366 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-12T01:46:28.471900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665734,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":751,"num_read_kb":500,"num_write":362,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360324,"kb_used_data":11076,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581500,"statfs":{"total":1151940427776,"available":1151571456000,"internally_reserved":0,"allocated":11341824,"data_stored":7243758,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9128,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":42,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002918"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472514+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:46:14.472514+0000","last_peered":"2024-09-12T01:46:14.472514+0000","last_clean":"2024-09-12T01:46:14.472514+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:46:14.472514+0000","last_undegraded":"2024-09-12T01:46:14.472514+0000","last_fullsized":"2024-09-12T01:46:14.472514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.451157+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:46:14.451157+0000","last_peered":"2024-09-12T01:46:14.451157+0000","last_clean":"2024-09-12T01:46:14.451157+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:46:14.451157+0000","last_undegraded":"2024-09-12T01:46:14.451157+0000","last_fullsized":"2024-09-12T01:46:14.451157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462071+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:46:14.462071+0000","last_peered":"2024-09-12T01:46:14.462071+0000","last_clean":"2024-09-12T01:46:14.462071+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:46:14.462071+0000","last_undegraded":"2024-09-12T01:46:14.462071+0000","last_fullsized":"2024-09-12T01:46:14.462071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462225+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:46:14.462225+0000","last_peered":"2024-09-12T01:46:14.462225+0000","last_clean":"2024-09-12T01:46:14.462225+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:46:14.462225+0000","last_undegraded":"2024-09-12T01:46:14.462225+0000","last_fullsized":"2024-09-12T01:46:14.462225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456436+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:46:14.456436+0000","last_peered":"2024-09-12T01:46:14.456436+0000","last_clean":"2024-09-12T01:46:14.456436+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:46:14.456436+0000","last_undegraded":"2024-09-12T01:46:14.456436+0000","last_fullsized":"2024-09-12T01:46:14.456436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470354+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:46:14.470354+0000","last_peered":"2024-09-12T01:46:14.470354+0000","last_clean":"2024-09-12T01:46:14.470354+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:46:14.470354+0000","last_undegraded":"2024-09-12T01:46:14.470354+0000","last_fullsized":"2024-09-12T01:46:14.470354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468414+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:46:14.468414+0000","last_peered":"2024-09-12T01:46:14.468414+0000","last_clean":"2024-09-12T01:46:14.468414+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:46:14.468414+0000","last_undegraded":"2024-09-12T01:46:14.468414+0000","last_fullsized":"2024-09-12T01:46:14.468414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470863+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:46:14.470863+0000","last_peered":"2024-09-12T01:46:14.470863+0000","last_clean":"2024-09-12T01:46:14.470863+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:46:14.470863+0000","last_undegraded":"2024-09-12T01:46:14.470863+0000","last_fullsized":"2024-09-12T01:46:14.470863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467613+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:46:14.467613+0000","last_peered":"2024-09-12T01:46:14.467613+0000","last_clean":"2024-09-12T01:46:14.467613+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:46:14.467613+0000","last_undegraded":"2024-09-12T01:46:14.467613+0000","last_fullsized":"2024-09-12T01:46:14.467613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471418+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:46:14.471418+0000","last_peered":"2024-09-12T01:46:14.471418+0000","last_clean":"2024-09-12T01:46:14.471418+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:46:14.471418+0000","last_undegraded":"2024-09-12T01:46:14.471418+0000","last_fullsized":"2024-09-12T01:46:14.471418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473581+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:46:14.473581+0000","last_peered":"2024-09-12T01:46:14.473581+0000","last_clean":"2024-09-12T01:46:14.473581+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:46:14.473581+0000","last_undegraded":"2024-09-12T01:46:14.473581+0000","last_fullsized":"2024-09-12T01:46:14.473581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466586+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:46:14.466586+0000","last_peered":"2024-09-12T01:46:14.466586+0000","last_clean":"2024-09-12T01:46:14.466586+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:46:14.466586+0000","last_undegraded":"2024-09-12T01:46:14.466586+0000","last_fullsized":"2024-09-12T01:46:14.466586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470765+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:46:14.470765+0000","last_peered":"2024-09-12T01:46:14.470765+0000","last_clean":"2024-09-12T01:46:14.470765+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:46:14.470765+0000","last_undegraded":"2024-09-12T01:46:14.470765+0000","last_fullsized":"2024-09-12T01:46:14.470765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456811+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:46:14.456811+0000","last_peered":"2024-09-12T01:46:14.456811+0000","last_clean":"2024-09-12T01:46:14.456811+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:46:14.456811+0000","last_undegraded":"2024-09-12T01:46:14.456811+0000","last_fullsized":"2024-09-12T01:46:14.456811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462152+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:46:14.462152+0000","last_peered":"2024-09-12T01:46:14.462152+0000","last_clean":"2024-09-12T01:46:14.462152+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:46:14.462152+0000","last_undegraded":"2024-09-12T01:46:14.462152+0000","last_fullsized":"2024-09-12T01:46:14.462152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470248+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:46:14.470248+0000","last_peered":"2024-09-12T01:46:14.470248+0000","last_clean":"2024-09-12T01:46:14.470248+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:46:14.470248+0000","last_undegraded":"2024-09-12T01:46:14.470248+0000","last_fullsized":"2024-09-12T01:46:14.470248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472627+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:46:14.472627+0000","last_peered":"2024-09-12T01:46:14.472627+0000","last_clean":"2024-09-12T01:46:14.472627+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:46:14.472627+0000","last_undegraded":"2024-09-12T01:46:14.472627+0000","last_fullsized":"2024-09-12T01:46:14.472627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466306+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:46:14.466306+0000","last_peered":"2024-09-12T01:46:14.466306+0000","last_clean":"2024-09-12T01:46:14.466306+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:46:14.466306+0000","last_undegraded":"2024-09-12T01:46:14.466306+0000","last_fullsized":"2024-09-12T01:46:14.466306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457876+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:46:14.457876+0000","last_peered":"2024-09-12T01:46:14.457876+0000","last_clean":"2024-09-12T01:46:14.457876+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:46:14.457876+0000","last_undegraded":"2024-09-12T01:46:14.457876+0000","last_fullsized":"2024-09-12T01:46:14.457876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461219+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:46:14.461219+0000","last_peered":"2024-09-12T01:46:14.461219+0000","last_clean":"2024-09-12T01:46:14.461219+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:46:14.461219+0000","last_undegraded":"2024-09-12T01:46:14.461219+0000","last_fullsized":"2024-09-12T01:46:14.461219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469829+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:46:14.469829+0000","last_peered":"2024-09-12T01:46:14.469829+0000","last_clean":"2024-09-12T01:46:14.469829+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:46:14.469829+0000","last_undegraded":"2024-09-12T01:46:14.469829+0000","last_fullsized":"2024-09-12T01:46:14.469829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471133+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:46:14.471133+0000","last_peered":"2024-09-12T01:46:14.471133+0000","last_clean":"2024-09-12T01:46:14.471133+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:46:14.471133+0000","last_undegraded":"2024-09-12T01:46:14.471133+0000","last_fullsized":"2024-09-12T01:46:14.471133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460490+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:46:14.460490+0000","last_peered":"2024-09-12T01:46:14.460490+0000","last_clean":"2024-09-12T01:46:14.460490+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:46:14.460490+0000","last_undegraded":"2024-09-12T01:46:14.460490+0000","last_fullsized":"2024-09-12T01:46:14.460490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.455176+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:46:14.455176+0000","last_peered":"2024-09-12T01:46:14.455176+0000","last_clean":"2024-09-12T01:46:14.455176+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:46:14.455176+0000","last_undegraded":"2024-09-12T01:46:14.455176+0000","last_fullsized":"2024-09-12T01:46:14.455176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449237+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:46:14.449237+0000","last_peered":"2024-09-12T01:46:14.449237+0000","last_clean":"2024-09-12T01:46:14.449237+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:46:14.449237+0000","last_undegraded":"2024-09-12T01:46:14.449237+0000","last_fullsized":"2024-09-12T01:46:14.449237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459600+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:46:14.459600+0000","last_peered":"2024-09-12T01:46:14.459600+0000","last_clean":"2024-09-12T01:46:14.459600+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:46:14.459600+0000","last_undegraded":"2024-09-12T01:46:14.459600+0000","last_fullsized":"2024-09-12T01:46:14.459600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456999+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:46:14.456999+0000","last_peered":"2024-09-12T01:46:14.456999+0000","last_clean":"2024-09-12T01:46:14.456999+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:46:14.456999+0000","last_undegraded":"2024-09-12T01:46:14.456999+0000","last_fullsized":"2024-09-12T01:46:14.456999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.453782+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:46:14.453782+0000","last_peered":"2024-09-12T01:46:14.453782+0000","last_clean":"2024-09-12T01:46:14.453782+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:46:14.453782+0000","last_undegraded":"2024-09-12T01:46:14.453782+0000","last_fullsized":"2024-09-12T01:46:14.453782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457999+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:46:14.457999+0000","last_peered":"2024-09-12T01:46:14.457999+0000","last_clean":"2024-09-12T01:46:14.457999+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:46:14.457999+0000","last_undegraded":"2024-09-12T01:46:14.457999+0000","last_fullsized":"2024-09-12T01:46:14.457999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460598+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:46:14.460598+0000","last_peered":"2024-09-12T01:46:14.460598+0000","last_clean":"2024-09-12T01:46:14.460598+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:46:14.460598+0000","last_undegraded":"2024-09-12T01:46:14.460598+0000","last_fullsized":"2024-09-12T01:46:14.460598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463452+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:46:14.463452+0000","last_peered":"2024-09-12T01:46:14.463452+0000","last_clean":"2024-09-12T01:46:14.463452+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:46:14.463452+0000","last_undegraded":"2024-09-12T01:46:14.463452+0000","last_fullsized":"2024-09-12T01:46:14.463452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463643+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:46:14.463643+0000","last_peered":"2024-09-12T01:46:14.463643+0000","last_clean":"2024-09-12T01:46:14.463643+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:46:14.463643+0000","last_undegraded":"2024-09-12T01:46:14.463643+0000","last_fullsized":"2024-09-12T01:46:14.463643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457236+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:46:14.457236+0000","last_peered":"2024-09-12T01:46:14.457236+0000","last_clean":"2024-09-12T01:46:14.457236+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:46:14.457236+0000","last_undegraded":"2024-09-12T01:46:14.457236+0000","last_fullsized":"2024-09-12T01:46:14.457236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:21.973123+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:46:21.973123+0000","last_peered":"2024-09-12T01:46:21.973123+0000","last_clean":"2024-09-12T01:46:21.973123+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:46:21.973123+0000","last_undegraded":"2024-09-12T01:46:21.973123+0000","last_fullsized":"2024-09-12T01:46:21.973123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461733+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:46:14.461733+0000","last_peered":"2024-09-12T01:46:14.461733+0000","last_clean":"2024-09-12T01:46:14.461733+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:46:14.461733+0000","last_undegraded":"2024-09-12T01:46:14.461733+0000","last_fullsized":"2024-09-12T01:46:14.461733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.452061+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:46:14.452061+0000","last_peered":"2024-09-12T01:46:14.452061+0000","last_clean":"2024-09-12T01:46:14.452061+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:46:14.452061+0000","last_undegraded":"2024-09-12T01:46:14.452061+0000","last_fullsized":"2024-09-12T01:46:14.452061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466729+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:46:14.466729+0000","last_peered":"2024-09-12T01:46:14.466729+0000","last_clean":"2024-09-12T01:46:14.466729+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:46:14.466729+0000","last_undegraded":"2024-09-12T01:46:14.466729+0000","last_fullsized":"2024-09-12T01:46:14.466729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:23.975221+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:46:23.975221+0000","last_peered":"2024-09-12T01:46:23.975221+0000","last_clean":"2024-09-12T01:46:23.975221+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:46:23.975221+0000","last_undegraded":"2024-09-12T01:46:23.975221+0000","last_fullsized":"2024-09-12T01:46:23.975221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473143+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:46:14.473143+0000","last_peered":"2024-09-12T01:46:14.473143+0000","last_clean":"2024-09-12T01:46:14.473143+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:46:14.473143+0000","last_undegraded":"2024-09-12T01:46:14.473143+0000","last_fullsized":"2024-09-12T01:46:14.473143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460971+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:46:14.460971+0000","last_peered":"2024-09-12T01:46:14.460971+0000","last_clean":"2024-09-12T01:46:14.460971+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:46:14.460971+0000","last_undegraded":"2024-09-12T01:46:14.460971+0000","last_fullsized":"2024-09-12T01:46:14.460971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473209+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:46:14.473209+0000","last_peered":"2024-09-12T01:46:14.473209+0000","last_clean":"2024-09-12T01:46:14.473209+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:46:14.473209+0000","last_undegraded":"2024-09-12T01:46:14.473209+0000","last_fullsized":"2024-09-12T01:46:14.473209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":267,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:24.977041+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:46:24.977041+0000","last_peered":"2024-09-12T01:46:24.977041+0000","last_clean":"2024-09-12T01:46:24.977041+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:46:24.977041+0000","last_undegraded":"2024-09-12T01:46:24.977041+0000","last_fullsized":"2024-09-12T01:46:24.977041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":210,"num_read_kb":174,"num_write":36,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457287+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:46:14.457287+0000","last_peered":"2024-09-12T01:46:14.457287+0000","last_clean":"2024-09-12T01:46:14.457287+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:46:14.457287+0000","last_undegraded":"2024-09-12T01:46:14.457287+0000","last_fullsized":"2024-09-12T01:46:14.457287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470461+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:46:14.470461+0000","last_peered":"2024-09-12T01:46:14.470461+0000","last_clean":"2024-09-12T01:46:14.470461+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:46:14.470461+0000","last_undegraded":"2024-09-12T01:46:14.470461+0000","last_fullsized":"2024-09-12T01:46:14.470461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472069+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:46:14.472069+0000","last_peered":"2024-09-12T01:46:14.472069+0000","last_clean":"2024-09-12T01:46:14.472069+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:46:14.472069+0000","last_undegraded":"2024-09-12T01:46:14.472069+0000","last_fullsized":"2024-09-12T01:46:14.472069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:25.978847+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:46:25.978847+0000","last_peered":"2024-09-12T01:46:25.978847+0000","last_clean":"2024-09-12T01:46:25.978847+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:46:25.978847+0000","last_undegraded":"2024-09-12T01:46:25.978847+0000","last_fullsized":"2024-09-12T01:46:25.978847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468634+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:46:14.468634+0000","last_peered":"2024-09-12T01:46:14.468634+0000","last_clean":"2024-09-12T01:46:14.468634+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:46:14.468634+0000","last_undegraded":"2024-09-12T01:46:14.468634+0000","last_fullsized":"2024-09-12T01:46:14.468634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469018+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:46:14.469018+0000","last_peered":"2024-09-12T01:46:14.469018+0000","last_clean":"2024-09-12T01:46:14.469018+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:46:14.469018+0000","last_undegraded":"2024-09-12T01:46:14.469018+0000","last_fullsized":"2024-09-12T01:46:14.469018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:47.816560+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:47.816560+0000","last_peered":"2024-09-12T01:44:47.816560+0000","last_clean":"2024-09-12T01:44:47.816560+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:47.816560+0000","last_undegraded":"2024-09-12T01:44:47.816560+0000","last_fullsized":"2024-09-12T01:44:47.816560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.458089+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:46:14.458089+0000","last_peered":"2024-09-12T01:46:14.458089+0000","last_clean":"2024-09-12T01:46:14.458089+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:46:14.458089+0000","last_undegraded":"2024-09-12T01:46:14.458089+0000","last_fullsized":"2024-09-12T01:46:14.458089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467837+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:46:14.467837+0000","last_peered":"2024-09-12T01:46:14.467837+0000","last_clean":"2024-09-12T01:46:14.467837+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:46:14.467837+0000","last_undegraded":"2024-09-12T01:46:14.467837+0000","last_fullsized":"2024-09-12T01:46:14.467837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465783+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:46:14.465783+0000","last_peered":"2024-09-12T01:46:14.465783+0000","last_clean":"2024-09-12T01:46:14.465783+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:46:14.465783+0000","last_undegraded":"2024-09-12T01:46:14.465783+0000","last_fullsized":"2024-09-12T01:46:14.465783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473700+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:46:14.473700+0000","last_peered":"2024-09-12T01:46:14.473700+0000","last_clean":"2024-09-12T01:46:14.473700+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:46:14.473700+0000","last_undegraded":"2024-09-12T01:46:14.473700+0000","last_fullsized":"2024-09-12T01:46:14.473700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818582+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818582+0000","last_peered":"2024-09-12T01:44:45.818582+0000","last_clean":"2024-09-12T01:44:45.818582+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818582+0000","last_undegraded":"2024-09-12T01:44:45.818582+0000","last_fullsized":"2024-09-12T01:44:45.818582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.464071+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:46:14.464071+0000","last_peered":"2024-09-12T01:46:14.464071+0000","last_clean":"2024-09-12T01:46:14.464071+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:46:14.464071+0000","last_undegraded":"2024-09-12T01:46:14.464071+0000","last_fullsized":"2024-09-12T01:46:14.464071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449016+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:46:14.449016+0000","last_peered":"2024-09-12T01:46:14.449016+0000","last_clean":"2024-09-12T01:46:14.449016+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:46:14.449016+0000","last_undegraded":"2024-09-12T01:46:14.449016+0000","last_fullsized":"2024-09-12T01:46:14.449016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:21.973563+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:46:21.973563+0000","last_peered":"2024-09-12T01:46:21.973563+0000","last_clean":"2024-09-12T01:46:21.973563+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:46:21.973563+0000","last_undegraded":"2024-09-12T01:46:21.973563+0000","last_fullsized":"2024-09-12T01:46:21.973563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468646+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:46:14.468646+0000","last_peered":"2024-09-12T01:46:14.468646+0000","last_clean":"2024-09-12T01:46:14.468646+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:46:14.468646+0000","last_undegraded":"2024-09-12T01:46:14.468646+0000","last_fullsized":"2024-09-12T01:46:14.468646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.450592+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:46:14.450592+0000","last_peered":"2024-09-12T01:46:14.450592+0000","last_clean":"2024-09-12T01:46:14.450592+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:46:14.450592+0000","last_undegraded":"2024-09-12T01:46:14.450592+0000","last_fullsized":"2024-09-12T01:46:14.450592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468594+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:46:14.468594+0000","last_peered":"2024-09-12T01:46:14.468594+0000","last_clean":"2024-09-12T01:46:14.468594+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:46:14.468594+0000","last_undegraded":"2024-09-12T01:46:14.468594+0000","last_fullsized":"2024-09-12T01:46:14.468594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449242+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:46:14.449242+0000","last_peered":"2024-09-12T01:46:14.449242+0000","last_clean":"2024-09-12T01:46:14.449242+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:46:14.449242+0000","last_undegraded":"2024-09-12T01:46:14.449242+0000","last_fullsized":"2024-09-12T01:46:14.449242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:18.793448+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:46:18.793448+0000","last_peered":"2024-09-12T01:46:18.793448+0000","last_clean":"2024-09-12T01:46:18.793448+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:46:18.793448+0000","last_undegraded":"2024-09-12T01:46:18.793448+0000","last_fullsized":"2024-09-12T01:46:18.793448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465155+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:46:14.465155+0000","last_peered":"2024-09-12T01:46:14.465155+0000","last_clean":"2024-09-12T01:46:14.465155+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:46:14.465155+0000","last_undegraded":"2024-09-12T01:46:14.465155+0000","last_fullsized":"2024-09-12T01:46:14.465155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457998+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:46:14.457998+0000","last_peered":"2024-09-12T01:46:14.457998+0000","last_clean":"2024-09-12T01:46:14.457998+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:46:14.457998+0000","last_undegraded":"2024-09-12T01:46:14.457998+0000","last_fullsized":"2024-09-12T01:46:14.457998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.440457+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:46:14.440457+0000","last_peered":"2024-09-12T01:46:14.440457+0000","last_clean":"2024-09-12T01:46:14.440457+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:46:14.440457+0000","last_undegraded":"2024-09-12T01:46:14.440457+0000","last_fullsized":"2024-09-12T01:46:14.440457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463449+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:46:14.463449+0000","last_peered":"2024-09-12T01:46:14.463449+0000","last_clean":"2024-09-12T01:46:14.463449+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:46:14.463449+0000","last_undegraded":"2024-09-12T01:46:14.463449+0000","last_fullsized":"2024-09-12T01:46:14.463449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470639+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:46:14.470639+0000","last_peered":"2024-09-12T01:46:14.470639+0000","last_clean":"2024-09-12T01:46:14.470639+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:46:14.470639+0000","last_undegraded":"2024-09-12T01:46:14.470639+0000","last_fullsized":"2024-09-12T01:46:14.470639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473199+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:46:14.473199+0000","last_peered":"2024-09-12T01:46:14.473199+0000","last_clean":"2024-09-12T01:46:14.473199+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:46:14.473199+0000","last_undegraded":"2024-09-12T01:46:14.473199+0000","last_fullsized":"2024-09-12T01:46:14.473199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469618+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:46:14.469618+0000","last_peered":"2024-09-12T01:46:14.469618+0000","last_clean":"2024-09-12T01:46:14.469618+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:46:14.469618+0000","last_undegraded":"2024-09-12T01:46:14.469618+0000","last_fullsized":"2024-09-12T01:46:14.469618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473089+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:46:14.473089+0000","last_peered":"2024-09-12T01:46:14.473089+0000","last_clean":"2024-09-12T01:46:14.473089+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:46:14.473089+0000","last_undegraded":"2024-09-12T01:46:14.473089+0000","last_fullsized":"2024-09-12T01:46:14.473089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.454649+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:46:14.454649+0000","last_peered":"2024-09-12T01:46:14.454649+0000","last_clean":"2024-09-12T01:46:14.454649+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:46:14.454649+0000","last_undegraded":"2024-09-12T01:46:14.454649+0000","last_fullsized":"2024-09-12T01:46:14.454649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459700+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:46:14.459700+0000","last_peered":"2024-09-12T01:46:14.459700+0000","last_clean":"2024-09-12T01:46:14.459700+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:46:14.459700+0000","last_undegraded":"2024-09-12T01:46:14.459700+0000","last_fullsized":"2024-09-12T01:46:14.459700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816629+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:45.816629+0000","last_peered":"2024-09-12T01:44:45.816629+0000","last_clean":"2024-09-12T01:44:45.816629+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:45.816629+0000","last_undegraded":"2024-09-12T01:44:45.816629+0000","last_fullsized":"2024-09-12T01:44:45.816629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75132,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":625,"num_read_kb":391,"num_write":179,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":225396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1100798,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1030417,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561084,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438784,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":509124,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":507808,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:46:31.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:46:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:31.889 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:31 smithi122 ceph-mon[30925]: pgmap v336: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T01:46:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:31 smithi183 ceph-mon[35127]: pgmap v336: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T01:46:32.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 -- 172.21.15.122:0/3625620431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0101de0 msgr2=0x7ff4f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:32.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/3625620431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0101de0 0x7ff4f01021c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009900 tx=0x7ff4ec02a410 comp rx=0 tx=0).stop 2024-09-12T01:46:32.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 -- 172.21.15.122:0/3625620431 shutdown_connections 2024-09-12T01:46:32.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/3625620431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f0106f10 0x7ff4f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/3625620431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0102790 0x7ff4f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/3625620431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0101de0 0x7ff4f01021c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 -- 172.21.15.122:0/3625620431 >> 172.21.15.122:0/3625620431 conn(0x7ff4f00fd8c0 msgr2=0x7ff4f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:32.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 -- 172.21.15.122:0/3625620431 shutdown_connections 2024-09-12T01:46:32.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.100+0000 7ff4f7de0640 1 -- 172.21.15.122:0/3625620431 wait complete. 2024-09-12T01:46:32.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 Processor -- start 2024-09-12T01:46:32.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 -- start start 2024-09-12T01:46:32.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f0101de0 0x7ff4f0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:32.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 0x7ff4f01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:32.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0106f10 0x7ff4f019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:32.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0111f30 con 0x7ff4f0102790 2024-09-12T01:46:32.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0111db0 con 0x7ff4f0106f10 2024-09-12T01:46:32.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f7de0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01120b0 con 0x7ff4f0101de0 2024-09-12T01:46:32.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 0x7ff4f01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:32.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 0x7ff4f01988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52466/0 (socket says 172.21.15.122:52466) 2024-09-12T01:46:32.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 -- 172.21.15.122:0/4001736619 learned_addr learned my addr 172.21.15.122:0/4001736619 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:32.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f5b7c640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f0101de0 0x7ff4f0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f637d640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0106f10 0x7ff4f019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 -- 172.21.15.122:0/4001736619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f0101de0 msgr2=0x7ff4f0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f0101de0 0x7ff4f0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 -- 172.21.15.122:0/4001736619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0106f10 msgr2=0x7ff4f019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0106f10 0x7ff4f019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f537b640 1 -- 172.21.15.122:0/4001736619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f019d340 con 0x7ff4f0102790 2024-09-12T01:46:32.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.101+0000 7ff4f637d640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0106f10 0x7ff4f019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:32.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.102+0000 7ff4f537b640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 0x7ff4f01988d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c920 tx=0x7ff4e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:32.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.102+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0007c70 con 0x7ff4f0102790 2024-09-12T01:46:32.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.102+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e0007e10 con 0x7ff4f0102790 2024-09-12T01:46:32.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.102+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f019d630 con 0x7ff4f0102790 2024-09-12T01:46:32.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.102+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0018890 con 0x7ff4f0102790 2024-09-12T01:46:32.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.102+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01a4f10 con 0x7ff4f0102790 2024-09-12T01:46:32.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.103+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f01021c0 con 0x7ff4f0102790 2024-09-12T01:46:32.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.104+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7ff4e0018a50 con 0x7ff4f0102790 2024-09-12T01:46:32.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.105+0000 7ff4e6ffd640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4dc0a2c10 0x7ff4dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:32.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.105+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff4e00cf5b0 con 0x7ff4f0102790 2024-09-12T01:46:32.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.105+0000 7ff4f5b7c640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4dc0a2c10 0x7ff4dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:32.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.105+0000 7ff4f5b7c640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4dc0a2c10 0x7ff4dc0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009230 tx=0x7ff4ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:32.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.108+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4e0014030 con 0x7ff4f0102790 2024-09-12T01:46:32.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:31 smithi151 ceph-mon[35229]: pgmap v336: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T01:46:32.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.273+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4f0062630 con 0x7ff4dc0a2c10 2024-09-12T01:46:32.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.279+0000 7ff4e6ffd640 1 -- 172.21.15.122:0/4001736619 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259070 (secure 0 0 0) 0x7ff4f0062630 con 0x7ff4dc0a2c10 2024-09-12T01:46:32.282 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:46:32.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4dc0a2c10 msgr2=0x7ff4dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:32.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4dc0a2c10 0x7ff4dc0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009230 tx=0x7ff4ec0058d0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 msgr2=0x7ff4f01988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:32.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 0x7ff4f01988d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c920 tx=0x7ff4e000cde0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 shutdown_connections 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4dc0a2c10 0x7ff4dc0a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4f0106f10 0x7ff4f019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4f0102790 0x7ff4f01988d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 --2- 172.21.15.122:0/4001736619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f0101de0 0x7ff4f0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 >> 172.21.15.122:0/4001736619 conn(0x7ff4f00fd8c0 msgr2=0x7ff4f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 shutdown_connections 2024-09-12T01:46:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:32.282+0000 7ff4f7de0640 1 -- 172.21.15.122:0/4001736619 wait complete. 2024-09-12T01:46:33.050 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-12T01:46:30.472355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665734,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":762,"num_read_kb":506,"num_write":362,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360324,"kb_used_data":11076,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581500,"statfs":{"total":1151940427776,"available":1151571456000,"internally_reserved":0,"allocated":11341824,"data_stored":7243758,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":47,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002976"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472514+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:46:14.472514+0000","last_peered":"2024-09-12T01:46:14.472514+0000","last_clean":"2024-09-12T01:46:14.472514+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:46:14.472514+0000","last_undegraded":"2024-09-12T01:46:14.472514+0000","last_fullsized":"2024-09-12T01:46:14.472514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.451157+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:46:14.451157+0000","last_peered":"2024-09-12T01:46:14.451157+0000","last_clean":"2024-09-12T01:46:14.451157+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:46:14.451157+0000","last_undegraded":"2024-09-12T01:46:14.451157+0000","last_fullsized":"2024-09-12T01:46:14.451157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462071+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:46:14.462071+0000","last_peered":"2024-09-12T01:46:14.462071+0000","last_clean":"2024-09-12T01:46:14.462071+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:46:14.462071+0000","last_undegraded":"2024-09-12T01:46:14.462071+0000","last_fullsized":"2024-09-12T01:46:14.462071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462225+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:46:14.462225+0000","last_peered":"2024-09-12T01:46:14.462225+0000","last_clean":"2024-09-12T01:46:14.462225+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:46:14.462225+0000","last_undegraded":"2024-09-12T01:46:14.462225+0000","last_fullsized":"2024-09-12T01:46:14.462225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456436+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:46:14.456436+0000","last_peered":"2024-09-12T01:46:14.456436+0000","last_clean":"2024-09-12T01:46:14.456436+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:46:14.456436+0000","last_undegraded":"2024-09-12T01:46:14.456436+0000","last_fullsized":"2024-09-12T01:46:14.456436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470354+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:46:14.470354+0000","last_peered":"2024-09-12T01:46:14.470354+0000","last_clean":"2024-09-12T01:46:14.470354+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:46:14.470354+0000","last_undegraded":"2024-09-12T01:46:14.470354+0000","last_fullsized":"2024-09-12T01:46:14.470354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468414+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:46:14.468414+0000","last_peered":"2024-09-12T01:46:14.468414+0000","last_clean":"2024-09-12T01:46:14.468414+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:46:14.468414+0000","last_undegraded":"2024-09-12T01:46:14.468414+0000","last_fullsized":"2024-09-12T01:46:14.468414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470863+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:46:14.470863+0000","last_peered":"2024-09-12T01:46:14.470863+0000","last_clean":"2024-09-12T01:46:14.470863+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:46:14.470863+0000","last_undegraded":"2024-09-12T01:46:14.470863+0000","last_fullsized":"2024-09-12T01:46:14.470863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467613+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:46:14.467613+0000","last_peered":"2024-09-12T01:46:14.467613+0000","last_clean":"2024-09-12T01:46:14.467613+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:46:14.467613+0000","last_undegraded":"2024-09-12T01:46:14.467613+0000","last_fullsized":"2024-09-12T01:46:14.467613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471418+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:46:14.471418+0000","last_peered":"2024-09-12T01:46:14.471418+0000","last_clean":"2024-09-12T01:46:14.471418+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:46:14.471418+0000","last_undegraded":"2024-09-12T01:46:14.471418+0000","last_fullsized":"2024-09-12T01:46:14.471418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473581+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:46:14.473581+0000","last_peered":"2024-09-12T01:46:14.473581+0000","last_clean":"2024-09-12T01:46:14.473581+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:46:14.473581+0000","last_undegraded":"2024-09-12T01:46:14.473581+0000","last_fullsized":"2024-09-12T01:46:14.473581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466586+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:46:14.466586+0000","last_peered":"2024-09-12T01:46:14.466586+0000","last_clean":"2024-09-12T01:46:14.466586+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:46:14.466586+0000","last_undegraded":"2024-09-12T01:46:14.466586+0000","last_fullsized":"2024-09-12T01:46:14.466586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470765+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:46:14.470765+0000","last_peered":"2024-09-12T01:46:14.470765+0000","last_clean":"2024-09-12T01:46:14.470765+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:46:14.470765+0000","last_undegraded":"2024-09-12T01:46:14.470765+0000","last_fullsized":"2024-09-12T01:46:14.470765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456811+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:46:14.456811+0000","last_peered":"2024-09-12T01:46:14.456811+0000","last_clean":"2024-09-12T01:46:14.456811+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:46:14.456811+0000","last_undegraded":"2024-09-12T01:46:14.456811+0000","last_fullsized":"2024-09-12T01:46:14.456811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462152+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:46:14.462152+0000","last_peered":"2024-09-12T01:46:14.462152+0000","last_clean":"2024-09-12T01:46:14.462152+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:46:14.462152+0000","last_undegraded":"2024-09-12T01:46:14.462152+0000","last_fullsized":"2024-09-12T01:46:14.462152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470248+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:46:14.470248+0000","last_peered":"2024-09-12T01:46:14.470248+0000","last_clean":"2024-09-12T01:46:14.470248+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:46:14.470248+0000","last_undegraded":"2024-09-12T01:46:14.470248+0000","last_fullsized":"2024-09-12T01:46:14.470248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472627+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:46:14.472627+0000","last_peered":"2024-09-12T01:46:14.472627+0000","last_clean":"2024-09-12T01:46:14.472627+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:46:14.472627+0000","last_undegraded":"2024-09-12T01:46:14.472627+0000","last_fullsized":"2024-09-12T01:46:14.472627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466306+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:46:14.466306+0000","last_peered":"2024-09-12T01:46:14.466306+0000","last_clean":"2024-09-12T01:46:14.466306+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:46:14.466306+0000","last_undegraded":"2024-09-12T01:46:14.466306+0000","last_fullsized":"2024-09-12T01:46:14.466306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457876+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:46:14.457876+0000","last_peered":"2024-09-12T01:46:14.457876+0000","last_clean":"2024-09-12T01:46:14.457876+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:46:14.457876+0000","last_undegraded":"2024-09-12T01:46:14.457876+0000","last_fullsized":"2024-09-12T01:46:14.457876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461219+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:46:14.461219+0000","last_peered":"2024-09-12T01:46:14.461219+0000","last_clean":"2024-09-12T01:46:14.461219+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:46:14.461219+0000","last_undegraded":"2024-09-12T01:46:14.461219+0000","last_fullsized":"2024-09-12T01:46:14.461219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469829+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:46:14.469829+0000","last_peered":"2024-09-12T01:46:14.469829+0000","last_clean":"2024-09-12T01:46:14.469829+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:46:14.469829+0000","last_undegraded":"2024-09-12T01:46:14.469829+0000","last_fullsized":"2024-09-12T01:46:14.469829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471133+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:46:14.471133+0000","last_peered":"2024-09-12T01:46:14.471133+0000","last_clean":"2024-09-12T01:46:14.471133+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:46:14.471133+0000","last_undegraded":"2024-09-12T01:46:14.471133+0000","last_fullsized":"2024-09-12T01:46:14.471133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460490+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:46:14.460490+0000","last_peered":"2024-09-12T01:46:14.460490+0000","last_clean":"2024-09-12T01:46:14.460490+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:46:14.460490+0000","last_undegraded":"2024-09-12T01:46:14.460490+0000","last_fullsized":"2024-09-12T01:46:14.460490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.455176+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:46:14.455176+0000","last_peered":"2024-09-12T01:46:14.455176+0000","last_clean":"2024-09-12T01:46:14.455176+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:46:14.455176+0000","last_undegraded":"2024-09-12T01:46:14.455176+0000","last_fullsized":"2024-09-12T01:46:14.455176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449237+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:46:14.449237+0000","last_peered":"2024-09-12T01:46:14.449237+0000","last_clean":"2024-09-12T01:46:14.449237+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:46:14.449237+0000","last_undegraded":"2024-09-12T01:46:14.449237+0000","last_fullsized":"2024-09-12T01:46:14.449237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459600+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:46:14.459600+0000","last_peered":"2024-09-12T01:46:14.459600+0000","last_clean":"2024-09-12T01:46:14.459600+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:46:14.459600+0000","last_undegraded":"2024-09-12T01:46:14.459600+0000","last_fullsized":"2024-09-12T01:46:14.459600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456999+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:46:14.456999+0000","last_peered":"2024-09-12T01:46:14.456999+0000","last_clean":"2024-09-12T01:46:14.456999+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:46:14.456999+0000","last_undegraded":"2024-09-12T01:46:14.456999+0000","last_fullsized":"2024-09-12T01:46:14.456999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.453782+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:46:14.453782+0000","last_peered":"2024-09-12T01:46:14.453782+0000","last_clean":"2024-09-12T01:46:14.453782+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:46:14.453782+0000","last_undegraded":"2024-09-12T01:46:14.453782+0000","last_fullsized":"2024-09-12T01:46:14.453782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457999+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:46:14.457999+0000","last_peered":"2024-09-12T01:46:14.457999+0000","last_clean":"2024-09-12T01:46:14.457999+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:46:14.457999+0000","last_undegraded":"2024-09-12T01:46:14.457999+0000","last_fullsized":"2024-09-12T01:46:14.457999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460598+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:46:14.460598+0000","last_peered":"2024-09-12T01:46:14.460598+0000","last_clean":"2024-09-12T01:46:14.460598+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:46:14.460598+0000","last_undegraded":"2024-09-12T01:46:14.460598+0000","last_fullsized":"2024-09-12T01:46:14.460598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463452+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:46:14.463452+0000","last_peered":"2024-09-12T01:46:14.463452+0000","last_clean":"2024-09-12T01:46:14.463452+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:46:14.463452+0000","last_undegraded":"2024-09-12T01:46:14.463452+0000","last_fullsized":"2024-09-12T01:46:14.463452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463643+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:46:14.463643+0000","last_peered":"2024-09-12T01:46:14.463643+0000","last_clean":"2024-09-12T01:46:14.463643+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:46:14.463643+0000","last_undegraded":"2024-09-12T01:46:14.463643+0000","last_fullsized":"2024-09-12T01:46:14.463643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457236+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:46:14.457236+0000","last_peered":"2024-09-12T01:46:14.457236+0000","last_clean":"2024-09-12T01:46:14.457236+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:46:14.457236+0000","last_undegraded":"2024-09-12T01:46:14.457236+0000","last_fullsized":"2024-09-12T01:46:14.457236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:26.979885+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:46:26.979885+0000","last_peered":"2024-09-12T01:46:26.979885+0000","last_clean":"2024-09-12T01:46:26.979885+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:46:26.979885+0000","last_undegraded":"2024-09-12T01:46:26.979885+0000","last_fullsized":"2024-09-12T01:46:26.979885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461733+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:46:14.461733+0000","last_peered":"2024-09-12T01:46:14.461733+0000","last_clean":"2024-09-12T01:46:14.461733+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:46:14.461733+0000","last_undegraded":"2024-09-12T01:46:14.461733+0000","last_fullsized":"2024-09-12T01:46:14.461733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.452061+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:46:14.452061+0000","last_peered":"2024-09-12T01:46:14.452061+0000","last_clean":"2024-09-12T01:46:14.452061+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:46:14.452061+0000","last_undegraded":"2024-09-12T01:46:14.452061+0000","last_fullsized":"2024-09-12T01:46:14.452061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466729+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:46:14.466729+0000","last_peered":"2024-09-12T01:46:14.466729+0000","last_clean":"2024-09-12T01:46:14.466729+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:46:14.466729+0000","last_undegraded":"2024-09-12T01:46:14.466729+0000","last_fullsized":"2024-09-12T01:46:14.466729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:23.975221+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:46:23.975221+0000","last_peered":"2024-09-12T01:46:23.975221+0000","last_clean":"2024-09-12T01:46:23.975221+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:46:23.975221+0000","last_undegraded":"2024-09-12T01:46:23.975221+0000","last_fullsized":"2024-09-12T01:46:23.975221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473143+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:46:14.473143+0000","last_peered":"2024-09-12T01:46:14.473143+0000","last_clean":"2024-09-12T01:46:14.473143+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:46:14.473143+0000","last_undegraded":"2024-09-12T01:46:14.473143+0000","last_fullsized":"2024-09-12T01:46:14.473143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460971+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:46:14.460971+0000","last_peered":"2024-09-12T01:46:14.460971+0000","last_clean":"2024-09-12T01:46:14.460971+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:46:14.460971+0000","last_undegraded":"2024-09-12T01:46:14.460971+0000","last_fullsized":"2024-09-12T01:46:14.460971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473209+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:46:14.473209+0000","last_peered":"2024-09-12T01:46:14.473209+0000","last_clean":"2024-09-12T01:46:14.473209+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:46:14.473209+0000","last_undegraded":"2024-09-12T01:46:14.473209+0000","last_fullsized":"2024-09-12T01:46:14.473209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":267,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:24.977041+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:46:24.977041+0000","last_peered":"2024-09-12T01:46:24.977041+0000","last_clean":"2024-09-12T01:46:24.977041+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:46:24.977041+0000","last_undegraded":"2024-09-12T01:46:24.977041+0000","last_fullsized":"2024-09-12T01:46:24.977041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":210,"num_read_kb":174,"num_write":36,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457287+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:46:14.457287+0000","last_peered":"2024-09-12T01:46:14.457287+0000","last_clean":"2024-09-12T01:46:14.457287+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:46:14.457287+0000","last_undegraded":"2024-09-12T01:46:14.457287+0000","last_fullsized":"2024-09-12T01:46:14.457287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470461+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:46:14.470461+0000","last_peered":"2024-09-12T01:46:14.470461+0000","last_clean":"2024-09-12T01:46:14.470461+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:46:14.470461+0000","last_undegraded":"2024-09-12T01:46:14.470461+0000","last_fullsized":"2024-09-12T01:46:14.470461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472069+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:46:14.472069+0000","last_peered":"2024-09-12T01:46:14.472069+0000","last_clean":"2024-09-12T01:46:14.472069+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:46:14.472069+0000","last_undegraded":"2024-09-12T01:46:14.472069+0000","last_fullsized":"2024-09-12T01:46:14.472069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:25.978847+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:46:25.978847+0000","last_peered":"2024-09-12T01:46:25.978847+0000","last_clean":"2024-09-12T01:46:25.978847+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:46:25.978847+0000","last_undegraded":"2024-09-12T01:46:25.978847+0000","last_fullsized":"2024-09-12T01:46:25.978847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468634+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:46:14.468634+0000","last_peered":"2024-09-12T01:46:14.468634+0000","last_clean":"2024-09-12T01:46:14.468634+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:46:14.468634+0000","last_undegraded":"2024-09-12T01:46:14.468634+0000","last_fullsized":"2024-09-12T01:46:14.468634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469018+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:46:14.469018+0000","last_peered":"2024-09-12T01:46:14.469018+0000","last_clean":"2024-09-12T01:46:14.469018+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:46:14.469018+0000","last_undegraded":"2024-09-12T01:46:14.469018+0000","last_fullsized":"2024-09-12T01:46:14.469018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:47.816560+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:47.816560+0000","last_peered":"2024-09-12T01:44:47.816560+0000","last_clean":"2024-09-12T01:44:47.816560+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:47.816560+0000","last_undegraded":"2024-09-12T01:44:47.816560+0000","last_fullsized":"2024-09-12T01:44:47.816560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.458089+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:46:14.458089+0000","last_peered":"2024-09-12T01:46:14.458089+0000","last_clean":"2024-09-12T01:46:14.458089+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:46:14.458089+0000","last_undegraded":"2024-09-12T01:46:14.458089+0000","last_fullsized":"2024-09-12T01:46:14.458089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467837+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:46:14.467837+0000","last_peered":"2024-09-12T01:46:14.467837+0000","last_clean":"2024-09-12T01:46:14.467837+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:46:14.467837+0000","last_undegraded":"2024-09-12T01:46:14.467837+0000","last_fullsized":"2024-09-12T01:46:14.467837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465783+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:46:14.465783+0000","last_peered":"2024-09-12T01:46:14.465783+0000","last_clean":"2024-09-12T01:46:14.465783+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:46:14.465783+0000","last_undegraded":"2024-09-12T01:46:14.465783+0000","last_fullsized":"2024-09-12T01:46:14.465783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473700+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:46:14.473700+0000","last_peered":"2024-09-12T01:46:14.473700+0000","last_clean":"2024-09-12T01:46:14.473700+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:46:14.473700+0000","last_undegraded":"2024-09-12T01:46:14.473700+0000","last_fullsized":"2024-09-12T01:46:14.473700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818582+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818582+0000","last_peered":"2024-09-12T01:44:45.818582+0000","last_clean":"2024-09-12T01:44:45.818582+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818582+0000","last_undegraded":"2024-09-12T01:44:45.818582+0000","last_fullsized":"2024-09-12T01:44:45.818582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.464071+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:46:14.464071+0000","last_peered":"2024-09-12T01:46:14.464071+0000","last_clean":"2024-09-12T01:46:14.464071+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:46:14.464071+0000","last_undegraded":"2024-09-12T01:46:14.464071+0000","last_fullsized":"2024-09-12T01:46:14.464071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449016+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:46:14.449016+0000","last_peered":"2024-09-12T01:46:14.449016+0000","last_clean":"2024-09-12T01:46:14.449016+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:46:14.449016+0000","last_undegraded":"2024-09-12T01:46:14.449016+0000","last_fullsized":"2024-09-12T01:46:14.449016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:26.979692+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:46:26.979692+0000","last_peered":"2024-09-12T01:46:26.979692+0000","last_clean":"2024-09-12T01:46:26.979692+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:46:26.979692+0000","last_undegraded":"2024-09-12T01:46:26.979692+0000","last_fullsized":"2024-09-12T01:46:26.979692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468646+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:46:14.468646+0000","last_peered":"2024-09-12T01:46:14.468646+0000","last_clean":"2024-09-12T01:46:14.468646+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:46:14.468646+0000","last_undegraded":"2024-09-12T01:46:14.468646+0000","last_fullsized":"2024-09-12T01:46:14.468646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.450592+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:46:14.450592+0000","last_peered":"2024-09-12T01:46:14.450592+0000","last_clean":"2024-09-12T01:46:14.450592+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:46:14.450592+0000","last_undegraded":"2024-09-12T01:46:14.450592+0000","last_fullsized":"2024-09-12T01:46:14.450592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468594+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:46:14.468594+0000","last_peered":"2024-09-12T01:46:14.468594+0000","last_clean":"2024-09-12T01:46:14.468594+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:46:14.468594+0000","last_undegraded":"2024-09-12T01:46:14.468594+0000","last_fullsized":"2024-09-12T01:46:14.468594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449242+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:46:14.449242+0000","last_peered":"2024-09-12T01:46:14.449242+0000","last_clean":"2024-09-12T01:46:14.449242+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:46:14.449242+0000","last_undegraded":"2024-09-12T01:46:14.449242+0000","last_fullsized":"2024-09-12T01:46:14.449242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:18.793448+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:46:18.793448+0000","last_peered":"2024-09-12T01:46:18.793448+0000","last_clean":"2024-09-12T01:46:18.793448+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:46:18.793448+0000","last_undegraded":"2024-09-12T01:46:18.793448+0000","last_fullsized":"2024-09-12T01:46:18.793448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465155+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:46:14.465155+0000","last_peered":"2024-09-12T01:46:14.465155+0000","last_clean":"2024-09-12T01:46:14.465155+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:46:14.465155+0000","last_undegraded":"2024-09-12T01:46:14.465155+0000","last_fullsized":"2024-09-12T01:46:14.465155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457998+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:46:14.457998+0000","last_peered":"2024-09-12T01:46:14.457998+0000","last_clean":"2024-09-12T01:46:14.457998+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:46:14.457998+0000","last_undegraded":"2024-09-12T01:46:14.457998+0000","last_fullsized":"2024-09-12T01:46:14.457998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.440457+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:46:14.440457+0000","last_peered":"2024-09-12T01:46:14.440457+0000","last_clean":"2024-09-12T01:46:14.440457+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:46:14.440457+0000","last_undegraded":"2024-09-12T01:46:14.440457+0000","last_fullsized":"2024-09-12T01:46:14.440457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463449+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:46:14.463449+0000","last_peered":"2024-09-12T01:46:14.463449+0000","last_clean":"2024-09-12T01:46:14.463449+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:46:14.463449+0000","last_undegraded":"2024-09-12T01:46:14.463449+0000","last_fullsized":"2024-09-12T01:46:14.463449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470639+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:46:14.470639+0000","last_peered":"2024-09-12T01:46:14.470639+0000","last_clean":"2024-09-12T01:46:14.470639+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:46:14.470639+0000","last_undegraded":"2024-09-12T01:46:14.470639+0000","last_fullsized":"2024-09-12T01:46:14.470639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473199+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:46:14.473199+0000","last_peered":"2024-09-12T01:46:14.473199+0000","last_clean":"2024-09-12T01:46:14.473199+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:46:14.473199+0000","last_undegraded":"2024-09-12T01:46:14.473199+0000","last_fullsized":"2024-09-12T01:46:14.473199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469618+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:46:14.469618+0000","last_peered":"2024-09-12T01:46:14.469618+0000","last_clean":"2024-09-12T01:46:14.469618+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:46:14.469618+0000","last_undegraded":"2024-09-12T01:46:14.469618+0000","last_fullsized":"2024-09-12T01:46:14.469618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473089+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:46:14.473089+0000","last_peered":"2024-09-12T01:46:14.473089+0000","last_clean":"2024-09-12T01:46:14.473089+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:46:14.473089+0000","last_undegraded":"2024-09-12T01:46:14.473089+0000","last_fullsized":"2024-09-12T01:46:14.473089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.454649+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:46:14.454649+0000","last_peered":"2024-09-12T01:46:14.454649+0000","last_clean":"2024-09-12T01:46:14.454649+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:46:14.454649+0000","last_undegraded":"2024-09-12T01:46:14.454649+0000","last_fullsized":"2024-09-12T01:46:14.454649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459700+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:46:14.459700+0000","last_peered":"2024-09-12T01:46:14.459700+0000","last_clean":"2024-09-12T01:46:14.459700+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:46:14.459700+0000","last_undegraded":"2024-09-12T01:46:14.459700+0000","last_fullsized":"2024-09-12T01:46:14.459700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.841955+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:46.841955+0000","last_peered":"2024-09-12T01:44:46.841955+0000","last_clean":"2024-09-12T01:44:46.841955+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:46.841955+0000","last_undegraded":"2024-09-12T01:44:46.841955+0000","last_fullsized":"2024-09-12T01:44:46.841955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75132,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":636,"num_read_kb":397,"num_write":179,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":225396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1100798,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1030417,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438784,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":509124,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":507808,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:46:33.055 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T01:46:33.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T01:46:33.298 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:33.695 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:33 smithi122 ceph-mon[30925]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:33.695 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:33 smithi122 ceph-mon[30925]: pgmap v337: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T01:46:33.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 -- 172.21.15.122:0/2973601613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60106f10 msgr2=0x7f5f6010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 --2- 172.21.15.122:0/2973601613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60106f10 0x7f5f6010ea90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009ca0 tx=0x7f5f5802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 -- 172.21.15.122:0/2973601613 shutdown_connections 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 --2- 172.21.15.122:0/2973601613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60106f10 0x7f5f6010ea90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 --2- 172.21.15.122:0/2973601613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60102790 0x7f5f601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 --2- 172.21.15.122:0/2973601613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f60101de0 0x7f5f601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 -- 172.21.15.122:0/2973601613 >> 172.21.15.122:0/2973601613 conn(0x7f5f600fd8c0 msgr2=0x7f5f600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 -- 172.21.15.122:0/2973601613 shutdown_connections 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.776+0000 7f5f655f6640 1 -- 172.21.15.122:0/2973601613 wait complete. 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 Processor -- start 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 -- start start 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 0x7f5f60198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:33.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f60102790 0x7f5f601989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60106f10 0x7f5f6019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60111f90 con 0x7f5f60101de0 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60111e10 con 0x7f5f60106f10 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f655f6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60112110 con 0x7f5f60102790 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 0x7f5f60198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 0x7f5f60198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52476/0 (socket says 172.21.15.122:52476) 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 -- 172.21.15.122:0/3872496359 learned_addr learned my addr 172.21.15.122:0/3872496359 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5e7fc640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f60102790 0x7f5f601989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5f7fe640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60106f10 0x7f5f6019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 -- 172.21.15.122:0/3872496359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f60102790 msgr2=0x7f5f601989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:33.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f60102790 0x7f5f601989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 -- 172.21.15.122:0/3872496359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60106f10 msgr2=0x7f5f6019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60106f10 0x7f5f6019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5effd640 1 -- 172.21.15.122:0/3872496359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6019d420 con 0x7f5f60101de0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.777+0000 7f5f5f7fe640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60106f10 0x7f5f6019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.778+0000 7f5f5effd640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 0x7f5f60198470 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000b4a0 tx=0x7f5f5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.778+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50014070 con 0x7f5f60101de0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.778+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f50010af0 con 0x7f5f60101de0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.778+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6019d710 con 0x7f5f60101de0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.778+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50017d10 con 0x7f5f60101de0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.778+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f601a4ff0 con 0x7f5f60101de0 2024-09-12T01:46:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.779+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f601021c0 con 0x7f5f60101de0 2024-09-12T01:46:33.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.780+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5f50017070 con 0x7f5f60101de0 2024-09-12T01:46:33.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.781+0000 7f5f47fff640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f480a2c10 0x7f5f480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:33.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.781+0000 7f5f5e7fc640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f480a2c10 0x7f5f480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:33.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.781+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5f500ce430 con 0x7f5f60101de0 2024-09-12T01:46:33.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.781+0000 7f5f5e7fc640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f480a2c10 0x7f5f480a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f601088e0 tx=0x7f5f5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:33.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.785+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f5009b340 con 0x7f5f60101de0 2024-09-12T01:46:33.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.950+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 --> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f60062630 con 0x7f5f480a2c10 2024-09-12T01:46:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:33 smithi183 ceph-mon[35127]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:33 smithi183 ceph-mon[35127]: pgmap v337: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T01:46:33.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.956+0000 7f5f47fff640 1 -- 172.21.15.122:0/3872496359 <== mgr.14150 v2:172.21.15.122:6800/3245830997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259071 (secure 0 0 0) 0x7f5f60062630 con 0x7f5f480a2c10 2024-09-12T01:46:33.958 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:33.959 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-12T01:46:33.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f480a2c10 msgr2=0x7f5f480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f480a2c10 0x7f5f480a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f601088e0 tx=0x7f5f5400b390 comp rx=0 tx=0).stop 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 msgr2=0x7f5f60198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 0x7f5f60198470 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000b4a0 tx=0x7f5f5000b960 comp rx=0 tx=0).stop 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 shutdown_connections 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f480a2c10 0x7f5f480a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f60106f10 0x7f5f6019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f60102790 0x7f5f601989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 --2- 172.21.15.122:0/3872496359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f60101de0 0x7f5f60198470 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 >> 172.21.15.122:0/3872496359 conn(0x7f5f600fd8c0 msgr2=0x7f5f600fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 shutdown_connections 2024-09-12T01:46:33.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:33.960+0000 7f5f655f6640 1 -- 172.21.15.122:0/3872496359 wait complete. 2024-09-12T01:46:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:33 smithi151 ceph-mon[35229]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:33 smithi151 ceph-mon[35229]: pgmap v337: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T01:46:34.737 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-12T01:46:32.472751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665734,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":778,"num_read_kb":517,"num_write":362,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360324,"kb_used_data":11076,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581500,"statfs":{"total":1151940427776,"available":1151571456000,"internally_reserved":0,"allocated":11341824,"data_stored":7243758,"data_compressed":176769,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":53,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002927"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472514+0000","last_change":"2024-09-12T01:44:35.753142+0000","last_active":"2024-09-12T01:46:14.472514+0000","last_peered":"2024-09-12T01:46:14.472514+0000","last_clean":"2024-09-12T01:46:14.472514+0000","last_became_active":"2024-09-12T01:44:35.752761+0000","last_became_peered":"2024-09-12T01:44:35.752761+0000","last_unstale":"2024-09-12T01:46:14.472514+0000","last_undegraded":"2024-09-12T01:46:14.472514+0000","last_fullsized":"2024-09-12T01:46:14.472514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:57:31.755261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.451157+0000","last_change":"2024-09-12T01:44:35.757315+0000","last_active":"2024-09-12T01:46:14.451157+0000","last_peered":"2024-09-12T01:46:14.451157+0000","last_clean":"2024-09-12T01:46:14.451157+0000","last_became_active":"2024-09-12T01:44:35.756999+0000","last_became_peered":"2024-09-12T01:44:35.756999+0000","last_unstale":"2024-09-12T01:46:14.451157+0000","last_undegraded":"2024-09-12T01:46:14.451157+0000","last_fullsized":"2024-09-12T01:46:14.451157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:01.282073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462071+0000","last_change":"2024-09-12T01:44:35.763684+0000","last_active":"2024-09-12T01:46:14.462071+0000","last_peered":"2024-09-12T01:46:14.462071+0000","last_clean":"2024-09-12T01:46:14.462071+0000","last_became_active":"2024-09-12T01:44:35.763445+0000","last_became_peered":"2024-09-12T01:44:35.763445+0000","last_unstale":"2024-09-12T01:46:14.462071+0000","last_undegraded":"2024-09-12T01:46:14.462071+0000","last_fullsized":"2024-09-12T01:46:14.462071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:07.802277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462225+0000","last_change":"2024-09-12T01:44:35.765575+0000","last_active":"2024-09-12T01:46:14.462225+0000","last_peered":"2024-09-12T01:46:14.462225+0000","last_clean":"2024-09-12T01:46:14.462225+0000","last_became_active":"2024-09-12T01:44:35.765244+0000","last_became_peered":"2024-09-12T01:44:35.765244+0000","last_unstale":"2024-09-12T01:46:14.462225+0000","last_undegraded":"2024-09-12T01:46:14.462225+0000","last_fullsized":"2024-09-12T01:46:14.462225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:25.251479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456436+0000","last_change":"2024-09-12T01:44:35.758062+0000","last_active":"2024-09-12T01:46:14.456436+0000","last_peered":"2024-09-12T01:46:14.456436+0000","last_clean":"2024-09-12T01:46:14.456436+0000","last_became_active":"2024-09-12T01:44:35.756978+0000","last_became_peered":"2024-09-12T01:44:35.756978+0000","last_unstale":"2024-09-12T01:46:14.456436+0000","last_undegraded":"2024-09-12T01:46:14.456436+0000","last_fullsized":"2024-09-12T01:46:14.456436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:12.394224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470354+0000","last_change":"2024-09-12T01:44:35.760416+0000","last_active":"2024-09-12T01:46:14.470354+0000","last_peered":"2024-09-12T01:46:14.470354+0000","last_clean":"2024-09-12T01:46:14.470354+0000","last_became_active":"2024-09-12T01:44:35.760086+0000","last_became_peered":"2024-09-12T01:44:35.760086+0000","last_unstale":"2024-09-12T01:46:14.470354+0000","last_undegraded":"2024-09-12T01:46:14.470354+0000","last_fullsized":"2024-09-12T01:46:14.470354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:11.969978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468414+0000","last_change":"2024-09-12T01:44:46.077967+0000","last_active":"2024-09-12T01:46:14.468414+0000","last_peered":"2024-09-12T01:46:14.468414+0000","last_clean":"2024-09-12T01:46:14.468414+0000","last_became_active":"2024-09-12T01:44:46.077646+0000","last_became_peered":"2024-09-12T01:44:46.077646+0000","last_unstale":"2024-09-12T01:46:14.468414+0000","last_undegraded":"2024-09-12T01:46:14.468414+0000","last_fullsized":"2024-09-12T01:46:14.468414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:53:53.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470863+0000","last_change":"2024-09-12T01:44:35.759376+0000","last_active":"2024-09-12T01:46:14.470863+0000","last_peered":"2024-09-12T01:46:14.470863+0000","last_clean":"2024-09-12T01:46:14.470863+0000","last_became_active":"2024-09-12T01:44:35.758602+0000","last_became_peered":"2024-09-12T01:44:35.758602+0000","last_unstale":"2024-09-12T01:46:14.470863+0000","last_undegraded":"2024-09-12T01:46:14.470863+0000","last_fullsized":"2024-09-12T01:46:14.470863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.486454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467613+0000","last_change":"2024-09-12T01:44:35.761965+0000","last_active":"2024-09-12T01:46:14.467613+0000","last_peered":"2024-09-12T01:46:14.467613+0000","last_clean":"2024-09-12T01:46:14.467613+0000","last_became_active":"2024-09-12T01:44:35.761661+0000","last_became_peered":"2024-09-12T01:44:35.761661+0000","last_unstale":"2024-09-12T01:46:14.467613+0000","last_undegraded":"2024-09-12T01:46:14.467613+0000","last_fullsized":"2024-09-12T01:46:14.467613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:22.276373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471418+0000","last_change":"2024-09-12T01:44:45.824934+0000","last_active":"2024-09-12T01:46:14.471418+0000","last_peered":"2024-09-12T01:46:14.471418+0000","last_clean":"2024-09-12T01:46:14.471418+0000","last_became_active":"2024-09-12T01:44:45.824494+0000","last_became_peered":"2024-09-12T01:44:45.824494+0000","last_unstale":"2024-09-12T01:46:14.471418+0000","last_undegraded":"2024-09-12T01:46:14.471418+0000","last_fullsized":"2024-09-12T01:46:14.471418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:17.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473581+0000","last_change":"2024-09-12T01:44:35.767901+0000","last_active":"2024-09-12T01:46:14.473581+0000","last_peered":"2024-09-12T01:46:14.473581+0000","last_clean":"2024-09-12T01:46:14.473581+0000","last_became_active":"2024-09-12T01:44:35.767557+0000","last_became_peered":"2024-09-12T01:44:35.767557+0000","last_unstale":"2024-09-12T01:46:14.473581+0000","last_undegraded":"2024-09-12T01:46:14.473581+0000","last_fullsized":"2024-09-12T01:46:14.473581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:13.871499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466586+0000","last_change":"2024-09-12T01:44:45.821285+0000","last_active":"2024-09-12T01:46:14.466586+0000","last_peered":"2024-09-12T01:46:14.466586+0000","last_clean":"2024-09-12T01:46:14.466586+0000","last_became_active":"2024-09-12T01:44:45.820950+0000","last_became_peered":"2024-09-12T01:44:45.820950+0000","last_unstale":"2024-09-12T01:46:14.466586+0000","last_undegraded":"2024-09-12T01:46:14.466586+0000","last_fullsized":"2024-09-12T01:46:14.466586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:08.624483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470765+0000","last_change":"2024-09-12T01:44:35.758083+0000","last_active":"2024-09-12T01:46:14.470765+0000","last_peered":"2024-09-12T01:46:14.470765+0000","last_clean":"2024-09-12T01:46:14.470765+0000","last_became_active":"2024-09-12T01:44:35.757718+0000","last_became_peered":"2024-09-12T01:44:35.757718+0000","last_unstale":"2024-09-12T01:46:14.470765+0000","last_undegraded":"2024-09-12T01:46:14.470765+0000","last_fullsized":"2024-09-12T01:46:14.470765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:02.691729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456811+0000","last_change":"2024-09-12T01:44:35.755757+0000","last_active":"2024-09-12T01:46:14.456811+0000","last_peered":"2024-09-12T01:46:14.456811+0000","last_clean":"2024-09-12T01:46:14.456811+0000","last_became_active":"2024-09-12T01:44:35.755304+0000","last_became_peered":"2024-09-12T01:44:35.755304+0000","last_unstale":"2024-09-12T01:46:14.456811+0000","last_undegraded":"2024-09-12T01:46:14.456811+0000","last_fullsized":"2024-09-12T01:46:14.456811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:28.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.462152+0000","last_change":"2024-09-12T01:44:35.764185+0000","last_active":"2024-09-12T01:46:14.462152+0000","last_peered":"2024-09-12T01:46:14.462152+0000","last_clean":"2024-09-12T01:46:14.462152+0000","last_became_active":"2024-09-12T01:44:35.763859+0000","last_became_peered":"2024-09-12T01:44:35.763859+0000","last_unstale":"2024-09-12T01:46:14.462152+0000","last_undegraded":"2024-09-12T01:46:14.462152+0000","last_fullsized":"2024-09-12T01:46:14.462152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:29:32.739941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470248+0000","last_change":"2024-09-12T01:44:35.769458+0000","last_active":"2024-09-12T01:46:14.470248+0000","last_peered":"2024-09-12T01:46:14.470248+0000","last_clean":"2024-09-12T01:46:14.470248+0000","last_became_active":"2024-09-12T01:44:35.769162+0000","last_became_peered":"2024-09-12T01:44:35.769162+0000","last_unstale":"2024-09-12T01:46:14.470248+0000","last_undegraded":"2024-09-12T01:46:14.470248+0000","last_fullsized":"2024-09-12T01:46:14.470248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:29.282404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472627+0000","last_change":"2024-09-12T01:44:35.753378+0000","last_active":"2024-09-12T01:46:14.472627+0000","last_peered":"2024-09-12T01:46:14.472627+0000","last_clean":"2024-09-12T01:46:14.472627+0000","last_became_active":"2024-09-12T01:44:35.752934+0000","last_became_peered":"2024-09-12T01:44:35.752934+0000","last_unstale":"2024-09-12T01:46:14.472627+0000","last_undegraded":"2024-09-12T01:46:14.472627+0000","last_fullsized":"2024-09-12T01:46:14.472627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:05.089581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466306+0000","last_change":"2024-09-12T01:44:35.755600+0000","last_active":"2024-09-12T01:46:14.466306+0000","last_peered":"2024-09-12T01:46:14.466306+0000","last_clean":"2024-09-12T01:46:14.466306+0000","last_became_active":"2024-09-12T01:44:35.755286+0000","last_became_peered":"2024-09-12T01:44:35.755286+0000","last_unstale":"2024-09-12T01:46:14.466306+0000","last_undegraded":"2024-09-12T01:46:14.466306+0000","last_fullsized":"2024-09-12T01:46:14.466306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:23.552352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457876+0000","last_change":"2024-09-12T01:44:35.762491+0000","last_active":"2024-09-12T01:46:14.457876+0000","last_peered":"2024-09-12T01:46:14.457876+0000","last_clean":"2024-09-12T01:46:14.457876+0000","last_became_active":"2024-09-12T01:44:35.762188+0000","last_became_peered":"2024-09-12T01:44:35.762188+0000","last_unstale":"2024-09-12T01:46:14.457876+0000","last_undegraded":"2024-09-12T01:46:14.457876+0000","last_fullsized":"2024-09-12T01:46:14.457876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:49:17.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461219+0000","last_change":"2024-09-12T01:44:35.760198+0000","last_active":"2024-09-12T01:46:14.461219+0000","last_peered":"2024-09-12T01:46:14.461219+0000","last_clean":"2024-09-12T01:46:14.461219+0000","last_became_active":"2024-09-12T01:44:35.759701+0000","last_became_peered":"2024-09-12T01:44:35.759701+0000","last_unstale":"2024-09-12T01:46:14.461219+0000","last_undegraded":"2024-09-12T01:46:14.461219+0000","last_fullsized":"2024-09-12T01:46:14.461219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:37.314769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469829+0000","last_change":"2024-09-12T01:44:35.760694+0000","last_active":"2024-09-12T01:46:14.469829+0000","last_peered":"2024-09-12T01:46:14.469829+0000","last_clean":"2024-09-12T01:46:14.469829+0000","last_became_active":"2024-09-12T01:44:35.760381+0000","last_became_peered":"2024-09-12T01:44:35.760381+0000","last_unstale":"2024-09-12T01:46:14.469829+0000","last_undegraded":"2024-09-12T01:46:14.469829+0000","last_fullsized":"2024-09-12T01:46:14.469829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:09.968320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.471133+0000","last_change":"2024-09-12T01:44:35.766442+0000","last_active":"2024-09-12T01:46:14.471133+0000","last_peered":"2024-09-12T01:46:14.471133+0000","last_clean":"2024-09-12T01:46:14.471133+0000","last_became_active":"2024-09-12T01:44:35.766138+0000","last_became_peered":"2024-09-12T01:44:35.766138+0000","last_unstale":"2024-09-12T01:46:14.471133+0000","last_undegraded":"2024-09-12T01:46:14.471133+0000","last_fullsized":"2024-09-12T01:46:14.471133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:42.002241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460490+0000","last_change":"2024-09-12T01:44:35.761980+0000","last_active":"2024-09-12T01:46:14.460490+0000","last_peered":"2024-09-12T01:46:14.460490+0000","last_clean":"2024-09-12T01:46:14.460490+0000","last_became_active":"2024-09-12T01:44:35.760610+0000","last_became_peered":"2024-09-12T01:44:35.760610+0000","last_unstale":"2024-09-12T01:46:14.460490+0000","last_undegraded":"2024-09-12T01:46:14.460490+0000","last_fullsized":"2024-09-12T01:46:14.460490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:20.133859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.455176+0000","last_change":"2024-09-12T01:44:45.823417+0000","last_active":"2024-09-12T01:46:14.455176+0000","last_peered":"2024-09-12T01:46:14.455176+0000","last_clean":"2024-09-12T01:46:14.455176+0000","last_became_active":"2024-09-12T01:44:45.822891+0000","last_became_peered":"2024-09-12T01:44:45.822891+0000","last_unstale":"2024-09-12T01:46:14.455176+0000","last_undegraded":"2024-09-12T01:46:14.455176+0000","last_fullsized":"2024-09-12T01:46:14.455176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:44.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449237+0000","last_change":"2024-09-12T01:44:35.763032+0000","last_active":"2024-09-12T01:46:14.449237+0000","last_peered":"2024-09-12T01:46:14.449237+0000","last_clean":"2024-09-12T01:46:14.449237+0000","last_became_active":"2024-09-12T01:44:35.762525+0000","last_became_peered":"2024-09-12T01:44:35.762525+0000","last_unstale":"2024-09-12T01:46:14.449237+0000","last_undegraded":"2024-09-12T01:46:14.449237+0000","last_fullsized":"2024-09-12T01:46:14.449237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:41.847416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459600+0000","last_change":"2024-09-12T01:44:35.768477+0000","last_active":"2024-09-12T01:46:14.459600+0000","last_peered":"2024-09-12T01:46:14.459600+0000","last_clean":"2024-09-12T01:46:14.459600+0000","last_became_active":"2024-09-12T01:44:35.768110+0000","last_became_peered":"2024-09-12T01:44:35.768110+0000","last_unstale":"2024-09-12T01:46:14.459600+0000","last_undegraded":"2024-09-12T01:46:14.459600+0000","last_fullsized":"2024-09-12T01:46:14.459600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:48.024620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.456999+0000","last_change":"2024-09-12T01:44:35.761814+0000","last_active":"2024-09-12T01:46:14.456999+0000","last_peered":"2024-09-12T01:46:14.456999+0000","last_clean":"2024-09-12T01:46:14.456999+0000","last_became_active":"2024-09-12T01:44:35.761534+0000","last_became_peered":"2024-09-12T01:44:35.761534+0000","last_unstale":"2024-09-12T01:46:14.456999+0000","last_undegraded":"2024-09-12T01:46:14.456999+0000","last_fullsized":"2024-09-12T01:46:14.456999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:14.356723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.453782+0000","last_change":"2024-09-12T01:44:35.755209+0000","last_active":"2024-09-12T01:46:14.453782+0000","last_peered":"2024-09-12T01:46:14.453782+0000","last_clean":"2024-09-12T01:46:14.453782+0000","last_became_active":"2024-09-12T01:44:35.754914+0000","last_became_peered":"2024-09-12T01:44:35.754914+0000","last_unstale":"2024-09-12T01:46:14.453782+0000","last_undegraded":"2024-09-12T01:46:14.453782+0000","last_fullsized":"2024-09-12T01:46:14.453782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:12.929867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457999+0000","last_change":"2024-09-12T01:44:35.757215+0000","last_active":"2024-09-12T01:46:14.457999+0000","last_peered":"2024-09-12T01:46:14.457999+0000","last_clean":"2024-09-12T01:46:14.457999+0000","last_became_active":"2024-09-12T01:44:35.756771+0000","last_became_peered":"2024-09-12T01:44:35.756771+0000","last_unstale":"2024-09-12T01:46:14.457999+0000","last_undegraded":"2024-09-12T01:46:14.457999+0000","last_fullsized":"2024-09-12T01:46:14.457999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:30:35.931714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460598+0000","last_change":"2024-09-12T01:44:35.765924+0000","last_active":"2024-09-12T01:46:14.460598+0000","last_peered":"2024-09-12T01:46:14.460598+0000","last_clean":"2024-09-12T01:46:14.460598+0000","last_became_active":"2024-09-12T01:44:35.765366+0000","last_became_peered":"2024-09-12T01:44:35.765366+0000","last_unstale":"2024-09-12T01:46:14.460598+0000","last_undegraded":"2024-09-12T01:46:14.460598+0000","last_fullsized":"2024-09-12T01:46:14.460598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:53.513977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463452+0000","last_change":"2024-09-12T01:44:35.763688+0000","last_active":"2024-09-12T01:46:14.463452+0000","last_peered":"2024-09-12T01:46:14.463452+0000","last_clean":"2024-09-12T01:46:14.463452+0000","last_became_active":"2024-09-12T01:44:35.763235+0000","last_became_peered":"2024-09-12T01:44:35.763235+0000","last_unstale":"2024-09-12T01:46:14.463452+0000","last_undegraded":"2024-09-12T01:46:14.463452+0000","last_fullsized":"2024-09-12T01:46:14.463452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:18:28.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463643+0000","last_change":"2024-09-12T01:44:35.758715+0000","last_active":"2024-09-12T01:46:14.463643+0000","last_peered":"2024-09-12T01:46:14.463643+0000","last_clean":"2024-09-12T01:46:14.463643+0000","last_became_active":"2024-09-12T01:44:35.758309+0000","last_became_peered":"2024-09-12T01:44:35.758309+0000","last_unstale":"2024-09-12T01:46:14.463643+0000","last_undegraded":"2024-09-12T01:46:14.463643+0000","last_fullsized":"2024-09-12T01:46:14.463643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:17.625289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457236+0000","last_change":"2024-09-12T01:44:35.759337+0000","last_active":"2024-09-12T01:46:14.457236+0000","last_peered":"2024-09-12T01:46:14.457236+0000","last_clean":"2024-09-12T01:46:14.457236+0000","last_became_active":"2024-09-12T01:44:35.759019+0000","last_became_peered":"2024-09-12T01:44:35.759019+0000","last_unstale":"2024-09-12T01:46:14.457236+0000","last_undegraded":"2024-09-12T01:46:14.457236+0000","last_fullsized":"2024-09-12T01:46:14.457236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:18.662544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:26.979885+0000","last_change":"2024-09-12T01:44:33.752541+0000","last_active":"2024-09-12T01:46:26.979885+0000","last_peered":"2024-09-12T01:46:26.979885+0000","last_clean":"2024-09-12T01:46:26.979885+0000","last_became_active":"2024-09-12T01:44:33.752133+0000","last_became_peered":"2024-09-12T01:44:33.752133+0000","last_unstale":"2024-09-12T01:46:26.979885+0000","last_undegraded":"2024-09-12T01:46:26.979885+0000","last_fullsized":"2024-09-12T01:46:26.979885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:51.371819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.461733+0000","last_change":"2024-09-12T01:44:35.763826+0000","last_active":"2024-09-12T01:46:14.461733+0000","last_peered":"2024-09-12T01:46:14.461733+0000","last_clean":"2024-09-12T01:46:14.461733+0000","last_became_active":"2024-09-12T01:44:35.763509+0000","last_became_peered":"2024-09-12T01:44:35.763509+0000","last_unstale":"2024-09-12T01:46:14.461733+0000","last_undegraded":"2024-09-12T01:46:14.461733+0000","last_fullsized":"2024-09-12T01:46:14.461733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:32.450790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818423+0000","last_change":"2024-09-12T01:44:33.755985+0000","last_active":"2024-09-12T01:44:45.818423+0000","last_peered":"2024-09-12T01:44:45.818423+0000","last_clean":"2024-09-12T01:44:45.818423+0000","last_became_active":"2024-09-12T01:44:33.755550+0000","last_became_peered":"2024-09-12T01:44:33.755550+0000","last_unstale":"2024-09-12T01:44:45.818423+0000","last_undegraded":"2024-09-12T01:44:45.818423+0000","last_fullsized":"2024-09-12T01:44:45.818423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:30.854174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077078+0000","last_change":"2024-09-12T01:44:33.743709+0000","last_active":"2024-09-12T01:44:46.077078+0000","last_peered":"2024-09-12T01:44:46.077078+0000","last_clean":"2024-09-12T01:44:46.077078+0000","last_became_active":"2024-09-12T01:44:33.743390+0000","last_became_peered":"2024-09-12T01:44:33.743390+0000","last_unstale":"2024-09-12T01:44:46.077078+0000","last_undegraded":"2024-09-12T01:44:46.077078+0000","last_fullsized":"2024-09-12T01:44:46.077078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:17.876451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.452061+0000","last_change":"2024-09-12T01:44:35.750827+0000","last_active":"2024-09-12T01:46:14.452061+0000","last_peered":"2024-09-12T01:46:14.452061+0000","last_clean":"2024-09-12T01:46:14.452061+0000","last_became_active":"2024-09-12T01:44:35.750551+0000","last_became_peered":"2024-09-12T01:44:35.750551+0000","last_unstale":"2024-09-12T01:46:14.452061+0000","last_undegraded":"2024-09-12T01:46:14.452061+0000","last_fullsized":"2024-09-12T01:46:14.452061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:06:50.673842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815771+0000","last_change":"2024-09-12T01:44:33.750264+0000","last_active":"2024-09-12T01:44:45.815771+0000","last_peered":"2024-09-12T01:44:45.815771+0000","last_clean":"2024-09-12T01:44:45.815771+0000","last_became_active":"2024-09-12T01:44:33.749935+0000","last_became_peered":"2024-09-12T01:44:33.749935+0000","last_unstale":"2024-09-12T01:44:45.815771+0000","last_undegraded":"2024-09-12T01:44:45.815771+0000","last_fullsized":"2024-09-12T01:44:45.815771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:31.509175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.466729+0000","last_change":"2024-09-12T01:44:35.754793+0000","last_active":"2024-09-12T01:46:14.466729+0000","last_peered":"2024-09-12T01:46:14.466729+0000","last_clean":"2024-09-12T01:46:14.466729+0000","last_became_active":"2024-09-12T01:44:35.754492+0000","last_became_peered":"2024-09-12T01:44:35.754492+0000","last_unstale":"2024-09-12T01:46:14.466729+0000","last_undegraded":"2024-09-12T01:46:14.466729+0000","last_fullsized":"2024-09-12T01:46:14.466729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:20.436299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:28.982131+0000","last_change":"2024-09-12T01:44:33.753599+0000","last_active":"2024-09-12T01:46:28.982131+0000","last_peered":"2024-09-12T01:46:28.982131+0000","last_clean":"2024-09-12T01:46:28.982131+0000","last_became_active":"2024-09-12T01:44:33.753229+0000","last_became_peered":"2024-09-12T01:44:33.753229+0000","last_unstale":"2024-09-12T01:46:28.982131+0000","last_undegraded":"2024-09-12T01:46:28.982131+0000","last_fullsized":"2024-09-12T01:46:28.982131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:03.987013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473143+0000","last_change":"2024-09-12T01:44:35.765340+0000","last_active":"2024-09-12T01:46:14.473143+0000","last_peered":"2024-09-12T01:46:14.473143+0000","last_clean":"2024-09-12T01:46:14.473143+0000","last_became_active":"2024-09-12T01:44:35.764093+0000","last_became_peered":"2024-09-12T01:44:35.764093+0000","last_unstale":"2024-09-12T01:46:14.473143+0000","last_undegraded":"2024-09-12T01:46:14.473143+0000","last_fullsized":"2024-09-12T01:46:14.473143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:51.144263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818170+0000","last_change":"2024-09-12T01:44:33.752522+0000","last_active":"2024-09-12T01:44:45.818170+0000","last_peered":"2024-09-12T01:44:45.818170+0000","last_clean":"2024-09-12T01:44:45.818170+0000","last_became_active":"2024-09-12T01:44:33.752090+0000","last_became_peered":"2024-09-12T01:44:33.752090+0000","last_unstale":"2024-09-12T01:44:45.818170+0000","last_undegraded":"2024-09-12T01:44:45.818170+0000","last_fullsized":"2024-09-12T01:44:45.818170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:38:52.404453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.460971+0000","last_change":"2024-09-12T01:44:35.764346+0000","last_active":"2024-09-12T01:46:14.460971+0000","last_peered":"2024-09-12T01:46:14.460971+0000","last_clean":"2024-09-12T01:46:14.460971+0000","last_became_active":"2024-09-12T01:44:35.763840+0000","last_became_peered":"2024-09-12T01:44:35.763840+0000","last_unstale":"2024-09-12T01:46:14.460971+0000","last_undegraded":"2024-09-12T01:46:14.460971+0000","last_fullsized":"2024-09-12T01:46:14.460971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:20.848867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816682+0000","last_change":"2024-09-12T01:44:33.785497+0000","last_active":"2024-09-12T01:44:45.816682+0000","last_peered":"2024-09-12T01:44:45.816682+0000","last_clean":"2024-09-12T01:44:45.816682+0000","last_became_active":"2024-09-12T01:44:33.785235+0000","last_became_peered":"2024-09-12T01:44:33.785235+0000","last_unstale":"2024-09-12T01:44:45.816682+0000","last_undegraded":"2024-09-12T01:44:45.816682+0000","last_fullsized":"2024-09-12T01:44:45.816682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:28.924363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473209+0000","last_change":"2024-09-12T01:44:35.767093+0000","last_active":"2024-09-12T01:46:14.473209+0000","last_peered":"2024-09-12T01:46:14.473209+0000","last_clean":"2024-09-12T01:46:14.473209+0000","last_became_active":"2024-09-12T01:44:35.766726+0000","last_became_peered":"2024-09-12T01:44:35.766726+0000","last_unstale":"2024-09-12T01:46:14.473209+0000","last_undegraded":"2024-09-12T01:46:14.473209+0000","last_fullsized":"2024-09-12T01:46:14.473209+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:14.872363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":278,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:29.983561+0000","last_change":"2024-09-12T01:44:33.752331+0000","last_active":"2024-09-12T01:46:29.983561+0000","last_peered":"2024-09-12T01:46:29.983561+0000","last_clean":"2024-09-12T01:46:29.983561+0000","last_became_active":"2024-09-12T01:44:33.752012+0000","last_became_peered":"2024-09-12T01:44:33.752012+0000","last_unstale":"2024-09-12T01:46:29.983561+0000","last_undegraded":"2024-09-12T01:46:29.983561+0000","last_fullsized":"2024-09-12T01:46:29.983561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:16.751999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":180,"num_write":36,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457287+0000","last_change":"2024-09-12T01:44:35.767092+0000","last_active":"2024-09-12T01:46:14.457287+0000","last_peered":"2024-09-12T01:46:14.457287+0000","last_clean":"2024-09-12T01:46:14.457287+0000","last_became_active":"2024-09-12T01:44:35.766680+0000","last_became_peered":"2024-09-12T01:44:35.766680+0000","last_unstale":"2024-09-12T01:46:14.457287+0000","last_undegraded":"2024-09-12T01:46:14.457287+0000","last_fullsized":"2024-09-12T01:46:14.457287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:11.777338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.829691+0000","last_change":"2024-09-12T01:44:45.829356+0000","last_active":"2024-09-12T01:44:45.829691+0000","last_peered":"2024-09-12T01:44:45.829691+0000","last_clean":"2024-09-12T01:44:45.829691+0000","last_became_active":"2024-09-12T01:44:45.821676+0000","last_became_peered":"2024-09-12T01:44:45.821676+0000","last_unstale":"2024-09-12T01:44:45.829691+0000","last_undegraded":"2024-09-12T01:44:45.829691+0000","last_fullsized":"2024-09-12T01:44:45.829691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:11.134394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470461+0000","last_change":"2024-09-12T01:44:35.760720+0000","last_active":"2024-09-12T01:46:14.470461+0000","last_peered":"2024-09-12T01:46:14.470461+0000","last_clean":"2024-09-12T01:46:14.470461+0000","last_became_active":"2024-09-12T01:44:35.760414+0000","last_became_peered":"2024-09-12T01:44:35.760414+0000","last_unstale":"2024-09-12T01:46:14.470461+0000","last_undegraded":"2024-09-12T01:46:14.470461+0000","last_fullsized":"2024-09-12T01:46:14.470461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:08.075214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815472+0000","last_change":"2024-09-12T01:44:33.749091+0000","last_active":"2024-09-12T01:44:45.815472+0000","last_peered":"2024-09-12T01:44:45.815472+0000","last_clean":"2024-09-12T01:44:45.815472+0000","last_became_active":"2024-09-12T01:44:33.748543+0000","last_became_peered":"2024-09-12T01:44:33.748543+0000","last_unstale":"2024-09-12T01:44:45.815472+0000","last_undegraded":"2024-09-12T01:44:45.815472+0000","last_fullsized":"2024-09-12T01:44:45.815472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:30.643750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.472069+0000","last_change":"2024-09-12T01:44:35.768122+0000","last_active":"2024-09-12T01:46:14.472069+0000","last_peered":"2024-09-12T01:46:14.472069+0000","last_clean":"2024-09-12T01:46:14.472069+0000","last_became_active":"2024-09-12T01:44:35.767826+0000","last_became_peered":"2024-09-12T01:44:35.767826+0000","last_unstale":"2024-09-12T01:46:14.472069+0000","last_undegraded":"2024-09-12T01:46:14.472069+0000","last_fullsized":"2024-09-12T01:46:14.472069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:06.297221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:25.978847+0000","last_change":"2024-09-12T01:44:33.749799+0000","last_active":"2024-09-12T01:46:25.978847+0000","last_peered":"2024-09-12T01:46:25.978847+0000","last_clean":"2024-09-12T01:46:25.978847+0000","last_became_active":"2024-09-12T01:44:33.749301+0000","last_became_peered":"2024-09-12T01:44:33.749301+0000","last_unstale":"2024-09-12T01:46:25.978847+0000","last_undegraded":"2024-09-12T01:46:25.978847+0000","last_fullsized":"2024-09-12T01:46:25.978847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.304896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468634+0000","last_change":"2024-09-12T01:44:35.760789+0000","last_active":"2024-09-12T01:46:14.468634+0000","last_peered":"2024-09-12T01:46:14.468634+0000","last_clean":"2024-09-12T01:46:14.468634+0000","last_became_active":"2024-09-12T01:44:35.760422+0000","last_became_peered":"2024-09-12T01:44:35.760422+0000","last_unstale":"2024-09-12T01:46:14.468634+0000","last_undegraded":"2024-09-12T01:46:14.468634+0000","last_fullsized":"2024-09-12T01:46:14.468634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:51.924495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819329+0000","last_change":"2024-09-12T01:44:33.786495+0000","last_active":"2024-09-12T01:44:45.819329+0000","last_peered":"2024-09-12T01:44:45.819329+0000","last_clean":"2024-09-12T01:44:45.819329+0000","last_became_active":"2024-09-12T01:44:33.785862+0000","last_became_peered":"2024-09-12T01:44:33.785862+0000","last_unstale":"2024-09-12T01:44:45.819329+0000","last_undegraded":"2024-09-12T01:44:45.819329+0000","last_fullsized":"2024-09-12T01:44:45.819329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:44.273644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469018+0000","last_change":"2024-09-12T01:44:45.822535+0000","last_active":"2024-09-12T01:46:14.469018+0000","last_peered":"2024-09-12T01:46:14.469018+0000","last_clean":"2024-09-12T01:46:14.469018+0000","last_became_active":"2024-09-12T01:44:45.822187+0000","last_became_peered":"2024-09-12T01:44:45.822187+0000","last_unstale":"2024-09-12T01:46:14.469018+0000","last_undegraded":"2024-09-12T01:46:14.469018+0000","last_fullsized":"2024-09-12T01:46:14.469018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:29.172788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:47.816560+0000","last_change":"2024-09-12T01:44:33.747651+0000","last_active":"2024-09-12T01:44:47.816560+0000","last_peered":"2024-09-12T01:44:47.816560+0000","last_clean":"2024-09-12T01:44:47.816560+0000","last_became_active":"2024-09-12T01:44:33.747010+0000","last_became_peered":"2024-09-12T01:44:33.747010+0000","last_unstale":"2024-09-12T01:44:47.816560+0000","last_undegraded":"2024-09-12T01:44:47.816560+0000","last_fullsized":"2024-09-12T01:44:47.816560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.215448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.458089+0000","last_change":"2024-09-12T01:44:35.763519+0000","last_active":"2024-09-12T01:46:14.458089+0000","last_peered":"2024-09-12T01:46:14.458089+0000","last_clean":"2024-09-12T01:46:14.458089+0000","last_became_active":"2024-09-12T01:44:35.763173+0000","last_became_peered":"2024-09-12T01:44:35.763173+0000","last_unstale":"2024-09-12T01:46:14.458089+0000","last_undegraded":"2024-09-12T01:46:14.458089+0000","last_fullsized":"2024-09-12T01:46:14.458089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:05.905154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.077271+0000","last_change":"2024-09-12T01:44:33.749679+0000","last_active":"2024-09-12T01:44:46.077271+0000","last_peered":"2024-09-12T01:44:46.077271+0000","last_clean":"2024-09-12T01:44:46.077271+0000","last_became_active":"2024-09-12T01:44:33.749337+0000","last_became_peered":"2024-09-12T01:44:33.749337+0000","last_unstale":"2024-09-12T01:44:46.077271+0000","last_undegraded":"2024-09-12T01:44:46.077271+0000","last_fullsized":"2024-09-12T01:44:46.077271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:31.290490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.467837+0000","last_change":"2024-09-12T01:44:35.761414+0000","last_active":"2024-09-12T01:46:14.467837+0000","last_peered":"2024-09-12T01:46:14.467837+0000","last_clean":"2024-09-12T01:46:14.467837+0000","last_became_active":"2024-09-12T01:44:35.760783+0000","last_became_peered":"2024-09-12T01:44:35.760783+0000","last_unstale":"2024-09-12T01:46:14.467837+0000","last_undegraded":"2024-09-12T01:46:14.467837+0000","last_fullsized":"2024-09-12T01:46:14.467837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:42.850107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816726+0000","last_change":"2024-09-12T01:44:33.752013+0000","last_active":"2024-09-12T01:44:45.816726+0000","last_peered":"2024-09-12T01:44:45.816726+0000","last_clean":"2024-09-12T01:44:45.816726+0000","last_became_active":"2024-09-12T01:44:33.751614+0000","last_became_peered":"2024-09-12T01:44:33.751614+0000","last_unstale":"2024-09-12T01:44:45.816726+0000","last_undegraded":"2024-09-12T01:44:45.816726+0000","last_fullsized":"2024-09-12T01:44:45.816726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:30.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465783+0000","last_change":"2024-09-12T01:44:45.819405+0000","last_active":"2024-09-12T01:46:14.465783+0000","last_peered":"2024-09-12T01:46:14.465783+0000","last_clean":"2024-09-12T01:46:14.465783+0000","last_became_active":"2024-09-12T01:44:45.819030+0000","last_became_peered":"2024-09-12T01:44:45.819030+0000","last_unstale":"2024-09-12T01:46:14.465783+0000","last_undegraded":"2024-09-12T01:46:14.465783+0000","last_fullsized":"2024-09-12T01:46:14.465783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:29.973416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821453+0000","last_change":"2024-09-12T01:44:34.054542+0000","last_active":"2024-09-12T01:44:45.821453+0000","last_peered":"2024-09-12T01:44:45.821453+0000","last_clean":"2024-09-12T01:44:45.821453+0000","last_became_active":"2024-09-12T01:44:34.054202+0000","last_became_peered":"2024-09-12T01:44:34.054202+0000","last_unstale":"2024-09-12T01:44:45.821453+0000","last_undegraded":"2024-09-12T01:44:45.821453+0000","last_fullsized":"2024-09-12T01:44:45.821453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:39:50.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473700+0000","last_change":"2024-09-12T01:44:35.761352+0000","last_active":"2024-09-12T01:46:14.473700+0000","last_peered":"2024-09-12T01:46:14.473700+0000","last_clean":"2024-09-12T01:46:14.473700+0000","last_became_active":"2024-09-12T01:44:35.761005+0000","last_became_peered":"2024-09-12T01:44:35.761005+0000","last_unstale":"2024-09-12T01:46:14.473700+0000","last_undegraded":"2024-09-12T01:46:14.473700+0000","last_fullsized":"2024-09-12T01:46:14.473700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:31.574943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818582+0000","last_change":"2024-09-12T01:43:22.746799+0000","last_active":"2024-09-12T01:44:45.818582+0000","last_peered":"2024-09-12T01:44:45.818582+0000","last_clean":"2024-09-12T01:44:45.818582+0000","last_became_active":"2024-09-12T01:43:22.746321+0000","last_became_peered":"2024-09-12T01:43:22.746321+0000","last_unstale":"2024-09-12T01:44:45.818582+0000","last_undegraded":"2024-09-12T01:44:45.818582+0000","last_fullsized":"2024-09-12T01:44:45.818582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:39:59.329065+0000","last_clean_scrub_stamp":"2024-09-12T01:39:59.329065+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:26.820603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.835768+0000","last_change":"2024-09-12T01:44:33.747401+0000","last_active":"2024-09-12T01:44:46.835768+0000","last_peered":"2024-09-12T01:44:46.835768+0000","last_clean":"2024-09-12T01:44:46.835768+0000","last_became_active":"2024-09-12T01:44:33.746847+0000","last_became_peered":"2024-09-12T01:44:33.746847+0000","last_unstale":"2024-09-12T01:44:46.835768+0000","last_undegraded":"2024-09-12T01:44:46.835768+0000","last_fullsized":"2024-09-12T01:44:46.835768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:18.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.464071+0000","last_change":"2024-09-12T01:44:35.759168+0000","last_active":"2024-09-12T01:46:14.464071+0000","last_peered":"2024-09-12T01:46:14.464071+0000","last_clean":"2024-09-12T01:46:14.464071+0000","last_became_active":"2024-09-12T01:44:35.758490+0000","last_became_peered":"2024-09-12T01:44:35.758490+0000","last_unstale":"2024-09-12T01:46:14.464071+0000","last_undegraded":"2024-09-12T01:46:14.464071+0000","last_fullsized":"2024-09-12T01:46:14.464071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:34.492084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818361+0000","last_change":"2024-09-12T01:44:33.753337+0000","last_active":"2024-09-12T01:44:45.818361+0000","last_peered":"2024-09-12T01:44:45.818361+0000","last_clean":"2024-09-12T01:44:45.818361+0000","last_became_active":"2024-09-12T01:44:33.752980+0000","last_became_peered":"2024-09-12T01:44:33.752980+0000","last_unstale":"2024-09-12T01:44:45.818361+0000","last_undegraded":"2024-09-12T01:44:45.818361+0000","last_fullsized":"2024-09-12T01:44:45.818361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:58.873914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449016+0000","last_change":"2024-09-12T01:44:35.758295+0000","last_active":"2024-09-12T01:46:14.449016+0000","last_peered":"2024-09-12T01:46:14.449016+0000","last_clean":"2024-09-12T01:46:14.449016+0000","last_became_active":"2024-09-12T01:44:35.757979+0000","last_became_peered":"2024-09-12T01:44:35.757979+0000","last_unstale":"2024-09-12T01:46:14.449016+0000","last_undegraded":"2024-09-12T01:46:14.449016+0000","last_fullsized":"2024-09-12T01:46:14.449016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:51.629649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:26.979692+0000","last_change":"2024-09-12T01:44:33.784822+0000","last_active":"2024-09-12T01:46:26.979692+0000","last_peered":"2024-09-12T01:46:26.979692+0000","last_clean":"2024-09-12T01:46:26.979692+0000","last_became_active":"2024-09-12T01:44:33.784517+0000","last_became_peered":"2024-09-12T01:44:33.784517+0000","last_unstale":"2024-09-12T01:46:26.979692+0000","last_undegraded":"2024-09-12T01:46:26.979692+0000","last_fullsized":"2024-09-12T01:46:26.979692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:23.094164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468646+0000","last_change":"2024-09-12T01:44:45.824468+0000","last_active":"2024-09-12T01:46:14.468646+0000","last_peered":"2024-09-12T01:46:14.468646+0000","last_clean":"2024-09-12T01:46:14.468646+0000","last_became_active":"2024-09-12T01:44:45.824103+0000","last_became_peered":"2024-09-12T01:44:45.824103+0000","last_unstale":"2024-09-12T01:46:14.468646+0000","last_undegraded":"2024-09-12T01:46:14.468646+0000","last_fullsized":"2024-09-12T01:46:14.468646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:45.884834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.818308+0000","last_change":"2024-09-12T01:44:33.748895+0000","last_active":"2024-09-12T01:44:45.818308+0000","last_peered":"2024-09-12T01:44:45.818308+0000","last_clean":"2024-09-12T01:44:45.818308+0000","last_became_active":"2024-09-12T01:44:33.748523+0000","last_became_peered":"2024-09-12T01:44:33.748523+0000","last_unstale":"2024-09-12T01:44:45.818308+0000","last_undegraded":"2024-09-12T01:44:45.818308+0000","last_fullsized":"2024-09-12T01:44:45.818308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:16.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.450592+0000","last_change":"2024-09-12T01:44:35.761582+0000","last_active":"2024-09-12T01:46:14.450592+0000","last_peered":"2024-09-12T01:46:14.450592+0000","last_clean":"2024-09-12T01:46:14.450592+0000","last_became_active":"2024-09-12T01:44:35.761260+0000","last_became_peered":"2024-09-12T01:44:35.761260+0000","last_unstale":"2024-09-12T01:46:14.450592+0000","last_undegraded":"2024-09-12T01:46:14.450592+0000","last_fullsized":"2024-09-12T01:46:14.450592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:01.361605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.816583+0000","last_change":"2024-09-12T01:44:33.786279+0000","last_active":"2024-09-12T01:44:45.816583+0000","last_peered":"2024-09-12T01:44:45.816583+0000","last_clean":"2024-09-12T01:44:45.816583+0000","last_became_active":"2024-09-12T01:44:33.785902+0000","last_became_peered":"2024-09-12T01:44:33.785902+0000","last_unstale":"2024-09-12T01:44:45.816583+0000","last_undegraded":"2024-09-12T01:44:45.816583+0000","last_fullsized":"2024-09-12T01:44:45.816583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:29.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.468594+0000","last_change":"2024-09-12T01:44:35.758535+0000","last_active":"2024-09-12T01:46:14.468594+0000","last_peered":"2024-09-12T01:46:14.468594+0000","last_clean":"2024-09-12T01:46:14.468594+0000","last_became_active":"2024-09-12T01:44:35.758222+0000","last_became_peered":"2024-09-12T01:44:35.758222+0000","last_unstale":"2024-09-12T01:46:14.468594+0000","last_undegraded":"2024-09-12T01:46:14.468594+0000","last_fullsized":"2024-09-12T01:46:14.468594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:16.388047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820465+0000","last_change":"2024-09-12T01:44:33.786243+0000","last_active":"2024-09-12T01:44:45.820465+0000","last_peered":"2024-09-12T01:44:45.820465+0000","last_clean":"2024-09-12T01:44:45.820465+0000","last_became_active":"2024-09-12T01:44:33.785873+0000","last_became_peered":"2024-09-12T01:44:33.785873+0000","last_unstale":"2024-09-12T01:44:45.820465+0000","last_undegraded":"2024-09-12T01:44:45.820465+0000","last_fullsized":"2024-09-12T01:44:45.820465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:21.338967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.449242+0000","last_change":"2024-09-12T01:44:35.761617+0000","last_active":"2024-09-12T01:46:14.449242+0000","last_peered":"2024-09-12T01:46:14.449242+0000","last_clean":"2024-09-12T01:46:14.449242+0000","last_became_active":"2024-09-12T01:44:35.761325+0000","last_became_peered":"2024-09-12T01:44:35.761325+0000","last_unstale":"2024-09-12T01:46:14.449242+0000","last_undegraded":"2024-09-12T01:46:14.449242+0000","last_fullsized":"2024-09-12T01:46:14.449242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:40.704127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:18.793448+0000","last_change":"2024-09-12T01:44:33.751104+0000","last_active":"2024-09-12T01:46:18.793448+0000","last_peered":"2024-09-12T01:46:18.793448+0000","last_clean":"2024-09-12T01:46:18.793448+0000","last_became_active":"2024-09-12T01:44:33.750761+0000","last_became_peered":"2024-09-12T01:44:33.750761+0000","last_unstale":"2024-09-12T01:46:18.793448+0000","last_undegraded":"2024-09-12T01:46:18.793448+0000","last_fullsized":"2024-09-12T01:46:18.793448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:56:23.284751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.465155+0000","last_change":"2024-09-12T01:44:35.758738+0000","last_active":"2024-09-12T01:46:14.465155+0000","last_peered":"2024-09-12T01:46:14.465155+0000","last_clean":"2024-09-12T01:46:14.465155+0000","last_became_active":"2024-09-12T01:44:35.758385+0000","last_became_peered":"2024-09-12T01:44:35.758385+0000","last_unstale":"2024-09-12T01:46:14.465155+0000","last_undegraded":"2024-09-12T01:46:14.465155+0000","last_fullsized":"2024-09-12T01:46:14.465155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:00.468993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.815866+0000","last_change":"2024-09-12T01:44:34.054760+0000","last_active":"2024-09-12T01:44:45.815866+0000","last_peered":"2024-09-12T01:44:45.815866+0000","last_clean":"2024-09-12T01:44:45.815866+0000","last_became_active":"2024-09-12T01:44:34.054385+0000","last_became_peered":"2024-09-12T01:44:34.054385+0000","last_unstale":"2024-09-12T01:44:45.815866+0000","last_undegraded":"2024-09-12T01:44:45.815866+0000","last_fullsized":"2024-09-12T01:44:45.815866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:08.747141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.457998+0000","last_change":"2024-09-12T01:44:35.763415+0000","last_active":"2024-09-12T01:46:14.457998+0000","last_peered":"2024-09-12T01:46:14.457998+0000","last_clean":"2024-09-12T01:46:14.457998+0000","last_became_active":"2024-09-12T01:44:35.763130+0000","last_became_peered":"2024-09-12T01:44:35.763130+0000","last_unstale":"2024-09-12T01:46:14.457998+0000","last_undegraded":"2024-09-12T01:46:14.457998+0000","last_fullsized":"2024-09-12T01:46:14.457998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:10.013344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820592+0000","last_change":"2024-09-12T01:44:33.786072+0000","last_active":"2024-09-12T01:44:45.820592+0000","last_peered":"2024-09-12T01:44:45.820592+0000","last_clean":"2024-09-12T01:44:45.820592+0000","last_became_active":"2024-09-12T01:44:33.785674+0000","last_became_peered":"2024-09-12T01:44:33.785674+0000","last_unstale":"2024-09-12T01:44:45.820592+0000","last_undegraded":"2024-09-12T01:44:45.820592+0000","last_fullsized":"2024-09-12T01:44:45.820592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:07.724084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.440457+0000","last_change":"2024-09-12T01:44:35.761373+0000","last_active":"2024-09-12T01:46:14.440457+0000","last_peered":"2024-09-12T01:46:14.440457+0000","last_clean":"2024-09-12T01:46:14.440457+0000","last_became_active":"2024-09-12T01:44:35.760777+0000","last_became_peered":"2024-09-12T01:44:35.760777+0000","last_unstale":"2024-09-12T01:46:14.440457+0000","last_undegraded":"2024-09-12T01:46:14.440457+0000","last_fullsized":"2024-09-12T01:46:14.440457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:26:33.733842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821575+0000","last_change":"2024-09-12T01:44:33.751087+0000","last_active":"2024-09-12T01:44:45.821575+0000","last_peered":"2024-09-12T01:44:45.821575+0000","last_clean":"2024-09-12T01:44:45.821575+0000","last_became_active":"2024-09-12T01:44:33.750763+0000","last_became_peered":"2024-09-12T01:44:33.750763+0000","last_unstale":"2024-09-12T01:44:45.821575+0000","last_undegraded":"2024-09-12T01:44:45.821575+0000","last_fullsized":"2024-09-12T01:44:45.821575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.968535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.463449+0000","last_change":"2024-09-12T01:44:35.756192+0000","last_active":"2024-09-12T01:46:14.463449+0000","last_peered":"2024-09-12T01:46:14.463449+0000","last_clean":"2024-09-12T01:46:14.463449+0000","last_became_active":"2024-09-12T01:44:35.755875+0000","last_became_peered":"2024-09-12T01:44:35.755875+0000","last_unstale":"2024-09-12T01:46:14.463449+0000","last_undegraded":"2024-09-12T01:46:14.463449+0000","last_fullsized":"2024-09-12T01:46:14.463449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:43.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821035+0000","last_change":"2024-09-12T01:44:33.749546+0000","last_active":"2024-09-12T01:44:45.821035+0000","last_peered":"2024-09-12T01:44:45.821035+0000","last_clean":"2024-09-12T01:44:45.821035+0000","last_became_active":"2024-09-12T01:44:33.749154+0000","last_became_peered":"2024-09-12T01:44:33.749154+0000","last_unstale":"2024-09-12T01:44:45.821035+0000","last_undegraded":"2024-09-12T01:44:45.821035+0000","last_fullsized":"2024-09-12T01:44:45.821035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:50:46.185074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.470639+0000","last_change":"2024-09-12T01:44:35.756974+0000","last_active":"2024-09-12T01:46:14.470639+0000","last_peered":"2024-09-12T01:46:14.470639+0000","last_clean":"2024-09-12T01:46:14.470639+0000","last_became_active":"2024-09-12T01:44:35.756654+0000","last_became_peered":"2024-09-12T01:44:35.756654+0000","last_unstale":"2024-09-12T01:46:14.470639+0000","last_undegraded":"2024-09-12T01:46:14.470639+0000","last_fullsized":"2024-09-12T01:46:14.470639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:06:32.518925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.819813+0000","last_change":"2024-09-12T01:44:33.755012+0000","last_active":"2024-09-12T01:44:45.819813+0000","last_peered":"2024-09-12T01:44:45.819813+0000","last_clean":"2024-09-12T01:44:45.819813+0000","last_became_active":"2024-09-12T01:44:33.754279+0000","last_became_peered":"2024-09-12T01:44:33.754279+0000","last_unstale":"2024-09-12T01:44:45.819813+0000","last_undegraded":"2024-09-12T01:44:45.819813+0000","last_fullsized":"2024-09-12T01:44:45.819813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:32.249679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473199+0000","last_change":"2024-09-12T01:44:35.760916+0000","last_active":"2024-09-12T01:46:14.473199+0000","last_peered":"2024-09-12T01:46:14.473199+0000","last_clean":"2024-09-12T01:46:14.473199+0000","last_became_active":"2024-09-12T01:44:35.759979+0000","last_became_peered":"2024-09-12T01:44:35.759979+0000","last_unstale":"2024-09-12T01:46:14.473199+0000","last_undegraded":"2024-09-12T01:46:14.473199+0000","last_fullsized":"2024-09-12T01:46:14.473199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:15.061540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.469618+0000","last_change":"2024-09-12T01:44:35.767487+0000","last_active":"2024-09-12T01:46:14.469618+0000","last_peered":"2024-09-12T01:46:14.469618+0000","last_clean":"2024-09-12T01:46:14.469618+0000","last_became_active":"2024-09-12T01:44:35.767162+0000","last_became_peered":"2024-09-12T01:44:35.767162+0000","last_unstale":"2024-09-12T01:46:14.469618+0000","last_undegraded":"2024-09-12T01:46:14.469618+0000","last_fullsized":"2024-09-12T01:46:14.469618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:01.113322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.831580+0000","last_change":"2024-09-12T01:44:45.831580+0000","last_active":"2024-09-12T01:44:45.831580+0000","last_peered":"2024-09-12T01:44:45.831580+0000","last_clean":"2024-09-12T01:44:45.831580+0000","last_became_active":"2024-09-12T01:44:45.821864+0000","last_became_peered":"2024-09-12T01:44:45.821864+0000","last_unstale":"2024-09-12T01:44:45.831580+0000","last_undegraded":"2024-09-12T01:44:45.831580+0000","last_fullsized":"2024-09-12T01:44:45.831580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:12:09.473405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.473089+0000","last_change":"2024-09-12T01:44:35.765012+0000","last_active":"2024-09-12T01:46:14.473089+0000","last_peered":"2024-09-12T01:46:14.473089+0000","last_clean":"2024-09-12T01:46:14.473089+0000","last_became_active":"2024-09-12T01:44:35.763639+0000","last_became_peered":"2024-09-12T01:44:35.763639+0000","last_unstale":"2024-09-12T01:46:14.473089+0000","last_undegraded":"2024-09-12T01:46:14.473089+0000","last_fullsized":"2024-09-12T01:46:14.473089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:09.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.820370+0000","last_change":"2024-09-12T01:44:33.754478+0000","last_active":"2024-09-12T01:44:45.820370+0000","last_peered":"2024-09-12T01:44:45.820370+0000","last_clean":"2024-09-12T01:44:45.820370+0000","last_became_active":"2024-09-12T01:44:33.754121+0000","last_became_peered":"2024-09-12T01:44:33.754121+0000","last_unstale":"2024-09-12T01:44:45.820370+0000","last_undegraded":"2024-09-12T01:44:45.820370+0000","last_fullsized":"2024-09-12T01:44:45.820370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:46.649374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.454649+0000","last_change":"2024-09-12T01:44:35.759467+0000","last_active":"2024-09-12T01:46:14.454649+0000","last_peered":"2024-09-12T01:46:14.454649+0000","last_clean":"2024-09-12T01:46:14.454649+0000","last_became_active":"2024-09-12T01:44:35.759106+0000","last_became_peered":"2024-09-12T01:44:35.759106+0000","last_unstale":"2024-09-12T01:46:14.454649+0000","last_undegraded":"2024-09-12T01:46:14.454649+0000","last_fullsized":"2024-09-12T01:46:14.454649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:05.196249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:45.821067+0000","last_change":"2024-09-12T01:44:33.785282+0000","last_active":"2024-09-12T01:44:45.821067+0000","last_peered":"2024-09-12T01:44:45.821067+0000","last_clean":"2024-09-12T01:44:45.821067+0000","last_became_active":"2024-09-12T01:44:33.785012+0000","last_became_peered":"2024-09-12T01:44:33.785012+0000","last_unstale":"2024-09-12T01:44:45.821067+0000","last_undegraded":"2024-09-12T01:44:45.821067+0000","last_fullsized":"2024-09-12T01:44:45.821067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:48:57.158407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:46:14.459700+0000","last_change":"2024-09-12T01:44:35.761581+0000","last_active":"2024-09-12T01:46:14.459700+0000","last_peered":"2024-09-12T01:46:14.459700+0000","last_clean":"2024-09-12T01:46:14.459700+0000","last_became_active":"2024-09-12T01:44:35.760103+0000","last_became_peered":"2024-09-12T01:44:35.760103+0000","last_unstale":"2024-09-12T01:46:14.459700+0000","last_undegraded":"2024-09-12T01:46:14.459700+0000","last_fullsized":"2024-09-12T01:46:14.459700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:34.737495+0000","last_clean_scrub_stamp":"2024-09-12T01:44:34.737495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:38.839560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T01:44:46.841955+0000","last_change":"2024-09-12T01:44:33.748547+0000","last_active":"2024-09-12T01:44:46.841955+0000","last_peered":"2024-09-12T01:44:46.841955+0000","last_clean":"2024-09-12T01:44:46.841955+0000","last_became_active":"2024-09-12T01:44:33.748278+0000","last_became_peered":"2024-09-12T01:44:33.748278+0000","last_unstale":"2024-09-12T01:44:46.841955+0000","last_undegraded":"2024-09-12T01:44:46.841955+0000","last_fullsized":"2024-09-12T01:44:46.841955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T01:44:32.733830+0000","last_clean_scrub_stamp":"2024-09-12T01:44:32.733830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:30.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75132,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":652,"num_read_kb":408,"num_write":179,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":225396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247745,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437829,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1100798,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438148,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":437951,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1030417,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":437420,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029174,"data_compressed":41625,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":438784,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437614,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":509124,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":507808,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438691,"data_compressed":5766,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34225,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35859,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T01:46:34.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1227730840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88100680 msgr2=0x7f9b88100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1227730840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88100680 0x7f9b88100ae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009900 tx=0x7f9b7c02a410 comp rx=0 tx=0).stop 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1227730840 shutdown_connections 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1227730840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88101020 0x7f9b8810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1227730840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88100680 0x7f9b88100ae0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1227730840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b88105220 0x7f9b88105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1227730840 >> 172.21.15.122:0/1227730840 conn(0x7f9b88077210 msgr2=0x7f9b88077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1227730840 shutdown_connections 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.447+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1227730840 wait complete. 2024-09-12T01:46:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 Processor -- start 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 -- start start 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88100680 0x7f9b881a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b88101020 0x7f9b881a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 0x7f9b8819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88111030 con 0x7f9b88105220 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88110eb0 con 0x7f9b88100680 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8fcee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881111b0 con 0x7f9b88101020 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 0x7f9b8819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 0x7f9b8819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52506/0 (socket says 172.21.15.122:52506) 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 -- 172.21.15.122:0/1262428373 learned_addr learned my addr 172.21.15.122:0/1262428373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8da8a640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88100680 0x7f9b881a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:35.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8d289640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b88101020 0x7f9b881a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 -- 172.21.15.122:0/1262428373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b88101020 msgr2=0x7f9b881a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b88101020 0x7f9b881a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 -- 172.21.15.122:0/1262428373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88100680 msgr2=0x7f9b881a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88100680 0x7f9b881a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 -- 172.21.15.122:0/1262428373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8819acc0 con 0x7f9b88105220 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8da8a640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88100680 0x7f9b881a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.448+0000 7f9b8e28b640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 0x7f9b8819a590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b970 tx=0x7f9b7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.449+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78019070 con 0x7f9b88105220 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.449+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7800cb40 con 0x7f9b88105220 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.449+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78007650 con 0x7f9b88105220 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.449+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8819afb0 con 0x7f9b88105220 2024-09-12T01:46:35.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.449+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b881a72a0 con 0x7f9b88105220 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.450+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b88100ae0 con 0x7f9b88105220 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.451+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f9b7801c070 con 0x7f9b88105220 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.451+0000 7f9b86ffd640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9b700a2c10 0x7f9b700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.452+0000 7f9b8da8a640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9b700a2c10 0x7f9b700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.452+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9b780cec10 con 0x7f9b88105220 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.452+0000 7f9b8da8a640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9b700a2c10 0x7f9b700a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b80006fd0 tx=0x7f9b80008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:35.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.455+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b7809bb20 con 0x7f9b88105220 2024-09-12T01:46:35.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.658+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b8819bac0 con 0x7f9b88105220 2024-09-12T01:46:35.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.659+0000 7f9b86ffd640 1 -- 172.21.15.122:0/1262428373 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f9b8819bac0 con 0x7f9b88105220 2024-09-12T01:46:35.659 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:35.659 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:46:35.660 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:46:35.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9b700a2c10 msgr2=0x7f9b700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9b700a2c10 0x7f9b700a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9b80006fd0 tx=0x7f9b80008040 comp rx=0 tx=0).stop 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 msgr2=0x7f9b8819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 0x7f9b8819a590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b970 tx=0x7f9b7800be30 comp rx=0 tx=0).stop 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 shutdown_connections 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9b700a2c10 0x7f9b700a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b88105220 0x7f9b8819a590 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b88101020 0x7f9b881a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 --2- 172.21.15.122:0/1262428373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b88100680 0x7f9b881a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.661+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 >> 172.21.15.122:0/1262428373 conn(0x7f9b88077210 msgr2=0x7f9b8810daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.662+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 shutdown_connections 2024-09-12T01:46:35.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:35.662+0000 7f9b8fcee640 1 -- 172.21.15.122:0/1262428373 wait complete. 2024-09-12T01:46:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:35 smithi183 ceph-mon[35127]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:35 smithi183 ceph-mon[35127]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T01:46:35.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:35 smithi122 ceph-mon[30925]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:35 smithi122 ceph-mon[30925]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T01:46:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:35 smithi151 ceph-mon[35229]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T01:46:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:35 smithi151 ceph-mon[35229]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T01:46:36.410 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:36.654 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:36.905 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:36 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1262428373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:36.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:36 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1262428373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:36 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1262428373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:37.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 -- 172.21.15.122:0/4199996903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 msgr2=0x7f6a601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:37.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 --2- 172.21.15.122:0/4199996903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601021c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009ca0 tx=0x7f6a5802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 -- 172.21.15.122:0/4199996903 shutdown_connections 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 --2- 172.21.15.122:0/4199996903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a60106f10 0x7f6a6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 --2- 172.21.15.122:0/4199996903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a60102790 0x7f6a601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 --2- 172.21.15.122:0/4199996903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601021c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 -- 172.21.15.122:0/4199996903 >> 172.21.15.122:0/4199996903 conn(0x7f6a600fd8c0 msgr2=0x7f6a600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 -- 172.21.15.122:0/4199996903 shutdown_connections 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.116+0000 7f6a64c12640 1 -- 172.21.15.122:0/4199996903 wait complete. 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 Processor -- start 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 -- start start 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a60102790 0x7f6a60198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:37.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a60106f10 0x7f6a6019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60112010 con 0x7f6a60101de0 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60111e90 con 0x7f6a60106f10 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a64c12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60112190 con 0x7f6a60102790 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52512/0 (socket says 172.21.15.122:52512) 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 -- 172.21.15.122:0/1665109191 learned_addr learned my addr 172.21.15.122:0/1665109191 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 -- 172.21.15.122:0/1665109191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a60102790 msgr2=0x7f6a60198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a60102790 0x7f6a60198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 -- 172.21.15.122:0/1665109191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a60106f10 msgr2=0x7f6a6019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a60106f10 0x7f6a6019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.117+0000 7f6a5e59c640 1 -- 172.21.15.122:0/1665109191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6019d560 con 0x7f6a60101de0 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.118+0000 7f6a5e59c640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601985b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097b0 tx=0x7f6a58031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:37.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.118+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5802fba0 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.118+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6019d7f0 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.118+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5802fd40 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.118+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5803b930 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.118+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a5130 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.119+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a60102240 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.120+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6a58030cc0 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.121+0000 7f6a577fe640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.121+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6a580f06a0 con 0x7f6a60101de0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.121+0000 7f6a5dd9b640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.121+0000 7f6a5dd9b640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a600ffde0 tx=0x7f6a48006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:37.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.125+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a58043330 con 0x7f6a60101de0 2024-09-12T01:46:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.327+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a60109b60 con 0x7f6a60101de0 2024-09-12T01:46:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.328+0000 7f6a577fe640 1 -- 172.21.15.122:0/1665109191 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f6a580bd570 con 0x7f6a60101de0 2024-09-12T01:46:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:37.328 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:46:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6a4c0a2c10 msgr2=0x7f6a4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a600ffde0 tx=0x7f6a48006c90 comp rx=0 tx=0).stop 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 msgr2=0x7f6a601985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601985b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097b0 tx=0x7f6a58031b10 comp rx=0 tx=0).stop 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 shutdown_connections 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a60106f10 0x7f6a6019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a60102790 0x7f6a60198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 --2- 172.21.15.122:0/1665109191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a60101de0 0x7f6a601985b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 >> 172.21.15.122:0/1665109191 conn(0x7f6a600fd8c0 msgr2=0x7f6a600fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 shutdown_connections 2024-09-12T01:46:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:37.330+0000 7f6a64c12640 1 -- 172.21.15.122:0/1665109191 wait complete. 2024-09-12T01:46:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:37 smithi183 ceph-mon[35127]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T01:46:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:37 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1665109191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:37.976 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:37 smithi122 ceph-mon[30925]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T01:46:37.977 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:37 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1665109191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:38.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:46:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:37 smithi151 ceph-mon[35229]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T01:46:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:37 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1665109191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:38.248 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:38.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 -- 172.21.15.122:0/2256623182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 msgr2=0x7fdcec10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:38.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/2256623182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec10eab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009ca0 tx=0x7fdce402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 -- 172.21.15.122:0/2256623182 shutdown_connections 2024-09-12T01:46:38.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/2256623182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec10eab0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/2256623182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec100cc0 0x7fdcec106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/2256623182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100310 0x7fdcec1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 -- 172.21.15.122:0/2256623182 >> 172.21.15.122:0/2256623182 conn(0x7fdcec077210 msgr2=0x7fdcec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 -- 172.21.15.122:0/2256623182 shutdown_connections 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.702+0000 7fdcf31bd640 1 -- 172.21.15.122:0/2256623182 wait complete. 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 Processor -- start 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 -- start start 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec100310 0x7fdcec073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100cc0 0x7fdcec06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec111f90 con 0x7fdcec106f30 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec111e10 con 0x7fdcec100310 2024-09-12T01:46:38.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf31bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec112110 con 0x7fdcec100cc0 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf175a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf175a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec06c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52518/0 (socket says 172.21.15.122:52518) 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf175a640 1 -- 172.21.15.122:0/1995906954 learned_addr learned my addr 172.21.15.122:0/1995906954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcebfff640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100cc0 0x7fdcec06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf0f59640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec100310 0x7fdcec073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf175a640 1 -- 172.21.15.122:0/1995906954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100cc0 msgr2=0x7fdcec06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf175a640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100cc0 0x7fdcec06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.703+0000 7fdcf175a640 1 -- 172.21.15.122:0/1995906954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec100310 msgr2=0x7fdcec073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdcf175a640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec100310 0x7fdcec073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdcf175a640 1 -- 172.21.15.122:0/1995906954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec06ca40 con 0x7fdcec106f30 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdcebfff640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100cc0 0x7fdcec06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdcf175a640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec06c220 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdce4031a20 tx=0x7fdce4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003c30 con 0x7fdcec106f30 2024-09-12T01:46:38.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1ad1a0 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce4003dd0 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce402fc40 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.704+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1ad730 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.705+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec1033d0 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.707+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fdce4004050 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.708+0000 7fdce9ffb640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcdc0a2c10 0x7fdcdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.708+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdce40f0470 con 0x7fdcec106f30 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.708+0000 7fdcf0f59640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcdc0a2c10 0x7fdcdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.708+0000 7fdcf0f59640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcdc0a2c10 0x7fdcdc0a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009bd0 tx=0x7fdce0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:38.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.711+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdce40bd410 con 0x7fdcec106f30 2024-09-12T01:46:38.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.881+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcec10a170 con 0x7fdcec106f30 2024-09-12T01:46:38.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.883+0000 7fdce9ffb640 1 -- 172.21.15.122:0/1995906954 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22783 (secure 0 0 0) 0x7fdce40bcd10 con 0x7fdcec106f30 2024-09-12T01:46:38.883 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:38.883 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":83,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T01:44:45.811494+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:46:38.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcdc0a2c10 msgr2=0x7fdcdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:38.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcdc0a2c10 0x7fdcdc0a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009bd0 tx=0x7fdce0005f90 comp rx=0 tx=0).stop 2024-09-12T01:46:38.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 msgr2=0x7fdcec06c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:38.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec06c220 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdce4031a20 tx=0x7fdce4031a50 comp rx=0 tx=0).stop 2024-09-12T01:46:38.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 shutdown_connections 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcdc0a2c10 0x7fdcdc0a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcec106f30 0x7fdcec06c220 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcec100cc0 0x7fdcec06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 --2- 172.21.15.122:0/1995906954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcec100310 0x7fdcec073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 >> 172.21.15.122:0/1995906954 conn(0x7fdcec077210 msgr2=0x7fdcec105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 shutdown_connections 2024-09-12T01:46:38.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:38.885+0000 7fdcf31bd640 1 -- 172.21.15.122:0/1995906954 wait complete. 2024-09-12T01:46:39.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T01:46:39.844 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:39 smithi183 ceph-mon[35127]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T01:46:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:39 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1995906954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:39.964 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:39 smithi122 ceph-mon[30925]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T01:46:39.964 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:39 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1995906954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:39 smithi151 ceph-mon[35229]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T01:46:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:39 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1995906954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 -- 172.21.15.122:0/758042146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641013e0 msgr2=0x7f2b64101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 --2- 172.21.15.122:0/758042146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641013e0 0x7f2b64101860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009900 tx=0x7f2b5802a410 comp rx=0 tx=0).stop 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 -- 172.21.15.122:0/758042146 shutdown_connections 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 --2- 172.21.15.122:0/758042146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b64101da0 0x7f2b64108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 --2- 172.21.15.122:0/758042146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641013e0 0x7f2b64101860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 --2- 172.21.15.122:0/758042146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641001f0 0x7f2b641005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 -- 172.21.15.122:0/758042146 >> 172.21.15.122:0/758042146 conn(0x7f2b640fb970 msgr2=0x7f2b640fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 -- 172.21.15.122:0/758042146 shutdown_connections 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 -- 172.21.15.122:0/758042146 wait complete. 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.289+0000 7f2b69549640 1 Processor -- start 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 -- start start 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 0x7f2b64104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641013e0 0x7f2b641027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b64101da0 0x7f2b64102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410b620 con 0x7f2b641001f0 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410b4a0 con 0x7f2b64101da0 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b69549640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410b7a0 con 0x7f2b641013e0 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 0x7f2b64104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 0x7f2b64104180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46842/0 (socket says 172.21.15.122:46842) 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 -- 172.21.15.122:0/3734229739 learned_addr learned my addr 172.21.15.122:0/3734229739 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b627fc640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641013e0 0x7f2b641027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 -- 172.21.15.122:0/3734229739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641013e0 msgr2=0x7f2b641027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641013e0 0x7f2b641027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 -- 172.21.15.122:0/3734229739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b64101da0 msgr2=0x7f2b64102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b64101da0 0x7f2b64102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 -- 172.21.15.122:0/3734229739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64103590 con 0x7f2b641001f0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b627fc640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641013e0 0x7f2b641027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.290+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 0x7f2b64104180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c970 tx=0x7f2b5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.291+0000 7f2b4bfff640 1 -- 172.21.15.122:0/3734229739 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54004220 con 0x7f2b641001f0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.291+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641a1f80 con 0x7f2b641001f0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.291+0000 7f2b4bfff640 1 -- 172.21.15.122:0/3734229739 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b540043c0 con 0x7f2b641001f0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.291+0000 7f2b4bfff640 1 -- 172.21.15.122:0/3734229739 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400f450 con 0x7f2b641001f0 2024-09-12T01:46:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.291+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a23f0 con 0x7f2b641001f0 2024-09-12T01:46:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.293+0000 7f2b4bfff640 1 -- 172.21.15.122:0/3734229739 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2b540079b0 con 0x7f2b641001f0 2024-09-12T01:46:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.293+0000 7f2b4bfff640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2b4c0a2cd0 0x7f2b4c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b627fc640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2b4c0a2cd0 0x7f2b4c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b4bfff640 1 -- 172.21.15.122:0/3734229739 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2b540cef80 con 0x7f2b641001f0 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b627fc640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2b4c0a2cd0 0x7f2b4c0a5190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b58000c00 tx=0x7f2b58009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 Processor -- start 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 -- start start 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da059a0 0x55f73da05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da0b370 0x55f73da0d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f73d9692f0 con 0x55f73da062e0 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f73d911270 con 0x55f73da059a0 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.294+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f73d9652b0 con 0x55f73da0b370 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da0ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da0ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46858/0 (socket says 172.21.15.122:46858) 2024-09-12T01:46:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 -- 172.21.15.122:0/685158037 learned_addr learned my addr 172.21.15.122:0/685158037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b637fe640 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da0b370 0x55f73da0d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da059a0 0x55f73da05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 -- 172.21.15.122:0/685158037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da0b370 msgr2=0x55f73da0d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da0b370 0x55f73da0d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 -- 172.21.15.122:0/685158037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da059a0 msgr2=0x55f73da05da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da059a0 0x55f73da05da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 -- 172.21.15.122:0/685158037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73d910340 con 0x55f73da062e0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b627fc640 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da0ada0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b58032fc0 tx=0x7f2b5803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bf3e88ee49e351 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b497fa640 1 -- 172.21.15.122:0/685158037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58035050 con 0x55f73da062e0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b497fa640 1 -- 172.21.15.122:0/685158037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5804c070 con 0x55f73da062e0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.295+0000 7f2b497fa640 1 -- 172.21.15.122:0/685158037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58004bc0 con 0x55f73da062e0 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/685158037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 msgr2=0x55f73da0ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da0ada0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b58032fc0 tx=0x7f2b5803b000 comp rx=0 tx=0).stop 2024-09-12T01:46:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/685158037 shutdown_connections 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da0b370 0x55f73da0d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da0ada0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/685158037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da059a0 0x55f73da05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/685158037 >> 172.21.15.122:0/685158037 conn(0x55f73da01010 msgr2=0x55f73da01410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/685158037 shutdown_connections 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/685158037 wait complete. 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.296+0000 7f2b6c3ba740 1 -- start start 2024-09-12T01:46:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 Processor -- start 2024-09-12T01:46:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da10070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f73da0e2e0 con 0x55f73da0b370 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f73d8f2290 con 0x55f73da062e0 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f73da0e460 con 0x55f73da059a0 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b6c3ba740 10 client.0 _ll_register_callbacks cb 0x55f73d923140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da170b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46860/0 (socket says 172.21.15.122:46860) 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 -- 172.21.15.122:0/3749180632 learned_addr learned my addr 172.21.15.122:0/3749180632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b627fc640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da10070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 -- 172.21.15.122:0/3749180632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 msgr2=0x55f73da0fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 -- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 msgr2=0x55f73da10070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da10070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b637fe640 1 -- 172.21.15.122:0/3749180632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73d67c210 con 0x55f73da0b370 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.297+0000 7f2b627fc640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da10070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b637fe640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da170b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00ea90 tx=0x7f2b5c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef266621c7671c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b46ff5640 1 -- 172.21.15.122:0/3749180632 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00dd90 con 0x55f73da0b370 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b46ff5640 1 -- 172.21.15.122:0/3749180632 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2b5c004b20 con 0x55f73da0b370 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5804c070 con 0x55f73da0b370 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b46ff5640 1 -- 172.21.15.122:0/3749180632 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c004d20 con 0x55f73da0b370 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f73d748620 con 0x55f73da0b370 2024-09-12T01:46:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b46ff5640 1 -- 172.21.15.122:0/3749180632 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5804c070 con 0x55f73da0b370 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b46ff5640 1 -- 172.21.15.122:0/3749180632 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2b5c00b040 con 0x55f73da0b370 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 20 client.15297 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da1b870 0x55f73da1dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 4 client.15297 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.298+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73d976a20 con 0x55f73da1b870 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.299+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da1b870 0x55f73da1dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.299+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da1b870 0x55f73da1dc80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42102bd35f1aee5a server_cookie=4d94c6a58ea97284 in_seq=0 out_seq=0 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.299+0000 7f2b46ff5640 10 client.15297 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b46ff5640 1 -- 172.21.15.122:0/3749180632 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f73d976a20 con 0x55f73da1b870 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b46ff5640 10 client.15297 handle_command_reply: tid=0 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 client.15297 shutdown 2024-09-12T01:46:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 msgr2=0x55f73da170b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da170b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00ea90 tx=0x7f2b5c00ef50 comp rx=0 tx=0).stop 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 shutdown_connections 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da1b870 0x55f73da1dc80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da170b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da10070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b46ff5640 0 client.15297 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/3749180632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 >> 172.21.15.122:0/3749180632 conn(0x55f73da01010 msgr2=0x55f73da02820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 shutdown_connections 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/3749180632 wait complete. 2024-09-12T01:46:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.300+0000 7f2b6c3ba740 20 client.15297 trim_cache size 0 max 16384 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 Processor -- start 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 -- start start 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da17070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da0b370 0x55f73da000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641adbd0 con 0x55f73da062e0 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ab4a0 con 0x55f73da0b370 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641a92d0 con 0x55f73da059a0 2024-09-12T01:46:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da17070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da17070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46876/0 (socket says 172.21.15.122:46876) 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 -- 172.21.15.122:0/210624690 learned_addr learned my addr 172.21.15.122:0/210624690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b637fe640 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da0b370 0x55f73da000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 -- 172.21.15.122:0/210624690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 msgr2=0x55f73da0e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 -- 172.21.15.122:0/210624690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da0b370 msgr2=0x55f73da000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.329+0000 7f2b627fc640 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da0b370 0x55f73da000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.330+0000 7f2b627fc640 1 -- 172.21.15.122:0/210624690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73da10210 con 0x55f73da062e0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.330+0000 7f2b627fc640 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da17070 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55f73da06d10 tx=0x7f2b58004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9182759fd21ccc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.330+0000 7f2b48ff9640 1 -- 172.21.15.122:0/210624690 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58035040 con 0x55f73da062e0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.330+0000 7f2b48ff9640 1 -- 172.21.15.122:0/210624690 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5804c070 con 0x55f73da062e0 2024-09-12T01:46:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.330+0000 7f2b48ff9640 1 -- 172.21.15.122:0/210624690 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580035f0 con 0x55f73da062e0 2024-09-12T01:46:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/210624690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 msgr2=0x55f73da17070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da17070 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55f73da06d10 tx=0x7f2b58004250 comp rx=0 tx=0).stop 2024-09-12T01:46:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/210624690 shutdown_connections 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da0b370 0x55f73da000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da062e0 0x55f73da17070 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/210624690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da0e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/210624690 >> 172.21.15.122:0/210624690 conn(0x55f73da01010 msgr2=0x55f73d9ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/210624690 shutdown_connections 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/210624690 wait complete. 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.331+0000 7f2b6c3ba740 1 -- start start 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 Processor -- start 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da14460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da15960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641a9960 con 0x55f73da0b370 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641aadd0 con 0x55f73da062e0 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641aa030 con 0x55f73da059a0 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b6c3ba740 10 client.0 _ll_register_callbacks cb 0x7f2b5c00a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da15960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da15960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46878/0 (socket says 172.21.15.122:46878) 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 -- 172.21.15.122:0/367658839 learned_addr learned my addr 172.21.15.122:0/367658839 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da14460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b627fc640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 -- 172.21.15.122:0/367658839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 msgr2=0x55f73da14460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da14460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 -- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 msgr2=0x55f73da149a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da149a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 -- 172.21.15.122:0/367658839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73da15f70 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b637fe640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da15960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c000f80 tx=0x7f2b5c004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d29d553d6441309 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.332+0000 7f2b46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b46ff5640 1 -- 172.21.15.122:0/367658839 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c026040 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b46ff5640 1 -- 172.21.15.122:0/367658839 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2b5c004b20 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5804c070 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b46ff5640 1 -- 172.21.15.122:0/367658839 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c0173e0 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f73da17280 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b46ff5640 1 -- 172.21.15.122:0/367658839 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5804c070 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b46ff5640 1 -- 172.21.15.122:0/367658839 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2b5c022410 con 0x55f73da0b370 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 20 client.15303 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da86b60 0x55f73da88f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 4 client.15303 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:46:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f73d970d70 con 0x55f73da86b60 2024-09-12T01:46:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.333+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da86b60 0x55f73da88f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.334+0000 7f2b46ff5640 10 client.15303 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.334+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da86b60 0x55f73da88f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df80ddd78cdb11a server_cookie=44f0566ee9b4d4c0 in_seq=0 out_seq=0 2024-09-12T01:46:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.344+0000 7f2b46ff5640 1 -- 172.21.15.122:0/367658839 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f73d970d70 con 0x55f73da86b60 2024-09-12T01:46:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.344+0000 7f2b46ff5640 10 client.15303 handle_command_reply: tid=0 2024-09-12T01:46:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.344+0000 7f2b6c3ba740 1 client.15303 shutdown 2024-09-12T01:46:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 msgr2=0x55f73da15960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da15960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c000f80 tx=0x7f2b5c004980 comp rx=0 tx=0).stop 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 shutdown_connections 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f73da86b60 0x55f73da88f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f73da0b370 0x55f73da15960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f73da062e0 0x55f73da149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b46ff5640 0 client.15303 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 --2- 172.21.15.122:0/367658839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73da059a0 0x55f73da14460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 >> 172.21.15.122:0/367658839 conn(0x55f73da01010 msgr2=0x55f73da001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 shutdown_connections 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 1 -- 172.21.15.122:0/367658839 wait complete. 2024-09-12T01:46:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b6c3ba740 20 client.15303 trim_cache size 0 max 16384 2024-09-12T01:46:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:46:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:46:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:46:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2b4c0a2cd0 msgr2=0x7f2b4c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b69549640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2b4c0a2cd0 0x7f2b4c0a5190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b58000c00 tx=0x7f2b58009560 comp rx=0 tx=0).stop 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 msgr2=0x7f2b64104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.345+0000 7f2b69549640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 0x7f2b64104180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c970 tx=0x7f2b5400ce30 comp rx=0 tx=0).stop 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 shutdown_connections 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2b4c0a2cd0 0x7f2b4c0a5190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b64101da0 0x7f2b64102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b641013e0 0x7f2b641027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 --2- 172.21.15.122:0/3734229739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641001f0 0x7f2b64104180 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 >> 172.21.15.122:0/3734229739 conn(0x7f2b640fb970 msgr2=0x7f2b64106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 shutdown_connections 2024-09-12T01:46:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:40.346+0000 7f2b69549640 1 -- 172.21.15.122:0/3734229739 wait complete. 2024-09-12T01:46:40.978 INFO:teuthology.orchestra.run.smithi122.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T01:46:40.979 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T01:46:40.979 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T01:46:40.979 DEBUG:tasks.quiescer:Yielding 2024-09-12T01:46:40.980 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T01:46:40.997 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T01:46:40.997 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T01:46:40.997 INFO:tasks.workunit:timeout=3h 2024-09-12T01:46:40.997 INFO:tasks.workunit:cleanup=True 2024-09-12T01:46:40.998 DEBUG:teuthology.orchestra.run.smithi122:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:40.998 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:46:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T01:46:41.015 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T01:46:41.015 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T01:46:41.015 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T01:46:41.016 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-12 01:44:54.396949303 +0000 2024-09-12T01:46:41.016 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-12 01:44:54.396949303 +0000 2024-09-12T01:46:41.016 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-12 01:46:18.388189878 +0000 2024-09-12T01:46:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-12 01:44:54.396949303 +0000 2024-09-12T01:46:41.016 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T01:46:41.016 DEBUG:teuthology.orchestra.run.smithi122:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T01:46:41.086 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T01:46:41.144 INFO:tasks.workunit.client.0.smithi122.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T01:46:41.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:41 smithi122 ceph-mon[30925]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T01:46:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:41 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:46:41.341 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:41 smithi151 ceph-mon[35229]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T01:46:41.341 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:41 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:46:41.366 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:41 smithi183 ceph-mon[35127]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T01:46:41.366 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:41 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:46:41.912 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:42.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:42 smithi151 ceph-mon[35229]: scrub summary: idle 2024-09-12T01:46:42.376 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:42 smithi122 ceph-mon[30925]: scrub summary: idle 2024-09-12T01:46:42.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:42 smithi183 ceph-mon[35127]: scrub summary: idle 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 -- 172.21.15.122:0/3502906271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 msgr2=0x7fea40103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 --2- 172.21.15.122:0/3502906271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea40103110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009930 tx=0x7fea3c02a420 comp rx=0 tx=0).stop 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 -- 172.21.15.122:0/3502906271 shutdown_connections 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 --2- 172.21.15.122:0/3502906271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 0x7fea4010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 --2- 172.21.15.122:0/3502906271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea401036e0 0x7fea40107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 --2- 172.21.15.122:0/3502906271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea40103110 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 -- 172.21.15.122:0/3502906271 >> 172.21.15.122:0/3502906271 conn(0x7fea400fc7d0 msgr2=0x7fea400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 -- 172.21.15.122:0/3502906271 shutdown_connections 2024-09-12T01:46:42.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.457+0000 7fea47f14640 1 -- 172.21.15.122:0/3502906271 wait complete. 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 Processor -- start 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 -- start start 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea401985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea401036e0 0x7fea40198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 0x7fea4019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea401120a0 con 0x7fea40102d30 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea40111f20 con 0x7fea401036e0 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea47f14640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea40112220 con 0x7fea401082a0 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea45cb0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea401985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea45cb0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea401985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46884/0 (socket says 172.21.15.122:46884) 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea45cb0640 1 -- 172.21.15.122:0/1064155652 learned_addr learned my addr 172.21.15.122:0/1064155652 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.458+0000 7fea454af640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea401036e0 0x7fea40198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea464b1640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 0x7fea4019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:42.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea45cb0640 1 -- 172.21.15.122:0/1064155652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 msgr2=0x7fea4019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea45cb0640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 0x7fea4019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea45cb0640 1 -- 172.21.15.122:0/1064155652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea401036e0 msgr2=0x7fea40198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea45cb0640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea401036e0 0x7fea40198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea45cb0640 1 -- 172.21.15.122:0/1064155652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4019d570 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea464b1640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 0x7fea4019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea45cb0640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea401985c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009230 tx=0x7fea3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c03f030 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c003ad0 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4019d800 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c035070 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.459+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea401a5140 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.460+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4006a850 con 0x7fea40102d30 2024-09-12T01:46:42.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.461+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fea3c042030 con 0x7fea40102d30 2024-09-12T01:46:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.462+0000 7fea36ffd640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea2c0a2c10 0x7fea2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.462+0000 7fea454af640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea2c0a2c10 0x7fea2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.462+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fea3c0efb00 con 0x7fea40102d30 2024-09-12T01:46:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.462+0000 7fea454af640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea2c0a2c10 0x7fea2c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea400fee00 tx=0x7fea3000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.465+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea3c0bc900 con 0x7fea40102d30 2024-09-12T01:46:42.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.684+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea40103110 con 0x7fea40102d30 2024-09-12T01:46:42.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.685+0000 7fea36ffd640 1 -- 172.21.15.122:0/1064155652 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7fea3c0bc200 con 0x7fea40102d30 2024-09-12T01:46:42.685 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:46:42.685 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:46:42.685 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:46:42.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea2c0a2c10 msgr2=0x7fea2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:42.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea2c0a2c10 0x7fea2c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea400fee00 tx=0x7fea3000b410 comp rx=0 tx=0).stop 2024-09-12T01:46:42.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 msgr2=0x7fea401985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:42.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea401985c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009230 tx=0x7fea3c004270 comp rx=0 tx=0).stop 2024-09-12T01:46:42.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 shutdown_connections 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fea2c0a2c10 0x7fea2c0a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea401082a0 0x7fea4019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea401036e0 0x7fea40198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 --2- 172.21.15.122:0/1064155652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea40102d30 0x7fea401985c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 >> 172.21.15.122:0/1064155652 conn(0x7fea400fc7d0 msgr2=0x7fea40105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 shutdown_connections 2024-09-12T01:46:42.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:42.687+0000 7fea47f14640 1 -- 172.21.15.122:0/1064155652 wait complete. 2024-09-12T01:46:43.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:43 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:43 smithi151 ceph-mon[35229]: pgmap v342: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T01:46:43.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:43 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1064155652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:43.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:43 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.433 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:43 smithi122 ceph-mon[30925]: pgmap v342: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T01:46:43.434 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:43 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1064155652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:43.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:43 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:43 smithi183 ceph-mon[35127]: pgmap v342: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T01:46:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:43 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1064155652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:46:43.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T01:44:36.280693+0000', 'modified': '2024-09-12T01:46:19.050885+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24436, 'mds_2': 24428}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/3975453954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 3975453954}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 3975453954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/2236567181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 2236567181}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 2236567181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1190566748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1190566748}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1190566748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.151:6837/3984907767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3984907767}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3984907767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6837/3258783936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3258783936}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3258783936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.151:6839/1184770501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 1184770501}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 1184770501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24452, 24436, 24428]} 2024-09-12T01:46:43.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T01:46:43.461 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T01:46:43.778 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:44.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2647089148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 msgr2=0x7f1e2010ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2647089148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e2010ba00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009ca0 tx=0x7f1e1002a950 comp rx=0 tx=0).stop 2024-09-12T01:46:44.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2647089148 shutdown_connections 2024-09-12T01:46:44.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2647089148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2010bfd0 0x7f1e2010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2647089148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e2010ba00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2647089148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2647089148 >> 172.21.15.122:0/2647089148 conn(0x7f1e200fc610 msgr2=0x7f1e200fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:44.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2647089148 shutdown_connections 2024-09-12T01:46:44.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.258+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2647089148 wait complete. 2024-09-12T01:46:44.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 Processor -- start 2024-09-12T01:46:44.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 -- start start 2024-09-12T01:46:44.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e201987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2010bfd0 0x7f1e2019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20111db0 con 0x7f1e20102bf0 2024-09-12T01:46:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20111c30 con 0x7f1e20103510 2024-09-12T01:46:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e25ba0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20111f30 con 0x7f1e2010bfd0 2024-09-12T01:46:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e24b9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e24b9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46892/0 (socket says 172.21.15.122:46892) 2024-09-12T01:46:44.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e24b9e640 1 -- 172.21.15.122:0/2666868578 learned_addr learned my addr 172.21.15.122:0/2666868578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:44.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e201987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e2539f640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2010bfd0 0x7f1e2019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e24b9e640 1 -- 172.21.15.122:0/2666868578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2010bfd0 msgr2=0x7f1e2019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.259+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2010bfd0 0x7f1e2019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e24b9e640 1 -- 172.21.15.122:0/2666868578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 msgr2=0x7f1e201987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e201987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e24b9e640 1 -- 172.21.15.122:0/2666868578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2019d220 con 0x7f1e20102bf0 2024-09-12T01:46:44.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e201987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:44.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20198270 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e9f0 tx=0x7f1e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:44.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e1dffb640 1 -- 172.21.15.122:0/2666868578 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800cd80 con 0x7f1e20102bf0 2024-09-12T01:46:44.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e1dffb640 1 -- 172.21.15.122:0/2666868578 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08010910 con 0x7f1e20102bf0 2024-09-12T01:46:44.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2019d510 con 0x7f1e20102bf0 2024-09-12T01:46:44.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e1dffb640 1 -- 172.21.15.122:0/2666868578 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08018990 con 0x7f1e20102bf0 2024-09-12T01:46:44.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.260+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a4df0 con 0x7f1e20102bf0 2024-09-12T01:46:44.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 Processor -- start 2024-09-12T01:46:44.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 -- start start 2024-09-12T01:46:44.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063225bd0 0x560063225fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x560063230870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x5600632331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56006318c090 con 0x5600632264f0 2024-09-12T01:46:44.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560063133a70 con 0x560063230db0 2024-09-12T01:46:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e291db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560063188050 con 0x560063225bd0 2024-09-12T01:46:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e2539f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063225bd0 0x560063225fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e2539f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063225bd0 0x560063225fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:47222/0 (socket says 172.21.15.122:47222) 2024-09-12T01:46:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e2539f640 1 -- 172.21.15.122:0/1662669715 learned_addr learned my addr 172.21.15.122:0/1662669715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.263+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x5600632331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1dffb640 1 -- 172.21.15.122:0/2666868578 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1e08010ac0 con 0x7f1e20102bf0 2024-09-12T01:46:44.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1662669715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063225bd0 msgr2=0x560063225fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063225bd0 0x560063225fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1662669715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 msgr2=0x560063230870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x560063230870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1662669715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560063140130 con 0x560063230db0 2024-09-12T01:46:44.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x5600632331a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002ae70 tx=0x7f1e10005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2192ff086cff760c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:44.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e16ffd640 1 -- 172.21.15.122:0/1662669715 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10048070 con 0x560063230db0 2024-09-12T01:46:44.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e16ffd640 1 -- 172.21.15.122:0/1662669715 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e100098a0 con 0x560063230db0 2024-09-12T01:46:44.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.264+0000 7f1e16ffd640 1 -- 172.21.15.122:0/1662669715 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10034040 con 0x560063230db0 2024-09-12T01:46:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e1dffb640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1e0c0a2c10 0x7f1e0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e2539f640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1e0c0a2c10 0x7f1e0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e1dffb640 1 -- 172.21.15.122:0/2666868578 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1e080cf5b0 con 0x7f1e20102bf0 2024-09-12T01:46:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e2539f640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1e0c0a2c10 0x7f1e0c0a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d90 tx=0x7f1e18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e291db740 1 -- 172.21.15.122:0/1662669715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 msgr2=0x5600632331a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e291db740 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x5600632331a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002ae70 tx=0x7f1e10005ef0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e291db740 1 -- 172.21.15.122:0/1662669715 shutdown_connections 2024-09-12T01:46:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e291db740 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x5600632331a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.265+0000 7f1e291db740 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x560063230870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 --2- 172.21.15.122:0/1662669715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063225bd0 0x560063225fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 -- 172.21.15.122:0/1662669715 >> 172.21.15.122:0/1662669715 conn(0x560063222160 msgr2=0x560063222540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 -- 172.21.15.122:0/1662669715 shutdown_connections 2024-09-12T01:46:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 -- 172.21.15.122:0/1662669715 wait complete. 2024-09-12T01:46:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 -- start start 2024-09-12T01:46:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 Processor -- start 2024-09-12T01:46:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x5600632306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600632264f0 0x56006322da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x56006322df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.266+0000 7f1e291db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560063233980 con 0x560063225bd0 2024-09-12T01:46:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e291db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560063146f20 con 0x560063230db0 2024-09-12T01:46:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e291db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560063233b00 con 0x5600632264f0 2024-09-12T01:46:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e24b9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x5600632306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e24b9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x5600632306d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46896/0 (socket says 172.21.15.122:46896) 2024-09-12T01:46:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e291db740 10 client.0 _ll_register_callbacks cb 0x560063110030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e24b9e640 1 -- 172.21.15.122:0/710145819 learned_addr learned my addr 172.21.15.122:0/710145819 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600632264f0 0x56006322da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e2539f640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x56006322df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 -- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 msgr2=0x56006322df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x56006322df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 -- 172.21.15.122:0/710145819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 msgr2=0x5600632306d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x5600632306d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 -- 172.21.15.122:0/710145819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560062efd2b0 con 0x5600632264f0 2024-09-12T01:46:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x5600632306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:44.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600632264f0 0x56006322da20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005e30 tx=0x7f1e10003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e594dd118a2b9930 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:44.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1dfffff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:46:44.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1dfffff640 1 -- 172.21.15.122:0/710145819 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10048070 con 0x5600632264f0 2024-09-12T01:46:44.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e100098a0 con 0x5600632264f0 2024-09-12T01:46:44.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.267+0000 7f1dfffff640 1 -- 172.21.15.122:0/710145819 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1e100041b0 con 0x5600632264f0 2024-09-12T01:46:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1dfffff640 1 -- 172.21.15.122:0/710145819 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e10054050 con 0x5600632264f0 2024-09-12T01:46:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 10 client.24814 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560063233980 con 0x5600632264f0 2024-09-12T01:46:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1dfffff640 1 -- 172.21.15.122:0/710145819 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e100098a0 con 0x5600632264f0 2024-09-12T01:46:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1dfffff640 1 -- 172.21.15.122:0/710145819 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1e10030890 con 0x5600632264f0 2024-09-12T01:46:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 10 client.24814 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 10 client.24814 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:46:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 20 client.24814 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560063241360 0x5600632427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 4 client.24814 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560063199600 con 0x560063241360 2024-09-12T01:46:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.268+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560063241360 0x5600632427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.269+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560063241360 0x5600632427d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7194d8b1ffeb7634 server_cookie=19fbfd4b757743e in_seq=0 out_seq=0 2024-09-12T01:46:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.269+0000 7f1dfffff640 10 client.24814 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.269+0000 7f1dfffff640 1 -- 172.21.15.122:0/710145819 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560063199600 con 0x560063241360 2024-09-12T01:46:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.269+0000 7f1dfffff640 10 client.24814 handle_command_reply: tid=0 2024-09-12T01:46:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.269+0000 7f1e291db740 1 client.24814 shutdown 2024-09-12T01:46:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600632264f0 msgr2=0x56006322da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600632264f0 0x56006322da20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005e30 tx=0x7f1e10003ba0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 shutdown_connections 2024-09-12T01:46:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560063241360 0x5600632427d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063230db0 0x56006322df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600632264f0 0x56006322da20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1dfffff640 0 client.24814 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 --2- 172.21.15.122:0/710145819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x5600632306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 >> 172.21.15.122:0/710145819 conn(0x560063222160 msgr2=0x560063227e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 shutdown_connections 2024-09-12T01:46:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 1 -- 172.21.15.122:0/710145819 wait complete. 2024-09-12T01:46:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.270+0000 7f1e291db740 20 client.24814 trim_cache size 0 max 16384 2024-09-12T01:46:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.287+0000 7f1e291db740 1 Processor -- start 2024-09-12T01:46:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.287+0000 7f1e291db740 1 -- start start 2024-09-12T01:46:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e291db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x560063228760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e291db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600632264f0 0x560063228ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e291db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e291db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560063240b10 con 0x560063225bd0 2024-09-12T01:46:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e291db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5600632411e0 con 0x5600632264f0 2024-09-12T01:46:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e291db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600632418b0 con 0x560063230db0 2024-09-12T01:46:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x560063228760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x560063228760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46912/0 (socket says 172.21.15.122:46912) 2024-09-12T01:46:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 -- 172.21.15.122:0/1749979122 learned_addr learned my addr 172.21.15.122:0/1749979122 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600632264f0 0x560063228ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 -- 172.21.15.122:0/1749979122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 msgr2=0x5600632291e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 -- 172.21.15.122:0/1749979122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600632264f0 msgr2=0x560063228ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600632264f0 0x560063228ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e24b9e640 1 -- 172.21.15.122:0/1749979122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560063221c00 con 0x560063225bd0 2024-09-12T01:46:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.288+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600632264f0 0x560063228ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.289+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x560063228760 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e0809bc40 tx=0x7f1e080cf870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6df1a5506f9e457 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.289+0000 7f1e167fc640 1 -- 172.21.15.122:0/1749979122 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080a4920 con 0x560063225bd0 2024-09-12T01:46:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.289+0000 7f1e167fc640 1 -- 172.21.15.122:0/1749979122 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08099d50 con 0x560063225bd0 2024-09-12T01:46:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.289+0000 7f1e167fc640 1 -- 172.21.15.122:0/1749979122 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0809b100 con 0x560063225bd0 2024-09-12T01:46:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 -- 172.21.15.122:0/1749979122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 msgr2=0x560063228760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x560063228760 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e0809bc40 tx=0x7f1e080cf870 comp rx=0 tx=0).stop 2024-09-12T01:46:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 -- 172.21.15.122:0/1749979122 shutdown_connections 2024-09-12T01:46:44.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632291e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600632264f0 0x560063228ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 --2- 172.21.15.122:0/1749979122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560063225bd0 0x560063228760 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 -- 172.21.15.122:0/1749979122 >> 172.21.15.122:0/1749979122 conn(0x560063222160 msgr2=0x56006322cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:44.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 -- 172.21.15.122:0/1749979122 shutdown_connections 2024-09-12T01:46:44.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 -- 172.21.15.122:0/1749979122 wait complete. 2024-09-12T01:46:44.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 -- start start 2024-09-12T01:46:44.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.290+0000 7f1e291db740 1 Processor -- start 2024-09-12T01:46:44.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063225bd0 0x560063243f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x5600632444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560063242650 con 0x5600632264f0 2024-09-12T01:46:44.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560063241f80 con 0x560063225bd0 2024-09-12T01:46:44.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560063242d20 con 0x560063230db0 2024-09-12T01:46:44.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e291db740 10 client.0 _ll_register_callbacks cb 0x7f1e10046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:44.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x5600632444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x5600632444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46918/0 (socket says 172.21.15.122:46918) 2024-09-12T01:46:44.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1206903665 learned_addr learned my addr 172.21.15.122:0/1206903665 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:44.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e2539f640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063225bd0 0x560063243f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1206903665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 msgr2=0x5600632a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063225bd0 msgr2=0x560063243f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063225bd0 0x560063243f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e1ffff640 1 -- 172.21.15.122:0/1206903665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600632a9fa0 con 0x5600632264f0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.291+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063225bd0 0x560063243f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e1ffff640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x5600632444b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1e10002230 tx=0x7f1e100306f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ae71916a8affd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1dfffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1dfffff640 1 -- 172.21.15.122:0/1206903665 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1005d060 con 0x5600632264f0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e08099d50 con 0x5600632264f0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1dfffff640 1 -- 172.21.15.122:0/1206903665 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1e10003fa0 con 0x5600632264f0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1dfffff640 1 -- 172.21.15.122:0/1206903665 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e10048040 con 0x5600632264f0 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e291db740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600632ab250 con 0x5600632264f0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1dfffff640 1 -- 172.21.15.122:0/1206903665 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08099d50 con 0x5600632264f0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1dfffff640 1 -- 172.21.15.122:0/1206903665 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1e10062070 con 0x5600632264f0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e291db740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e291db740 10 client.15324 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.292+0000 7f1e291db740 20 client.15324 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.293+0000 7f1e291db740 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5600632afcd0 0x5600632ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.293+0000 7f1e291db740 4 client.15324 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.293+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "status"}) -- 0x560063193b10 con 0x5600632afcd0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.293+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5600632afcd0 0x5600632ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.293+0000 7f1dfffff640 10 client.15324 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.293+0000 7f1e24b9e640 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5600632afcd0 0x5600632ae9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a89ddf1906cf0260 server_cookie=b7c9dbfcebeed7d8 in_seq=0 out_seq=0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1dfffff640 1 -- 172.21.15.122:0/1206903665 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560063193b10 con 0x5600632afcd0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1dfffff640 10 client.15324 handle_command_reply: tid=0 2024-09-12T01:46:44.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 client.15324 shutdown 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 msgr2=0x5600632444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x5600632444b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1e10002230 tx=0x7f1e100306f0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 shutdown_connections 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5600632afcd0 0x5600632ae9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560063230db0 0x5600632a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1dfffff640 0 client.15324 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600632264f0 0x5600632444b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 --2- 172.21.15.122:0/1206903665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560063225bd0 0x560063243f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 >> 172.21.15.122:0/1206903665 conn(0x560063222160 msgr2=0x5600632286d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 shutdown_connections 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 1 -- 172.21.15.122:0/1206903665 wait complete. 2024-09-12T01:46:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.294+0000 7f1e291db740 20 client.15324 trim_cache size 0 max 16384 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24428, 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:46:44.333 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 119.483582368, 2024-09-12T01:46:44.334 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 26, 2024-09-12T01:46:44.334 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:46:44.334 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:46:44.334 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 149.61552365700001 2024-09-12T01:46:44.334 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1e0c0a2c10 msgr2=0x7f1e0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1e0c0a2c10 0x7f1e0c0a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d90 tx=0x7f1e18005f90 comp rx=0 tx=0).stop 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 msgr2=0x7f1e20198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20198270 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e9f0 tx=0x7f1e0800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 shutdown_connections 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1e0c0a2c10 0x7f1e0c0a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2010bfd0 0x7f1e2019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20103510 0x7f1e201987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 --2- 172.21.15.122:0/2666868578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e20102bf0 0x7f1e20198270 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 >> 172.21.15.122:0/2666868578 conn(0x7f1e200fc610 msgr2=0x7f1e200fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 shutdown_connections 2024-09-12T01:46:44.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:44.295+0000 7f1e25ba0640 1 -- 172.21.15.122:0/2666868578 wait complete. 2024-09-12T01:46:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:46:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:46:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:46:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:46:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:44.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T01:46:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:46:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:46:45.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:46:45.212 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 -- 172.21.15.122:0/4111059409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c102790 msgr2=0x7fb75c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 --2- 172.21.15.122:0/4111059409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c102790 0x7fb75c1067e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb750009900 tx=0x7fb75002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 -- 172.21.15.122:0/4111059409 shutdown_connections 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 --2- 172.21.15.122:0/4111059409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb75c106f10 0x7fb75c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 --2- 172.21.15.122:0/4111059409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c102790 0x7fb75c1067e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 --2- 172.21.15.122:0/4111059409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c101de0 0x7fb75c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 -- 172.21.15.122:0/4111059409 >> 172.21.15.122:0/4111059409 conn(0x7fb75c0fd8c0 msgr2=0x7fb75c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 -- 172.21.15.122:0/4111059409 shutdown_connections 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.723+0000 7fb76130e640 1 -- 172.21.15.122:0/4111059409 wait complete. 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 Processor -- start 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 -- start start 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb75c101de0 0x7fb75c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c102790 0x7fb75c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 0x7fb75c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c111f70 con 0x7fb75c106f10 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c111df0 con 0x7fb75c102790 2024-09-12T01:46:45.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb76130e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c1120f0 con 0x7fb75c101de0 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb75b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 0x7fb75c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb75b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 0x7fb75c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46946/0 (socket says 172.21.15.122:46946) 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb75b7fe640 1 -- 172.21.15.122:0/2648070715 learned_addr learned my addr 172.21.15.122:0/2648070715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.724+0000 7fb75affd640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb75c101de0 0x7fb75c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c102790 0x7fb75c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75b7fe640 1 -- 172.21.15.122:0/2648070715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb75c101de0 msgr2=0x7fb75c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75b7fe640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb75c101de0 0x7fb75c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75b7fe640 1 -- 172.21.15.122:0/2648070715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c102790 msgr2=0x7fb75c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75b7fe640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c102790 0x7fb75c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75b7fe640 1 -- 172.21.15.122:0/2648070715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c19d320 con 0x7fb75c106f10 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c102790 0x7fb75c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb75b7fe640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 0x7fb75c19cc40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb75400eaa0 tx=0x7fb75400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb743fff640 1 -- 172.21.15.122:0/2648070715 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75400cdb0 con 0x7fb75c106f10 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c19d610 con 0x7fb75c106f10 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb743fff640 1 -- 172.21.15.122:0/2648070715 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb754005020 con 0x7fb75c106f10 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb743fff640 1 -- 172.21.15.122:0/2648070715 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540116d0 con 0x7fb75c106f10 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.725+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c1a4ef0 con 0x7fb75c106f10 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.727+0000 7fb743fff640 1 -- 172.21.15.122:0/2648070715 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb7540054d0 con 0x7fb75c106f10 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.727+0000 7fb7636e7740 1 Processor -- start 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 -- start start 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x565439332db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x56543933ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565439299090 con 0x5654393332f0 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565439240a70 con 0x56543933dbb0 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb7636e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565439295050 con 0x5654393329d0 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46948/0 (socket says 172.21.15.122:46948) 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 -- 172.21.15.122:0/4128030638 learned_addr learned my addr 172.21.15.122:0/4128030638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x565439332db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb743fff640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c10 0x7fb7440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 -- 172.21.15.122:0/4128030638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 msgr2=0x565439332db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x565439332db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 -- 172.21.15.122:0/4128030638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 msgr2=0x56543933ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x56543933ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x56543933ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 -- 172.21.15.122:0/4128030638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543924d130 con 0x5654393332f0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75b7fe640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c10 0x7fb7440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb743fff640 1 -- 172.21.15.122:0/2648070715 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb7540cf480 con 0x7fb75c106f10 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.728+0000 7fb75affd640 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933d670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00cc60 tx=0x7fb74c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9b2ce682d7ab58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7417fa640 1 -- 172.21.15.122:0/4128030638 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c014070 con 0x5654393332f0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7417fa640 1 -- 172.21.15.122:0/4128030638 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c0047a0 con 0x5654393332f0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7417fa640 1 -- 172.21.15.122:0/4128030638 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c010790 con 0x5654393332f0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 -- 172.21.15.122:0/4128030638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 msgr2=0x56543933d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933d670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00cc60 tx=0x7fb74c00e720 comp rx=0 tx=0).stop 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb75b7fe640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c10 0x7fb7440a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb75409b880 tx=0x7fb75401eb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 -- 172.21.15.122:0/4128030638 shutdown_connections 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x56543933ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933d670 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4128030638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x565439332db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 -- 172.21.15.122:0/4128030638 >> 172.21.15.122:0/4128030638 conn(0x56543932ef60 msgr2=0x56543932f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 -- 172.21.15.122:0/4128030638 shutdown_connections 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.729+0000 7fb7636e7740 1 -- 172.21.15.122:0/4128030638 wait complete. 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 -- start start 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 Processor -- start 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393329d0 0x56543933d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56543933dbb0 0x56543933ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565439340780 con 0x5654393332f0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565439253f20 con 0x5654393329d0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565439340900 con 0x56543933dbb0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb7636e7740 10 client.0 _ll_register_callbacks cb 0x56543921d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46952/0 (socket says 172.21.15.122:46952) 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 -- 172.21.15.122:0/4129350657 learned_addr learned my addr 172.21.15.122:0/4129350657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393329d0 0x56543933d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56543933dbb0 0x56543933ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 -- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56543933dbb0 msgr2=0x56543933ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56543933dbb0 0x56543933ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 -- 172.21.15.122:0/4129350657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393329d0 msgr2=0x56543933d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393329d0 0x56543933d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.730+0000 7fb75affd640 1 -- 172.21.15.122:0/4129350657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543900a2b0 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb75affd640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0057c0 tx=0x7fb74c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cea25c02d0a892 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb73eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb73eff5640 1 -- 172.21.15.122:0/4129350657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c027060 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb73eff5640 1 -- 172.21.15.122:0/4129350657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb74c018620 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74c0047a0 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb73eff5640 1 -- 172.21.15.122:0/4129350657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb74c014070 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 10 client.15336 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56543934c0a0 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb73eff5640 1 -- 172.21.15.122:0/4129350657 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c0047a0 con 0x5654393332f0 2024-09-12T01:46:45.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb73eff5640 1 -- 172.21.15.122:0/4129350657 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb74c02c070 con 0x5654393332f0 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 10 client.15336 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 10 client.15336 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 20 client.15336 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.731+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x56543934e360 0x56543934f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.732+0000 7fb7636e7740 4 client.15336 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.732+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654392a6600 con 0x56543934e360 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.732+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x56543934e360 0x56543934f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.732+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x56543934e360 0x56543934f7d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=830b2caa99187abf server_cookie=7ed5eb15a12400b5 in_seq=0 out_seq=0 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.732+0000 7fb73eff5640 10 client.15336 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb73eff5640 1 -- 172.21.15.122:0/4129350657 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654392a6600 con 0x56543934e360 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb73eff5640 10 client.15336 handle_command_reply: tid=0 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 client.15336 shutdown 2024-09-12T01:46:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 msgr2=0x56543933a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0057c0 tx=0x7fb74c010c20 comp rx=0 tx=0).stop 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 shutdown_connections 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x56543934e360 0x56543934f7d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56543933dbb0 0x56543933ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393332f0 0x56543933a7a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb73eff5640 0 client.15336 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 --2- 172.21.15.122:0/4129350657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393329d0 0x56543933d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 >> 172.21.15.122:0/4129350657 conn(0x56543932ef60 msgr2=0x565439334c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 shutdown_connections 2024-09-12T01:46:45.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 1 -- 172.21.15.122:0/4129350657 wait complete. 2024-09-12T01:46:45.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.733+0000 7fb7636e7740 20 client.15336 trim_cache size 0 max 16384 2024-09-12T01:46:45.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 Processor -- start 2024-09-12T01:46:45.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 -- start start 2024-09-12T01:46:45.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x56543933c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393332f0 0x565439337a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 0x565439337f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56543934db10 con 0x56543933dbb0 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56543934e1e0 con 0x5654393332f0 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb7636e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56543934e8b0 con 0x5654393329d0 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 0x565439337f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 0x565439337f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46954/0 (socket says 172.21.15.122:46954) 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 -- 172.21.15.122:0/750588235 learned_addr learned my addr 172.21.15.122:0/750588235 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75affd640 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393332f0 0x565439337a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x56543933c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 -- 172.21.15.122:0/750588235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 msgr2=0x56543933c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x56543933c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 -- 172.21.15.122:0/750588235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393332f0 msgr2=0x565439337a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393332f0 0x565439337a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 -- 172.21.15.122:0/750588235 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543932faa0 con 0x56543933dbb0 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.751+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 0x565439337f40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb750009900 tx=0x7fb75002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558c8a35a55ab250 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb740ff9640 1 -- 172.21.15.122:0/750588235 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75004e030 con 0x56543933dbb0 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb740ff9640 1 -- 172.21.15.122:0/750588235 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750031920 con 0x56543933dbb0 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb740ff9640 1 -- 172.21.15.122:0/750588235 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750035040 con 0x56543933dbb0 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 -- 172.21.15.122:0/750588235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 msgr2=0x565439337f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 0x565439337f40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb750009900 tx=0x7fb75002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 -- 172.21.15.122:0/750588235 shutdown_connections 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56543933dbb0 0x565439337f40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654393332f0 0x565439337a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 --2- 172.21.15.122:0/750588235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393329d0 0x56543933c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 -- 172.21.15.122:0/750588235 >> 172.21.15.122:0/750588235 conn(0x56543932ef60 msgr2=0x56543933d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 -- 172.21.15.122:0/750588235 shutdown_connections 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.752+0000 7fb7636e7740 1 -- 172.21.15.122:0/750588235 wait complete. 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 -- start start 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 Processor -- start 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 0x565439353870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393332f0 0x565439353db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x5654393542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56543934f650 con 0x5654393329d0 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56543934ef80 con 0x56543933dbb0 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56543934fd20 con 0x5654393332f0 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb7636e7740 10 client.0 _ll_register_callbacks cb 0x7fb74c00cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 0x565439353870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 0x565439353870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46962/0 (socket says 172.21.15.122:46962) 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 -- 172.21.15.122:0/3999170378 learned_addr learned my addr 172.21.15.122:0/3999170378 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75affd640 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393332f0 0x565439353db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 -- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393332f0 msgr2=0x565439353db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393332f0 0x565439353db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 -- 172.21.15.122:0/3999170378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 msgr2=0x5654393542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:45.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x5654393542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.753+0000 7fb75a7fc640 1 -- 172.21.15.122:0/3999170378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565439332800 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb75a7fc640 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 0x565439353870 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb750004610 tx=0x7fb750032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25879166b89824ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb73eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb73eff5640 1 -- 172.21.15.122:0/3999170378 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750057070 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb73eff5640 1 -- 172.21.15.122:0/3999170378 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb750005300 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb750031920 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb73eff5640 1 -- 172.21.15.122:0/3999170378 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb750062050 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 10 client.15342 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5654393b8bb0 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb73eff5640 1 -- 172.21.15.122:0/3999170378 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750031920 con 0x5654393329d0 2024-09-12T01:46:45.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb73eff5640 1 -- 172.21.15.122:0/3999170378 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb75003a040 con 0x5654393329d0 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 10 client.15342 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 10 client.15342 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 20 client.15342 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5654393bd1a0 0x5654393bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 4 client.15342 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.754+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "status"}) -- 0x5654392a0b10 con 0x5654393bd1a0 2024-09-12T01:46:45.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.755+0000 7fb75affd640 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5654393bd1a0 0x5654393bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:45.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.755+0000 7fb73eff5640 10 client.15342 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:45.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.755+0000 7fb75affd640 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5654393bd1a0 0x5654393bbeb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dda3be32dbbcf12 server_cookie=58f34e8e37890288 in_seq=0 out_seq=0 2024-09-12T01:46:45.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.755+0000 7fb73eff5640 1 -- 172.21.15.122:0/3999170378 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5654392a0b10 con 0x5654393bd1a0 2024-09-12T01:46:45.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.755+0000 7fb73eff5640 10 client.15342 handle_command_reply: tid=0 2024-09-12T01:46:45.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.755+0000 7fb7636e7740 1 client.15342 shutdown 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 msgr2=0x565439353870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 0x565439353870 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb750004610 tx=0x7fb750032d50 comp rx=0 tx=0).stop 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 shutdown_connections 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x5654393bd1a0 0x5654393bbeb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56543933dbb0 0x5654393542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5654393332f0 0x565439353db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb73eff5640 0 client.15342 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 --2- 172.21.15.122:0/3999170378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654393329d0 0x565439353870 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 >> 172.21.15.122:0/3999170378 conn(0x56543932ef60 msgr2=0x56543932e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 shutdown_connections 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 1 -- 172.21.15.122:0/3999170378 wait complete. 2024-09-12T01:46:45.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb7636e7740 20 client.15342 trim_cache size 0 max 16384 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24436, 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 121.9566765, 2024-09-12T01:46:45.763 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 26, 2024-09-12T01:46:45.764 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:46:45.764 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:46:45.764 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 153.85153122700001 2024-09-12T01:46:45.764 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:45.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c10 msgr2=0x7fb7440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c10 0x7fb7440a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb75409b880 tx=0x7fb75401eb40 comp rx=0 tx=0).stop 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 msgr2=0x7fb75c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 0x7fb75c19cc40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb75400eaa0 tx=0x7fb75400ef60 comp rx=0 tx=0).stop 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 shutdown_connections 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c10 0x7fb7440a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb75c106f10 0x7fb75c19cc40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c102790 0x7fb75c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 --2- 172.21.15.122:0/2648070715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb75c101de0 0x7fb75c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.756+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 >> 172.21.15.122:0/2648070715 conn(0x7fb75c0fd8c0 msgr2=0x7fb75c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.757+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 shutdown_connections 2024-09-12T01:46:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:45.757+0000 7fb76130e640 1 -- 172.21.15.122:0/2648070715 wait complete. 2024-09-12T01:46:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:45 smithi183 ceph-mon[35127]: pgmap v343: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:46:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:45 smithi122 ceph-mon[30925]: pgmap v343: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:46:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:45 smithi151 ceph-mon[35229]: pgmap v343: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:46:46.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T01:46:46.689 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:46:47.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 -- 172.21.15.122:0/1140272429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374102790 msgr2=0x7f63741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 --2- 172.21.15.122:0/1140272429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374102790 0x7f63741067e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637002a410 comp rx=0 tx=0).stop 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 -- 172.21.15.122:0/1140272429 shutdown_connections 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 --2- 172.21.15.122:0/1140272429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 0x7f637410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 --2- 172.21.15.122:0/1140272429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374102790 0x7f63741067e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 --2- 172.21.15.122:0/1140272429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374101de0 0x7f63741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 -- 172.21.15.122:0/1140272429 >> 172.21.15.122:0/1140272429 conn(0x7f63740fd8c0 msgr2=0x7f63740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 -- 172.21.15.122:0/1140272429 shutdown_connections 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.199+0000 7f637bcbc640 1 -- 172.21.15.122:0/1140272429 wait complete. 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 Processor -- start 2024-09-12T01:46:47.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 -- start start 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 0x7f63740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374102790 0x7f6374074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 0x7f6374074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6374111f70 con 0x7f6374101de0 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6374111df0 con 0x7f6374106f10 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f637bcbc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63741120f0 con 0x7f6374102790 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 0x7f63740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 0x7f63740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46974/0 (socket says 172.21.15.122:46974) 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f6379a58640 1 -- 172.21.15.122:0/3421902475 learned_addr learned my addr 172.21.15.122:0/3421902475 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.200+0000 7f6379257640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374102790 0x7f6374074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f637a259640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 0x7f6374074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f6379a58640 1 -- 172.21.15.122:0/3421902475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374102790 msgr2=0x7f6374074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f6379a58640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374102790 0x7f6374074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f6379a58640 1 -- 172.21.15.122:0/3421902475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 msgr2=0x7f6374074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f6379a58640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 0x7f6374074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f6379a58640 1 -- 172.21.15.122:0/3421902475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a6970 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f637a259640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 0x7f6374074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f6379a58640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 0x7f63740744f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f636400d770 tx=0x7f636400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f636affd640 1 -- 172.21.15.122:0/3421902475 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364017070 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f636affd640 1 -- 172.21.15.122:0/3421902475 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6364005500 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a6c60 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f636affd640 1 -- 172.21.15.122:0/3421902475 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364004e00 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.201+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a7150 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f636affd640 1 -- 172.21.15.122:0/3421902475 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6364004fa0 con 0x7f6374101de0 2024-09-12T01:46:47.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 Processor -- start 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 -- start start 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e499d0 0x559064e49db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e56fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559064db0090 con 0x559064e4a2f0 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559064d57a70 con 0x559064e54bb0 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.204+0000 7f637e095740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559064dac050 con 0x559064e499d0 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e54670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46980/0 (socket says 172.21.15.122:46980) 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 -- 172.21.15.122:0/1576927489 learned_addr learned my addr 172.21.15.122:0/1576927489 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f637a259640 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e499d0 0x559064e49db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379257640 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e56fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f636affd640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63600a2c10 0x7f63600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 -- 172.21.15.122:0/1576927489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e499d0 msgr2=0x559064e49db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e499d0 0x559064e49db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 -- 172.21.15.122:0/1576927489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 msgr2=0x559064e56fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e56fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 -- 172.21.15.122:0/1576927489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559064d64130 con 0x559064e4a2f0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379257640 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e56fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f637a259640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63600a2c10 0x7f63600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f636affd640 1 -- 172.21.15.122:0/3421902475 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f63640ced00 con 0x7f6374101de0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f6379a58640 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e54670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f63640cc5f0 tx=0x7f6364065fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dc15793a9cc151 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f635ffff640 1 -- 172.21.15.122:0/1576927489 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636409af40 con 0x559064e4a2f0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f635ffff640 1 -- 172.21.15.122:0/1576927489 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636407ab70 con 0x559064e4a2f0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f635ffff640 1 -- 172.21.15.122:0/1576927489 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640b0cf0 con 0x559064e4a2f0 2024-09-12T01:46:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.205+0000 7f637a259640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63600a2c10 0x7f63600a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f636c005fd0 tx=0x7f636c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 -- 172.21.15.122:0/1576927489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 msgr2=0x559064e54670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e54670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f63640cc5f0 tx=0x7f6364065fd0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 -- 172.21.15.122:0/1576927489 shutdown_connections 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e56fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e54670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 --2- 172.21.15.122:0/1576927489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e499d0 0x559064e49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 -- 172.21.15.122:0/1576927489 >> 172.21.15.122:0/1576927489 conn(0x559064e45f60 msgr2=0x559064e46340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 -- 172.21.15.122:0/1576927489 shutdown_connections 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.206+0000 7f637e095740 1 -- 172.21.15.122:0/1576927489 wait complete. 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 -- start start 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 Processor -- start 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e499d0 0x559064e543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e51c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559064e57780 con 0x559064e4a2f0 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559064d6af20 con 0x559064e499d0 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559064e57900 con 0x559064e54bb0 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637e095740 10 client.0 _ll_register_callbacks cb 0x559064d34030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f6379257640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f6379257640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e51720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46994/0 (socket says 172.21.15.122:46994) 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f6379257640 1 -- 172.21.15.122:0/150259191 learned_addr learned my addr 172.21.15.122:0/150259191 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f6379a58640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e499d0 0x559064e543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.207+0000 7f637a259640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e51c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379257640 1 -- 172.21.15.122:0/150259191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 msgr2=0x559064e51c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379257640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e51c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379257640 1 -- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e499d0 msgr2=0x559064e543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379257640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e499d0 0x559064e543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379257640 1 -- 172.21.15.122:0/150259191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559064b212b0 con 0x559064e4a2f0 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379a58640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e499d0 0x559064e543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:47.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f6379257640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e51720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f637002a930 tx=0x7f6370031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899e5b7e509a4520 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f635d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f635d7fa640 1 -- 172.21.15.122:0/150259191 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370035070 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f635d7fa640 1 -- 172.21.15.122:0/150259191 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6370004240 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559064dbd600 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f635d7fa640 1 -- 172.21.15.122:0/150259191 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6370004440 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f637e095740 10 client.15354 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559064e63080 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.208+0000 7f635d7fa640 1 -- 172.21.15.122:0/150259191 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559064dbd600 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f635d7fa640 1 -- 172.21.15.122:0/150259191 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6370004b40 con 0x559064e4a2f0 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f637e095740 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f637e095740 10 client.15354 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f637e095740 20 client.15354 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f637e095740 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064e656a0 0x559064e66b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f637e095740 4 client.15354 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559064db7b10 con 0x559064e656a0 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f6379a58640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064e656a0 0x559064e66b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f635d7fa640 10 client.15354 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.209+0000 7f6379a58640 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064e656a0 0x559064e66b10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c25bef5f3a173a2 server_cookie=d84f88ddba0a988b in_seq=0 out_seq=0 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f635d7fa640 1 -- 172.21.15.122:0/150259191 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559064db7b10 con 0x559064e656a0 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f635d7fa640 10 client.15354 handle_command_reply: tid=0 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 client.15354 shutdown 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 msgr2=0x559064e51720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e51720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f637002a930 tx=0x7f6370031860 comp rx=0 tx=0).stop 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 shutdown_connections 2024-09-12T01:46:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064e656a0 0x559064e66b10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e51c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e4a2f0 0x559064e51720 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 --2- 172.21.15.122:0/150259191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e499d0 0x559064e543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f635d7fa640 0 client.15354 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 >> 172.21.15.122:0/150259191 conn(0x559064e45f60 msgr2=0x559064e4bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 shutdown_connections 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 1 -- 172.21.15.122:0/150259191 wait complete. 2024-09-12T01:46:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.210+0000 7f637e095740 20 client.15354 trim_cache size 0 max 16384 2024-09-12T01:46:47.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 Processor -- start 2024-09-12T01:46:47.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 -- start start 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e4e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e4a2f0 0x559064e4eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e6edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559064e64e50 con 0x559064e499d0 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559064e65520 con 0x559064e54bb0 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637e095740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559064e65bf0 con 0x559064e4a2f0 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e4e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e4e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46996/0 (socket says 172.21.15.122:46996) 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 -- 172.21.15.122:0/3447679455 learned_addr learned my addr 172.21.15.122:0/3447679455 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f637a259640 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e6edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 -- 172.21.15.122:0/3447679455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e4a2f0 msgr2=0x559064e4eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e4a2f0 0x559064e4eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 -- 172.21.15.122:0/3447679455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 msgr2=0x559064e6edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e6edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.228+0000 7f6379a58640 1 -- 172.21.15.122:0/3447679455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559064e57890 con 0x559064e499d0 2024-09-12T01:46:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f6379a58640 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e4e980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f636400dfa0 tx=0x7f63640d2830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980bdd3685a33cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:47.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f635f7fe640 1 -- 172.21.15.122:0/3447679455 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640d0070 con 0x559064e499d0 2024-09-12T01:46:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f635f7fe640 1 -- 172.21.15.122:0/3447679455 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63640046c0 con 0x559064e499d0 2024-09-12T01:46:47.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f635f7fe640 1 -- 172.21.15.122:0/3447679455 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364020040 con 0x559064e499d0 2024-09-12T01:46:47.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 -- 172.21.15.122:0/3447679455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 msgr2=0x559064e4e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e4e980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f636400dfa0 tx=0x7f63640d2830 comp rx=0 tx=0).stop 2024-09-12T01:46:47.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 -- 172.21.15.122:0/3447679455 shutdown_connections 2024-09-12T01:46:47.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e54bb0 0x559064e6edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e4a2f0 0x559064e4eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 --2- 172.21.15.122:0/3447679455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e4e980 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 -- 172.21.15.122:0/3447679455 >> 172.21.15.122:0/3447679455 conn(0x559064e45f60 msgr2=0x559064e4fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 -- 172.21.15.122:0/3447679455 shutdown_connections 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.229+0000 7f637e095740 1 -- 172.21.15.122:0/3447679455 wait complete. 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 -- start start 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 Processor -- start 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e69d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e4a2f0 0x559064e6a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e6a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559064e66990 con 0x559064e499d0 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559064e662c0 con 0x559064e4a2f0 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559064e67060 con 0x559064e54bb0 2024-09-12T01:46:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637e095740 10 client.0 _ll_register_callbacks cb 0x7f6370009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e69d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f6379a58640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e69d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46998/0 (socket says 172.21.15.122:46998) 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f6379a58640 1 -- 172.21.15.122:0/2855801796 learned_addr learned my addr 172.21.15.122:0/2855801796 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f6379257640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e4a2f0 0x559064e6a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.230+0000 7f637a259640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e6a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f6379a58640 1 -- 172.21.15.122:0/2855801796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 msgr2=0x559064e6a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f6379a58640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e6a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f6379a58640 1 -- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e4a2f0 msgr2=0x559064e6a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f6379a58640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e4a2f0 0x559064e6a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f6379a58640 1 -- 172.21.15.122:0/2855801796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559064ecdf70 con 0x559064e499d0 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637a259640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e6a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f6379a58640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e69d00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6364009610 tx=0x7f63640048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea69345798479dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f635d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f635d7fa640 1 -- 172.21.15.122:0/2855801796 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640d0070 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f635d7fa640 1 -- 172.21.15.122:0/2855801796 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6364005290 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559064d340e0 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f635d7fa640 1 -- 172.21.15.122:0/2855801796 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6364020050 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 10 client.15360 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559064ecf250 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f635d7fa640 1 -- 172.21.15.122:0/2855801796 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559064d340e0 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f635d7fa640 1 -- 172.21.15.122:0/2855801796 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6364097a70 con 0x559064e499d0 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 10 client.15360 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 10 client.15360 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 20 client.15360 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064ed3700 0x559064ed2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 4 client.15360 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T01:46:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.231+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "status"}) -- 0x559064d81750 con 0x559064ed3700 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f6379257640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064ed3700 0x559064ed2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f635d7fa640 10 client.15360 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f6379257640 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064ed3700 0x559064ed2410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acec6d4ef84c54f server_cookie=a91a98e07dedcd0b in_seq=0 out_seq=0 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f635d7fa640 1 -- 172.21.15.122:0/2855801796 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559064d81750 con 0x559064ed3700 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f635d7fa640 10 client.15360 handle_command_reply: tid=0 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f637e095740 1 client.15360 shutdown 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 msgr2=0x559064e69d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.232+0000 7f637e095740 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e69d00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6364009610 tx=0x7f63640048a0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 shutdown_connections 2024-09-12T01:46:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559064ed3700 0x559064ed2410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559064e54bb0 0x559064e6a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559064e4a2f0 0x559064e6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f635d7fa640 0 client.15360 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 --2- 172.21.15.122:0/2855801796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559064e499d0 0x559064e69d00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 >> 172.21.15.122:0/2855801796 conn(0x559064e45f60 msgr2=0x559064e447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 shutdown_connections 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 1 -- 172.21.15.122:0/2855801796 wait complete. 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637e095740 20 client.15360 trim_cache size 0 max 16384 2024-09-12T01:46:47.241 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "5731f69a-70a7-11ef-bcea-c7b262605968", 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24452, 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 130.463619784, 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 26, 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 83, 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 141.05912037799999 2024-09-12T01:46:47.242 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:46:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63600a2c10 msgr2=0x7f63600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63600a2c10 0x7f63600a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f636c005fd0 tx=0x7f636c004a40 comp rx=0 tx=0).stop 2024-09-12T01:46:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 msgr2=0x7f63740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 0x7f63740744f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f636400d770 tx=0x7f636400dc30 comp rx=0 tx=0).stop 2024-09-12T01:46:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 shutdown_connections 2024-09-12T01:46:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63600a2c10 0x7f63600a50d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374106f10 0x7f6374074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374102790 0x7f6374074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 --2- 172.21.15.122:0/3421902475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6374101de0 0x7f63740744f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:47.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 >> 172.21.15.122:0/3421902475 conn(0x7f63740fd8c0 msgr2=0x7f63740fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:47.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 shutdown_connections 2024-09-12T01:46:47.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:46:47.233+0000 7f637bcbc640 1 -- 172.21.15.122:0/3421902475 wait complete. 2024-09-12T01:46:47.919 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:47 smithi122 ceph-mon[30925]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:47.946 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T01:46:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:47 smithi183 ceph-mon[35127]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:47 smithi151 ceph-mon[35229]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:49 smithi183 ceph-mon[35127]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T01:46:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:49 smithi122 ceph-mon[30925]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T01:46:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:49 smithi151 ceph-mon[35229]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T01:46:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:51 smithi183 ceph-mon[35127]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:51.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:51 smithi122 ceph-mon[30925]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:51 smithi151 ceph-mon[35229]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/3710372016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 msgr2=0x7fb3ec109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/3710372016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec109cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009ca0 tx=0x7fb3e402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/3710372016 shutdown_connections 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/3710372016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec109cb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/3710372016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3ec104f10 0x7fb3ec107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/3710372016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ec1025e0 0x7fb3ec1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/3710372016 >> 172.21.15.122:0/3710372016 conn(0x7fb3ec0fc1e0 msgr2=0x7fb3ec0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/3710372016 shutdown_connections 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.442+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/3710372016 wait complete. 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 Processor -- start 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 -- start start 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ec1025e0 0x7fb3ec197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3ec104f10 0x7fb3ec198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec10d870 con 0x7fb3ec107840 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec10d6f0 con 0x7fb3ec104f10 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f2e7b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec10d9f0 con 0x7fb3ec1025e0 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f1418640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f1418640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41674/0 (socket says 172.21.15.122:41674) 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.443+0000 7fb3f1418640 1 -- 172.21.15.122:0/218837901 learned_addr learned my addr 172.21.15.122:0/218837901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3ec104f10 0x7fb3ec198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ec1025e0 0x7fb3ec197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f1418640 1 -- 172.21.15.122:0/218837901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ec1025e0 msgr2=0x7fb3ec197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f1418640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ec1025e0 0x7fb3ec197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f1418640 1 -- 172.21.15.122:0/218837901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3ec104f10 msgr2=0x7fb3ec198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f1418640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3ec104f10 0x7fb3ec198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f1418640 1 -- 172.21.15.122:0/218837901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec19fa30 con 0x7fb3ec107840 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f1418640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec19f3c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4031ae0 tx=0x7fb3e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3e9ffb640 1 -- 172.21.15.122:0/218837901 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4004040 con 0x7fb3ec107840 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ec19fcc0 con 0x7fb3ec107840 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3e9ffb640 1 -- 172.21.15.122:0/218837901 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e40046c0 con 0x7fb3ec107840 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.444+0000 7fb3e9ffb640 1 -- 172.21.15.122:0/218837901 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e402fea0 con 0x7fb3ec107840 2024-09-12T01:46:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.445+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ec1a01c0 con 0x7fb3ec107840 2024-09-12T01:46:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3e9ffb640 1 -- 172.21.15.122:0/218837901 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb3e402f610 con 0x7fb3ec107840 2024-09-12T01:46:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 Processor -- start 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 -- start start 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd05c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556a3cc5d2f0 con 0x556a3ccfddb0 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556a3cc05270 con 0x556a3ccfe6f0 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556a3cc592b0 con 0x556a3cd03780 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3e9ffb640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3dc0a2cd0 0x7fb3dc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41690/0 (socket says 172.21.15.122:41690) 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3ebfff640 1 -- 172.21.15.122:0/2630215002 learned_addr learned my addr 172.21.15.122:0/2630215002 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.447+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd05c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3f1418640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3dc0a2cd0 0x7fb3dc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3e9ffb640 1 -- 172.21.15.122:0/218837901 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb3e40f0780 con 0x7fb3ec107840 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 -- 172.21.15.122:0/2630215002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 msgr2=0x556a3cd05c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd05c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 -- 172.21.15.122:0/2630215002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 msgr2=0x556a3cd031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 -- 172.21.15.122:0/2630215002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3cc04340 con 0x556a3ccfddb0 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfe1b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80134b0 tx=0x7fb3d800a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8957e745aeef5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3f1418640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3dc0a2cd0 0x7fb3dc0a5190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40bcc90 tx=0x7fb3e40dd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3d6ffd640 1 -- 172.21.15.122:0/2630215002 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8019030 con 0x556a3ccfddb0 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3d6ffd640 1 -- 172.21.15.122:0/2630215002 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8005500 con 0x556a3ccfddb0 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.448+0000 7fb3d6ffd640 1 -- 172.21.15.122:0/2630215002 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80116b0 con 0x556a3ccfddb0 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 -- 172.21.15.122:0/2630215002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 msgr2=0x556a3ccfe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfe1b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80134b0 tx=0x7fb3d800a720 comp rx=0 tx=0).stop 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 -- 172.21.15.122:0/2630215002 shutdown_connections 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd05c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/2630215002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfe1b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 -- 172.21.15.122:0/2630215002 >> 172.21.15.122:0/2630215002 conn(0x556a3ccf5010 msgr2=0x556a3ccf5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 -- 172.21.15.122:0/2630215002 shutdown_connections 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 -- 172.21.15.122:0/2630215002 wait complete. 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 -- start start 2024-09-12T01:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.449+0000 7fb3f5c19740 1 Processor -- start 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd07f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd0f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556a3cd06750 con 0x556a3ccfddb0 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556a3cbe6290 con 0x556a3ccfe6f0 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556a3cd068d0 con 0x556a3cd03780 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f5c19740 10 client.0 _ll_register_callbacks cb 0x556a3cc17140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd07f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd07f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41694/0 (socket says 172.21.15.122:41694) 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 -- 172.21.15.122:0/1635532145 learned_addr learned my addr 172.21.15.122:0/1635532145 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 -- 172.21.15.122:0/1635532145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 msgr2=0x556a3cd0f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd0f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 -- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 msgr2=0x556a3cd084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 -- 172.21.15.122:0/1635532145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3c970210 con 0x556a3ccfddb0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd07f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8004e80 tx=0x7fb3d8004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102a6d90e2553126 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.450+0000 7fb3cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3cbfff640 1 -- 172.21.15.122:0/1635532145 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8019040 con 0x556a3ccfddb0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3cbfff640 1 -- 172.21.15.122:0/1635532145 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb3d80051d0 con 0x556a3ccfddb0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d8005500 con 0x556a3ccfddb0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3cbfff640 1 -- 172.21.15.122:0/1635532145 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3d8004020 con 0x556a3ccfddb0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 10 client.15372 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a3ca3c620 con 0x556a3ccfddb0 2024-09-12T01:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3cbfff640 1 -- 172.21.15.122:0/1635532145 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8005500 con 0x556a3ccfddb0 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3cbfff640 1 -- 172.21.15.122:0/1635532145 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb3d8025070 con 0x556a3ccfddb0 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 20 client.15372 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd15db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 4 client.15372 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.451+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a3cc6aa20 con 0x556a3cd13970 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.452+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd15db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.452+0000 7fb3cbfff640 10 client.15372 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.452+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd15db0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ead3cc20444dc1 server_cookie=b1633ab88dbe25fa in_seq=0 out_seq=0 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3cbfff640 1 -- 172.21.15.122:0/1635532145 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a3cc6aa20 con 0x556a3cd13970 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3cbfff640 10 client.15372 handle_command_reply: tid=0 2024-09-12T01:46:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 client.15372 shutdown 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 msgr2=0x556a3cd07f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd07f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8004e80 tx=0x7fb3d8004eb0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 shutdown_connections 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd15db0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3cd0f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3cd084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3cbfff640 0 client.15372 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/1635532145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd07f70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 >> 172.21.15.122:0/1635532145 conn(0x556a3ccf5010 msgr2=0x556a3ccf6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 shutdown_connections 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 1 -- 172.21.15.122:0/1635532145 wait complete. 2024-09-12T01:46:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.453+0000 7fb3f5c19740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 Processor -- start 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 -- start start 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3ccf49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3ccfd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1a9690 con 0x556a3ccfddb0 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1a6f60 con 0x556a3ccfe6f0 2024-09-12T01:46:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1a4d90 con 0x556a3cd03780 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd08540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41702/0 (socket says 172.21.15.122:41702) 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.467+0000 7fb3ebfff640 1 -- 172.21.15.122:0/105181537 learned_addr learned my addr 172.21.15.122:0/105181537 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3ccf49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3ebfff640 1 -- 172.21.15.122:0/105181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 msgr2=0x556a3ccfd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3ccfd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3ebfff640 1 -- 172.21.15.122:0/105181537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 msgr2=0x556a3ccf49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3ccf49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3ebfff640 1 -- 172.21.15.122:0/105181537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3cd06630 con 0x556a3ccfddb0 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3ccf49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd08540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8005dc0 tx=0x7fb3d8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c166a63c6ec01836 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3d67fc640 1 -- 172.21.15.122:0/105181537 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8004810 con 0x556a3ccfddb0 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3d67fc640 1 -- 172.21.15.122:0/105181537 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d800a2d0 con 0x556a3ccfddb0 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3d67fc640 1 -- 172.21.15.122:0/105181537 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8003880 con 0x556a3ccfddb0 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 -- 172.21.15.122:0/105181537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 msgr2=0x556a3cd08540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd08540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8005dc0 tx=0x7fb3d8011e50 comp rx=0 tx=0).stop 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 -- 172.21.15.122:0/105181537 shutdown_connections 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3cd03780 0x556a3ccfd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3ccfe6f0 0x556a3ccf49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/105181537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3cd08540 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.468+0000 7fb3f5c19740 1 -- 172.21.15.122:0/105181537 >> 172.21.15.122:0/105181537 conn(0x556a3ccf5010 msgr2=0x556a3ccf5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 -- 172.21.15.122:0/105181537 shutdown_connections 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 -- 172.21.15.122:0/105181537 wait complete. 2024-09-12T01:46:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 -- start start 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 Processor -- start 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3ccfe6f0 0x556a3ccfcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3cd03780 0x556a3cd0f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1a5420 con 0x556a3ccfddb0 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1a6890 con 0x556a3cd03780 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1a5af0 con 0x556a3ccfe6f0 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3f5c19740 10 client.0 _ll_register_callbacks cb 0x7fb3e000a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41716/0 (socket says 172.21.15.122:41716) 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.469+0000 7fb3ebfff640 1 -- 172.21.15.122:0/3002715770 learned_addr learned my addr 172.21.15.122:0/3002715770 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:46:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3ccfe6f0 0x556a3ccfcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3ebfff640 1 -- 172.21.15.122:0/3002715770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3ccfe6f0 msgr2=0x556a3ccfcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3ccfe6f0 0x556a3ccfcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3ebfff640 1 -- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3cd03780 msgr2=0x556a3cd0f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3cd03780 0x556a3cd0f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3ebfff640 1 -- 172.21.15.122:0/3002715770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3cd80e30 con 0x556a3ccfddb0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3ebfff640 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfc670 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8001eb0 tx=0x7fb3d800a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12145ea7a689c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3cbfff640 1 -- 172.21.15.122:0/3002715770 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80043c0 con 0x556a3ccfddb0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3cbfff640 1 -- 172.21.15.122:0/3002715770 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb3d8013130 con 0x556a3ccfddb0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d800a2d0 con 0x556a3ccfddb0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3cbfff640 1 -- 172.21.15.122:0/3002715770 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3d8019020 con 0x556a3ccfddb0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f5c19740 10 client.15378 fetch_fsmap learned FSMap version 26 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a3cd82010 con 0x556a3ccfddb0 2024-09-12T01:46:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3cbfff640 1 -- 172.21.15.122:0/3002715770 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d800a2d0 con 0x556a3ccfddb0 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3cbfff640 1 -- 172.21.15.122:0/3002715770 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb3d8025040 con 0x556a3ccfddb0 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f5c19740 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f5c19740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.470+0000 7fb3f5c19740 20 client.15378 populate_metadata read hostname 'smithi122' 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.471+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd849a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.471+0000 7fb3f5c19740 4 client.15378 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.471+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a3cc64d70 con 0x556a3cd13970 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.471+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.471+0000 7fb3cbfff640 10 client.15378 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.471+0000 7fb3f0c17640 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd849a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cffc263aa676623d server_cookie=fb5be7372fe61083 in_seq=0 out_seq=0 2024-09-12T01:46:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3cbfff640 1 -- 172.21.15.122:0/3002715770 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a3cc64d70 con 0x556a3cd13970 2024-09-12T01:46:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3cbfff640 10 client.15378 handle_command_reply: tid=0 2024-09-12T01:46:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 client.15378 shutdown 2024-09-12T01:46:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 msgr2=0x556a3ccfc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfc670 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8001eb0 tx=0x7fb3d800a590 comp rx=0 tx=0).stop 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 shutdown_connections 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556a3cd13970 0x556a3cd849a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a3cd03780 0x556a3cd0f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556a3ccfe6f0 0x556a3ccfcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3cbfff640 0 client.15378 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 --2- 172.21.15.122:0/3002715770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a3ccfddb0 0x556a3ccfc670 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 >> 172.21.15.122:0/3002715770 conn(0x556a3ccf5010 msgr2=0x556a3ccf66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 shutdown_connections 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 1 -- 172.21.15.122:0/3002715770 wait complete. 2024-09-12T01:46:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.479+0000 7fb3f5c19740 20 client.15378 trim_cache size 0 max 16384 2024-09-12T01:46:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:46:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:46:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:46:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3dc0a2cd0 msgr2=0x7fb3dc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3dc0a2cd0 0x7fb3dc0a5190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40bcc90 tx=0x7fb3e40dd040 comp rx=0 tx=0).stop 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 msgr2=0x7fb3ec19f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec19f3c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4031ae0 tx=0x7fb3e4003ba0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 shutdown_connections 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3dc0a2cd0 0x7fb3dc0a5190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3ec107840 0x7fb3ec19f3c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3ec104f10 0x7fb3ec198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 --2- 172.21.15.122:0/218837901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ec1025e0 0x7fb3ec197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 >> 172.21.15.122:0/218837901 conn(0x7fb3ec0fc1e0 msgr2=0x7fb3ec102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 shutdown_connections 2024-09-12T01:46:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:46:53.480+0000 7fb3f2e7b640 1 -- 172.21.15.122:0/218837901 wait complete. 2024-09-12T01:46:53.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:53 smithi151 ceph-mon[35229]: pgmap v347: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:53 smithi183 ceph-mon[35127]: pgmap v347: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:53.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:53 smithi122 ceph-mon[30925]: pgmap v347: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T01:46:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:55 smithi183 ceph-mon[35127]: pgmap v348: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-12T01:46:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:55 smithi122 ceph-mon[30925]: pgmap v348: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-12T01:46:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:55 smithi151 ceph-mon[35229]: pgmap v348: 97 pgs: 97 active+clean; 656 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-12T01:46:56.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:56 smithi183 ceph-mon[35127]: pgmap v349: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T01:46:56.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:56 smithi122 ceph-mon[30925]: pgmap v349: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T01:46:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:56 smithi151 ceph-mon[35229]: pgmap v349: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-12T01:46:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:46:59 smithi151 ceph-mon[35229]: pgmap v350: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T01:46:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:46:59 smithi183 ceph-mon[35127]: pgmap v350: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T01:46:59.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:46:59 smithi122 ceph-mon[30925]: pgmap v350: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T01:47:00.789 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:47:01.043 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:01.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 -- 172.21.15.122:0/1973704145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 msgr2=0x7f2c401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 --2- 172.21.15.122:0/1973704145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c401029b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009900 tx=0x7f2c2802a480 comp rx=0 tx=0).stop 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 -- 172.21.15.122:0/1973704145 shutdown_connections 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 --2- 172.21.15.122:0/1973704145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c40102ef0 0x7f2c4010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 --2- 172.21.15.122:0/1973704145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c401029b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 --2- 172.21.15.122:0/1973704145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 0x7f2c40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 -- 172.21.15.122:0/1973704145 >> 172.21.15.122:0/1973704145 conn(0x7f2c400fc7d0 msgr2=0x7f2c400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 -- 172.21.15.122:0/1973704145 shutdown_connections 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.577+0000 7f2c44e83640 1 -- 172.21.15.122:0/1973704145 wait complete. 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 Processor -- start 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 -- start start 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 0x7f2c40198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c40198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c40102ef0 0x7f2c4019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40110ff0 con 0x7f2c40102570 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.578+0000 7f2c44e83640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40110e70 con 0x7f2c40100e20 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c44e83640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40111170 con 0x7f2c40102ef0 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c40198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c40198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46206/0 (socket says 172.21.15.122:46206) 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 -- 172.21.15.122:0/286864342 learned_addr learned my addr 172.21.15.122:0/286864342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 0x7f2c40198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c40102ef0 0x7f2c4019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 -- 172.21.15.122:0/286864342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c40102ef0 msgr2=0x7f2c4019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c40102ef0 0x7f2c4019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 -- 172.21.15.122:0/286864342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 msgr2=0x7f2c40198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 0x7f2c40198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 -- 172.21.15.122:0/286864342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4019d4b0 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 0x7f2c40198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c37fff640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c40198a40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2c28006010 tx=0x7f2c28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c377fe640 1 -- 172.21.15.122:0/286864342 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2803f070 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4019d740 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c377fe640 1 -- 172.21.15.122:0/286864342 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c28003e30 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.579+0000 7f2c377fe640 1 -- 172.21.15.122:0/286864342 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28035030 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.580+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401a5080 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.582+0000 7f2c377fe640 1 -- 172.21.15.122:0/286864342 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2c28042070 con 0x7f2c40102570 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.582+0000 7f2c4725c740 1 Processor -- start 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.582+0000 7f2c4725c740 1 -- start start 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c4725c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b13972b2e0 con 0x55b1397c53a0 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c4725c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b1396d2ce0 con 0x55b1397c5cc0 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c4725c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1397272a0 con 0x55b1397d4a90 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46220/0 (socket says 172.21.15.122:46220) 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 -- 172.21.15.122:0/3875453748 learned_addr learned my addr 172.21.15.122:0/3875453748 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c377fe640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c2c0a2c10 0x7f2c2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c37fff640 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c2c0a2c10 0x7f2c2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c377fe640 1 -- 172.21.15.122:0/286864342 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2c280efc20 con 0x7f2c40102570 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 -- 172.21.15.122:0/3875453748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 msgr2=0x55b1397d6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 -- 172.21.15.122:0/3875453748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 msgr2=0x55b1397d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3ed9d640 1 -- 172.21.15.122:0/3875453748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1396b7440 con 0x55b1397c53a0 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.583+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.584+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397c5780 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800ae40 tx=0x7f2c38017640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c862193f1f49e73e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.584+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c2c0a2c10 0x7f2c2c0a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2c38017b40 tx=0x7f2c3801b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.584+0000 7f2c34ff9640 1 -- 172.21.15.122:0/3875453748 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38027070 con 0x55b1397c53a0 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.584+0000 7f2c34ff9640 1 -- 172.21.15.122:0/3875453748 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c38017e10 con 0x55b1397c53a0 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.584+0000 7f2c34ff9640 1 -- 172.21.15.122:0/3875453748 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38020350 con 0x55b1397c53a0 2024-09-12T01:47:01.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 -- 172.21.15.122:0/3875453748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 msgr2=0x55b1397c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397c5780 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800ae40 tx=0x7f2c38017640 comp rx=0 tx=0).stop 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 -- 172.21.15.122:0/3875453748 shutdown_connections 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3875453748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397c5780 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 -- 172.21.15.122:0/3875453748 >> 172.21.15.122:0/3875453748 conn(0x55b1397c1a30 msgr2=0x55b1397c1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 -- 172.21.15.122:0/3875453748 shutdown_connections 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 -- 172.21.15.122:0/3875453748 wait complete. 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 -- start start 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.585+0000 7f2c4725c740 1 Processor -- start 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c53a0 0x55b1397d4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b1397d7660 con 0x55b1397c5cc0 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b1396e6050 con 0x55b1397c53a0 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1397d77e0 con 0x55b1397d4a90 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 10 client.0 _ll_register_callbacks cb 0x55b139726160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46226/0 (socket says 172.21.15.122:46226) 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 -- 172.21.15.122:0/1235682018 learned_addr learned my addr 172.21.15.122:0/1235682018 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 -- 172.21.15.122:0/1235682018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 msgr2=0x55b1397d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 -- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c53a0 msgr2=0x55b1397d4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c53a0 0x55b1397d4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c53a0 0x55b1397d4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 -- 172.21.15.122:0/1235682018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13949c3c0 con 0x55b1397c5cc0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c53a0 0x55b1397d4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c37fff640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d16f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c280b8cb0 tx=0x7f2c2808b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be41019842b7b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c25ffb640 1 -- 172.21.15.122:0/1235682018 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28049070 con 0x55b1397c5cc0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c25ffb640 1 -- 172.21.15.122:0/1235682018 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2c280ed7e0 con 0x55b1397c5cc0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.586+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c38017e10 con 0x55b1397c5cc0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c25ffb640 1 -- 172.21.15.122:0/1235682018 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c280bc9e0 con 0x55b1397c5cc0 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:01.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1397e30b0 con 0x55b1397c5cc0 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c25ffb640 1 -- 172.21.15.122:0/1235682018 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c38017e10 con 0x55b1397c5cc0 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c25ffb640 1 -- 172.21.15.122:0/1235682018 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2c280c4ac0 con 0x55b1397c5cc0 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 20 client.15390 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b1397e4510 0x55b1397e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 4 client.15390 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b139738850 con 0x55b1397e4510 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.587+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b1397e4510 0x55b1397e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.588+0000 7f2c25ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.588+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b1397e4510 0x55b1397e6940 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6011df44ea6cc57 server_cookie=facc9ceeb3777d7b in_seq=0 out_seq=0 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.588+0000 7f2c25ffb640 1 -- 172.21.15.122:0/1235682018 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b139738850 con 0x55b1397e4510 2024-09-12T01:47:01.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.588+0000 7f2c25ffb640 10 client.15390 handle_command_reply: tid=0 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 client.15390 shutdown 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 msgr2=0x55b1397d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d16f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c280b8cb0 tx=0x7f2c2808b260 comp rx=0 tx=0).stop 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 shutdown_connections 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b1397e4510 0x55b1397e6940 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d16f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 --2- 172.21.15.122:0/1235682018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c53a0 0x55b1397d4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 >> 172.21.15.122:0/1235682018 conn(0x55b1397c1a30 msgr2=0x55b1397c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c25ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 shutdown_connections 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 1 -- 172.21.15.122:0/1235682018 wait complete. 2024-09-12T01:47:01.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.589+0000 7f2c4725c740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 Processor -- start 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 -- start start 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401adc60 con 0x55b1397c53a0 2024-09-12T01:47:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401ab530 con 0x55b1397c5cc0 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.608+0000 7f2c4725c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401a9360 con 0x55b1397d4a90 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397e0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46242/0 (socket says 172.21.15.122:46242) 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 -- 172.21.15.122:0/2211565225 learned_addr learned my addr 172.21.15.122:0/2211565225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c37fff640 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 -- 172.21.15.122:0/2211565225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 msgr2=0x55b1397d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 -- 172.21.15.122:0/2211565225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 msgr2=0x55b1397d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 -- 172.21.15.122:0/2211565225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1397d06a0 con 0x55b1397c53a0 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c37fff640 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397e0900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55b1397c8090 tx=0x7f2c3000a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f34fad721b8730 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c27fff640 1 -- 172.21.15.122:0/2211565225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300043d0 con 0x55b1397c53a0 2024-09-12T01:47:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c27fff640 1 -- 172.21.15.122:0/2211565225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c30003ad0 con 0x55b1397c53a0 2024-09-12T01:47:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.609+0000 7f2c27fff640 1 -- 172.21.15.122:0/2211565225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3001bb70 con 0x55b1397c53a0 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 -- 172.21.15.122:0/2211565225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 msgr2=0x55b1397e0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397e0900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55b1397c8090 tx=0x7f2c3000a600 comp rx=0 tx=0).stop 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 -- 172.21.15.122:0/2211565225 shutdown_connections 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397d4a90 0x55b1397d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397c5cc0 0x55b1397d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 --2- 172.21.15.122:0/2211565225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c53a0 0x55b1397e0900 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 -- 172.21.15.122:0/2211565225 >> 172.21.15.122:0/2211565225 conn(0x55b1397c1a30 msgr2=0x55b1397d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.610+0000 7f2c4725c740 1 -- 172.21.15.122:0/2211565225 shutdown_connections 2024-09-12T01:47:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 -- 172.21.15.122:0/2211565225 wait complete. 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 -- start start 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 Processor -- start 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397c53a0 0x55b1397d9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397d4a90 0x55b1398471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401a99f0 con 0x55b1397c5cc0 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401aae60 con 0x55b1397d4a90 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401aa0c0 con 0x55b1397c53a0 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c4725c740 10 client.0 _ll_register_callbacks cb 0x7f2c28009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46256/0 (socket says 172.21.15.122:46256) 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c37fff640 1 -- 172.21.15.122:0/3893348432 learned_addr learned my addr 172.21.15.122:0/3893348432 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.611+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397c53a0 0x55b1397d9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397d4a90 0x55b1398471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c37fff640 1 -- 172.21.15.122:0/3893348432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397c53a0 msgr2=0x55b1397d9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c37fff640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397c53a0 0x55b1397d9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c37fff640 1 -- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397d4a90 msgr2=0x55b1398471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c37fff640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397d4a90 0x55b1398471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c37fff640 1 -- 172.21.15.122:0/3893348432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b139847870 con 0x55b1397c5cc0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c3ed9d640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397d4a90 0x55b1398471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c37fff640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d9870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2c280c5080 tx=0x7f2c280c4cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ffe24b4cc4648b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c25ffb640 1 -- 172.21.15.122:0/3893348432 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28049070 con 0x55b1397c5cc0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c25ffb640 1 -- 172.21.15.122:0/3893348432 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2c280dc2b0 con 0x55b1397c5cc0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c30003ad0 con 0x55b1397c5cc0 2024-09-12T01:47:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c25ffb640 1 -- 172.21.15.122:0/3893348432 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c280bbe20 con 0x55b1397c5cc0 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c4725c740 10 client.15396 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b139847870 con 0x55b1397c5cc0 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c25ffb640 1 -- 172.21.15.122:0/3893348432 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c280b6e50 con 0x55b1397c5cc0 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.612+0000 7f2c25ffb640 1 -- 172.21.15.122:0/3893348432 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2c2808f040 con 0x55b1397c5cc0 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c4725c740 10 client.15396 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c4725c740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c4725c740 20 client.15396 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b139850110 0x55b139852530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c4725c740 4 client.15396 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2c30003ad0 con 0x55b139850110 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b139850110 0x55b139852530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c25ffb640 10 client.15396 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.613+0000 7f2c3e59c640 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b139850110 0x55b139852530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f7bf88aaa13449 server_cookie=712e2a2a56edac20 in_seq=0 out_seq=0 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c25ffb640 1 -- 172.21.15.122:0/3893348432 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f2c30003ad0 con 0x55b139850110 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c25ffb640 10 client.15396 handle_command_reply: tid=0 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 client.15396 shutdown 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 msgr2=0x55b1397d9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d9870 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2c280c5080 tx=0x7f2c280c4cd0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 shutdown_connections 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b139850110 0x55b139852530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1397d4a90 0x55b1398471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1397c5cc0 0x55b1397d9870 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c25ffb640 0 client.15396 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 --2- 172.21.15.122:0/3893348432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1397c53a0 0x55b1397d9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 >> 172.21.15.122:0/3893348432 conn(0x55b1397c1a30 msgr2=0x55b1397c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 shutdown_connections 2024-09-12T01:47:01.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 1 -- 172.21.15.122:0/3893348432 wait complete. 2024-09-12T01:47:01.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c4725c740 20 client.15396 trim_cache size 0 max 16384 2024-09-12T01:47:01.619 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:01.619 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-09-12T01:47:01.619 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-09-12T01:47:01.619 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:01.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.614+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c2c0a2c10 msgr2=0x7f2c2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c2c0a2c10 0x7f2c2c0a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2c38017b40 tx=0x7f2c3801b000 comp rx=0 tx=0).stop 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 msgr2=0x7f2c40198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c40198a40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2c28006010 tx=0x7f2c28003c10 comp rx=0 tx=0).stop 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 shutdown_connections 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c2c0a2c10 0x7f2c2c0a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c40102ef0 0x7f2c4019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c40102570 0x7f2c40198a40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 --2- 172.21.15.122:0/286864342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c40100e20 0x7f2c40198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 >> 172.21.15.122:0/286864342 conn(0x7f2c400fc7d0 msgr2=0x7f2c4006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 shutdown_connections 2024-09-12T01:47:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:01.615+0000 7f2c44e83640 1 -- 172.21.15.122:0/286864342 wait complete. 2024-09-12T01:47:01.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:01 smithi122 ceph-mon[30925]: pgmap v351: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:01 smithi151 ceph-mon[35229]: pgmap v351: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:01 smithi183 ceph-mon[35127]: pgmap v351: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:02.315 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:47:02.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:47:02.584 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:02.948 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T01:47:02.948 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 -- 172.21.15.122:0/4180474112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c102d20 msgr2=0x7f510c103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 --2- 172.21.15.122:0/4180474112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c102d20 0x7f510c103100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009900 tx=0x7f50fc02a480 comp rx=0 tx=0).stop 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 -- 172.21.15.122:0/4180474112 shutdown_connections 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 --2- 172.21.15.122:0/4180474112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 0x7f510c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 --2- 172.21.15.122:0/4180474112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c103640 0x7f510c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 --2- 172.21.15.122:0/4180474112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c102d20 0x7f510c103100 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 -- 172.21.15.122:0/4180474112 >> 172.21.15.122:0/4180474112 conn(0x7f510c0fc780 msgr2=0x7f510c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 -- 172.21.15.122:0/4180474112 shutdown_connections 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.095+0000 7f5112843640 1 -- 172.21.15.122:0/4180474112 wait complete. 2024-09-12T01:47:03.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 Processor -- start 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 -- start start 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c102d20 0x7f510c100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 0x7f510c100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 0x7f510c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f510c111f50 con 0x7f510c103640 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f510c111dd0 con 0x7f510c10c150 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5112843640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f510c1120d0 con 0x7f510c102d20 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f510b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 0x7f510c100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f510b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 0x7f510c100ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46276/0 (socket says 172.21.15.122:46276) 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f510b7fe640 1 -- 172.21.15.122:0/2798888739 learned_addr learned my addr 172.21.15.122:0/2798888739 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:03.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f510bfff640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c102d20 0x7f510c100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.096+0000 7f5110de0640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 0x7f510c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510b7fe640 1 -- 172.21.15.122:0/2798888739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c102d20 msgr2=0x7f510c100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510b7fe640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c102d20 0x7f510c100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510b7fe640 1 -- 172.21.15.122:0/2798888739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 msgr2=0x7f510c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510b7fe640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 0x7f510c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510b7fe640 1 -- 172.21.15.122:0/2798888739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a68b0 con 0x7f510c103640 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f5110de0640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 0x7f510c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510bfff640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c102d20 0x7f510c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f510b7fe640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 0x7f510c100ec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f510000c970 tx=0x7f510000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100007bb0 con 0x7f510c103640 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5100007d50 con 0x7f510c103640 2024-09-12T01:47:03.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100018820 con 0x7f510c103640 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a6ba0 con 0x7f510c103640 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.097+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a7130 con 0x7f510c103640 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.098+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c103100 con 0x7f510c103640 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.100+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f51000189c0 con 0x7f510c103640 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.100+0000 7f51097fa640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f50f40a2c90 0x7f50f40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.101+0000 7f510bfff640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f50f40a2c90 0x7f50f40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.101+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f51000cf2a0 con 0x7f510c103640 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.101+0000 7f510bfff640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f50f40a2c90 0x7f50f40a5150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005ab0 tx=0x7f50fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:03.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.105+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5100014030 con 0x7f510c103640 2024-09-12T01:47:03.226 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:03.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.339+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f510c101eb0 con 0x7f510c103640 2024-09-12T01:47:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.340+0000 7f51097fa640 1 -- 172.21.15.122:0/2798888739 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f510009c0a0 con 0x7f510c103640 2024-09-12T01:47:03.340 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:47:03.340 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:47:03.341 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f50f40a2c90 msgr2=0x7f50f40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f50f40a2c90 0x7f50f40a5150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005ab0 tx=0x7f50fc005a20 comp rx=0 tx=0).stop 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 msgr2=0x7f510c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 0x7f510c100ec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f510000c970 tx=0x7f510000ce30 comp rx=0 tx=0).stop 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 shutdown_connections 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f50f40a2c90 0x7f50f40a5150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f510c10c150 0x7f510c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c103640 0x7f510c100ec0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 --2- 172.21.15.122:0/2798888739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f510c102d20 0x7f510c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 >> 172.21.15.122:0/2798888739 conn(0x7f510c0fc780 msgr2=0x7f510c0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 shutdown_connections 2024-09-12T01:47:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:03.342+0000 7f5112843640 1 -- 172.21.15.122:0/2798888739 wait complete. 2024-09-12T01:47:03.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:03 smithi122 ceph-mon[30925]: pgmap v352: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:03.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:03 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2798888739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:03 smithi151 ceph-mon[35229]: pgmap v352: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:03 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2798888739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:03 smithi183 ceph-mon[35127]: pgmap v352: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:03 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2798888739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:05 smithi151 ceph-mon[35229]: pgmap v353: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:05 smithi183 ceph-mon[35127]: pgmap v353: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:05.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:05 smithi122 ceph-mon[30925]: pgmap v353: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.570+0000 7f034d81e640 1 -- 172.21.15.122:0/3022253370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 msgr2=0x7f03481020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.570+0000 7f034d81e640 1 --2- 172.21.15.122:0/3022253370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f03481020b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f033c009900 tx=0x7f033c02a410 comp rx=0 tx=0).stop 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 -- 172.21.15.122:0/3022253370 shutdown_connections 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 --2- 172.21.15.122:0/3022253370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03481025f0 0x7f0348108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 --2- 172.21.15.122:0/3022253370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f03481020b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 --2- 172.21.15.122:0/3022253370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0348100a40 0x7f0348100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 -- 172.21.15.122:0/3022253370 >> 172.21.15.122:0/3022253370 conn(0x7f03480fc1e0 msgr2=0x7f03480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 -- 172.21.15.122:0/3022253370 shutdown_connections 2024-09-12T01:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 -- 172.21.15.122:0/3022253370 wait complete. 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 Processor -- start 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.571+0000 7f034d81e640 1 -- start start 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0348100a40 0x7f0348197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f0348198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03481025f0 0x7f034819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f034810d810 con 0x7f0348101c30 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f034810d690 con 0x7f0348100a40 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f034810d990 con 0x7f03481025f0 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f0348198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f0348198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46292/0 (socket says 172.21.15.122:46292) 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 -- 172.21.15.122:0/3938654445 learned_addr learned my addr 172.21.15.122:0/3938654445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f0346ffd640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0348100a40 0x7f0348197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03477fe640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03481025f0 0x7f034819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 -- 172.21.15.122:0/3938654445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03481025f0 msgr2=0x7f034819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03481025f0 0x7f034819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 -- 172.21.15.122:0/3938654445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0348100a40 msgr2=0x7f0348197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0348100a40 0x7f0348197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 -- 172.21.15.122:0/3938654445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034819f9f0 con 0x7f0348101c30 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f03467fc640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f0348198310 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f033c009230 tx=0x7f033c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034c81c640 1 -- 172.21.15.122:0/3938654445 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c03f030 con 0x7f0348101c30 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034c81c640 1 -- 172.21.15.122:0/3938654445 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c003ce0 con 0x7f0348101c30 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034819fc80 con 0x7f0348101c30 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034c81c640 1 -- 172.21.15.122:0/3938654445 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c035070 con 0x7f0348101c30 2024-09-12T01:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.572+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a01c0 con 0x7f0348101c30 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f034c81c640 1 -- 172.21.15.122:0/3938654445 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f033c042030 con 0x7f0348101c30 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 Processor -- start 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 -- start start 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9444da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 0x562fd9449da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd944c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562fd93a82f0 con 0x562fd944a370 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562fd9350270 con 0x562fd94449a0 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f035068e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562fd93a42b0 con 0x562fd94452e0 2024-09-12T01:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.575+0000 7f034c81c640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f03300a2d50 0x7f03300a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd944c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 0x562fd9449da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 0x562fd9449da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43112/0 (socket says 172.21.15.122:43112) 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 -- 172.21.15.122:0/517576704 learned_addr learned my addr 172.21.15.122:0/517576704 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f03477fe640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f03300a2d50 0x7f03300a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f03477fe640 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9444da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 -- 172.21.15.122:0/517576704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 msgr2=0x562fd9444da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9444da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 -- 172.21.15.122:0/517576704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 msgr2=0x562fd944c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd944c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f034c81c640 1 -- 172.21.15.122:0/3938654445 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f033c0ef780 con 0x7f0348101c30 2024-09-12T01:47:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 -- 172.21.15.122:0/517576704 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd934f340 con 0x562fd94452e0 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f03477fe640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f03300a2d50 0x7f03300a5210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0340013ae0 tx=0x7f034000b3a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f0346ffd640 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 0x562fd9449da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f033800bcb0 tx=0x7f03380107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6799351fd87a2c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f032dffb640 1 -- 172.21.15.122:0/517576704 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338004020 con 0x562fd94452e0 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f032dffb640 1 -- 172.21.15.122:0/517576704 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0338005500 con 0x562fd94452e0 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.576+0000 7f032dffb640 1 -- 172.21.15.122:0/517576704 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338004cc0 con 0x562fd94452e0 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 -- 172.21.15.122:0/517576704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 msgr2=0x562fd9449da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 0x562fd9449da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f033800bcb0 tx=0x7f03380107a0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 -- 172.21.15.122:0/517576704 shutdown_connections 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd944c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94452e0 0x562fd9449da0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 --2- 172.21.15.122:0/517576704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9444da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 -- 172.21.15.122:0/517576704 >> 172.21.15.122:0/517576704 conn(0x562fd9440010 msgr2=0x562fd9440410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 -- 172.21.15.122:0/517576704 shutdown_connections 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.577+0000 7f035068e740 1 -- 172.21.15.122:0/517576704 wait complete. 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 -- start start 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 Processor -- start 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd944ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd944f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd944a370 0x562fd9456180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562fd944d340 con 0x562fd94452e0 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562fd9331290 con 0x562fd944a370 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562fd944d4c0 con 0x562fd94449a0 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f035068e740 10 client.0 _ll_register_callbacks cb 0x562fd9362140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd944f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd944f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46302/0 (socket says 172.21.15.122:46302) 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f03467fc640 1 -- 172.21.15.122:0/3072958955 learned_addr learned my addr 172.21.15.122:0/3072958955 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.578+0000 7f0346ffd640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd944ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03477fe640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd944a370 0x562fd9456180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03467fc640 1 -- 172.21.15.122:0/3072958955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 msgr2=0x562fd944ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03467fc640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd944ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03467fc640 1 -- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd944a370 msgr2=0x562fd9456180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03467fc640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd944a370 0x562fd9456180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03467fc640 1 -- 172.21.15.122:0/3072958955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd90bb210 con 0x562fd94452e0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f03467fc640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd944f140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f033c0314e0 tx=0x7f033c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbd12732d1c4859 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f0326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f0326ffd640 1 -- 172.21.15.122:0/3072958955 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c049040 con 0x562fd94452e0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f0326ffd640 1 -- 172.21.15.122:0/3072958955 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f033c0b84f0 con 0x562fd94452e0 2024-09-12T01:47:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0338005500 con 0x562fd94452e0 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f0326ffd640 1 -- 172.21.15.122:0/3072958955 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f033c0bbe80 con 0x562fd94452e0 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f035068e740 10 client.15414 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fd9187620 con 0x562fd94452e0 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f0326ffd640 1 -- 172.21.15.122:0/3072958955 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0338005500 con 0x562fd94452e0 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.579+0000 7f0326ffd640 1 -- 172.21.15.122:0/3072958955 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f033c0c4660 con 0x562fd94452e0 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f035068e740 10 client.15414 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f035068e740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f035068e740 20 client.15414 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f035068e740 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd945ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f035068e740 4 client.15414 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fd93b5a20 con 0x562fd945a890 2024-09-12T01:47:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f0346ffd640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd945ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f0326ffd640 10 client.15414 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.580+0000 7f0346ffd640 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd945ccd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346b866face0a3cc server_cookie=9028049925edfea3 in_seq=0 out_seq=0 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f0326ffd640 1 -- 172.21.15.122:0/3072958955 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fd93b5a20 con 0x562fd945a890 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f0326ffd640 10 client.15414 handle_command_reply: tid=0 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 client.15414 shutdown 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 msgr2=0x562fd944f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd944f140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f033c0314e0 tx=0x7f033c009260 comp rx=0 tx=0).stop 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 shutdown_connections 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd945ccd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f0326ffd640 0 client.15414 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd944a370 0x562fd9456180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd944f140 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 --2- 172.21.15.122:0/3072958955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd944ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 >> 172.21.15.122:0/3072958955 conn(0x562fd9440010 msgr2=0x562fd9441820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 shutdown_connections 2024-09-12T01:47:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 1 -- 172.21.15.122:0/3072958955 wait complete. 2024-09-12T01:47:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.581+0000 7f035068e740 20 client.15414 trim_cache size 0 max 16384 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 Processor -- start 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 -- start start 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd9440ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94452e0 0x562fd9456140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd9441400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03481a96f0 con 0x562fd944a370 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03481a6fc0 con 0x562fd94452e0 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f035068e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03481a4df0 con 0x562fd94449a0 2024-09-12T01:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f03477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd9441400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f0346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd9440ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f03477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd9441400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46304/0 (socket says 172.21.15.122:46304) 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f03477fe640 1 -- 172.21.15.122:0/2018061277 learned_addr learned my addr 172.21.15.122:0/2018061277 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.596+0000 7f03467fc640 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94452e0 0x562fd9456140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03477fe640 1 -- 172.21.15.122:0/2018061277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 msgr2=0x562fd9440ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03477fe640 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd9440ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03477fe640 1 -- 172.21.15.122:0/2018061277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94452e0 msgr2=0x562fd9456140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03477fe640 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94452e0 0x562fd9456140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03477fe640 1 -- 172.21.15.122:0/2018061277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd9440ca0 con 0x562fd944a370 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f0346ffd640 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd9440ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03467fc640 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94452e0 0x562fd9456140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f03477fe640 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd9441400 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0340013270 tx=0x7f0340007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119f08d6797ea53b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f032d7fa640 1 -- 172.21.15.122:0/2018061277 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340025030 con 0x562fd944a370 2024-09-12T01:47:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f032d7fa640 1 -- 172.21.15.122:0/2018061277 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f034000ed40 con 0x562fd944a370 2024-09-12T01:47:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.597+0000 7f032d7fa640 1 -- 172.21.15.122:0/2018061277 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340004310 con 0x562fd944a370 2024-09-12T01:47:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 -- 172.21.15.122:0/2018061277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 msgr2=0x562fd9441400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd9441400 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0340013270 tx=0x7f0340007220 comp rx=0 tx=0).stop 2024-09-12T01:47:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 -- 172.21.15.122:0/2018061277 shutdown_connections 2024-09-12T01:47:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd944a370 0x562fd9441400 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94452e0 0x562fd9456140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 --2- 172.21.15.122:0/2018061277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd94449a0 0x562fd9440ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 -- 172.21.15.122:0/2018061277 >> 172.21.15.122:0/2018061277 conn(0x562fd9440010 msgr2=0x562fd9446480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 -- 172.21.15.122:0/2018061277 shutdown_connections 2024-09-12T01:47:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 -- 172.21.15.122:0/2018061277 wait complete. 2024-09-12T01:47:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 -- start start 2024-09-12T01:47:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.598+0000 7f035068e740 1 Processor -- start 2024-09-12T01:47:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9453960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd9453ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd944a370 0x562fd9454e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03481a5480 con 0x562fd94452e0 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03481a68f0 con 0x562fd94449a0 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03481a5b50 con 0x562fd944a370 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 10 client.0 _ll_register_callbacks cb 0x7f033c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd9453ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd9453ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46306/0 (socket says 172.21.15.122:46306) 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 -- 172.21.15.122:0/1848665555 learned_addr learned my addr 172.21.15.122:0/1848665555 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f0346ffd640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9453960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03477fe640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd944a370 0x562fd9454e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 -- 172.21.15.122:0/1848665555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd944a370 msgr2=0x562fd9454e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd944a370 0x562fd9454e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 -- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 msgr2=0x562fd9453960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9453960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 -- 172.21.15.122:0/1848665555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd9455470 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f03467fc640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd9453ea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f033c0ba130 tx=0x7f033c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1839ac8e87022a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f0326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f0326ffd640 1 -- 172.21.15.122:0/1848665555 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c049070 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f0326ffd640 1 -- 172.21.15.122:0/1848665555 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f033c03b4b0 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034000ed40 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f0326ffd640 1 -- 172.21.15.122:0/1848665555 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f033c0c4af0 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.599+0000 7f035068e740 10 client.15420 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fd94c56e0 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f0326ffd640 1 -- 172.21.15.122:0/1848665555 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f034000ed40 con 0x562fd94452e0 2024-09-12T01:47:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f0326ffd640 1 -- 172.21.15.122:0/1848665555 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f033c031b80 con 0x562fd94452e0 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 20 client.15420 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd94c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 4 client.15420 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fd93afd70 con 0x562fd945a890 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f0346ffd640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd94c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f0326ffd640 10 client.15420 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.600+0000 7f0346ffd640 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd94c7f10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f00939241c822e server_cookie=105ff7695758d372 in_seq=0 out_seq=0 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f0326ffd640 1 -- 172.21.15.122:0/1848665555 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fd93afd70 con 0x562fd945a890 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f0326ffd640 10 client.15420 handle_command_reply: tid=0 2024-09-12T01:47:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 client.15420 shutdown 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 msgr2=0x562fd9453ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd9453ea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f033c0ba130 tx=0x7f033c005c90 comp rx=0 tx=0).stop 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 shutdown_connections 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562fd945a890 0x562fd94c7f10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562fd944a370 0x562fd9454e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562fd94452e0 0x562fd9453ea0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 --2- 172.21.15.122:0/1848665555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fd94449a0 0x562fd9453960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 >> 172.21.15.122:0/1848665555 conn(0x562fd9440010 msgr2=0x562fd943e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f0326ffd640 0 client.15420 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 shutdown_connections 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.604+0000 7f035068e740 1 -- 172.21.15.122:0/1848665555 wait complete. 2024-09-12T01:47:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f035068e740 20 client.15420 trim_cache size 0 max 16384 2024-09-12T01:47:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:47:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:47:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:47:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f03300a2d50 msgr2=0x7f03300a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f03300a2d50 0x7f03300a5210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0340013ae0 tx=0x7f034000b3a0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 msgr2=0x7f0348198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f0348198310 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f033c009230 tx=0x7f033c004360 comp rx=0 tx=0).stop 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 shutdown_connections 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f03300a2d50 0x7f03300a5210 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03481025f0 0x7f034819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0348101c30 0x7f0348198310 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 --2- 172.21.15.122:0/3938654445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0348100a40 0x7f0348197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 >> 172.21.15.122:0/3938654445 conn(0x7f03480fc1e0 msgr2=0x7f0348104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 shutdown_connections 2024-09-12T01:47:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:06.605+0000 7f034d81e640 1 -- 172.21.15.122:0/3938654445 wait complete. 2024-09-12T01:47:07.626 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:47:07.626 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:47:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 -- 172.21.15.122:0/1821514953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c100e20 msgr2=0x7f1d7c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 --2- 172.21.15.122:0/1821514953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c100e20 0x7f1d7c101200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009900 tx=0x7f1d6c02a480 comp rx=0 tx=0).stop 2024-09-12T01:47:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 -- 172.21.15.122:0/1821514953 shutdown_connections 2024-09-12T01:47:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 --2- 172.21.15.122:0/1821514953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 0x7f1d7c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 --2- 172.21.15.122:0/1821514953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d7c102570 0x7f1d7c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 --2- 172.21.15.122:0/1821514953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c100e20 0x7f1d7c101200 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 -- 172.21.15.122:0/1821514953 >> 172.21.15.122:0/1821514953 conn(0x7f1d7c0fc7d0 msgr2=0x7f1d7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 -- 172.21.15.122:0/1821514953 shutdown_connections 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 -- 172.21.15.122:0/1821514953 wait complete. 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 Processor -- start 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.872+0000 7f1d814b8640 1 -- start start 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d814b8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d7c100e20 0x7f1d7c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d814b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 0x7f1d7c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d814b8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 0x7f1d7c1a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d814b8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c110f50 con 0x7f1d7c102570 2024-09-12T01:47:07.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d814b8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c110dd0 con 0x7f1d7c102ef0 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d814b8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1110d0 con 0x7f1d7c100e20 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 0x7f1d7c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 0x7f1d7c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46316/0 (socket says 172.21.15.122:46316) 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 -- 172.21.15.122:0/3030107757 learned_addr learned my addr 172.21.15.122:0/3030107757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7affd640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d7c100e20 0x7f1d7c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7b7fe640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 0x7f1d7c1a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 -- 172.21.15.122:0/3030107757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d7c100e20 msgr2=0x7f1d7c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d7c100e20 0x7f1d7c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 -- 172.21.15.122:0/3030107757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 msgr2=0x7f1d7c1a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 0x7f1d7c1a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 -- 172.21.15.122:0/3030107757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c1a4860 con 0x7f1d7c102570 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7b7fe640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 0x7f1d7c1a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:07.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d7a7fc640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 0x7f1d7c1997e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c920 tx=0x7f1d7000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.873+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70007c70 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.874+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c1a4a30 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.874+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d70007e10 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.874+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70018890 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.874+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c1a4f70 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.874+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7c101200 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.876+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1d70018a70 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.876+0000 7f1d63fff640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d640a2c10 0x7f1d640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.876+0000 7f1d7affd640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d640a2c10 0x7f1d640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.876+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1d700cf3d0 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.877+0000 7f1d7affd640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d640a2c10 0x7f1d640a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c005ab0 tx=0x7f1d6c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:06.880+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d70014030 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.093+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d7c109090 con 0x7f1d7c102570 2024-09-12T01:47:07.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.094+0000 7f1d63fff640 1 -- 172.21.15.122:0/3030107757 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f1d7000de20 con 0x7f1d7c102570 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d640a2c10 msgr2=0x7f1d640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d640a2c10 0x7f1d640a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c005ab0 tx=0x7f1d6c005a20 comp rx=0 tx=0).stop 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 msgr2=0x7f1d7c1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 0x7f1d7c1997e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c920 tx=0x7f1d7000cde0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 shutdown_connections 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d640a2c10 0x7f1d640a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d7c102ef0 0x7f1d7c1a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d7c102570 0x7f1d7c1997e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 --2- 172.21.15.122:0/3030107757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d7c100e20 0x7f1d7c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 >> 172.21.15.122:0/3030107757 conn(0x7f1d7c0fc7d0 msgr2=0x7f1d7c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 shutdown_connections 2024-09-12T01:47:07.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:07.097+0000 7f1d814b8640 1 -- 172.21.15.122:0/3030107757 wait complete. 2024-09-12T01:47:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:07 smithi151 ceph-mon[35229]: pgmap v354: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:07 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3030107757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:07 smithi183 ceph-mon[35127]: pgmap v354: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:07 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3030107757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:07.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:07 smithi122 ceph-mon[30925]: pgmap v354: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T01:47:07.972 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:07 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3030107757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:08.317 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T01:47:08.317 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T01:47:08.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:47:08.584 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 -- 172.21.15.122:0/3085890210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102570 msgr2=0x7f352c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 --2- 172.21.15.122:0/3085890210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102570 0x7f352c1029b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3514009900 tx=0x7f351402a480 comp rx=0 tx=0).stop 2024-09-12T01:47:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 -- 172.21.15.122:0/3085890210 shutdown_connections 2024-09-12T01:47:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 --2- 172.21.15.122:0/3085890210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c102ef0 0x7f352c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 --2- 172.21.15.122:0/3085890210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102570 0x7f352c1029b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 --2- 172.21.15.122:0/3085890210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c100e20 0x7f352c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 -- 172.21.15.122:0/3085890210 >> 172.21.15.122:0/3085890210 conn(0x7f352c0fc7d0 msgr2=0x7f352c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 -- 172.21.15.122:0/3085890210 shutdown_connections 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.130+0000 7f35314bc640 1 -- 172.21.15.122:0/3085890210 wait complete. 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 Processor -- start 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 -- start start 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c100e20 0x7f352c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c102570 0x7f352c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 0x7f352c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f352c110ff0 con 0x7f352c102ef0 2024-09-12T01:47:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f352c110e70 con 0x7f352c102570 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f35314bc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f352c111170 con 0x7f352c100e20 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.131+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 0x7f352c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 0x7f352c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46336/0 (socket says 172.21.15.122:46336) 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 -- 172.21.15.122:0/3880662373 learned_addr learned my addr 172.21.15.122:0/3880662373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352affd640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c100e20 0x7f352c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f35227fc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c102570 0x7f352c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 -- 172.21.15.122:0/3880662373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c100e20 msgr2=0x7f352c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c100e20 0x7f352c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 -- 172.21.15.122:0/3880662373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c102570 msgr2=0x7f352c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c102570 0x7f352c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 -- 172.21.15.122:0/3880662373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c1a69c0 con 0x7f352c102ef0 2024-09-12T01:47:09.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f35227fc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c102570 0x7f352c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f352b7fe640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 0x7f352c0fd700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f352400b970 tx=0x7f352400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f3528ff9640 1 -- 172.21.15.122:0/3880662373 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524019070 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f3528ff9640 1 -- 172.21.15.122:0/3880662373 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524014440 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f3528ff9640 1 -- 172.21.15.122:0/3880662373 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240075d0 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c1a6cb0 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.132+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c1a7210 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.135+0000 7f3528ff9640 1 -- 172.21.15.122:0/3880662373 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f352401c070 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.135+0000 7f3528ff9640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35180a2c90 0x7f35180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.135+0000 7f352affd640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35180a2c90 0x7f35180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.135+0000 7f3528ff9640 1 -- 172.21.15.122:0/3880662373 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f35240ce650 con 0x7f352c102ef0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f352affd640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35180a2c90 0x7f35180a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f351c00abe0 tx=0x7f351c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 Processor -- start 2024-09-12T01:47:09.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 -- start start 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b6ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6b080 0x559c59b75400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b77d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559c59ad02a0 con 0x559c59b75940 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c59a77b10 con 0x559c59b6a760 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.136+0000 7f3533895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c59acc260 con 0x559c59b6b080 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b77d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b77d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46342/0 (socket says 172.21.15.122:46342) 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 -- 172.21.15.122:0/861214707 learned_addr learned my addr 172.21.15.122:0/861214707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:09.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f35227fc640 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6b080 0x559c59b75400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352affd640 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b6ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 -- 172.21.15.122:0/861214707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6b080 msgr2=0x559c59b75400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6b080 0x559c59b75400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 -- 172.21.15.122:0/861214707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 msgr2=0x559c59b6ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b6ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 -- 172.21.15.122:0/861214707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c59841b90 con 0x559c59b75940 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f352b7fe640 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b77d30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f352409ab60 tx=0x7f352409af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdeb2d3cbe3e847 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f35217fa640 1 -- 172.21.15.122:0/861214707 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240cc250 con 0x559c59b75940 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f35217fa640 1 -- 172.21.15.122:0/861214707 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35240149a0 con 0x559c59b75940 2024-09-12T01:47:09.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.137+0000 7f35217fa640 1 -- 172.21.15.122:0/861214707 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240a3840 con 0x559c59b75940 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 -- 172.21.15.122:0/861214707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 msgr2=0x559c59b77d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b77d30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f352409ab60 tx=0x7f352409af20 comp rx=0 tx=0).stop 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 -- 172.21.15.122:0/861214707 shutdown_connections 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b77d30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6b080 0x559c59b75400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 --2- 172.21.15.122:0/861214707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b6ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 -- 172.21.15.122:0/861214707 >> 172.21.15.122:0/861214707 conn(0x559c59b66cf0 msgr2=0x559c59b670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 -- 172.21.15.122:0/861214707 shutdown_connections 2024-09-12T01:47:09.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.138+0000 7f3533895740 1 -- 172.21.15.122:0/861214707 wait complete. 2024-09-12T01:47:09.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 -- start start 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 Processor -- start 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b75940 0x559c59b73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559c59b78510 con 0x559c59b6b080 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c59a586e0 con 0x559c59b75940 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c59b78690 con 0x559c59b6a760 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f3533895740 10 client.0 _ll_register_callbacks cb 0x559c59a7ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.139+0000 7f35227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b72d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46348/0 (socket says 172.21.15.122:46348) 2024-09-12T01:47:09.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 -- 172.21.15.122:0/2971708336 learned_addr learned my addr 172.21.15.122:0/2971708336 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f352b7fe640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b75940 0x559c59b73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f352affd640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 -- 172.21.15.122:0/2971708336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 msgr2=0x559c59b727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 -- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b75940 msgr2=0x559c59b73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b75940 0x559c59b73270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 -- 172.21.15.122:0/2971708336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c59b819d0 con 0x559c59b6b080 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f352b7fe640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b75940 0x559c59b73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35227fc640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b72d30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3514009230 tx=0x7f3514003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22955d7e6e3c080 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35127fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35127fc640 1 -- 172.21.15.122:0/2971708336 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514035040 con 0x559c59b6b080 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f35127fc640 1 -- 172.21.15.122:0/2971708336 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3514003cd0 con 0x559c59b6b080 2024-09-12T01:47:09.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.140+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35240149a0 con 0x559c59b6b080 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f35127fc640 1 -- 172.21.15.122:0/2971708336 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3514005020 con 0x559c59b6b080 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 10 client.15438 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c59b84360 con 0x559c59b6b080 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f35127fc640 1 -- 172.21.15.122:0/2971708336 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35240149a0 con 0x559c59b6b080 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f35127fc640 1 -- 172.21.15.122:0/2971708336 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3514031540 con 0x559c59b6b080 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 10 client.15438 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 10 client.15438 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 20 client.15438 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59b85870 0x559c59b87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 4 client.15438 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.141+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c59add9d0 con 0x559c59b85870 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.142+0000 7f352affd640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59b85870 0x559c59b87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.142+0000 7f352affd640 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59b85870 0x559c59b87c90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b7e3ec30c9c4b38 server_cookie=84b969b5beb2c2a6 in_seq=0 out_seq=0 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.142+0000 7f35127fc640 10 client.15438 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f35127fc640 1 -- 172.21.15.122:0/2971708336 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c59add9d0 con 0x559c59b85870 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f35127fc640 10 client.15438 handle_command_reply: tid=0 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 client.15438 shutdown 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 msgr2=0x559c59b72d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b72d30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3514009230 tx=0x7f3514003820 comp rx=0 tx=0).stop 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 shutdown_connections 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59b85870 0x559c59b87c90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b75940 0x559c59b73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b72d30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 --2- 172.21.15.122:0/2971708336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 >> 172.21.15.122:0/2971708336 conn(0x559c59b66cf0 msgr2=0x559c59b65e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:09.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f35127fc640 0 client.15438 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:09.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 shutdown_connections 2024-09-12T01:47:09.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 1 -- 172.21.15.122:0/2971708336 wait complete. 2024-09-12T01:47:09.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.143+0000 7f3533895740 20 client.15438 trim_cache size 0 max 16384 2024-09-12T01:47:09.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 Processor -- start 2024-09-12T01:47:09.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 -- start start 2024-09-12T01:47:09.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b74760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6b080 0x559c59b730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1b20f0 con 0x559c59b75940 2024-09-12T01:47:09.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1af9c0 con 0x559c59b6b080 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.163+0000 7f3533895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ad7f0 con 0x559c59b6a760 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b8d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46352/0 (socket says 172.21.15.122:46352) 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 -- 172.21.15.122:0/866958064 learned_addr learned my addr 172.21.15.122:0/866958064 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f35227fc640 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6b080 0x559c59b730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352affd640 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b74760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 -- 172.21.15.122:0/866958064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 msgr2=0x559c59b74760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b74760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 -- 172.21.15.122:0/866958064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6b080 msgr2=0x559c59b730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6b080 0x559c59b730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 -- 172.21.15.122:0/866958064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c59b66b20 con 0x559c59b75940 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f35227fc640 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6b080 0x559c59b730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:09.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f352b7fe640 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b8d560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f352400bfd0 tx=0x7f3524014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f1405057a735e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:09.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f3520ff9640 1 -- 172.21.15.122:0/866958064 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240b9070 con 0x559c59b75940 2024-09-12T01:47:09.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f3520ff9640 1 -- 172.21.15.122:0/866958064 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35240149a0 con 0x559c59b75940 2024-09-12T01:47:09.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.164+0000 7f3520ff9640 1 -- 172.21.15.122:0/866958064 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240cc2b0 con 0x559c59b75940 2024-09-12T01:47:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 -- 172.21.15.122:0/866958064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 msgr2=0x559c59b8d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b8d560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f352400bfd0 tx=0x7f3524014690 comp rx=0 tx=0).stop 2024-09-12T01:47:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 -- 172.21.15.122:0/866958064 shutdown_connections 2024-09-12T01:47:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b75940 0x559c59b8d560 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6b080 0x559c59b730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 --2- 172.21.15.122:0/866958064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b6a760 0x559c59b74760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.165+0000 7f3533895740 1 -- 172.21.15.122:0/866958064 >> 172.21.15.122:0/866958064 conn(0x559c59b66cf0 msgr2=0x559c59b74050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 -- 172.21.15.122:0/866958064 shutdown_connections 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 -- 172.21.15.122:0/866958064 wait complete. 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 -- start start 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 Processor -- start 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b7ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b79120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b75940 0x559c59b79660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ade80 con 0x559c59b6b080 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1af2f0 con 0x559c59b6a760 2024-09-12T01:47:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ae550 con 0x559c59b75940 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.166+0000 7f3533895740 10 client.0 _ll_register_callbacks cb 0x7f35140043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b79120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b79120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46362/0 (socket says 172.21.15.122:46362) 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 -- 172.21.15.122:0/3698501600 learned_addr learned my addr 172.21.15.122:0/3698501600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f352b7fe640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b75940 0x559c59b79660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 -- 172.21.15.122:0/3698501600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b75940 msgr2=0x559c59b79660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f352affd640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b7ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b75940 0x559c59b79660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 -- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 msgr2=0x559c59b7ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b7ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 -- 172.21.15.122:0/3698501600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c59b62260 con 0x559c59b6b080 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f352b7fe640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b75940 0x559c59b79660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35227fc640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b79120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3514031960 tx=0x7f3514031d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651f1997a5179c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35127fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35127fc640 1 -- 172.21.15.122:0/3698501600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514035050 con 0x559c59b6b080 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f35127fc640 1 -- 172.21.15.122:0/3698501600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f35140051c0 con 0x559c59b6b080 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.167+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35240149a0 con 0x559c59b6b080 2024-09-12T01:47:09.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f35127fc640 1 -- 172.21.15.122:0/3698501600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3514000e60 con 0x559c59b6b080 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 10 client.15444 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c59bf0110 con 0x559c59b6b080 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f35127fc640 1 -- 172.21.15.122:0/3698501600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35240149a0 con 0x559c59b6b080 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f35127fc640 1 -- 172.21.15.122:0/3698501600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f351404f460 con 0x559c59b6b080 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 10 client.15444 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 10 client.15444 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 20 client.15444 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59bf08d0 0x559c59bf2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 4 client.15444 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x559c59ad7d20 con 0x559c59bf08d0 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.168+0000 7f352affd640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59bf08d0 0x559c59bf2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.169+0000 7f35127fc640 10 client.15444 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.169+0000 7f352affd640 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59bf08d0 0x559c59bf2cf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b53061402559b8 server_cookie=97d9fc5a7127a8f7 in_seq=0 out_seq=0 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.182+0000 7f35127fc640 1 -- 172.21.15.122:0/3698501600 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559c59ad7d20 con 0x559c59bf08d0 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.182+0000 7f35127fc640 10 client.15444 handle_command_reply: tid=0 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.182+0000 7f3533895740 1 client.15444 shutdown 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.182+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 msgr2=0x559c59b79120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.182+0000 7f3533895740 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b79120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3514031960 tx=0x7f3514031d80 comp rx=0 tx=0).stop 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 shutdown_connections 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559c59bf08d0 0x559c59bf2cf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c59b75940 0x559c59b79660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c59b6b080 0x559c59b79120 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 --2- 172.21.15.122:0/3698501600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c59b6a760 0x559c59b7ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 >> 172.21.15.122:0/3698501600 conn(0x559c59b66cf0 msgr2=0x559c59b6c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f35127fc640 0 client.15444 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 shutdown_connections 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 1 -- 172.21.15.122:0/3698501600 wait complete. 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 26, 2024-09-12T01:47:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 2, 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "6977f67f": { 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 2, 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:47:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.5, 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:09.187 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.183+0000 7f3533895740 20 client.15444 trim_cache size 0 max 16384 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35180a2c90 msgr2=0x7f35180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35180a2c90 0x7f35180a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f351c00abe0 tx=0x7f351c00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 msgr2=0x7f352c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 0x7f352c0fd700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f352400b970 tx=0x7f352400bd70 comp rx=0 tx=0).stop 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 shutdown_connections 2024-09-12T01:47:09.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35180a2c90 0x7f35180a5150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352c102ef0 0x7f352c0fd700 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352c102570 0x7f352c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 --2- 172.21.15.122:0/3880662373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c100e20 0x7f352c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:09.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.184+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 >> 172.21.15.122:0/3880662373 conn(0x7f352c0fc7d0 msgr2=0x7f352c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:09.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.185+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 shutdown_connections 2024-09-12T01:47:09.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:09.185+0000 7f35314bc640 1 -- 172.21.15.122:0/3880662373 wait complete. 2024-09-12T01:47:09.194 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:09 smithi151 ceph-mon[35229]: pgmap v355: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T01:47:09.873 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:09 smithi122 ceph-mon[30925]: pgmap v355: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T01:47:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:09 smithi183 ceph-mon[35127]: pgmap v355: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T01:47:09.965 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6977f67f', quiesce took 7.0 seconds. Will release after: 17.5 2024-09-12T01:47:10.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 -- 172.21.15.122:0/2670703155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288106f20 msgr2=0x7f328810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 --2- 172.21.15.122:0/2670703155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288106f20 0x7f328810eaa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3280009ca0 tx=0x7f328002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 -- 172.21.15.122:0/2670703155 shutdown_connections 2024-09-12T01:47:10.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 --2- 172.21.15.122:0/2670703155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288106f20 0x7f328810eaa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 --2- 172.21.15.122:0/2670703155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32881027a0 0x7f32881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 --2- 172.21.15.122:0/2670703155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3288101df0 0x7f32881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 -- 172.21.15.122:0/2670703155 >> 172.21.15.122:0/2670703155 conn(0x7f32880fd8f0 msgr2=0x7f32880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 -- 172.21.15.122:0/2670703155 shutdown_connections 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.408+0000 7f329008d640 1 -- 172.21.15.122:0/2670703155 wait complete. 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 Processor -- start 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 -- start start 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 0x7f32881983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32881027a0 0x7f3288198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3288106f20 0x7f328819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3288111f80 con 0x7f3288101df0 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3288111e00 con 0x7f3288106f20 2024-09-12T01:47:10.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f329008d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3288112100 con 0x7f32881027a0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.409+0000 7f328de29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 0x7f32881983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 0x7f32881983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54236/0 (socket says 172.21.15.122:54236) 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 -- 172.21.15.122:0/3027642978 learned_addr learned my addr 172.21.15.122:0/3027642978 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328e62a640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3288106f20 0x7f328819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 -- 172.21.15.122:0/3027642978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32881027a0 msgr2=0x7f3288198900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32881027a0 0x7f3288198900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 -- 172.21.15.122:0/3027642978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3288106f20 msgr2=0x7f328819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3288106f20 0x7f328819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 -- 172.21.15.122:0/3027642978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328819d370 con 0x7f3288101df0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f328de29640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 0x7f32881983c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f328400d950 tx=0x7f328400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f327effd640 1 -- 172.21.15.122:0/3027642978 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284017070 con 0x7f3288101df0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f327effd640 1 -- 172.21.15.122:0/3027642978 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3284009bb0 con 0x7f3288101df0 2024-09-12T01:47:10.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.410+0000 7f327effd640 1 -- 172.21.15.122:0/3027642978 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400f600 con 0x7f3288101df0 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.411+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328819d660 con 0x7f3288101df0 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.411+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3288074420 con 0x7f3288101df0 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.414+0000 7f327effd640 1 -- 172.21.15.122:0/3027642978 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3284010460 con 0x7f3288101df0 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.414+0000 7f3292466740 1 Processor -- start 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.414+0000 7f3292466740 1 -- start start 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f3292466740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390648c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f3292466740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648c5d0 0x56390648ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f3292466740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x563906499760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f3292466740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639063f22f0 con 0x563906497370 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f3292466740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563906399cf0 con 0x56390648c5d0 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f3292466740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639063ee2b0 con 0x56390648db40 2024-09-12T01:47:10.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328e62a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390648c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328de29640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648c5d0 0x56390648ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328e62a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390648c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34114/0 (socket says 172.21.15.122:34114) 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328e62a640 1 -- 172.21.15.122:0/3623820643 learned_addr learned my addr 172.21.15.122:0/3623820643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328d628640 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x563906499760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f327effd640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f32740a2ce0 0x7f32740a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328e62a640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f32740a2ce0 0x7f32740a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f327effd640 1 -- 172.21.15.122:0/3027642978 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f32840ce620 con 0x7f3288101df0 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328d628640 1 -- 172.21.15.122:0/3623820643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 msgr2=0x56390648c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328d628640 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390648c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328d628640 1 -- 172.21.15.122:0/3623820643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648c5d0 msgr2=0x56390648ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328d628640 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648c5d0 0x56390648ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328d628640 1 -- 172.21.15.122:0/3623820643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390637e3a0 con 0x563906497370 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.415+0000 7f328e62a640 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390648c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.416+0000 7f328de29640 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648c5d0 0x56390648ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.416+0000 7f328d628640 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x563906499760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f32880fdd10 tx=0x7f327800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c29fef4819c24e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.416+0000 7f328e62a640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f32740a2ce0 0x7f32740a51a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3280009ca0 tx=0x7f328002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.416+0000 7f3273fff640 1 -- 172.21.15.122:0/3623820643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278016070 con 0x563906497370 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.416+0000 7f3273fff640 1 -- 172.21.15.122:0/3623820643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327800dda0 con 0x563906497370 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.416+0000 7f3273fff640 1 -- 172.21.15.122:0/3623820643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003dd0 con 0x563906497370 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 -- 172.21.15.122:0/3623820643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 msgr2=0x563906499760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x563906499760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f32880fdd10 tx=0x7f327800bdb0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 -- 172.21.15.122:0/3623820643 shutdown_connections 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x563906499760 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648c5d0 0x56390648ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 --2- 172.21.15.122:0/3623820643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390648c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 -- 172.21.15.122:0/3623820643 >> 172.21.15.122:0/3623820643 conn(0x563906455310 msgr2=0x563906488c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 -- 172.21.15.122:0/3623820643 shutdown_connections 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.417+0000 7f3292466740 1 -- 172.21.15.122:0/3623820643 wait complete. 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.418+0000 7f3292466740 1 -- start start 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.418+0000 7f3292466740 1 Processor -- start 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.418+0000 7f3292466740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x563906496cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f3292466740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648db40 0x563906493f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f3292466740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x5639064944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f3292466740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56390649a070 con 0x563906497370 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f3292466740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5639063ad060 con 0x56390648db40 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f3292466740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56390649a1f0 con 0x56390648c5d0 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f3292466740 10 client.0 _ll_register_callbacks cb 0x5639061008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x5639064944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x5639064944e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54252/0 (socket says 172.21.15.122:54252) 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 -- 172.21.15.122:0/567389916 learned_addr learned my addr 172.21.15.122:0/567389916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328de29640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x563906496cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328d628640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648db40 0x563906493f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 -- 172.21.15.122:0/567389916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 msgr2=0x563906496cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x563906496cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 -- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648db40 msgr2=0x563906493f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648db40 0x563906493f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 -- 172.21.15.122:0/567389916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639061633c0 con 0x563906497370 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328de29640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x563906496cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f328e62a640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x5639064944e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f328002f330 tx=0x7f32800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e25668d1a9050 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.419+0000 7f32717fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f32717fa640 1 -- 172.21.15.122:0/567389916 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280045070 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f32717fa640 1 -- 172.21.15.122:0/567389916 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f32800305c0 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327800dda0 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f32717fa640 1 -- 172.21.15.122:0/567389916 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32800307c0 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 10 client.15456 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639064a5a80 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f32717fa640 1 -- 172.21.15.122:0/567389916 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3280050070 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f32717fa640 1 -- 172.21.15.122:0/567389916 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f328004baa0 con 0x563906497370 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 10 client.15456 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 20 client.15456 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639064a6f40 0x5639064a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 4 client.15456 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:10.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.420+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f327800dda0 con 0x5639064a6f40 2024-09-12T01:47:10.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.421+0000 7f328de29640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639064a6f40 0x5639064a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.421+0000 7f328de29640 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639064a6f40 0x5639064a9370 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49a55f7a69aa9b0d server_cookie=5982489f8ad3ff04 in_seq=0 out_seq=0 2024-09-12T01:47:10.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.421+0000 7f32717fa640 10 client.15456 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:10.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f32717fa640 1 -- 172.21.15.122:0/567389916 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f327800dda0 con 0x5639064a6f40 2024-09-12T01:47:10.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f32717fa640 10 client.15456 handle_command_reply: tid=0 2024-09-12T01:47:10.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 client.15456 shutdown 2024-09-12T01:47:10.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 msgr2=0x5639064944e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x5639064944e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f328002f330 tx=0x7f32800095c0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 shutdown_connections 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639064a6f40 0x5639064a9370 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563906497370 0x5639064944e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56390648db40 0x563906493f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f32717fa640 0 client.15456 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 --2- 172.21.15.122:0/567389916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x563906496cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.422+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 >> 172.21.15.122:0/567389916 conn(0x563906455310 msgr2=0x56390648e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.423+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 shutdown_connections 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.423+0000 7f3292466740 1 -- 172.21.15.122:0/567389916 wait complete. 2024-09-12T01:47:10.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.423+0000 7f3292466740 20 client.15456 trim_cache size 0 max 16384 2024-09-12T01:47:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 Processor -- start 2024-09-12T01:47:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 -- start start 2024-09-12T01:47:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x5639064a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 0x56390648bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x5639064aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32881b2170 con 0x56390648db40 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32881afa40 con 0x563906497370 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.447+0000 7f3292466740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32881ad870 con 0x56390648c5d0 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 0x56390648bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 0x56390648bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54254/0 (socket says 172.21.15.122:54254) 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 -- 172.21.15.122:0/1574834841 learned_addr learned my addr 172.21.15.122:0/1574834841 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328de29640 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x5639064a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 -- 172.21.15.122:0/1574834841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 msgr2=0x5639064a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x5639064a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 -- 172.21.15.122:0/1574834841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 msgr2=0x5639064aec40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:47:10.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x5639064aec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 -- 172.21.15.122:0/1574834841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639064a99f0 con 0x56390648db40 2024-09-12T01:47:10.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f328d628640 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 0x56390648bbd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x56390648d070 tx=0x7f32780035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab856725585abe78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:10.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.448+0000 7f32737fe640 1 -- 172.21.15.122:0/1574834841 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278016040 con 0x56390648db40 2024-09-12T01:47:10.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.449+0000 7f32737fe640 1 -- 172.21.15.122:0/1574834841 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3278003760 con 0x56390648db40 2024-09-12T01:47:10.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.449+0000 7f32737fe640 1 -- 172.21.15.122:0/1574834841 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278005540 con 0x56390648db40 2024-09-12T01:47:10.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 -- 172.21.15.122:0/1574834841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 msgr2=0x56390648bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 0x56390648bbd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x56390648d070 tx=0x7f32780035f0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 -- 172.21.15.122:0/1574834841 shutdown_connections 2024-09-12T01:47:10.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x5639064aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648db40 0x56390648bbd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 --2- 172.21.15.122:0/1574834841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648c5d0 0x5639064a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 -- 172.21.15.122:0/1574834841 >> 172.21.15.122:0/1574834841 conn(0x563906455310 msgr2=0x563906492850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 -- 172.21.15.122:0/1574834841 shutdown_connections 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.450+0000 7f3292466740 1 -- 172.21.15.122:0/1574834841 wait complete. 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f3292466740 1 -- start start 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f3292466740 1 Processor -- start 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f3292466740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 0x56390649b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f3292466740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390649b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f328de29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 0x56390649b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f328de29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 0x56390649b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54268/0 (socket says 172.21.15.122:54268) 2024-09-12T01:47:10.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f328de29640 1 -- 172.21.15.122:0/2466452986 learned_addr learned my addr 172.21.15.122:0/2466452986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:10.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.451+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x56390649be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32881adf00 con 0x56390648c5d0 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32881af370 con 0x563906497370 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32881ae5d0 con 0x56390648db40 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f3292466740 10 client.0 _ll_register_callbacks cb 0x7f3280031ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328d628640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390649b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328de29640 1 -- 172.21.15.122:0/2466452986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 msgr2=0x56390649b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328de29640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390649b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328e62a640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x56390649be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328de29640 1 -- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 msgr2=0x56390649be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328de29640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x56390649be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328de29640 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563906510e10 con 0x56390648c5d0 2024-09-12T01:47:10.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328e62a640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x56390649be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f328de29640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 0x56390649b3e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3284097180 tx=0x7f3284000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b04605f6c960da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.452+0000 7f32717fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.453+0000 7f32717fa640 1 -- 172.21.15.122:0/2466452986 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284020040 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.453+0000 7f32717fa640 1 -- 172.21.15.122:0/2466452986 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3284097390 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.453+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278003760 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.453+0000 7f32717fa640 1 -- 172.21.15.122:0/2466452986 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f328409f0d0 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.453+0000 7f3292466740 10 client.15462 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.453+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639065120d0 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f32717fa640 1 -- 172.21.15.122:0/2466452986 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32840a3990 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f32717fa640 1 -- 172.21.15.122:0/2466452986 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f328409a290 con 0x56390648c5d0 2024-09-12T01:47:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f3292466740 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f3292466740 10 client.15462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:47:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f3292466740 20 client.15462 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639065127f0 0x563906514c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f3292466740 4 client.15462 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:47:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3278003760 con 0x5639065127f0 2024-09-12T01:47:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.454+0000 7f328d628640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639065127f0 0x563906514c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f32717fa640 10 client.15462 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f328d628640 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639065127f0 0x563906514c00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95a901e03ed2d174 server_cookie=a750ba7cacc6564d in_seq=0 out_seq=0 2024-09-12T01:47:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f32717fa640 1 -- 172.21.15.122:0/2466452986 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3278003760 con 0x5639065127f0 2024-09-12T01:47:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f32717fa640 10 client.15462 handle_command_reply: tid=0 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f3292466740 1 client.15462 shutdown 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 msgr2=0x56390649b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.455+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 0x56390649b3e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3284097180 tx=0x7f3284000bc0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 shutdown_connections 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x5639065127f0 0x563906514c00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563906497370 0x56390649be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56390648db40 0x56390649b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 --2- 172.21.15.122:0/2466452986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56390648c5d0 0x56390649b3e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 >> 172.21.15.122:0/2466452986 conn(0x563906455310 msgr2=0x5639064874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f32717fa640 0 client.15462 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 shutdown_connections 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 1 -- 172.21.15.122:0/2466452986 wait complete. 2024-09-12T01:47:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.456+0000 7f3292466740 20 client.15462 trim_cache size 0 max 16384 2024-09-12T01:47:10.460 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:47:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f32740a2ce0 msgr2=0x7f32740a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f32740a2ce0 0x7f32740a51a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3280009ca0 tx=0x7f328002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 msgr2=0x7f32881983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 0x7f32881983c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f328400d950 tx=0x7f328400de10 comp rx=0 tx=0).stop 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 shutdown_connections 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f32740a2ce0 0x7f32740a51a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3288106f20 0x7f328819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32881027a0 0x7f3288198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 --2- 172.21.15.122:0/3027642978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3288101df0 0x7f32881983c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 >> 172.21.15.122:0/3027642978 conn(0x7f32880fd8f0 msgr2=0x7f32880fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 shutdown_connections 2024-09-12T01:47:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:10.457+0000 7f329008d640 1 -- 172.21.15.122:0/3027642978 wait complete. 2024-09-12T01:47:11.125 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:47:11.390 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:11.795 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:11 smithi122 ceph-mon[30925]: pgmap v356: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T01:47:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:11 smithi151 ceph-mon[35229]: pgmap v356: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 -- 172.21.15.122:0/1582330366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8106a70 msgr2=0x7f07a8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1582330366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8106a70 0x7f07a8106e50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f079c009930 tx=0x7f079c02a420 comp rx=0 tx=0).stop 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 -- 172.21.15.122:0/1582330366 shutdown_connections 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1582330366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8101410 0x7f07a810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1582330366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 0x7f07a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1582330366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8106a70 0x7f07a8106e50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 -- 172.21.15.122:0/1582330366 >> 172.21.15.122:0/1582330366 conn(0x7f07a80fc790 msgr2=0x7f07a80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 -- 172.21.15.122:0/1582330366 shutdown_connections 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.898+0000 7f07aefb1640 1 -- 172.21.15.122:0/1582330366 wait complete. 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.899+0000 7f07aefb1640 1 Processor -- start 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.899+0000 7f07aefb1640 1 -- start start 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.899+0000 7f07aefb1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 0x7f07a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.899+0000 7f07aefb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8101410 0x7f07a819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.899+0000 7f07aefb1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8106a70 0x7f07a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07acd4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 0x7f07a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8101410 0x7f07a819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07acd4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 0x7f07a819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34146/0 (socket says 172.21.15.122:34146) 2024-09-12T01:47:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07acd4d640 1 -- 172.21.15.122:0/1045358356 learned_addr learned my addr 172.21.15.122:0/1045358356 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8110e70 con 0x7f07a8101410 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8110cf0 con 0x7f07a8106a70 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8110ff0 con 0x7f07a8100a70 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07ad54e640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8106a70 0x7f07a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07a7fff640 1 -- 172.21.15.122:0/1045358356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 msgr2=0x7f07a819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07a7fff640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 0x7f07a819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07a7fff640 1 -- 172.21.15.122:0/1045358356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8106a70 msgr2=0x7f07a819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07a7fff640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8106a70 0x7f07a819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07a7fff640 1 -- 172.21.15.122:0/1045358356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a819f620 con 0x7f07a8101410 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.900+0000 7f07ad54e640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8106a70 0x7f07a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.901+0000 7f07a7fff640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8101410 0x7f07a819abb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0794005fd0 tx=0x7f079400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.901+0000 7f07a5ffb640 1 -- 172.21.15.122:0/1045358356 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794009c30 con 0x7f07a8101410 2024-09-12T01:47:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.901+0000 7f07a5ffb640 1 -- 172.21.15.122:0/1045358356 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0794009dd0 con 0x7f07a8101410 2024-09-12T01:47:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.901+0000 7f07a5ffb640 1 -- 172.21.15.122:0/1045358356 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794010460 con 0x7f07a8101410 2024-09-12T01:47:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.901+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a819f910 con 0x7f07a8101410 2024-09-12T01:47:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.901+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a81a71f0 con 0x7f07a8101410 2024-09-12T01:47:11.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.904+0000 7f07b138a740 1 Processor -- start 2024-09-12T01:47:11.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.904+0000 7f07b138a740 1 -- start start 2024-09-12T01:47:11.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.904+0000 7f07a5ffb640 1 -- 172.21.15.122:0/1045358356 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f0794003f50 con 0x7f07a8101410 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.905+0000 7f07a5ffb640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07980a2d10 0x7f07980a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.905+0000 7f07ad54e640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07980a2d10 0x7f07980a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07a5ffb640 1 -- 172.21.15.122:0/1045358356 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f07940cf020 con 0x7f07a8101410 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07b138a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8333d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07b138a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8333da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07b138a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db8334a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8333d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07b138a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55db832a32f0 con 0x55db83348370 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07b138a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55db8324acf0 con 0x55db8333d5d0 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07b138a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55db8329f2b0 con 0x55db8333eb40 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8333d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34152/0 (socket says 172.21.15.122:34152) 2024-09-12T01:47:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 -- 172.21.15.122:0/1720026475 learned_addr learned my addr 172.21.15.122:0/1720026475 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07ad54e640 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db8334a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07a7fff640 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8333da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07ad54e640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07980a2d10 0x7f07980a51d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009fd0 tx=0x7f07a0010a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 -- 172.21.15.122:0/1720026475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 msgr2=0x55db8333da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8333da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 -- 172.21.15.122:0/1720026475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 msgr2=0x55db8334a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db8334a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07acd4d640 1 -- 172.21.15.122:0/1720026475 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8322f3a0 con 0x55db8333eb40 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.906+0000 7f07a7fff640 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8333da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.907+0000 7f07acd4d640 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8333d090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f079c006010 tx=0x7f079c042b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46226268de4d3a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:11.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.907+0000 7f0792ffd640 1 -- 172.21.15.122:0/1720026475 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c035040 con 0x55db8333eb40 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.907+0000 7f0792ffd640 1 -- 172.21.15.122:0/1720026475 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f079c031870 con 0x55db8333eb40 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.907+0000 7f0792ffd640 1 -- 172.21.15.122:0/1720026475 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c004a00 con 0x55db8333eb40 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 -- 172.21.15.122:0/1720026475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 msgr2=0x55db8333d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8333d090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f079c006010 tx=0x7f079c042b70 comp rx=0 tx=0).stop 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 -- 172.21.15.122:0/1720026475 shutdown_connections 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db8334a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8333da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 --2- 172.21.15.122:0/1720026475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8333d090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 -- 172.21.15.122:0/1720026475 >> 172.21.15.122:0/1720026475 conn(0x55db83306310 msgr2=0x55db83339c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 -- 172.21.15.122:0/1720026475 shutdown_connections 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 -- 172.21.15.122:0/1720026475 wait complete. 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.908+0000 7f07b138a740 1 -- start start 2024-09-12T01:47:11.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 Processor -- start 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db83347d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db83345020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db83345610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55db8334b010 con 0x55db83348370 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55db8325e060 con 0x55db8333d5d0 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07b138a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55db8334b190 con 0x55db8333eb40 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07ad54e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db83345610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07ad54e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db83345610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54310/0 (socket says 172.21.15.122:54310) 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.909+0000 7f07ad54e640 1 -- 172.21.15.122:0/2776117261 learned_addr learned my addr 172.21.15.122:0/2776117261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:11.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07b138a740 10 client.0 _ll_register_callbacks cb 0x55db82fb18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07a7fff640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db83345020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07acd4d640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db83347d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07ad54e640 1 -- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 msgr2=0x55db83345020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07ad54e640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db83345020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07ad54e640 1 -- 172.21.15.122:0/2776117261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 msgr2=0x55db83347d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07ad54e640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db83347d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07ad54e640 1 -- 172.21.15.122:0/2776117261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db830143c0 con 0x55db83348370 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07acd4d640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db83347d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07ad54e640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db83345610 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f07a001f460 tx=0x7f07a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c72f9ce4f3dffb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f0787fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f0787fff640 1 -- 172.21.15.122:0/2776117261 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0012850 con 0x55db83348370 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f079c031870 con 0x55db83348370 2024-09-12T01:47:11.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.910+0000 7f0787fff640 1 -- 172.21.15.122:0/2776117261 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f07a00129f0 con 0x55db83348370 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f0787fff640 1 -- 172.21.15.122:0/2776117261 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07a0012bf0 con 0x55db83348370 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 10 client.15474 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55db83356b10 con 0x55db83348370 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f0787fff640 1 -- 172.21.15.122:0/2776117261 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a0012d90 con 0x55db83348370 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f0787fff640 1 -- 172.21.15.122:0/2776117261 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f07a0026070 con 0x55db83348370 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 10 client.15474 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 10 client.15474 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 20 client.15474 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db83357f20 0x55db8335a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 4 client.15474 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.911+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f079c031870 con 0x55db83357f20 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.912+0000 7f07acd4d640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db83357f20 0x55db8335a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.912+0000 7f07acd4d640 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db83357f20 0x55db8335a3c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c64e7f437dd36e5 server_cookie=bb884d5bf4ff3e25 in_seq=0 out_seq=0 2024-09-12T01:47:11.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.912+0000 7f0787fff640 10 client.15474 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f0787fff640 1 -- 172.21.15.122:0/2776117261 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f079c031870 con 0x55db83357f20 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f0787fff640 10 client.15474 handle_command_reply: tid=0 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 client.15474 shutdown 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 msgr2=0x55db83345610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db83345610 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f07a001f460 tx=0x7f07a0004ab0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 shutdown_connections 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db83357f20 0x55db8335a3c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db83345610 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db83345020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 --2- 172.21.15.122:0/2776117261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db83347d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.913+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 >> 172.21.15.122:0/2776117261 conn(0x55db83306310 msgr2=0x55db8333f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.914+0000 7f0787fff640 0 client.15474 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:11.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.914+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 shutdown_connections 2024-09-12T01:47:11.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.914+0000 7f07b138a740 1 -- 172.21.15.122:0/2776117261 wait complete. 2024-09-12T01:47:11.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.914+0000 7f07b138a740 20 client.15474 trim_cache size 0 max 16384 2024-09-12T01:47:11.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.938+0000 7f07b138a740 1 Processor -- start 2024-09-12T01:47:11.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.938+0000 7f07b138a740 1 -- start start 2024-09-12T01:47:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.938+0000 7f07b138a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db8333d5d0 0x55db833542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07b138a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db833438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07b138a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db83348370 0x55db83343e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07b138a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81add10 con 0x55db8333d5d0 2024-09-12T01:47:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07b138a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81ab5e0 con 0x55db83348370 2024-09-12T01:47:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07b138a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81a9430 con 0x55db8333eb40 2024-09-12T01:47:11.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07ad54e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db83348370 0x55db83343e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07a7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db833438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07a7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db833438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34168/0 (socket says 172.21.15.122:34168) 2024-09-12T01:47:11.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07ad54e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db83348370 0x55db83343e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52014/0 (socket says 172.21.15.122:52014) 2024-09-12T01:47:11.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07a7fff640 1 -- 172.21.15.122:0/246999965 learned_addr learned my addr 172.21.15.122:0/246999965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07a7fff640 1 -- 172.21.15.122:0/246999965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db83348370 msgr2=0x55db83343e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07a7fff640 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db83348370 0x55db83343e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.939+0000 7f07a7fff640 1 -- 172.21.15.122:0/246999965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db8333d5d0 msgr2=0x55db833542b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07acd4d640 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db8333d5d0 0x55db833542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07a7fff640 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db8333d5d0 0x55db833542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07a7fff640 1 -- 172.21.15.122:0/246999965 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db83339d70 con 0x55db8333eb40 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07acd4d640 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db8333d5d0 0x55db833542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07a7fff640 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db833438f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0794005d00 tx=0x7f07940973f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2067176c22448d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07927fc640 1 -- 172.21.15.122:0/246999965 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794020040 con 0x55db8333eb40 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07927fc640 1 -- 172.21.15.122:0/246999965 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07940b4070 con 0x55db8333eb40 2024-09-12T01:47:11.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.940+0000 7f07927fc640 1 -- 172.21.15.122:0/246999965 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079409b9f0 con 0x55db8333eb40 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 -- 172.21.15.122:0/246999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 msgr2=0x55db833438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db833438f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0794005d00 tx=0x7f07940973f0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 -- 172.21.15.122:0/246999965 shutdown_connections 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db83348370 0x55db83343e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db833438f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 --2- 172.21.15.122:0/246999965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db8333d5d0 0x55db833542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 -- 172.21.15.122:0/246999965 >> 172.21.15.122:0/246999965 conn(0x55db83306310 msgr2=0x55db83344fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:11.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.941+0000 7f07b138a740 1 -- 172.21.15.122:0/246999965 shutdown_connections 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 -- 172.21.15.122:0/246999965 wait complete. 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 -- start start 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 Processor -- start 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8334b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8334b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db833c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81a9aa0 con 0x55db83348370 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81aaf10 con 0x55db8333d5d0 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81aa170 con 0x55db8333eb40 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.942+0000 7f07b138a740 10 client.0 _ll_register_callbacks cb 0x7f07a000ed00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db833c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db833c0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54330/0 (socket says 172.21.15.122:54330) 2024-09-12T01:47:11.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 -- 172.21.15.122:0/720564591 learned_addr learned my addr 172.21.15.122:0/720564591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07a7fff640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8334b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07acd4d640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8334b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 -- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 msgr2=0x55db8334b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8334b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 -- 172.21.15.122:0/720564591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 msgr2=0x55db8334b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8334b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 -- 172.21.15.122:0/720564591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8334c1f0 con 0x55db83348370 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07ad54e640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db833c0ca0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07a00329b0 tx=0x7f07a0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d268a8d0612346 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f0787fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f0787fff640 1 -- 172.21.15.122:0/720564591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0012850 con 0x55db83348370 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f0787fff640 1 -- 172.21.15.122:0/720564591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f07a00129f0 con 0x55db83348370 2024-09-12T01:47:11.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.943+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07940b4070 con 0x55db83348370 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.944+0000 7f0787fff640 1 -- 172.21.15.122:0/720564591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07a00329e0 con 0x55db83348370 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.944+0000 7f07b138a740 10 client.15477 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.944+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55db833c2220 con 0x55db83348370 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.944+0000 7f0787fff640 1 -- 172.21.15.122:0/720564591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a0032b80 con 0x55db83348370 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.944+0000 7f0787fff640 1 -- 172.21.15.122:0/720564591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f07a0026040 con 0x55db83348370 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07b138a740 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07b138a740 10 client.15477 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07b138a740 20 client.15477 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07b138a740 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db833c2940 0x55db833c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07b138a740 4 client.15477 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f07940b4070 con 0x55db833c2940 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07acd4d640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db833c2940 0x55db833c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:11.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f0787fff640 10 client.15477 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:11.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.945+0000 7f07acd4d640 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db833c2940 0x55db833c4d60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1663667b5660749 server_cookie=ddd02adda41f7230 in_seq=0 out_seq=0 2024-09-12T01:47:11.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f0787fff640 1 -- 172.21.15.122:0/720564591 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f07940b4070 con 0x55db833c2940 2024-09-12T01:47:11.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f0787fff640 10 client.15477 handle_command_reply: tid=0 2024-09-12T01:47:11.950 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:47:11.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 client.15477 shutdown 2024-09-12T01:47:11.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 msgr2=0x55db833c0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db833c0ca0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07a00329b0 tx=0x7f07a0005d20 comp rx=0 tx=0).stop 2024-09-12T01:47:11.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 shutdown_connections 2024-09-12T01:47:11.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x55db833c2940 0x55db833c4d60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55db83348370 0x55db833c0ca0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55db8333eb40 0x55db8334b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 --2- 172.21.15.122:0/720564591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55db8333d5d0 0x55db8334b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 >> 172.21.15.122:0/720564591 conn(0x55db83306310 msgr2=0x55db83343780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f0787fff640 0 client.15477 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 shutdown_connections 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 1 -- 172.21.15.122:0/720564591 wait complete. 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.946+0000 7f07b138a740 20 client.15477 trim_cache size 0 max 16384 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07980a2d10 msgr2=0x7f07980a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07980a2d10 0x7f07980a51d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009fd0 tx=0x7f07a0010a60 comp rx=0 tx=0).stop 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8101410 msgr2=0x7f07a819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8101410 0x7f07a819abb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0794005fd0 tx=0x7f079400dde0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 shutdown_connections 2024-09-12T01:47:11.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07980a2d10 0x7f07980a51d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07a8106a70 0x7f07a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07a8101410 0x7f07a819abb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 --2- 172.21.15.122:0/1045358356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07a8100a70 0x7f07a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:11.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 >> 172.21.15.122:0/1045358356 conn(0x7f07a80fc790 msgr2=0x7f07a80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:11.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 shutdown_connections 2024-09-12T01:47:11.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:11.947+0000 7f07aefb1640 1 -- 172.21.15.122:0/1045358356 wait complete. 2024-09-12T01:47:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:11 smithi183 ceph-mon[35127]: pgmap v356: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T01:47:12.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:47:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:13.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 -- 172.21.15.122:0/4196475140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 msgr2=0x7fb41010ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 --2- 172.21.15.122:0/4196475140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb41010ec10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb408009cd0 tx=0x7fb40802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 -- 172.21.15.122:0/4196475140 shutdown_connections 2024-09-12T01:47:13.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 --2- 172.21.15.122:0/4196475140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb41010ec10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 --2- 172.21.15.122:0/4196475140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb410103720 0x7fb410107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 --2- 172.21.15.122:0/4196475140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb410102d70 0x7fb410103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 -- 172.21.15.122:0/4196475140 >> 172.21.15.122:0/4196475140 conn(0x7fb4100fc7d0 msgr2=0x7fb4100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 -- 172.21.15.122:0/4196475140 shutdown_connections 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.517+0000 7fb416e2e640 1 -- 172.21.15.122:0/4196475140 wait complete. 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 Processor -- start 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 -- start start 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb410102d70 0x7fb4101009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb410103720 0x7fb410100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb410101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101120c0 con 0x7fb4101082a0 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb410111f40 con 0x7fb410102d70 2024-09-12T01:47:13.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb416e2e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112240 con 0x7fb410103720 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb4153cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb410101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb4153cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb410101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54360/0 (socket says 172.21.15.122:54360) 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb4153cb640 1 -- 172.21.15.122:0/2028315857 learned_addr learned my addr 172.21.15.122:0/2028315857 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.518+0000 7fb414bca640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb410102d70 0x7fb4101009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb4153cb640 1 -- 172.21.15.122:0/2028315857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb410103720 msgr2=0x7fb410100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb410103720 0x7fb410100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb4153cb640 1 -- 172.21.15.122:0/2028315857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb410102d70 msgr2=0x7fb4101009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb410102d70 0x7fb4101009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb4153cb640 1 -- 172.21.15.122:0/2028315857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a69f0 con 0x7fb4101082a0 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb410101450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb408005cc0 tx=0x7fb408003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb40dffb640 1 -- 172.21.15.122:0/2028315857 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408004040 con 0x7fb4101082a0 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb40dffb640 1 -- 172.21.15.122:0/2028315857 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4080046c0 con 0x7fb4101082a0 2024-09-12T01:47:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb40dffb640 1 -- 172.21.15.122:0/2028315857 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408046480 con 0x7fb4101082a0 2024-09-12T01:47:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.519+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a6c80 con 0x7fb4101082a0 2024-09-12T01:47:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.520+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a71e0 con 0x7fb4101082a0 2024-09-12T01:47:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.523+0000 7fb40dffb640 1 -- 172.21.15.122:0/2028315857 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb4080305a0 con 0x7fb4101082a0 2024-09-12T01:47:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.523+0000 7fb40dffb640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4000a2c90 0x7fb4000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.523+0000 7fb419207740 1 Processor -- start 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.523+0000 7fb40dffb640 1 -- 172.21.15.122:0/2028315857 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb4080f0760 con 0x7fb4101082a0 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.523+0000 7fb414bca640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4000a2c90 0x7fb4000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 -- start start 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa5964b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 0x55aa5964ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59658760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aa595b12f0 con 0x55aa5964b5d0 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa59558cf0 con 0x55aa5964cb40 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb419207740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aa595ad2b0 con 0x55aa59656370 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 0x55aa5964ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 0x55aa5964ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54364/0 (socket says 172.21.15.122:54364) 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb40ffff640 1 -- 172.21.15.122:0/2180609452 learned_addr learned my addr 172.21.15.122:0/2180609452 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb414bca640 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa5964b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59658760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb414bca640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4000a2c90 0x7fb4000a5150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb404005fd0 tx=0x7fb404005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.524+0000 7fb40ffff640 1 -- 172.21.15.122:0/2180609452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 msgr2=0x55aa59658760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59658760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb40ffff640 1 -- 172.21.15.122:0/2180609452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 msgr2=0x55aa5964b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa5964b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb40ffff640 1 -- 172.21.15.122:0/2180609452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa5953d3a0 con 0x55aa5964b5d0 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59658760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 0x55aa5964ba10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b9f0 tx=0x7fb3fc00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8f380b9517271a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb3faffd640 1 -- 172.21.15.122:0/2180609452 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc016070 con 0x55aa5964b5d0 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb3faffd640 1 -- 172.21.15.122:0/2180609452 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc00dbe0 con 0x55aa5964b5d0 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.525+0000 7fb3faffd640 1 -- 172.21.15.122:0/2180609452 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc00f620 con 0x55aa5964b5d0 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 -- 172.21.15.122:0/2180609452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 msgr2=0x55aa5964ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 0x55aa5964ba10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b9f0 tx=0x7fb3fc00beb0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 -- 172.21.15.122:0/2180609452 shutdown_connections 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59658760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964b5d0 0x55aa5964ba10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 --2- 172.21.15.122:0/2180609452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa5964b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 -- 172.21.15.122:0/2180609452 >> 172.21.15.122:0/2180609452 conn(0x55aa59614310 msgr2=0x55aa59647c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 -- 172.21.15.122:0/2180609452 shutdown_connections 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.526+0000 7fb419207740 1 -- 172.21.15.122:0/2180609452 wait complete. 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 -- start start 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 Processor -- start 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59655cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59652f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa596534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aa59659070 con 0x55aa59656370 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa5956c060 con 0x55aa5964cb40 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aa596591f0 con 0x55aa5964b5d0 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.527+0000 7fb419207740 10 client.0 _ll_register_callbacks cb 0x55aa592bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59655cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59655cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34202/0 (socket says 172.21.15.122:34202) 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 -- 172.21.15.122:0/2768604322 learned_addr learned my addr 172.21.15.122:0/2768604322 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa596534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59652f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 -- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 msgr2=0x55aa59652f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59652f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 -- 172.21.15.122:0/2768604322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 msgr2=0x55aa596534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa596534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 -- 172.21.15.122:0/2768604322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa593223c0 con 0x55aa5964b5d0 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59652f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa596534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb414bca640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59655cc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb404011670 tx=0x7fb4040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7a83f2d8f1e73e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb3effff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:47:13.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb3effff640 1 -- 172.21.15.122:0/2768604322 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40401f900 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3fc00dbe0 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.528+0000 7fb3effff640 1 -- 172.21.15.122:0/2768604322 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb40401faa0 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb3effff640 1 -- 172.21.15.122:0/2768604322 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb40401fca0 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 10 client.24940 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa5966d370 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb3effff640 1 -- 172.21.15.122:0/2768604322 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc00dbe0 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb3effff640 1 -- 172.21.15.122:0/2768604322 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb40401e4c0 con 0x55aa5964b5d0 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 10 client.24940 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 10 client.24940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 20 client.24940 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa59670c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 4 client.24940 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.529+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa595be860 con 0x55aa5966e780 2024-09-12T01:47:13.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.530+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa59670c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.530+0000 7fb3effff640 10 client.24940 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:13.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.530+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa59670c20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abcb254a9cf23c7 server_cookie=74577aa2650de40 in_seq=0 out_seq=0 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb3effff640 1 -- 172.21.15.122:0/2768604322 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa595be860 con 0x55aa5966e780 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb3effff640 10 client.24940 handle_command_reply: tid=0 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 client.24940 shutdown 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 msgr2=0x55aa59655cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59655cc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb404011670 tx=0x7fb4040046c0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 shutdown_connections 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa59670c20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa596534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59652f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb3effff640 0 client.24940 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 --2- 172.21.15.122:0/2768604322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59655cc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 >> 172.21.15.122:0/2768604322 conn(0x55aa59614310 msgr2=0x55aa5964d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:13.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 shutdown_connections 2024-09-12T01:47:13.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 1 -- 172.21.15.122:0/2768604322 wait complete. 2024-09-12T01:47:13.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.531+0000 7fb419207740 20 client.24940 trim_cache size 0 max 16384 2024-09-12T01:47:13.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 Processor -- start 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 -- start start 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59662340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59657760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa59657ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101adc00 con 0x55aa59656370 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101ab4d0 con 0x55aa5964cb40 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.544+0000 7fb419207740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101a9320 con 0x55aa5964b5d0 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa59657ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa59657ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54380/0 (socket says 172.21.15.122:54380) 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 -- 172.21.15.122:0/952866364 learned_addr learned my addr 172.21.15.122:0/952866364 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:13.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb414bca640 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59662340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb40ffff640 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59657760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 -- 172.21.15.122:0/952866364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 msgr2=0x55aa59662340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59662340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 -- 172.21.15.122:0/952866364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 msgr2=0x55aa59657760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59657760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 -- 172.21.15.122:0/952866364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa596514c0 con 0x55aa59656370 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb4153cb640 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa59657ca0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb4080bcc70 tx=0x7fb40803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf043370fd9965a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb3fa7fc640 1 -- 172.21.15.122:0/952866364 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080d7030 con 0x55aa59656370 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb3fa7fc640 1 -- 172.21.15.122:0/952866364 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408036040 con 0x55aa59656370 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.545+0000 7fb3fa7fc640 1 -- 172.21.15.122:0/952866364 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080c5be0 con 0x55aa59656370 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 -- 172.21.15.122:0/952866364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 msgr2=0x55aa59657ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa59657ca0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb4080bcc70 tx=0x7fb40803d040 comp rx=0 tx=0).stop 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 -- 172.21.15.122:0/952866364 shutdown_connections 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa59656370 0x55aa59657ca0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964cb40 0x55aa59657760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 --2- 172.21.15.122:0/952866364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa5964b5d0 0x55aa59662340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 -- 172.21.15.122:0/952866364 >> 172.21.15.122:0/952866364 conn(0x55aa59614310 msgr2=0x55aa59651840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.546+0000 7fb419207740 1 -- 172.21.15.122:0/952866364 shutdown_connections 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.547+0000 7fb419207740 1 -- 172.21.15.122:0/952866364 wait complete. 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.547+0000 7fb419207740 1 -- start start 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.547+0000 7fb419207740 1 Processor -- start 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.547+0000 7fb419207740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964b5d0 0x55aa5965e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.547+0000 7fb419207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 0x55aa5965ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb419207740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59659470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb419207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101a9990 con 0x55aa5964cb40 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb419207740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101aae00 con 0x55aa5964b5d0 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb419207740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101aa060 con 0x55aa59656370 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb419207740 10 client.0 _ll_register_callbacks cb 0x7fb40400b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 0x55aa5965ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 0x55aa5965ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54394/0 (socket says 172.21.15.122:54394) 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 -- 172.21.15.122:0/2579134745 learned_addr learned my addr 172.21.15.122:0/2579134745 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb414bca640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964b5d0 0x55aa5965e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb4153cb640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59659470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 -- 172.21.15.122:0/2579134745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 msgr2=0x55aa59659470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59659470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 -- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964b5d0 msgr2=0x55aa5965e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964b5d0 0x55aa5965e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 -- 172.21.15.122:0/2579134745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa596599b0 con 0x55aa5964cb40 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb40ffff640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 0x55aa5965ea00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00d6b0 tx=0x7fb3fc010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be7324eff93a82b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.548+0000 7fb3effff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb3effff640 1 -- 172.21.15.122:0/2579134745 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc016040 con 0x55aa5964cb40 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb3effff640 1 -- 172.21.15.122:0/2579134745 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb3fc004040 con 0x55aa5964cb40 2024-09-12T01:47:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb408036040 con 0x55aa5964cb40 2024-09-12T01:47:13.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb3effff640 1 -- 172.21.15.122:0/2579134745 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc005850 con 0x55aa5964cb40 2024-09-12T01:47:13.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb419207740 10 client.15495 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:13.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa596d9af0 con 0x55aa5964cb40 2024-09-12T01:47:13.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb3effff640 1 -- 172.21.15.122:0/2579134745 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408036040 con 0x55aa5964cb40 2024-09-12T01:47:13.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb3effff640 1 -- 172.21.15.122:0/2579134745 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb3fc005020 con 0x55aa5964cb40 2024-09-12T01:47:13.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb419207740 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.549+0000 7fb419207740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb419207740 20 client.15495 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb419207740 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa596dc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb419207740 4 client.15495 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa595b8d70 con 0x55aa5966e780 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb414bca640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa596dc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb3effff640 10 client.15495 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb414bca640 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa596dc020 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7805e214b465dc6e server_cookie=16a906688b23ac9a in_seq=0 out_seq=0 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb3effff640 1 -- 172.21.15.122:0/2579134745 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa595b8d70 con 0x55aa5966e780 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb3effff640 10 client.15495 handle_command_reply: tid=0 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.550+0000 7fb419207740 1 client.15495 shutdown 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 msgr2=0x55aa5965ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 0x55aa5965ea00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00d6b0 tx=0x7fb3fc010e10 comp rx=0 tx=0).stop 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 shutdown_connections 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55aa5966e780 0x55aa596dc020 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb3effff640 0 client.15495 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa59656370 0x55aa59659470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa5964cb40 0x55aa5965ea00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 --2- 172.21.15.122:0/2579134745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa5964b5d0 0x55aa5965e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 >> 172.21.15.122:0/2579134745 conn(0x55aa59614310 msgr2=0x55aa59646d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 shutdown_connections 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 1 -- 172.21.15.122:0/2579134745 wait complete. 2024-09-12T01:47:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.551+0000 7fb419207740 20 client.15495 trim_cache size 0 max 16384 2024-09-12T01:47:13.556 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:47:13.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4000a2c90 msgr2=0x7fb4000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4000a2c90 0x7fb4000a5150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb404005fd0 tx=0x7fb404005850 comp rx=0 tx=0).stop 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 msgr2=0x7fb410101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb410101450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb408005cc0 tx=0x7fb408003970 comp rx=0 tx=0).stop 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 shutdown_connections 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4000a2c90 0x7fb4000a5150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4101082a0 0x7fb410101450 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb410103720 0x7fb410100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 --2- 172.21.15.122:0/2028315857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb410102d70 0x7fb4101009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 >> 172.21.15.122:0/2028315857 conn(0x7fb4100fc7d0 msgr2=0x7fb410105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 shutdown_connections 2024-09-12T01:47:13.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:13.552+0000 7fb416e2e640 1 -- 172.21.15.122:0/2028315857 wait complete. 2024-09-12T01:47:13.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:13 smithi122 ceph-mon[30925]: pgmap v357: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:13 smithi151 ceph-mon[35229]: pgmap v357: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:13 smithi183 ceph-mon[35127]: pgmap v357: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:15.239 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:47:15.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:47:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:15.560 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:15 smithi122 ceph-mon[30925]: pgmap v358: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:15.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:15 smithi151 ceph-mon[35229]: pgmap v358: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:15 smithi183 ceph-mon[35127]: pgmap v358: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:16.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 -- 172.21.15.122:0/981874407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c81008f0 msgr2=0x7fd4c8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 --2- 172.21.15.122:0/981874407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c81008f0 0x7fd4c8100d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009930 tx=0x7fd4bc02a420 comp rx=0 tx=0).stop 2024-09-12T01:47:16.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 -- 172.21.15.122:0/981874407 shutdown_connections 2024-09-12T01:47:16.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 --2- 172.21.15.122:0/981874407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8101290 0x7fd4c810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 --2- 172.21.15.122:0/981874407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c81008f0 0x7fd4c8100d50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 --2- 172.21.15.122:0/981874407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4c81068f0 0x7fd4c8106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 -- 172.21.15.122:0/981874407 >> 172.21.15.122:0/981874407 conn(0x7fd4c80fc630 msgr2=0x7fd4c80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.117+0000 7fd4cd800640 1 -- 172.21.15.122:0/981874407 shutdown_connections 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.118+0000 7fd4cd800640 1 -- 172.21.15.122:0/981874407 wait complete. 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.119+0000 7fd4cd800640 1 Processor -- start 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.119+0000 7fd4cd800640 1 -- start start 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.119+0000 7fd4cd800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 0x7fd4c8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.119+0000 7fd4cd800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4c81068f0 0x7fd4c81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4cd800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8199e10 0x7fd4c81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4cd800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8110d40 con 0x7fd4c8101290 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4cd800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8110bc0 con 0x7fd4c8199e10 2024-09-12T01:47:16.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4cd800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8110ec0 con 0x7fd4c81068f0 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 0x7fd4c8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 0x7fd4c8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54414/0 (socket says 172.21.15.122:54414) 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3801764684 learned_addr learned my addr 172.21.15.122:0/3801764684 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4c81068f0 0x7fd4c81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3801764684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4c81068f0 msgr2=0x7fd4c81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4c81068f0 0x7fd4c81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3801764684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8199e10 msgr2=0x7fd4c81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8199e10 0x7fd4c81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8199e10 0x7fd4c81a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3801764684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c81a4be0 con 0x7fd4c8101290 2024-09-12T01:47:16.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.120+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 0x7fd4c8199390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800d770 tx=0x7fd4b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:16.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.121+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8199e10 0x7fd4c81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:16.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.121+0000 7fd4affff640 1 -- 172.21.15.122:0/3801764684 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8017070 con 0x7fd4c8101290 2024-09-12T01:47:16.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.121+0000 7fd4affff640 1 -- 172.21.15.122:0/3801764684 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b8005500 con 0x7fd4c8101290 2024-09-12T01:47:16.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.121+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c81a4ed0 con 0x7fd4c8101290 2024-09-12T01:47:16.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.121+0000 7fd4affff640 1 -- 172.21.15.122:0/3801764684 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8004e00 con 0x7fd4c8101290 2024-09-12T01:47:16.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.121+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c81a5330 con 0x7fd4c8101290 2024-09-12T01:47:16.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.124+0000 7fd4cfbd9740 1 Processor -- start 2024-09-12T01:47:16.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.124+0000 7fd4affff640 1 -- 172.21.15.122:0/3801764684 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd4b8004fa0 con 0x7fd4c8101290 2024-09-12T01:47:16.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.124+0000 7fd4cfbd9740 1 -- start start 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 0x55a951e95150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951e9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e9ff50 0x55a951ea2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a951dfa280 con 0x55a951e94d70 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a951da1b10 con 0x55a951e9ff50 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a951df6240 con 0x55a951e95690 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 0x55a951e95150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 0x55a951e95150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54428/0 (socket says 172.21.15.122:54428) 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4c77fe640 1 -- 172.21.15.122:0/3307585270 learned_addr learned my addr 172.21.15.122:0/3307585270 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951e9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e9ff50 0x55a951ea2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.125+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3307585270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e9ff50 msgr2=0x55a951ea2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e9ff50 0x55a951ea2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3307585270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 msgr2=0x55a951e95150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 0x55a951e95150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/3307585270 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a951dae1f0 con 0x55a951e95690 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e9ff50 0x55a951ea2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 0x55a951e95150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4affff640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd4b00a2d60 0x7fd4b00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd4b00a2d60 0x7fd4b00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4affff640 1 -- 172.21.15.122:0/3801764684 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd4b80ceae0 con 0x7fd4c8101290 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951e9fa10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cc490 tx=0x7fd4b80d7620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d1052a741060792 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4ad7fa640 1 -- 172.21.15.122:0/3307585270 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8020030 con 0x55a951e95690 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd4b00a2d60 0x7fd4b00a5220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fd0 tx=0x7fd4c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4ad7fa640 1 -- 172.21.15.122:0/3307585270 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b80046c0 con 0x55a951e95690 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.126+0000 7fd4ad7fa640 1 -- 172.21.15.122:0/3307585270 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b809afb0 con 0x55a951e95690 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.127+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3307585270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 msgr2=0x55a951e9fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.127+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951e9fa10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cc490 tx=0x7fd4b80d7620 comp rx=0 tx=0).stop 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3307585270 shutdown_connections 2024-09-12T01:47:16.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e9ff50 0x55a951ea2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951e9fa10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3307585270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e94d70 0x55a951e95150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3307585270 >> 172.21.15.122:0/3307585270 conn(0x55a951e91300 msgr2=0x55a951e916e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3307585270 shutdown_connections 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3307585270 wait complete. 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- start start 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 Processor -- start 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e94d70 0x55a951e99120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e95690 0x55a951e99660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951e99bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a951ea2ac0 con 0x55a951e9ff50 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a951db4f10 con 0x55a951e95690 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a951ea2c40 con 0x55a951e94d70 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.128+0000 7fd4cfbd9740 10 client.0 _ll_register_callbacks cb 0x55a951d7da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951e99bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e94d70 0x55a951e99120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951e99bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54444/0 (socket says 172.21.15.122:54444) 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e94d70 0x55a951e99120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34270/0 (socket says 172.21.15.122:34270) 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 -- 172.21.15.122:0/3347136960 learned_addr learned my addr 172.21.15.122:0/3347136960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e95690 0x55a951e99660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 -- 172.21.15.122:0/3347136960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e94d70 msgr2=0x55a951e99120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e94d70 0x55a951e99120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 -- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e95690 msgr2=0x55a951e99660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e95690 0x55a951e99660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 -- 172.21.15.122:0/3347136960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a951b6b6a0 con 0x55a951e9ff50 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e95690 0x55a951e99660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951e99bc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0010d40 tx=0x7fd4c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07d78b9f90f7a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4aaff5640 1 -- 172.21.15.122:0/3347136960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c002a060 con 0x55a951e9ff50 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4aaff5640 1 -- 172.21.15.122:0/3347136960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd4c001b460 con 0x55a951e9ff50 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a951e077f0 con 0x55a951e9ff50 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4aaff5640 1 -- 172.21.15.122:0/3347136960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4c001de50 con 0x55a951e9ff50 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4cfbd9740 10 client.15507 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.129+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a951ebf870 con 0x55a951e9ff50 2024-09-12T01:47:16.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4aaff5640 1 -- 172.21.15.122:0/3347136960 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4c001bb40 con 0x55a951e9ff50 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4aaff5640 1 -- 172.21.15.122:0/3347136960 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd4c002f070 con 0x55a951e9ff50 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4cfbd9740 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4cfbd9740 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4cfbd9740 20 client.15507 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ec1060 0x55a951ec34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4cfbd9740 4 client.15507 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a951e077f0 con 0x55a951ec1060 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.130+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ec1060 0x55a951ec34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.131+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ec1060 0x55a951ec34a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48df87577314c92 server_cookie=4bc3440dd2462a5c in_seq=0 out_seq=0 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.131+0000 7fd4aaff5640 10 client.15507 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.131+0000 7fd4aaff5640 1 -- 172.21.15.122:0/3347136960 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a951e077f0 con 0x55a951ec1060 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.131+0000 7fd4aaff5640 10 client.15507 handle_command_reply: tid=0 2024-09-12T01:47:16.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.131+0000 7fd4cfbd9740 1 client.15507 shutdown 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 msgr2=0x55a951e99bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951e99bc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0010d40 tx=0x7fd4c0004a40 comp rx=0 tx=0).stop 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 shutdown_connections 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ec1060 0x55a951ec34a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951e99bc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e95690 0x55a951e99660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/3347136960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e94d70 0x55a951e99120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 >> 172.21.15.122:0/3347136960 conn(0x55a951e91300 msgr2=0x55a951e97030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4aaff5640 0 client.15507 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 shutdown_connections 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/3347136960 wait complete. 2024-09-12T01:47:16.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.132+0000 7fd4cfbd9740 20 client.15507 trim_cache size 0 max 16384 2024-09-12T01:47:16.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.165+0000 7fd4cfbd9740 1 Processor -- start 2024-09-12T01:47:16.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.165+0000 7fd4cfbd9740 1 -- start start 2024-09-12T01:47:16.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951e999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951ec8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951ec9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c81a5890 con 0x55a951e9ff50 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8072020 con 0x55a951e94d70 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c806fe50 con 0x55a951e95690 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951ec9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951ec9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54454/0 (socket says 172.21.15.122:54454) 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c77fe640 1 -- 172.21.15.122:0/1481754468 learned_addr learned my addr 172.21.15.122:0/1481754468 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951e999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951ec8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/1481754468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 msgr2=0x55a951ec8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951ec8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/1481754468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 msgr2=0x55a951ec9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951ec9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 -- 172.21.15.122:0/1481754468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a951e91130 con 0x55a951e94d70 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.166+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951e999f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cc510 tx=0x7fd4b80d2610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=586b05a8d2d67f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4acff9640 1 -- 172.21.15.122:0/1481754468 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80ae070 con 0x55a951e94d70 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4acff9640 1 -- 172.21.15.122:0/1481754468 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b809b8e0 con 0x55a951e94d70 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4acff9640 1 -- 172.21.15.122:0/1481754468 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8020040 con 0x55a951e94d70 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1481754468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 msgr2=0x55a951e999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951e999f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cc510 tx=0x7fd4b80d2610 comp rx=0 tx=0).stop 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1481754468 shutdown_connections 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e9ff50 0x55a951ec9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e95690 0x55a951ec8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.167+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1481754468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951e999f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1481754468 >> 172.21.15.122:0/1481754468 conn(0x55a951e91300 msgr2=0x55a951e97030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1481754468 shutdown_connections 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1481754468 wait complete. 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- start start 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 Processor -- start 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951ea8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 0x55a951ea8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e9ff50 0x55a951f2ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c80704e0 con 0x55a951e95690 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8071950 con 0x55a951e94d70 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4c67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 0x55a951ea8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4c6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951ea8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8070bb0 con 0x55a951e9ff50 2024-09-12T01:47:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4c67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 0x55a951ea8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54468/0 (socket says 172.21.15.122:54468) 2024-09-12T01:47:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4c67fc640 1 -- 172.21.15.122:0/1524987927 learned_addr learned my addr 172.21.15.122:0/1524987927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.168+0000 7fd4cfbd9740 10 client.0 _ll_register_callbacks cb 0x7fd4c0009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c77fe640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e9ff50 0x55a951f2ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c67fc640 1 -- 172.21.15.122:0/1524987927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e9ff50 msgr2=0x55a951f2ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e9ff50 0x55a951f2ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c67fc640 1 -- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 msgr2=0x55a951ea8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951ea8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c67fc640 1 -- 172.21.15.122:0/1524987927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a951f2b060 con 0x55a951e95690 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951ea8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4c67fc640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 0x55a951ea8850 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006010 tx=0x7fd4bc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e4a6554d37403a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4aaff5640 1 -- 172.21.15.122:0/1524987927 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc035040 con 0x55a951e95690 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b809b8e0 con 0x55a951e95690 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4aaff5640 1 -- 172.21.15.122:0/1524987927 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd4bc005250 con 0x55a951e95690 2024-09-12T01:47:16.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4aaff5640 1 -- 172.21.15.122:0/1524987927 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4bc005450 con 0x55a951e95690 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4cfbd9740 10 client.15513 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.169+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a951eacd50 con 0x55a951e95690 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4aaff5640 1 -- 172.21.15.122:0/1524987927 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b809b8e0 con 0x55a951e95690 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4aaff5640 1 -- 172.21.15.122:0/1524987927 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd4bc0316d0 con 0x55a951e95690 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4cfbd9740 10 client.15513 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4cfbd9740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4cfbd9740 20 client.15513 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ead550 0x55a951eaf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4cfbd9740 4 client.15513 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a951e01d00 con 0x55a951ead550 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.170+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ead550 0x55a951eaf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:16.180 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:16.181 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-09-12T01:47:16.181 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "09bbce88-7a5b-405b-a560-64c028de30de", 2024-09-12T01:47:16.181 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-09-12T01:47:16.181 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.171+0000 7fd4c6ffd640 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ead550 0x55a951eaf970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5fc0d2bd5c856f server_cookie=b462bed8ac36ba4d in_seq=0 out_seq=0 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.171+0000 7fd4aaff5640 10 client.15513 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4aaff5640 1 -- 172.21.15.122:0/1524987927 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a951e01d00 con 0x55a951ead550 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4aaff5640 10 client.15513 handle_command_reply: tid=0 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 client.15513 shutdown 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 msgr2=0x55a951ea8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 0x55a951ea8850 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006010 tx=0x7fd4bc0043b0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 shutdown_connections 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a951ead550 0x55a951eaf970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a951e9ff50 0x55a951f2ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a951e95690 0x55a951ea8850 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 --2- 172.21.15.122:0/1524987927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a951e94d70 0x55a951ea8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 >> 172.21.15.122:0/1524987927 conn(0x55a951e91300 msgr2=0x55a951e8fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4aaff5640 0 client.15513 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 shutdown_connections 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 1 -- 172.21.15.122:0/1524987927 wait complete. 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.172+0000 7fd4cfbd9740 20 client.15513 trim_cache size 0 max 16384 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd4b00a2d60 msgr2=0x7fd4b00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd4b00a2d60 0x7fd4b00a5220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fd0 tx=0x7fd4c0009500 comp rx=0 tx=0).stop 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 msgr2=0x7fd4c8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 0x7fd4c8199390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800d770 tx=0x7fd4b800dc30 comp rx=0 tx=0).stop 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 shutdown_connections 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd4b00a2d60 0x7fd4b00a5220 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4c8199e10 0x7fd4c81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4c81068f0 0x7fd4c81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 --2- 172.21.15.122:0/3801764684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4c8101290 0x7fd4c8199390 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:16.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 >> 172.21.15.122:0/3801764684 conn(0x7fd4c80fc630 msgr2=0x7fd4c80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:16.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 shutdown_connections 2024-09-12T01:47:16.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:16.173+0000 7fd4cd800640 1 -- 172.21.15.122:0/3801764684 wait complete. 2024-09-12T01:47:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:16 smithi122 ceph-mon[30925]: scrub queued for path: / 2024-09-12T01:47:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:16 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:16 smithi122 ceph-mon[30925]: scrub summary: active paths [/] 2024-09-12T01:47:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:16 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:16 smithi122 ceph-mon[30925]: scrub summary: idle 2024-09-12T01:47:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:16 smithi151 ceph-mon[35229]: scrub queued for path: / 2024-09-12T01:47:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:16 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:16 smithi151 ceph-mon[35229]: scrub summary: active paths [/] 2024-09-12T01:47:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:16 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:16.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:16 smithi151 ceph-mon[35229]: scrub summary: idle 2024-09-12T01:47:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:16 smithi183 ceph-mon[35127]: scrub queued for path: / 2024-09-12T01:47:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:16 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:16 smithi183 ceph-mon[35127]: scrub summary: active paths [/] 2024-09-12T01:47:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:16 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:16 smithi183 ceph-mon[35127]: scrub summary: idle 2024-09-12T01:47:16.958 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:47:17.245 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:17.722 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:17 smithi122 ceph-mon[30925]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 -- 172.21.15.122:0/1894049144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1036e0 msgr2=0x7fb34c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 --2- 172.21.15.122:0/1894049144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1036e0 0x7fb34c107b90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb340009930 tx=0x7fb34002a420 comp rx=0 tx=0).stop 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 -- 172.21.15.122:0/1894049144 shutdown_connections 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 --2- 172.21.15.122:0/1894049144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c1082a0 0x7fb34c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 --2- 172.21.15.122:0/1894049144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1036e0 0x7fb34c107b90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 --2- 172.21.15.122:0/1894049144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb34c102d30 0x7fb34c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 -- 172.21.15.122:0/1894049144 >> 172.21.15.122:0/1894049144 conn(0x7fb34c0fc7d0 msgr2=0x7fb34c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 -- 172.21.15.122:0/1894049144 shutdown_connections 2024-09-12T01:47:17.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.815+0000 7fb3523d7640 1 -- 172.21.15.122:0/1894049144 wait complete. 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.816+0000 7fb3523d7640 1 Processor -- start 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.816+0000 7fb3523d7640 1 -- start start 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb3523d7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c102d30 0x7fb34c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb3523d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 0x7fb34c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb3523d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb34c19a820 0x7fb34c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb3523d7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1120a0 con 0x7fb34c1082a0 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb3523d7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c111f20 con 0x7fb34c102d30 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb3523d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c112220 con 0x7fb34c19a820 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 0x7fb34c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c102d30 0x7fb34c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 0x7fb34c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54478/0 (socket says 172.21.15.122:54478) 2024-09-12T01:47:17.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34b7fe640 1 -- 172.21.15.122:0/3530659642 learned_addr learned my addr 172.21.15.122:0/3530659642 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb350974640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb34c19a820 0x7fb34c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34bfff640 1 -- 172.21.15.122:0/3530659642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb34c19a820 msgr2=0x7fb34c19acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb34c19a820 0x7fb34c19acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34bfff640 1 -- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 msgr2=0x7fb34c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 0x7fb34c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34bfff640 1 -- 172.21.15.122:0/3530659642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c19b400 con 0x7fb34c102d30 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.817+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 0x7fb34c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.818+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c102d30 0x7fb34c1a06a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00d720 tx=0x7fb33c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.818+0000 7fb3497fa640 1 -- 172.21.15.122:0/3530659642 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c017070 con 0x7fb34c102d30 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.818+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c19b690 con 0x7fb34c102d30 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.818+0000 7fb3497fa640 1 -- 172.21.15.122:0/3530659642 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33c00de30 con 0x7fb34c102d30 2024-09-12T01:47:17.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.818+0000 7fb3497fa640 1 -- 172.21.15.122:0/3530659642 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0047a0 con 0x7fb34c102d30 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.818+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c1a77f0 con 0x7fb34c102d30 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.821+0000 7fb3497fa640 1 -- 172.21.15.122:0/3530659642 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb33c0099d0 con 0x7fb34c102d30 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 Processor -- start 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 -- start start 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 0x562187c6a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6adc0 0x562187c75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c75680 0x562187c77a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562187bd02e0 con 0x562187c6a4a0 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562187b77ce0 con 0x562187c6adc0 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb3547b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562187bcc2a0 con 0x562187c75680 2024-09-12T01:47:17.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 0x562187c6a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 0x562187c6a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54492/0 (socket says 172.21.15.122:54492) 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.822+0000 7fb350974640 1 -- 172.21.15.122:0/3315671062 learned_addr learned my addr 172.21.15.122:0/3315671062 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6adc0 0x562187c75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c75680 0x562187c77a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb3497fa640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3340a2c10 0x7fb3340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 -- 172.21.15.122:0/3315671062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c75680 msgr2=0x562187c77a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c75680 0x562187c77a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 -- 172.21.15.122:0/3315671062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6adc0 msgr2=0x562187c75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6adc0 0x562187c75140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 -- 172.21.15.122:0/3315671062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562187b5c440 con 0x562187c6a4a0 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3340a2c10 0x7fb3340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb350974640 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 0x562187c6a880 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb344009e60 tx=0x7fb344010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf40d6d776c4f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.823+0000 7fb3497fa640 1 -- 172.21.15.122:0/3530659642 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb33c0cee60 con 0x7fb34c102d30 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.824+0000 7fb3327fc640 1 -- 172.21.15.122:0/3315671062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344032070 con 0x562187c6a4a0 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.824+0000 7fb3327fc640 1 -- 172.21.15.122:0/3315671062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34402c470 con 0x562187c6a4a0 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.824+0000 7fb3327fc640 1 -- 172.21.15.122:0/3315671062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34402b6f0 con 0x562187c6a4a0 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 -- 172.21.15.122:0/3315671062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 msgr2=0x562187c6a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 0x562187c6a880 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb344009e60 tx=0x7fb344010740 comp rx=0 tx=0).stop 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb350974640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3340a2c10 0x7fb3340a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb344011010 tx=0x7fb34402bda0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 -- 172.21.15.122:0/3315671062 shutdown_connections 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c75680 0x562187c77a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6adc0 0x562187c75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3315671062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c6a4a0 0x562187c6a880 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.825+0000 7fb3547b0740 1 -- 172.21.15.122:0/3315671062 >> 172.21.15.122:0/3315671062 conn(0x562187c66a30 msgr2=0x562187c66e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:17.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.826+0000 7fb3547b0740 1 -- 172.21.15.122:0/3315671062 shutdown_connections 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.826+0000 7fb3547b0740 1 -- 172.21.15.122:0/3315671062 wait complete. 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.826+0000 7fb3547b0740 1 -- start start 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.826+0000 7fb3547b0740 1 Processor -- start 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c74f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c72740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562187c78250 con 0x562187c75680 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562187b8b050 con 0x562187c6a4a0 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562187c783d0 con 0x562187c6adc0 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb3547b0740 10 client.0 _ll_register_callbacks cb 0x562187bcb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c74f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c72740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c72740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54496/0 (socket says 172.21.15.122:54496) 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c72200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34330/0 (socket says 172.21.15.122:34330) 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb350974640 1 -- 172.21.15.122:0/4032340126 learned_addr learned my addr 172.21.15.122:0/4032340126 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb34b7fe640 1 -- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 msgr2=0x562187c74f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c74f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.827+0000 7fb34b7fe640 1 -- 172.21.15.122:0/4032340126 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 msgr2=0x562187c72740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c72740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb34b7fe640 1 -- 172.21.15.122:0/4032340126 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621879413c0 con 0x562187c6adc0 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb350974640 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c72740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c72200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb340009230 tx=0x7fb340031920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35299f85aae1496 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb32b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:47:17.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb32b7fe640 1 -- 172.21.15.122:0/4032340126 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340035040 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb32b7fe640 1 -- 172.21.15.122:0/4032340126 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb340004450 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb34402c470 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb32b7fe640 1 -- 172.21.15.122:0/4032340126 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb340005020 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb3547b0740 10 client.24976 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.828+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562187c83ba0 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.829+0000 7fb32b7fe640 1 -- 172.21.15.122:0/4032340126 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3400047a0 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb32b7fe640 1 -- 172.21.15.122:0/4032340126 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb340003680 con 0x562187c6adc0 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb3547b0740 10 client.24976 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb3547b0740 10 client.24976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb3547b0740 20 client.24976 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb3547b0740 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187c85050 0x562187c87470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb3547b0740 4 client.24976 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb34402c470 con 0x562187c85050 2024-09-12T01:47:17.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.830+0000 7fb34bfff640 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187c85050 0x562187c87470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.831+0000 7fb34bfff640 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187c85050 0x562187c87470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4874c51e32a0a597 server_cookie=a7dc56cd461eb7a1 in_seq=0 out_seq=0 2024-09-12T01:47:17.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.831+0000 7fb32b7fe640 10 client.24976 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.831+0000 7fb32b7fe640 1 -- 172.21.15.122:0/4032340126 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb34402c470 con 0x562187c85050 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.831+0000 7fb32b7fe640 10 client.24976 handle_command_reply: tid=0 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.831+0000 7fb3547b0740 1 client.24976 shutdown 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 msgr2=0x562187c72200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c72200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb340009230 tx=0x7fb340031920 comp rx=0 tx=0).stop 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 shutdown_connections 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187c85050 0x562187c87470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c72740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb32b7fe640 0 client.24976 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c72200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 --2- 172.21.15.122:0/4032340126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c74f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 >> 172.21.15.122:0/4032340126 conn(0x562187c66a30 msgr2=0x562187c6c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 shutdown_connections 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 1 -- 172.21.15.122:0/4032340126 wait complete. 2024-09-12T01:47:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.832+0000 7fb3547b0740 20 client.24976 trim_cache size 0 max 16384 2024-09-12T01:47:17.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.849+0000 7fb3547b0740 1 Processor -- start 2024-09-12T01:47:17.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.849+0000 7fb3547b0740 1 -- start start 2024-09-12T01:47:17.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.849+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.849+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c8d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb3547b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1ae400 con 0x562187c75680 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb3547b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1abcd0 con 0x562187c6a4a0 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb3547b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1a9b60 con 0x562187c6adc0 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c8d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c8d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54504/0 (socket says 172.21.15.122:54504) 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 -- 172.21.15.122:0/3328895283 learned_addr learned my addr 172.21.15.122:0/3328895283 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 -- 172.21.15.122:0/3328895283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 msgr2=0x562187c8cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c8cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 -- 172.21.15.122:0/3328895283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 msgr2=0x562187c814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 -- 172.21.15.122:0/3328895283 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562187c66860 con 0x562187c75680 2024-09-12T01:47:17.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.850+0000 7fb350974640 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c8d280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb34402a740 tx=0x7fb344033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a428faae6c5db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:17.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb331ffb640 1 -- 172.21.15.122:0/3328895283 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344032070 con 0x562187c75680 2024-09-12T01:47:17.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb331ffb640 1 -- 172.21.15.122:0/3328895283 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3440049e0 con 0x562187c75680 2024-09-12T01:47:17.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb331ffb640 1 -- 172.21.15.122:0/3328895283 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34402ab20 con 0x562187c75680 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb3547b0740 1 -- 172.21.15.122:0/3328895283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 msgr2=0x562187c8d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c8d280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb34402a740 tx=0x7fb344033000 comp rx=0 tx=0).stop 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb3547b0740 1 -- 172.21.15.122:0/3328895283 shutdown_connections 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.851+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c8d280 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3328895283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 -- 172.21.15.122:0/3328895283 >> 172.21.15.122:0/3328895283 conn(0x562187c66a30 msgr2=0x562187c70ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 -- 172.21.15.122:0/3328895283 shutdown_connections 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 -- 172.21.15.122:0/3328895283 wait complete. 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 -- start start 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 Processor -- start 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.852+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c6f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb3547b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c6fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c6f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c6f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52198/0 (socket says 172.21.15.122:52198) 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb350974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c6fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c6f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34336/0 (socket says 172.21.15.122:34336) 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb3547b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1aa1b0 con 0x562187c75680 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb3547b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1ab600 con 0x562187c6a4a0 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb3547b0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c1aa860 con 0x562187c6adc0 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb3547b0740 10 client.0 _ll_register_callbacks cb 0x7fb340005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34bfff640 1 -- 172.21.15.122:0/3860053444 learned_addr learned my addr 172.21.15.122:0/3860053444 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34b7fe640 1 -- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 msgr2=0x562187c6f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c6f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.853+0000 7fb34b7fe640 1 -- 172.21.15.122:0/3860053444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 msgr2=0x562187c6fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c6fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb34b7fe640 1 -- 172.21.15.122:0/3860053444 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562187ceedc0 con 0x562187c6adc0 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb350974640 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c6fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb34b7fe640 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c6f810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562187c737b0 tx=0x7fb340003fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d11542645126619 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:17.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb32b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb32b7fe640 1 -- 172.21.15.122:0/3860053444 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34005f060 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3440049e0 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb32b7fe640 1 -- 172.21.15.122:0/3860053444 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb3400037f0 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb32b7fe640 1 -- 172.21.15.122:0/3860053444 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb340056040 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb3547b0740 10 client.24982 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562187cf00b0 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.854+0000 7fb32b7fe640 1 -- 172.21.15.122:0/3860053444 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3440049e0 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb32b7fe640 1 -- 172.21.15.122:0/3860053444 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb340064070 con 0x562187c6adc0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb3547b0740 10 client.24982 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb3547b0740 10 client.24982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb3547b0740 20 client.24982 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187cf07d0 0x562187cf2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb3547b0740 4 client.24982 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562187c72ad0 con 0x562187cf07d0 2024-09-12T01:47:17.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.855+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187cf07d0 0x562187cf2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:17.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb34bfff640 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187cf07d0 0x562187cf2bf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599d0c25449e579 server_cookie=c25723f144980e7f in_seq=0 out_seq=0 2024-09-12T01:47:17.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb32b7fe640 10 client.24982 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:17.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb32b7fe640 1 -- 172.21.15.122:0/3860053444 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562187c72ad0 con 0x562187cf07d0 2024-09-12T01:47:17.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb32b7fe640 10 client.24982 handle_command_reply: tid=0 2024-09-12T01:47:17.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 client.24982 shutdown 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 msgr2=0x562187c6f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c6f810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562187c737b0 tx=0x7fb340003fd0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 shutdown_connections 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562187cf07d0 0x562187cf2bf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb32b7fe640 0 client.24982 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562187c75680 0x562187c6fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562187c6adc0 0x562187c6f810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 --2- 172.21.15.122:0/3860053444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562187c6a4a0 0x562187c6f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.856+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 >> 172.21.15.122:0/3860053444 conn(0x562187c66a30 msgr2=0x562187c65350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.857+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 shutdown_connections 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.857+0000 7fb3547b0740 1 -- 172.21.15.122:0/3860053444 wait complete. 2024-09-12T01:47:17.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.857+0000 7fb3547b0740 20 client.24982 trim_cache size 0 max 16384 2024-09-12T01:47:17.862 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:17.862 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-09-12T01:47:17.862 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-09-12T01:47:17.862 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:17.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3340a2c10 msgr2=0x7fb3340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3340a2c10 0x7fb3340a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb344011010 tx=0x7fb34402bda0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c102d30 msgr2=0x7fb34c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c102d30 0x7fb34c1a06a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00d720 tx=0x7fb33c00dbe0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 shutdown_connections 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb3340a2c10 0x7fb3340a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb34c19a820 0x7fb34c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb34c1082a0 0x7fb34c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 --2- 172.21.15.122:0/3530659642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34c102d30 0x7fb34c1a06a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 >> 172.21.15.122:0/3530659642 conn(0x7fb34c0fc7d0 msgr2=0x7fb34c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 shutdown_connections 2024-09-12T01:47:17.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:17.858+0000 7fb3523d7640 1 -- 172.21.15.122:0/3530659642 wait complete. 2024-09-12T01:47:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:17 smithi151 ceph-mon[35229]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:17 smithi183 ceph-mon[35127]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:18.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:47:18.572 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:47:18.837 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:19.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 -- 172.21.15.122:0/214944112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350102ef0 msgr2=0x7f335010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 --2- 172.21.15.122:0/214944112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350102ef0 0x7f335010da40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3348009ca0 tx=0x7f334802c840 comp rx=0 tx=0).stop 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 -- 172.21.15.122:0/214944112 shutdown_connections 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 --2- 172.21.15.122:0/214944112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350102ef0 0x7f335010da40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 --2- 172.21.15.122:0/214944112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 0x7f33501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 --2- 172.21.15.122:0/214944112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350100e20 0x7f3350101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 -- 172.21.15.122:0/214944112 >> 172.21.15.122:0/214944112 conn(0x7f33500fc7d0 msgr2=0x7f33500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.391+0000 7f3356df8640 1 -- 172.21.15.122:0/214944112 shutdown_connections 2024-09-12T01:47:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.392+0000 7f3356df8640 1 -- 172.21.15.122:0/214944112 wait complete. 2024-09-12T01:47:19.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3356df8640 1 Processor -- start 2024-09-12T01:47:19.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3356df8640 1 -- start start 2024-09-12T01:47:19.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3356df8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 0x7f3350196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3356df8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 0x7f33501967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3356df8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350102ef0 0x7f335019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3354b94640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 0x7f3350196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3354b94640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 0x7f3350196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54538/0 (socket says 172.21.15.122:54538) 2024-09-12T01:47:19.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3354b94640 1 -- 172.21.15.122:0/908838273 learned_addr learned my addr 172.21.15.122:0/908838273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:19.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.393+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fec0 con 0x7f3350100e20 2024-09-12T01:47:19.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fd40 con 0x7f3350102ef0 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3350110040 con 0x7f3350102570 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3355395640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350102ef0 0x7f335019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f334ffff640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 0x7f33501967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3355395640 1 -- 172.21.15.122:0/908838273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 msgr2=0x7f33501967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3355395640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 0x7f33501967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3355395640 1 -- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 msgr2=0x7f3350196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3355395640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 0x7f3350196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3355395640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335019b230 con 0x7f3350102ef0 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f334ffff640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 0x7f33501967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.394+0000 7f3354b94640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 0x7f3350196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.395+0000 7f3355395640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350102ef0 0x7f335019ab50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f334802ce30 tx=0x7f3348032fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.395+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348045030 con 0x7f3350102ef0 2024-09-12T01:47:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.395+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335019b4c0 con 0x7f3350102ef0 2024-09-12T01:47:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.395+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3348003e50 con 0x7f3350102ef0 2024-09-12T01:47:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.395+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348041a10 con 0x7f3350102ef0 2024-09-12T01:47:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.395+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33501a2e00 con 0x7f3350102ef0 2024-09-12T01:47:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.396+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3350101200 con 0x7f3350102ef0 2024-09-12T01:47:19.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.399+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3348052060 con 0x7f3350102ef0 2024-09-12T01:47:19.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.400+0000 7f334dffb640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33400a2c90 0x7f33400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.400+0000 7f3354b94640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33400a2c90 0x7f33400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.400+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3348050070 con 0x7f3350102ef0 2024-09-12T01:47:19.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.401+0000 7f3354b94640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33400a2c90 0x7f33400a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3344011550 tx=0x7f3344005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.402+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33480c0f90 con 0x7f3350102ef0 2024-09-12T01:47:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:19 smithi122 ceph-mon[30925]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 -- 172.21.15.122:0/3286417128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c1025f0 msgr2=0x7fc96c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3286417128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c1025f0 0x7fc96c108e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc964009ca0 tx=0x7fc96402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 -- 172.21.15.122:0/3286417128 shutdown_connections 2024-09-12T01:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3286417128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c1025f0 0x7fc96c108e80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3286417128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101c30 0x7fc96c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3286417128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c100a40 0x7fc96c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 -- 172.21.15.122:0/3286417128 >> 172.21.15.122:0/3286417128 conn(0x7fc96c0fc1e0 msgr2=0x7fc96c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 -- 172.21.15.122:0/3286417128 shutdown_connections 2024-09-12T01:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.731+0000 7fc973f6d640 1 -- 172.21.15.122:0/3286417128 wait complete. 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 Processor -- start 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 -- start start 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 0x7fc96c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc971d09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 0x7fc96c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc971d09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 0x7fc96c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54550/0 (socket says 172.21.15.122:54550) 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101c30 0x7fc96c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc971d09640 1 -- 172.21.15.122:0/3822586611 learned_addr learned my addr 172.21.15.122:0/3822586611 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c1025f0 0x7fc96c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c10c6f0 con 0x7fc96c100a40 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c10c570 con 0x7fc96c101c30 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.732+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c10c870 con 0x7fc96c1025f0 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971508640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101c30 0x7fc96c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc97250a640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c1025f0 0x7fc96c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971d09640 1 -- 172.21.15.122:0/3822586611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c1025f0 msgr2=0x7fc96c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971d09640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c1025f0 0x7fc96c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971d09640 1 -- 172.21.15.122:0/3822586611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101c30 msgr2=0x7fc96c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971d09640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101c30 0x7fc96c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971d09640 1 -- 172.21.15.122:0/3822586611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c19f9d0 con 0x7fc96c100a40 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc971d09640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 0x7fc96c06a410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc96800d950 tx=0x7fc96800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc962ffd640 1 -- 172.21.15.122:0/3822586611 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968017020 con 0x7fc96c100a40 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.733+0000 7fc962ffd640 1 -- 172.21.15.122:0/3822586611 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc968009c30 con 0x7fc96c100a40 2024-09-12T01:47:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.734+0000 7fc962ffd640 1 -- 172.21.15.122:0/3822586611 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96800f740 con 0x7fc96c100a40 2024-09-12T01:47:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.734+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c19fc60 con 0x7fc96c100a40 2024-09-12T01:47:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.734+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c1a0160 con 0x7fc96c100a40 2024-09-12T01:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.737+0000 7fc962ffd640 1 -- 172.21.15.122:0/3822586611 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc96801a020 con 0x7fc96c100a40 2024-09-12T01:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 Processor -- start 2024-09-12T01:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 -- start start 2024-09-12T01:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc962ffd640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9580a2e10 0x7fc9580a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b535dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b5362da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5365830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5594b52c12f0 con 0x5594b5363370 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5594b5269270 con 0x5594b535e2e0 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.738+0000 7fc976de4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5594b52bd2b0 con 0x5594b535d9a0 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971d09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b535dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc97250a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5365830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971d09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b535dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34384/0 (socket says 172.21.15.122:34384) 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc97250a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5365830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54558/0 (socket says 172.21.15.122:54558) 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971d09640 1 -- 172.21.15.122:0/3125696385 learned_addr learned my addr 172.21.15.122:0/3125696385 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b5362da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc97250a640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9580a2e10 0x7fc9580a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc962ffd640 1 -- 172.21.15.122:0/3822586611 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc9680ce3e0 con 0x7fc96c100a40 2024-09-12T01:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 -- 172.21.15.122:0/3125696385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 msgr2=0x5594b535dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b535dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 -- 172.21.15.122:0/3125696385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 msgr2=0x5594b5365830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5365830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 -- 172.21.15.122:0/3125696385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b5268340 con 0x5594b535e2e0 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc971508640 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b5362da0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc95c013ba0 tx=0x7fc95c009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e78abe78effffa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc957fff640 1 -- 172.21.15.122:0/3125696385 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c009e20 con 0x5594b535e2e0 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc97250a640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9580a2e10 0x7fc9580a52d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc964031c10 tx=0x7fc964005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc957fff640 1 -- 172.21.15.122:0/3125696385 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c00e990 con 0x5594b535e2e0 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.739+0000 7fc957fff640 1 -- 172.21.15.122:0/3125696385 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0054f0 con 0x5594b535e2e0 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 -- 172.21.15.122:0/3125696385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 msgr2=0x5594b5362da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b5362da0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc95c013ba0 tx=0x7fc95c009b70 comp rx=0 tx=0).stop 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 -- 172.21.15.122:0/3125696385 shutdown_connections 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5365830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b5362da0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 --2- 172.21.15.122:0/3125696385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b535dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.740+0000 7fc976de4740 1 -- 172.21.15.122:0/3125696385 >> 172.21.15.122:0/3125696385 conn(0x5594b5359010 msgr2=0x5594b5359410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.741+0000 7fc976de4740 1 -- 172.21.15.122:0/3125696385 shutdown_connections 2024-09-12T01:47:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.741+0000 7fc976de4740 1 -- 172.21.15.122:0/3125696385 wait complete. 2024-09-12T01:47:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.741+0000 7fc976de4740 1 -- start start 2024-09-12T01:47:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.741+0000 7fc976de4740 1 Processor -- start 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 0x5594b5367bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b5368130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971508640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b5368130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971508640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b5368130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54564/0 (socket says 172.21.15.122:54564) 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971d09640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 0x5594b5367bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b5363370 0x5594b536f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971d09640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 0x5594b5367bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52246/0 (socket says 172.21.15.122:52246) 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971508640 1 -- 172.21.15.122:0/1203804128 learned_addr learned my addr 172.21.15.122:0/1203804128 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5594b5366340 con 0x5594b535e2e0 2024-09-12T01:47:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5594b524a290 con 0x5594b535d9a0 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5594b53664c0 con 0x5594b5363370 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc976de4740 10 client.0 _ll_register_callbacks cb 0x5594b527b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc97250a640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b5363370 0x5594b536f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971d09640 1 -- 172.21.15.122:0/1203804128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b5363370 msgr2=0x5594b536f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971d09640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b5363370 0x5594b536f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.742+0000 7fc971d09640 1 -- 172.21.15.122:0/1203804128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 msgr2=0x5594b5368130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc971d09640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b5368130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc971d09640 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b4fd4210 con 0x5594b535d9a0 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc97250a640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b5363370 0x5594b536f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc971508640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b5368130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc971d09640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 0x5594b5367bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9680a32a0 tx=0x7fc968069b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc6ab0fc8f6c7530 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc9557fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:47:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc9557fa640 1 -- 172.21.15.122:0/1203804128 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9680b23d0 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c00e990 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.743+0000 7fc9557fa640 1 -- 172.21.15.122:0/1203804128 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc968010a40 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.744+0000 7fc9557fa640 1 -- 172.21.15.122:0/1203804128 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc9680b3d60 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.744+0000 7fc976de4740 10 client.25010 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.744+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594b50a0620 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.744+0000 7fc9557fa640 1 -- 172.21.15.122:0/1203804128 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c00e990 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc9557fa640 1 -- 172.21.15.122:0/1203804128 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc9680971d0 con 0x5594b535d9a0 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc976de4740 10 client.25010 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc976de4740 10 client.25010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc976de4740 20 client.25010 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc976de4740 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b5373840 0x5594b5375c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc976de4740 4 client.25010 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b52cea20 con 0x5594b5373840 2024-09-12T01:47:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.745+0000 7fc971508640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b5373840 0x5594b5375c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.746+0000 7fc971508640 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b5373840 0x5594b5375c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434c7fd1998672b0 server_cookie=d06b518129bdf890 in_seq=0 out_seq=0 2024-09-12T01:47:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.746+0000 7fc9557fa640 10 client.25010 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc9557fa640 1 -- 172.21.15.122:0/1203804128 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594b52cea20 con 0x5594b5373840 2024-09-12T01:47:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc9557fa640 10 client.25010 handle_command_reply: tid=0 2024-09-12T01:47:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 client.25010 shutdown 2024-09-12T01:47:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 msgr2=0x5594b5367bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 0x5594b5367bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9680a32a0 tx=0x7fc968069b60 comp rx=0 tx=0).stop 2024-09-12T01:47:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 shutdown_connections 2024-09-12T01:47:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b5373840 0x5594b5375c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b5363370 0x5594b536f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b5368130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 --2- 172.21.15.122:0/1203804128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535d9a0 0x5594b5367bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 >> 172.21.15.122:0/1203804128 conn(0x5594b5359010 msgr2=0x5594b535a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc9557fa640 0 client.25010 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 shutdown_connections 2024-09-12T01:47:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 1 -- 172.21.15.122:0/1203804128 wait complete. 2024-09-12T01:47:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.747+0000 7fc976de4740 20 client.25010 trim_cache size 0 max 16384 2024-09-12T01:47:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 Processor -- start 2024-09-12T01:47:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 -- start start 2024-09-12T01:47:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b53593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b536f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5359930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1a9670 con 0x5594b5363370 2024-09-12T01:47:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1a6f40 con 0x5594b535e2e0 2024-09-12T01:47:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.775+0000 7fc976de4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1a4d70 con 0x5594b535d9a0 2024-09-12T01:47:19.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc97250a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5359930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc97250a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5359930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54570/0 (socket says 172.21.15.122:54570) 2024-09-12T01:47:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc97250a640 1 -- 172.21.15.122:0/1568382595 learned_addr learned my addr 172.21.15.122:0/1568382595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971508640 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b536f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b53593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 -- 172.21.15.122:0/1568382595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 msgr2=0x5594b536f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b536f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 -- 172.21.15.122:0/1568382595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 msgr2=0x5594b5359930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5359930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 -- 172.21.15.122:0/1568382595 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b5368290 con 0x5594b535d9a0 2024-09-12T01:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc97250a640 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5359930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.776+0000 7fc971d09640 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b53593f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc96809ad20 tx=0x7fc96809aa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316a461c2a994e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.777+0000 7fc9577fe640 1 -- 172.21.15.122:0/1568382595 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9680b3f40 con 0x5594b535d9a0 2024-09-12T01:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.777+0000 7fc9577fe640 1 -- 172.21.15.122:0/1568382595 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96809b260 con 0x5594b535d9a0 2024-09-12T01:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.777+0000 7fc9577fe640 1 -- 172.21.15.122:0/1568382595 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9680a37d0 con 0x5594b535d9a0 2024-09-12T01:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 -- 172.21.15.122:0/1568382595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 msgr2=0x5594b53593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b53593f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc96809ad20 tx=0x7fc96809aa60 comp rx=0 tx=0).stop 2024-09-12T01:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 -- 172.21.15.122:0/1568382595 shutdown_connections 2024-09-12T01:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b5363370 0x5594b5359930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b535e2e0 0x5594b536f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 --2- 172.21.15.122:0/1568382595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b53593f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 -- 172.21.15.122:0/1568382595 >> 172.21.15.122:0/1568382595 conn(0x5594b5359010 msgr2=0x5594b5362b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 -- 172.21.15.122:0/1568382595 shutdown_connections 2024-09-12T01:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 -- 172.21.15.122:0/1568382595 wait complete. 2024-09-12T01:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 -- start start 2024-09-12T01:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.778+0000 7fc976de4740 1 Processor -- start 2024-09-12T01:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc976de4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b536bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc976de4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b536c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc971d09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b536bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc971d09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b536bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34404/0 (socket says 172.21.15.122:34404) 2024-09-12T01:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc971508640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b536c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc971508640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b536c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54572/0 (socket says 172.21.15.122:54572) 2024-09-12T01:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc971d09640 1 -- 172.21.15.122:0/1521274765 learned_addr learned my addr 172.21.15.122:0/1521274765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b5363370 0x5594b53dc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.779+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1a5400 con 0x5594b535e2e0 2024-09-12T01:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1a6870 con 0x5594b5363370 2024-09-12T01:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1a5ad0 con 0x5594b535d9a0 2024-09-12T01:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc97250a640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b5363370 0x5594b53dc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc971508640 1 -- 172.21.15.122:0/1521274765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 msgr2=0x5594b536bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc971508640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b536bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc971508640 1 -- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b5363370 msgr2=0x5594b53dc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc971508640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b5363370 0x5594b53dc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc971508640 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b536f3d0 con 0x5594b535e2e0 2024-09-12T01:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc97250a640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b5363370 0x5594b53dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc976de4740 10 client.0 _ll_register_callbacks cb 0x7fc9680cbd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96809b260 con 0x5594b535e2e0 2024-09-12T01:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc971508640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b536c330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0139e0 tx=0x7fc95c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f70acf61f20c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.780+0000 7fc9557fa640 10 client.15555 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc9557fa640 1 -- 172.21.15.122:0/1521274765 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c038050 con 0x5594b535e2e0 2024-09-12T01:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc9557fa640 1 -- 172.21.15.122:0/1521274765 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc95c019040 con 0x5594b535e2e0 2024-09-12T01:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc9557fa640 1 -- 172.21.15.122:0/1521274765 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc95c0050d0 con 0x5594b535e2e0 2024-09-12T01:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 10 client.15555 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5594b53dd7c0 con 0x5594b535e2e0 2024-09-12T01:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc9557fa640 1 -- 172.21.15.122:0/1521274765 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc95c03b020 con 0x5594b535e2e0 2024-09-12T01:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 10 client.15555 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 20 client.15555 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b53ddda0 0x5594b53e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 4 client.15555 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:47:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.781+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594b52c8d70 con 0x5594b53ddda0 2024-09-12T01:47:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.782+0000 7fc971d09640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b53ddda0 0x5594b53e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.782+0000 7fc9557fa640 10 client.15555 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.782+0000 7fc971d09640 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b53ddda0 0x5594b53e0260 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363b5d6e8df9485e server_cookie=ea03b29489775e9a in_seq=0 out_seq=0 2024-09-12T01:47:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc9557fa640 1 -- 172.21.15.122:0/1521274765 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594b52c8d70 con 0x5594b53ddda0 2024-09-12T01:47:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc9557fa640 10 client.15555 handle_command_reply: tid=0 2024-09-12T01:47:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 client.15555 shutdown 2024-09-12T01:47:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 msgr2=0x5594b536c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b536c330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0139e0 tx=0x7fc95c00ead0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 shutdown_connections 2024-09-12T01:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5594b53ddda0 0x5594b53e0260 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594b5363370 0x5594b53dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594b535e2e0 0x5594b536c330 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 --2- 172.21.15.122:0/1521274765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594b535d9a0 0x5594b536bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 >> 172.21.15.122:0/1521274765 conn(0x5594b5359010 msgr2=0x5594b53579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc9557fa640 0 client.15555 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 shutdown_connections 2024-09-12T01:47:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 1 -- 172.21.15.122:0/1521274765 wait complete. 2024-09-12T01:47:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.791+0000 7fc976de4740 20 client.15555 trim_cache size 0 max 16384 2024-09-12T01:47:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:47:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:47:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:47:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:47:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9580a2e10 msgr2=0x7fc9580a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9580a2e10 0x7fc9580a52d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc964031c10 tx=0x7fc964005ef0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 msgr2=0x7fc96c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 0x7fc96c06a410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc96800d950 tx=0x7fc96800de10 comp rx=0 tx=0).stop 2024-09-12T01:47:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 shutdown_connections 2024-09-12T01:47:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9580a2e10 0x7fc9580a52d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc96c1025f0 0x7fc96c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101c30 0x7fc96c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 --2- 172.21.15.122:0/3822586611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc96c100a40 0x7fc96c06a410 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.792+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 >> 172.21.15.122:0/3822586611 conn(0x7fc96c0fc1e0 msgr2=0x7fc96c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.793+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 shutdown_connections 2024-09-12T01:47:19.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:19.793+0000 7fc973f6d640 1 -- 172.21.15.122:0/3822586611 wait complete. 2024-09-12T01:47:19.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.805+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33501a2fd0 con 0x7f3350102ef0 2024-09-12T01:47:19.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.806+0000 7f334dffb640 1 -- 172.21.15.122:0/908838273 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f33480a9e20 con 0x7f3350102ef0 2024-09-12T01:47:19.811 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:47:19.811 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:47:19.812 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:47:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33400a2c90 msgr2=0x7f33400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33400a2c90 0x7f33400a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3344011550 tx=0x7f3344005f90 comp rx=0 tx=0).stop 2024-09-12T01:47:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350102ef0 msgr2=0x7f335019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350102ef0 0x7f335019ab50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f334802ce30 tx=0x7f3348032fe0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 shutdown_connections 2024-09-12T01:47:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33400a2c90 0x7f33400a5150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3350102ef0 0x7f335019ab50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3350102570 0x7f33501967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 --2- 172.21.15.122:0/908838273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3350100e20 0x7f3350196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:19.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 >> 172.21.15.122:0/908838273 conn(0x7f33500fc7d0 msgr2=0x7f33501068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:19.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 shutdown_connections 2024-09-12T01:47:19.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:19.809+0000 7f3356df8640 1 -- 172.21.15.122:0/908838273 wait complete. 2024-09-12T01:47:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:19 smithi151 ceph-mon[35229]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:19 smithi183 ceph-mon[35127]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T01:47:20.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:47:20.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/908838273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:20.768 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/908838273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/908838273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 -- 172.21.15.122:0/3419810209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0100a90 msgr2=0x7f4ea0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 --2- 172.21.15.122:0/3419810209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0100a90 0x7f4ea0100ef0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009930 tx=0x7f4e9402a420 comp rx=0 tx=0).stop 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 -- 172.21.15.122:0/3419810209 shutdown_connections 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 --2- 172.21.15.122:0/3419810209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ea0101430 0x7f4ea010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 --2- 172.21.15.122:0/3419810209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0100a90 0x7f4ea0100ef0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 --2- 172.21.15.122:0/3419810209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea0106a90 0x7f4ea0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.317+0000 7f4ea5199640 1 -- 172.21.15.122:0/3419810209 >> 172.21.15.122:0/3419810209 conn(0x7f4ea00fc7d0 msgr2=0x7f4ea00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.318+0000 7f4ea5199640 1 -- 172.21.15.122:0/3419810209 shutdown_connections 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.318+0000 7f4ea5199640 1 -- 172.21.15.122:0/3419810209 wait complete. 2024-09-12T01:47:21.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.318+0000 7f4ea5199640 1 Processor -- start 2024-09-12T01:47:21.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 -- start start 2024-09-12T01:47:21.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea0100a90 0x7f4ea0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 0x7f4ea01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ea0106a90 0x7f4ea01a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0110ea0 con 0x7f4ea0101430 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0110d20 con 0x7f4ea0100a90 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4ea5199640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0111020 con 0x7f4ea0106a90 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4e9e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 0x7f4ea01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4e9e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 0x7f4ea01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49446/0 (socket says 172.21.15.122:49446) 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2660376646 learned_addr learned my addr 172.21.15.122:0/2660376646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.319+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea0100a90 0x7f4ea0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ea0106a90 0x7f4ea01a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2660376646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ea0106a90 msgr2=0x7f4ea01a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ea0106a90 0x7f4ea01a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2660376646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea0100a90 msgr2=0x7f4ea0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea0100a90 0x7f4ea0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2660376646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea01a4850 con 0x7f4ea0101430 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 0x7f4ea01997d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4e9402a940 tx=0x7f4e94003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.320+0000 7f4e87fff640 1 -- 172.21.15.122:0/2660376646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9403f070 con 0x7f4ea0101430 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.321+0000 7f4e87fff640 1 -- 172.21.15.122:0/2660376646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e94031c90 con 0x7f4ea0101430 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.321+0000 7f4e87fff640 1 -- 172.21.15.122:0/2660376646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94035030 con 0x7f4ea0101430 2024-09-12T01:47:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.321+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea0197080 con 0x7f4ea0101430 2024-09-12T01:47:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.321+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01974b0 con 0x7f4ea0101430 2024-09-12T01:47:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.324+0000 7f4ea7572740 1 Processor -- start 2024-09-12T01:47:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.324+0000 7f4ea7572740 1 -- start start 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15277b40 0x560f15276090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f15276a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4e9ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f15276a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f15283760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4ea7572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560f151dc2f0 con 0x560f152765d0 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4e9ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f15276a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49460/0 (socket says 172.21.15.122:49460) 2024-09-12T01:47:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/2041726311 learned_addr learned my addr 172.21.15.122:0/2041726311 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f15183cf0 con 0x560f15281370 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.325+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f151d82b0 con 0x560f15277b40 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.326+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15277b40 0x560f15276090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.326+0000 7f4e87fff640 1 -- 172.21.15.122:0/2660376646 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f4e94042070 con 0x7f4ea0101430 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.326+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/2041726311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15277b40 msgr2=0x560f15276090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.326+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15277b40 0x560f15276090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.326+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/2041726311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 msgr2=0x560f15283760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f15283760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f15283760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/2041726311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f151683a0 con 0x560f152765d0 2024-09-12T01:47:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f15276a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000bc90 tx=0x7f4e900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ca9c64a69c6ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e857fa640 1 -- 172.21.15.122:0/2041726311 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9001c480 con 0x560f152765d0 2024-09-12T01:47:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e857fa640 1 -- 172.21.15.122:0/2041726311 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9001cb00 con 0x560f152765d0 2024-09-12T01:47:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.327+0000 7f4e857fa640 1 -- 172.21.15.122:0/2041726311 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90011c60 con 0x560f152765d0 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.328+0000 7f4e87fff640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4e880a7120 0x7f4e880a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.328+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4e880a7120 0x7f4e880a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.328+0000 7f4e87fff640 1 -- 172.21.15.122:0/2660376646 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4e940ef550 con 0x7f4ea0101430 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.328+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 msgr2=0x560f15276a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.328+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4e880a7120 0x7f4e880a95e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e98005fd0 tx=0x7f4e98009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.328+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f15276a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000bc90 tx=0x7f4e900107a0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 shutdown_connections 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f15283760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f15276a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2041726311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15277b40 0x560f15276090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 >> 172.21.15.122:0/2041726311 conn(0x560f1523f310 msgr2=0x560f15272c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 shutdown_connections 2024-09-12T01:47:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 -- 172.21.15.122:0/2041726311 wait complete. 2024-09-12T01:47:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.329+0000 7f4ea7572740 1 -- start start 2024-09-12T01:47:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 Processor -- start 2024-09-12T01:47:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f15280d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f1527dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560f15284070 con 0x560f15277b40 2024-09-12T01:47:21.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4e9e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f1527dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f15197060 con 0x560f15281370 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4ea7572740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f152841f0 con 0x560f152765d0 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.330+0000 7f4e9e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f1527dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49464/0 (socket says 172.21.15.122:49464) 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9e59c640 1 -- 172.21.15.122:0/1649255327 learned_addr learned my addr 172.21.15.122:0/1649255327 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f15280d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4ea7572740 10 client.0 _ll_register_callbacks cb 0x560f14eea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 -- 172.21.15.122:0/1649255327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 msgr2=0x560f15280d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f15280d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 -- 172.21.15.122:0/1649255327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 msgr2=0x560f1527dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f1527dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 -- 172.21.15.122:0/1649255327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f14f4d3c0 con 0x560f15281370 2024-09-12T01:47:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f1527dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527e570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e98003c10 tx=0x7f4e98004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4023a73248de32d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.331+0000 7f4e82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4e82ff5640 1 -- 172.21.15.122:0/1649255327 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98003e40 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9001cb00 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4e82ff5640 1 -- 172.21.15.122:0/1649255327 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4e98004b20 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4e82ff5640 1 -- 172.21.15.122:0/1649255327 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e98004d60 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4ea7572740 10 client.25025 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f1528fbe0 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4e82ff5640 1 -- 172.21.15.122:0/1649255327 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9001cb00 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4e82ff5640 1 -- 172.21.15.122:0/1649255327 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4e9800f040 con 0x560f15281370 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.332+0000 7f4ea7572740 10 client.25025 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.333+0000 7f4ea7572740 10 client.25025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:47:21.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.333+0000 7f4ea7572740 20 client.25025 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:21.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.333+0000 7f4ea7572740 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f15293920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.333+0000 7f4ea7572740 4 client.25025 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:21.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.333+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f151e9860 con 0x560f15291460 2024-09-12T01:47:21.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.333+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f15293920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.334+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f15293920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eed947c13a8a47ec server_cookie=51049025c11676b0 in_seq=0 out_seq=0 2024-09-12T01:47:21.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.334+0000 7f4e82ff5640 10 client.25025 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.334+0000 7f4e82ff5640 1 -- 172.21.15.122:0/1649255327 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f151e9860 con 0x560f15291460 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.334+0000 7f4e82ff5640 10 client.25025 handle_command_reply: tid=0 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.334+0000 7f4ea7572740 1 client.25025 shutdown 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 msgr2=0x560f1527e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527e570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e98003c10 tx=0x7f4e98004a40 comp rx=0 tx=0).stop 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 shutdown_connections 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f15293920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527e570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4e82ff5640 0 client.25025 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:21.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f1527dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 --2- 172.21.15.122:0/1649255327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f15280d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 >> 172.21.15.122:0/1649255327 conn(0x560f1523f310 msgr2=0x560f15278930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:21.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 shutdown_connections 2024-09-12T01:47:21.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 1 -- 172.21.15.122:0/1649255327 wait complete. 2024-09-12T01:47:21.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.335+0000 7f4ea7572740 20 client.25025 trim_cache size 0 max 16384 2024-09-12T01:47:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.360+0000 7f4ea7572740 1 Processor -- start 2024-09-12T01:47:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.360+0000 7f4ea7572740 1 -- start start 2024-09-12T01:47:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f1527ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15277b40 0x560f15279120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15281370 0x560f15279660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4ea7572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c010d60 con 0x560f152765d0 2024-09-12T01:47:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4ea7572740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00e630 con 0x560f15277b40 2024-09-12T01:47:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4ea7572740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00c460 con 0x560f15281370 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f1527ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f1527ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49476/0 (socket says 172.21.15.122:49476) 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/3874753261 learned_addr learned my addr 172.21.15.122:0/3874753261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15281370 0x560f15279660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/3874753261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15281370 msgr2=0x560f15279660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15281370 0x560f15279660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/3874753261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15277b40 msgr2=0x560f15279120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15277b40 0x560f15279120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9ed9d640 1 -- 172.21.15.122:0/3874753261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f15281090 con 0x560f152765d0 2024-09-12T01:47:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.361+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15281370 0x560f15279660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.362+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f1527ff40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000bc90 tx=0x7f4e9001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d26e32e539b8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.362+0000 7f4e84ff9640 1 -- 172.21.15.122:0/3874753261 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9002e030 con 0x560f152765d0 2024-09-12T01:47:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.362+0000 7f4e84ff9640 1 -- 172.21.15.122:0/3874753261 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e90011870 con 0x560f152765d0 2024-09-12T01:47:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.362+0000 7f4e84ff9640 1 -- 172.21.15.122:0/3874753261 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e900052a0 con 0x560f152765d0 2024-09-12T01:47:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 -- 172.21.15.122:0/3874753261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 msgr2=0x560f1527ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f1527ff40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000bc90 tx=0x7f4e9001f040 comp rx=0 tx=0).stop 2024-09-12T01:47:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 -- 172.21.15.122:0/3874753261 shutdown_connections 2024-09-12T01:47:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f15281370 0x560f15279660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15277b40 0x560f15279120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 --2- 172.21.15.122:0/3874753261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f152765d0 0x560f1527ff40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 -- 172.21.15.122:0/3874753261 >> 172.21.15.122:0/3874753261 conn(0x560f1523f310 msgr2=0x560f1527f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 -- 172.21.15.122:0/3874753261 shutdown_connections 2024-09-12T01:47:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.363+0000 7f4ea7572740 1 -- 172.21.15.122:0/3874753261 wait complete. 2024-09-12T01:47:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 -- start start 2024-09-12T01:47:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 Processor -- start 2024-09-12T01:47:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f152897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f15289d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4e9e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f15289d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4e9e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f15289d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49486/0 (socket says 172.21.15.122:49486) 2024-09-12T01:47:21.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2971911960 learned_addr learned my addr 172.21.15.122:0/2971911960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:21.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00caf0 con 0x560f15277b40 2024-09-12T01:47:21.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00df60 con 0x560f15281370 2024-09-12T01:47:21.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.364+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00d1c0 con 0x560f152765d0 2024-09-12T01:47:21.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4ea7572740 10 client.0 _ll_register_callbacks cb 0x7f4e98003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:21.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2971911960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 msgr2=0x560f152897d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:47:21.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f152897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 msgr2=0x560f1527a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:21.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9e59c640 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1527b130 con 0x560f15277b40 2024-09-12T01:47:21.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9f59e640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:21.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e9e59c640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f15289d10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4e9403ba00 tx=0x7f4e940bb860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1b6aeb05a3cc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:21.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:21.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e82ff5640 1 -- 172.21.15.122:0/2971911960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940ef310 con 0x560f15277b40 2024-09-12T01:47:21.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4e82ff5640 1 -- 172.21.15.122:0/2971911960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4e940c4500 con 0x560f15277b40 2024-09-12T01:47:21.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.365+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e90011870 con 0x560f15277b40 2024-09-12T01:47:21.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4e82ff5640 1 -- 172.21.15.122:0/2971911960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e940bbd20 con 0x560f15277b40 2024-09-12T01:47:21.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 10 client.15573 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:21.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f152fbce0 con 0x560f15277b40 2024-09-12T01:47:21.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4e82ff5640 1 -- 172.21.15.122:0/2971911960 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e940c00d0 con 0x560f15277b40 2024-09-12T01:47:21.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4e82ff5640 1 -- 172.21.15.122:0/2971911960 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4e940bb170 con 0x560f15277b40 2024-09-12T01:47:21.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 10 client.15573 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 10 client.15573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:47:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 20 client.15573 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f152fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 4 client.15573 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:47:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.366+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4e90011870 con 0x560f15291460 2024-09-12T01:47:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f152fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4e9ed9d640 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f152fe4a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99b67f5851ed824c server_cookie=8788bfefa7c1604c in_seq=0 out_seq=0 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4e82ff5640 10 client.15573 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4e82ff5640 1 -- 172.21.15.122:0/2971911960 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4e90011870 con 0x560f15291460 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4e82ff5640 10 client.15573 handle_command_reply: tid=0 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4ea7572740 1 client.15573 shutdown 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 msgr2=0x560f15289d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f15289d10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4e9403ba00 tx=0x7f4e940bb860 comp rx=0 tx=0).stop 2024-09-12T01:47:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 shutdown_connections 2024-09-12T01:47:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.367+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x560f15291460 0x560f152fe4a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4e82ff5640 0 client.15573 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:47:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f15281370 0x560f1527a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f15277b40 0x560f15289d10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 1 --2- 172.21.15.122:0/2971911960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f152765d0 0x560f152897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 >> 172.21.15.122:0/2971911960 conn(0x560f1523f310 msgr2=0x560f15271d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:21.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 shutdown_connections 2024-09-12T01:47:21.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 1 -- 172.21.15.122:0/2971911960 wait complete. 2024-09-12T01:47:21.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.368+0000 7f4ea7572740 20 client.15573 trim_cache size 0 max 16384 2024-09-12T01:47:21.379 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4e880a7120 msgr2=0x7f4e880a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4e880a7120 0x7f4e880a95e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e98005fd0 tx=0x7f4e98009600 comp rx=0 tx=0).stop 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 msgr2=0x7f4ea01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 0x7f4ea01997d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4e9402a940 tx=0x7f4e94003b90 comp rx=0 tx=0).stop 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 shutdown_connections 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4e880a7120 0x7f4e880a95e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ea0106a90 0x7f4ea01a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ea0101430 0x7f4ea01997d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 --2- 172.21.15.122:0/2660376646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea0100a90 0x7f4ea0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:21.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 >> 172.21.15.122:0/2660376646 conn(0x7f4ea00fc7d0 msgr2=0x7f4ea00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 shutdown_connections 2024-09-12T01:47:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:21.369+0000 7f4ea5199640 1 -- 172.21.15.122:0/2660376646 wait complete. 2024-09-12T01:47:21.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:21 smithi151 ceph-mon[35229]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T01:47:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:21 smithi183 ceph-mon[35127]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T01:47:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:21 smithi122 ceph-mon[30925]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T01:47:22.075 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:47:22.362 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 -- 172.21.15.122:0/569895374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990102d90 msgr2=0x7f5990103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 --2- 172.21.15.122:0/569895374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990102d90 0x7f5990103170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f598c009930 tx=0x7f598c02a420 comp rx=0 tx=0).stop 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 -- 172.21.15.122:0/569895374 shutdown_connections 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 --2- 172.21.15.122:0/569895374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 0x7f599010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 --2- 172.21.15.122:0/569895374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5990103740 0x7f5990107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 --2- 172.21.15.122:0/569895374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990102d90 0x7f5990103170 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 -- 172.21.15.122:0/569895374 >> 172.21.15.122:0/569895374 conn(0x7f59900fc7d0 msgr2=0x7f59900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:22.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 -- 172.21.15.122:0/569895374 shutdown_connections 2024-09-12T01:47:22.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.881+0000 7f5997c54640 1 -- 172.21.15.122:0/569895374 wait complete. 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 Processor -- start 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 -- start start 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5990102d90 0x7f599019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 0x7f599019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 0x7f599010dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59901120d0 con 0x7f5990103740 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5990111f50 con 0x7f5990108290 2024-09-12T01:47:22.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f5997c54640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5990112250 con 0x7f5990102d90 2024-09-12T01:47:22.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f59951ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 0x7f599019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f59951ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 0x7f599019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49508/0 (socket says 172.21.15.122:49508) 2024-09-12T01:47:22.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f59951ef640 1 -- 172.21.15.122:0/1955718409 learned_addr learned my addr 172.21.15.122:0/1955718409 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:22.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f59961f1640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 0x7f599010dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.882+0000 7f59959f0640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5990102d90 0x7f599019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59951ef640 1 -- 172.21.15.122:0/1955718409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5990102d90 msgr2=0x7f599019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59951ef640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5990102d90 0x7f599019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59951ef640 1 -- 172.21.15.122:0/1955718409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 msgr2=0x7f599010dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59951ef640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 0x7f599010dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59951ef640 1 -- 172.21.15.122:0/1955718409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599010e500 con 0x7f5990103740 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59961f1640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 0x7f599010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f59951ef640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 0x7f599019b490 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f5986ffd640 1 -- 172.21.15.122:0/1955718409 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598000ccc0 con 0x7f5990103740 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f5986ffd640 1 -- 172.21.15.122:0/1955718409 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5980005500 con 0x7f5990103740 2024-09-12T01:47:22.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599010e7f0 con 0x7f5990103740 2024-09-12T01:47:22.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f5986ffd640 1 -- 172.21.15.122:0/1955718409 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980010690 con 0x7f5990103740 2024-09-12T01:47:22.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.883+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599019ee30 con 0x7f5990103740 2024-09-12T01:47:22.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.886+0000 7f5986ffd640 1 -- 172.21.15.122:0/1955718409 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f59800046c0 con 0x7f5990103740 2024-09-12T01:47:22.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.887+0000 7f599a02d740 1 Processor -- start 2024-09-12T01:47:22.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 -- start start 2024-09-12T01:47:22.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f5986ffd640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f597c0a2c90 0x7f597c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fa7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2fb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556fb2f0d2f0 con 0x556fb2fb2370 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556fb2eb4cf0 con 0x556fb2fa75d0 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f599a02d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fb2f092b0 con 0x556fb2fa8b40 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f5986ffd640 1 -- 172.21.15.122:0/1955718409 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f59800cf630 con 0x7f5990103740 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f59961f1640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f597c0a2c90 0x7f597c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f59961f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2fb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f59951ef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fa7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f59951ef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fa7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:59114/0 (socket says 172.21.15.122:59114) 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.888+0000 7f59951ef640 1 -- 172.21.15.122:0/3770511356 learned_addr learned my addr 172.21.15.122:0/3770511356 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59959f0640 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59951ef640 1 -- 172.21.15.122:0/3770511356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 msgr2=0x556fb2fa7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59951ef640 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fa7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59951ef640 1 -- 172.21.15.122:0/3770511356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 msgr2=0x556fb2fb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59951ef640 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2fb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59951ef640 1 -- 172.21.15.122:0/3770511356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb2e993a0 con 0x556fb2fa75d0 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59951ef640 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fa7a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5980098700 tx=0x7f598009b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9519c8344ae92129 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59959f0640 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f59961f1640 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2fb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f597bfff640 1 -- 172.21.15.122:0/3770511356 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800120f0 con 0x556fb2fa75d0 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f597bfff640 1 -- 172.21.15.122:0/3770511356 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5980016e10 con 0x556fb2fa75d0 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.889+0000 7f597bfff640 1 -- 172.21.15.122:0/3770511356 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800cd230 con 0x556fb2fa75d0 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.890+0000 7f59961f1640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f597c0a2c90 0x7f597c0a5150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x556fb2fa8070 tx=0x7f5988009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 -- 172.21.15.122:0/3770511356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 msgr2=0x556fb2fa7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fa7a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5980098700 tx=0x7f598009b870 comp rx=0 tx=0).stop 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 -- 172.21.15.122:0/3770511356 shutdown_connections 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2fb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fa7a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 --2- 172.21.15.122:0/3770511356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 -- 172.21.15.122:0/3770511356 >> 172.21.15.122:0/3770511356 conn(0x556fb2f70310 msgr2=0x556fb2fa3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 -- 172.21.15.122:0/3770511356 shutdown_connections 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 -- 172.21.15.122:0/3770511356 wait complete. 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 -- start start 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.891+0000 7f599a02d740 1 Processor -- start 2024-09-12T01:47:22.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f599a02d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f59959f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f59959f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49526/0 (socket says 172.21.15.122:49526) 2024-09-12T01:47:22.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f59959f0640 1 -- 172.21.15.122:0/3556945458 learned_addr learned my addr 172.21.15.122:0/3556945458 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:22.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa8b40 0x556fb2faefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fb2370 0x556fb2faf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556fb2fb5070 con 0x556fb2fa75d0 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556fb2ec8060 con 0x556fb2fa8b40 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fb2fb51f0 con 0x556fb2fb2370 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.892+0000 7f59951ef640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa8b40 0x556fb2faefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59961f1640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fb2370 0x556fb2faf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59951ef640 1 -- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fb2370 msgr2=0x556fb2faf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59951ef640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fb2370 0x556fb2faf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59951ef640 1 -- 172.21.15.122:0/3556945458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 msgr2=0x556fb2fb1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59951ef640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59951ef640 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb2c7e3c0 con 0x556fb2fa8b40 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59961f1640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fb2370 0x556fb2faf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59959f0640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f599a02d740 10 client.0 _ll_register_callbacks cb 0x556fb2c1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5980016e10 con 0x556fb2fa8b40 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59951ef640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa8b40 0x556fb2faefe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59800a02a0 tx=0x7f59800ccf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e299bb93ab706db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59797fa640 10 client.25040 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59797fa640 1 -- 172.21.15.122:0/3556945458 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800b8020 con 0x556fb2fa8b40 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59797fa640 1 -- 172.21.15.122:0/3556945458 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5980013190 con 0x556fb2fa8b40 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f59797fa640 1 -- 172.21.15.122:0/3556945458 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5980013390 con 0x556fb2fa8b40 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f599a02d740 10 client.25040 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.893+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556fb2fc0710 con 0x556fb2fa8b40 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f59797fa640 1 -- 172.21.15.122:0/3556945458 <== mon.2 v2:172.21.15.151:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f59800b2020 con 0x556fb2fa8b40 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f599a02d740 10 client.25040 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f599a02d740 10 client.25040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f599a02d740 20 client.25040 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb2fc4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f599a02d740 4 client.25040 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb2f1a860 con 0x556fb2fc1e80 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.894+0000 7f59959f0640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb2fc4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.895+0000 7f59959f0640 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb2fc4310 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da0f3d7954e061c9 server_cookie=922449b6d0385559 in_seq=0 out_seq=0 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.895+0000 7f59797fa640 10 client.25040 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.895+0000 7f59797fa640 1 -- 172.21.15.122:0/3556945458 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fb2f1a860 con 0x556fb2fc1e80 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.895+0000 7f59797fa640 10 client.25040 handle_command_reply: tid=0 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.895+0000 7f599a02d740 1 client.25040 shutdown 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa8b40 msgr2=0x556fb2faefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa8b40 0x556fb2faefe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59800a02a0 tx=0x7f59800ccf50 comp rx=0 tx=0).stop 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 shutdown_connections 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb2fc4310 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fb2370 0x556fb2faf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa8b40 0x556fb2faefe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 --2- 172.21.15.122:0/3556945458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f59797fa640 0 client.25040 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 >> 172.21.15.122:0/3556945458 conn(0x556fb2f70310 msgr2=0x556fb2fa99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 shutdown_connections 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 1 -- 172.21.15.122:0/3556945458 wait complete. 2024-09-12T01:47:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.896+0000 7f599a02d740 20 client.25040 trim_cache size 0 max 16384 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 Processor -- start 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 -- start start 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2faa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59901bed20 con 0x556fb2fb2370 2024-09-12T01:47:22.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59901bc5f0 con 0x556fb2fa75d0 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f599a02d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59901ba420 con 0x556fb2fa8b40 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.921+0000 7f59961f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2faa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59961f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2faa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49534/0 (socket says 172.21.15.122:49534) 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59961f1640 1 -- 172.21.15.122:0/2719136945 learned_addr learned my addr 172.21.15.122:0/2719136945 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 -- 172.21.15.122:0/2719136945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 msgr2=0x556fb2fae8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fae8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 -- 172.21.15.122:0/2719136945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 msgr2=0x556fb2faa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2faa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 -- 172.21.15.122:0/2719136945 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb2fa4960 con 0x556fb2fa8b40 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59961f1640 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2faa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f59951ef640 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb0f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x556fb2fb0590 tx=0x7f59800b5000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd245938462e4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:22.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f597b7fe640 1 -- 172.21.15.122:0/2719136945 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800b8040 con 0x556fb2fa8b40 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f597b7fe640 1 -- 172.21.15.122:0/2719136945 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5980016e10 con 0x556fb2fa8b40 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.922+0000 7f597b7fe640 1 -- 172.21.15.122:0/2719136945 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598007aba0 con 0x556fb2fa8b40 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 -- 172.21.15.122:0/2719136945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 msgr2=0x556fb2fb0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb0f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x556fb2fb0590 tx=0x7f59800b5000 comp rx=0 tx=0).stop 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 -- 172.21.15.122:0/2719136945 shutdown_connections 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fb2370 0x556fb2faa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb0f50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 --2- 172.21.15.122:0/2719136945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fa75d0 0x556fb2fae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 -- 172.21.15.122:0/2719136945 >> 172.21.15.122:0/2719136945 conn(0x556fb2f70310 msgr2=0x556fb2fb0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 -- 172.21.15.122:0/2719136945 shutdown_connections 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.923+0000 7f599a02d740 1 -- 172.21.15.122:0/2719136945 wait complete. 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.924+0000 7f599a02d740 1 -- start start 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.924+0000 7f599a02d740 1 Processor -- start 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.924+0000 7f599a02d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.924+0000 7f599a02d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f599a02d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fb2370 0x556fb302abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f599a02d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59901baab0 con 0x556fb2fa75d0 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f599a02d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59901bbf20 con 0x556fb2fb2370 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f599a02d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59901bb180 con 0x556fb2fa8b40 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f599a02d740 10 client.0 _ll_register_callbacks cb 0x7f5980016fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49536/0 (socket says 172.21.15.122:49536) 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 -- 172.21.15.122:0/230821862 learned_addr learned my addr 172.21.15.122:0/230821862 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59961f1640 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fb2370 0x556fb302abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 -- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 msgr2=0x556fb2fb5f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb5f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 -- 172.21.15.122:0/230821862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fb2370 msgr2=0x556fb302abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fb2370 0x556fb302abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 -- 172.21.15.122:0/230821862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb302b220 con 0x556fb2fa75d0 2024-09-12T01:47:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59959f0640 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb59e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f598c00b0f0 tx=0x7f598c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97963993b112250 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.925+0000 7f59797fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f59797fa640 1 -- 172.21.15.122:0/230821862 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c035040 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f59797fa640 1 -- 172.21.15.122:0/230821862 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f598c0037f0 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5980016e10 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f59797fa640 1 -- 172.21.15.122:0/230821862 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f598c0039f0 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 10 client.15591 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fb302c500 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f59797fa640 1 -- 172.21.15.122:0/230821862 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c05b070 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f59797fa640 1 -- 172.21.15.122:0/230821862 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f598c045350 con 0x556fb2fa75d0 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 10 client.15591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 20 client.15591 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb302ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 4 client.15591 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.926+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5980016e10 con 0x556fb2fc1e80 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.927+0000 7f59951ef640 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb302ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f59951ef640 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb302ec80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1b715324ec85fb4 server_cookie=d50a399650701405 in_seq=0 out_seq=0 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f59797fa640 10 client.15591 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f59797fa640 1 -- 172.21.15.122:0/230821862 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5980016e10 con 0x556fb2fc1e80 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f59797fa640 10 client.15591 handle_command_reply: tid=0 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 client.15591 shutdown 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 msgr2=0x556fb2fb59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb59e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f598c00b0f0 tx=0x7f598c005760 comp rx=0 tx=0).stop 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 shutdown_connections 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x556fb2fc1e80 0x556fb302ec80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f59797fa640 0 client.15591 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fb2fb2370 0x556fb302abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb2fa8b40 0x556fb2fb5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 --2- 172.21.15.122:0/230821862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb2fa75d0 0x556fb2fb59e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.928+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 >> 172.21.15.122:0/230821862 conn(0x556fb2f70310 msgr2=0x556fb2fb0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 shutdown_connections 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f599a02d740 1 -- 172.21.15.122:0/230821862 wait complete. 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f599a02d740 20 client.15591 trim_cache size 0 max 16384 2024-09-12T01:47:22.933 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f597c0a2c90 msgr2=0x7f597c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f5997c54640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f597c0a2c90 0x7f597c0a5150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x556fb2fa8070 tx=0x7f5988009520 comp rx=0 tx=0).stop 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 msgr2=0x7f599019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.929+0000 7f5997c54640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 0x7f599019b490 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 shutdown_connections 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f597c0a2c90 0x7f597c0a5150 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5990108290 0x7f599010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5990103740 0x7f599019b490 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 --2- 172.21.15.122:0/1955718409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5990102d90 0x7f599019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 >> 172.21.15.122:0/1955718409 conn(0x7f59900fc7d0 msgr2=0x7f59900fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 shutdown_connections 2024-09-12T01:47:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:22.930+0000 7f5997c54640 1 -- 172.21.15.122:0/1955718409 wait complete. 2024-09-12T01:47:23.690 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:23 smithi122 ceph-mon[30925]: pgmap v362: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T01:47:23.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:47:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:23 smithi151 ceph-mon[35229]: pgmap v362: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T01:47:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:23 smithi183 ceph-mon[35127]: pgmap v362: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T01:47:24.071 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 -- 172.21.15.122:0/1311487064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec100430 msgr2=0x7f47ec100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 --2- 172.21.15.122:0/1311487064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec100430 0x7f47ec100810 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009900 tx=0x7f47dc02a410 comp rx=0 tx=0).stop 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 -- 172.21.15.122:0/1311487064 shutdown_connections 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 --2- 172.21.15.122:0/1311487064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec105560 0x7f47ec10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 --2- 172.21.15.122:0/1311487064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47ec100de0 0x7f47ec104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 --2- 172.21.15.122:0/1311487064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec100430 0x7f47ec100810 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 -- 172.21.15.122:0/1311487064 >> 172.21.15.122:0/1311487064 conn(0x7f47ec077230 msgr2=0x7f47ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:24.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 -- 172.21.15.122:0/1311487064 shutdown_connections 2024-09-12T01:47:24.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.607+0000 7f47f218c640 1 -- 172.21.15.122:0/1311487064 wait complete. 2024-09-12T01:47:24.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 Processor -- start 2024-09-12T01:47:24.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 -- start start 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 0x7f47ec110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47ec100de0 0x7f47ec110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec105560 0x7f47ec111000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec10fd10 con 0x7f47ec105560 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec10fb90 con 0x7f47ec100de0 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.609+0000 7f47f218c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec10fe90 con 0x7f47ec100430 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 0x7f47ec110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 0x7f47ec110580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:49026/0 (socket says 172.21.15.122:49026) 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 -- 172.21.15.122:0/31581388 learned_addr learned my addr 172.21.15.122:0/31581388 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:24.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47ebfff640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec105560 0x7f47ec111000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 -- 172.21.15.122:0/31581388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47ec100de0 msgr2=0x7f47ec110ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47ec100de0 0x7f47ec110ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 -- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec105560 msgr2=0x7f47ec111000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec105560 0x7f47ec111000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 -- 172.21.15.122:0/31581388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec1a6930 con 0x7f47ec100430 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.610+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 0x7f47ec110580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47dc006010 tx=0x7f47dc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.611+0000 7f47e97fa640 1 -- 172.21.15.122:0/31581388 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc03f030 con 0x7f47ec100430 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.611+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec1a6bc0 con 0x7f47ec100430 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.611+0000 7f47e97fa640 1 -- 172.21.15.122:0/31581388 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47dc003ce0 con 0x7f47ec100430 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.611+0000 7f47e97fa640 1 -- 172.21.15.122:0/31581388 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc035070 con 0x7f47ec100430 2024-09-12T01:47:24.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.611+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec1a70f0 con 0x7f47ec100430 2024-09-12T01:47:24.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.614+0000 7f47e97fa640 1 -- 172.21.15.122:0/31581388 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f47dc042030 con 0x7f47ec100430 2024-09-12T01:47:24.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 Processor -- start 2024-09-12T01:47:24.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 -- start start 2024-09-12T01:47:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x55899639a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x55899639aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589963a5370 0x5589963a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5589963002f0 con 0x55899639a5d0 2024-09-12T01:47:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5589962a7cf0 con 0x5589963a5370 2024-09-12T01:47:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.615+0000 7f47f4565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5589962fc2b0 con 0x55899639bb40 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47e97fa640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47d80a2c90 0x7f47d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x55899639a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x55899639a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:49040/0 (socket says 172.21.15.122:49040) 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 -- 172.21.15.122:0/2697333855 learned_addr learned my addr 172.21.15.122:0/2697333855 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47e3fff640 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589963a5370 0x5589963a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47d80a2c90 0x7f47d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47e97fa640 1 -- 172.21.15.122:0/31581388 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f47dc0ef380 con 0x7f47ec100430 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 -- 172.21.15.122:0/2697333855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589963a5370 msgr2=0x5589963a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589963a5370 0x5589963a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 -- 172.21.15.122:0/2697333855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 msgr2=0x55899639aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x55899639aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 -- 172.21.15.122:0/2697333855 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899628c3a0 con 0x55899639bb40 2024-09-12T01:47:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.616+0000 7f47ebfff640 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x55899639a090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47e400ed90 tx=0x7f47e400d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391271bcef8a5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.617+0000 7f47e1ffb640 1 -- 172.21.15.122:0/2697333855 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4021020 con 0x55899639bb40 2024-09-12T01:47:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.617+0000 7f47e1ffb640 1 -- 172.21.15.122:0/2697333855 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e401d960 con 0x55899639bb40 2024-09-12T01:47:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.617+0000 7f47e1ffb640 1 -- 172.21.15.122:0/2697333855 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e401c870 con 0x55899639bb40 2024-09-12T01:47:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 -- 172.21.15.122:0/2697333855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 msgr2=0x55899639a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47ebfff640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47d80a2c90 0x7f47d80a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47e4022000 tx=0x7f47e401ce40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x55899639a090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47e400ed90 tx=0x7f47e400d7b0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 -- 172.21.15.122:0/2697333855 shutdown_connections 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589963a5370 0x5589963a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x55899639aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 --2- 172.21.15.122:0/2697333855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x55899639a090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 -- 172.21.15.122:0/2697333855 >> 172.21.15.122:0/2697333855 conn(0x558996363310 msgr2=0x558996396c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 -- 172.21.15.122:0/2697333855 shutdown_connections 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 -- 172.21.15.122:0/2697333855 wait complete. 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.618+0000 7f47f4565740 1 -- start start 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 Processor -- start 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x5589963a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639bb40 0x5589963a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47eb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x5589963a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47eb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x5589963a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49570/0 (socket says 172.21.15.122:49570) 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5589963a8110 con 0x55899639a5d0 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5589962bb060 con 0x55899639bb40 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5589963a8290 con 0x5589963a5370 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47eb7fe640 1 -- 172.21.15.122:0/1138173771 learned_addr learned my addr 172.21.15.122:0/1138173771 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47f4565740 10 client.0 _ll_register_callbacks cb 0x55899600e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.619+0000 7f47e3fff640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639bb40 0x5589963a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47ebfff640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47eb7fe640 1 -- 172.21.15.122:0/1138173771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 msgr2=0x5589963a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47eb7fe640 1 -- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639bb40 msgr2=0x5589963a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639bb40 0x5589963a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47eb7fe640 1 -- 172.21.15.122:0/1138173771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589960713c0 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x5589963a4cb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f47dc005d00 tx=0x7f47dc0bb990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c3020b206df3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47caffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47caffd640 1 -- 172.21.15.122:0/1138173771 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc049030 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47caffd640 1 -- 172.21.15.122:0/1138173771 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f47dc0da2e0 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.620+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e401d960 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.621+0000 7f47caffd640 1 -- 172.21.15.122:0/1138173771 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47dc03bcb0 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.621+0000 7f47f4565740 10 client.15600 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.621+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589963b3b80 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.621+0000 7f47caffd640 1 -- 172.21.15.122:0/1138173771 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e401d960 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47caffd640 1 -- 172.21.15.122:0/1138173771 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f47dc04c060 con 0x55899639a5d0 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47f4565740 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:24.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47f4565740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47f4565740 20 client.15600 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47f4565740 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5589963b4fe0 0x5589963b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47f4565740 4 client.15600 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899630d860 con 0x5589963b4fe0 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.622+0000 7f47e3fff640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5589963b4fe0 0x5589963b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.623+0000 7f47e3fff640 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5589963b4fe0 0x5589963b73d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf34a42335386e15 server_cookie=f60a6fc27638a43c in_seq=0 out_seq=0 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.623+0000 7f47caffd640 10 client.15600 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.623+0000 7f47caffd640 1 -- 172.21.15.122:0/1138173771 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55899630d860 con 0x5589963b4fe0 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.623+0000 7f47caffd640 10 client.15600 handle_command_reply: tid=0 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.623+0000 7f47f4565740 1 client.15600 shutdown 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 msgr2=0x5589963a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x5589963a4cb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f47dc005d00 tx=0x7f47dc0bb990 comp rx=0 tx=0).stop 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 shutdown_connections 2024-09-12T01:47:24.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5589963b4fe0 0x5589963b73d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639bb40 0x5589963a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 --2- 172.21.15.122:0/1138173771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639a5d0 0x5589963a4cb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47caffd640 0 client.15600 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 >> 172.21.15.122:0/1138173771 conn(0x558996363310 msgr2=0x55899639c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 shutdown_connections 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 1 -- 172.21.15.122:0/1138173771 wait complete. 2024-09-12T01:47:24.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.624+0000 7f47f4565740 20 client.15600 trim_cache size 0 max 16384 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.649+0000 7f47f4565740 1 Processor -- start 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.649+0000 7f47f4565740 1 -- start start 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47f4565740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47f4565740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639bb40 0x558996396700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639bb40 0x558996396700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47f4565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47f4565740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1b1ed0 con 0x55899639bb40 2024-09-12T01:47:24.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47f4565740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1af7a0 con 0x55899639a5d0 2024-09-12T01:47:24.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47f4565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1ad5d0 con 0x5589963a5370 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639bb40 0x558996396700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49580/0 (socket says 172.21.15.122:49580) 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47e3fff640 1 -- 172.21.15.122:0/2994317505 learned_addr learned my addr 172.21.15.122:0/2994317505 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.650+0000 7f47ebfff640 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47eb7fe640 1 -- 172.21.15.122:0/2994317505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 msgr2=0x5589963bcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963bcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47eb7fe640 1 -- 172.21.15.122:0/2994317505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639bb40 msgr2=0x558996396700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639bb40 0x558996396700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47eb7fe640 1 -- 172.21.15.122:0/2994317505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558996397960 con 0x55899639a5d0 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963b1430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47dc03b510 tx=0x7f47dc0d7080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54efb2e9b5d8d35d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:24.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47e17fa640 1 -- 172.21.15.122:0/2994317505 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc049070 con 0x55899639a5d0 2024-09-12T01:47:24.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47e17fa640 1 -- 172.21.15.122:0/2994317505 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47dc004460 con 0x55899639a5d0 2024-09-12T01:47:24.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.651+0000 7f47e17fa640 1 -- 172.21.15.122:0/2994317505 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0b7da0 con 0x55899639a5d0 2024-09-12T01:47:24.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 -- 172.21.15.122:0/2994317505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 msgr2=0x5589963b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963b1430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47dc03b510 tx=0x7f47dc0d7080 comp rx=0 tx=0).stop 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 -- 172.21.15.122:0/2994317505 shutdown_connections 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589963a5370 0x5589963bcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55899639bb40 0x558996396700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 --2- 172.21.15.122:0/2994317505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963b1430 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.652+0000 7f47f4565740 1 -- 172.21.15.122:0/2994317505 >> 172.21.15.122:0/2994317505 conn(0x558996363310 msgr2=0x5589963a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.653+0000 7f47f4565740 1 -- 172.21.15.122:0/2994317505 shutdown_connections 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.653+0000 7f47f4565740 1 -- 172.21.15.122:0/2994317505 wait complete. 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.653+0000 7f47f4565740 1 -- start start 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.653+0000 7f47f4565740 1 Processor -- start 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47f4565740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47f4565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x5589963ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47f4565740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 0x55899639eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 0x55899639eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 0x55899639eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49584/0 (socket says 172.21.15.122:49584) 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 -- 172.21.15.122:0/3454318443 learned_addr learned my addr 172.21.15.122:0/3454318443 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1adc60 con 0x5589963a5370 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1af0d0 con 0x55899639a5d0 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1ae330 con 0x55899639bb40 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47e3fff640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x5589963ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 -- 172.21.15.122:0/3454318443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 msgr2=0x5589963ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x5589963ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 -- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 msgr2=0x5589963ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.654+0000 7f47ebfff640 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899639f340 con 0x5589963a5370 2024-09-12T01:47:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47f4565740 10 client.0 _ll_register_callbacks cb 0x7f47dc03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47dc004460 con 0x5589963a5370 2024-09-12T01:47:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47ebfff640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 0x55899639eb40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f47e400ed60 tx=0x7f47e400aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2bad7b5c66b9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47caffd640 10 client.15606 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47caffd640 1 -- 172.21.15.122:0/3454318443 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4034070 con 0x5589963a5370 2024-09-12T01:47:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47caffd640 1 -- 172.21.15.122:0/3454318443 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f47e4004d20 con 0x5589963a5370 2024-09-12T01:47:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.655+0000 7f47caffd640 1 -- 172.21.15.122:0/3454318443 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47e4021040 con 0x5589963a5370 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 10 client.15606 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55899641f660 con 0x5589963a5370 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47caffd640 1 -- 172.21.15.122:0/3454318443 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f47e400b040 con 0x5589963a5370 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 20 client.15606 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55899641fba0 0x558996422030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 4 client.15606 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558996307d70 con 0x55899641fba0 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.656+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55899641fba0 0x558996422030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.657+0000 7f47caffd640 10 client.15606 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.657+0000 7f47eb7fe640 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55899641fba0 0x558996422030 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc1bbea9d2aa7a4 server_cookie=73d40386f918d914 in_seq=0 out_seq=0 2024-09-12T01:47:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.657+0000 7f47caffd640 1 -- 172.21.15.122:0/3454318443 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558996307d70 con 0x55899641fba0 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.657+0000 7f47caffd640 10 client.15606 handle_command_reply: tid=0 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.657+0000 7f47f4565740 1 client.15606 shutdown 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 msgr2=0x55899639eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 0x55899639eb40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f47e400ed60 tx=0x7f47e400aa10 comp rx=0 tx=0).stop 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 shutdown_connections 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55899641fba0 0x558996422030 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5589963a5370 0x55899639eb40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55899639bb40 0x5589963ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 --2- 172.21.15.122:0/3454318443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55899639a5d0 0x5589963ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 >> 172.21.15.122:0/3454318443 conn(0x558996363310 msgr2=0x5589963954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47caffd640 0 client.15606 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:24.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 shutdown_connections 2024-09-12T01:47:24.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 1 -- 172.21.15.122:0/3454318443 wait complete. 2024-09-12T01:47:24.661 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:47:24.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.658+0000 7f47f4565740 20 client.15606 trim_cache size 0 max 16384 2024-09-12T01:47:24.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47d80a2c90 msgr2=0x7f47d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47d80a2c90 0x7f47d80a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47e4022000 tx=0x7f47e401ce40 comp rx=0 tx=0).stop 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 msgr2=0x7f47ec110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 0x7f47ec110580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47dc006010 tx=0x7f47dc0042f0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 shutdown_connections 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f47d80a2c90 0x7f47d80a5150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47ec105560 0x7f47ec111000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47ec100de0 0x7f47ec110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 --2- 172.21.15.122:0/31581388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ec100430 0x7f47ec110580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 >> 172.21.15.122:0/31581388 conn(0x7f47ec077230 msgr2=0x7f47ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 shutdown_connections 2024-09-12T01:47:24.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:24.659+0000 7f47f218c640 1 -- 172.21.15.122:0/31581388 wait complete. 2024-09-12T01:47:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:25 smithi151 ceph-mon[35229]: pgmap v363: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T01:47:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:25 smithi183 ceph-mon[35127]: pgmap v363: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T01:47:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:25 smithi122 ceph-mon[30925]: pgmap v363: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T01:47:26.538 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:47:26.539 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:47:26.818 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:27.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/541418602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f481082a0 msgr2=0x7f2f4810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/541418602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f481082a0 0x7f2f4810ec40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009cd0 tx=0x7f2f4002a8f0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/541418602 shutdown_connections 2024-09-12T01:47:27.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/541418602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f481082a0 0x7f2f4810ec40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/541418602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f481036e0 0x7f2f48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/541418602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f48102d30 0x7f2f48103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.354+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/541418602 >> 172.21.15.122:0/541418602 conn(0x7f2f480fc7d0 msgr2=0x7f2f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.355+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/541418602 shutdown_connections 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.355+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/541418602 wait complete. 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.355+0000 7f2f4c9aa640 1 Processor -- start 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 -- start start 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 0x7f2f481984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f481036e0 0x7f2f48198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f481082a0 0x7f2f4819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f481120a0 con 0x7f2f48102d30 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48111f20 con 0x7f2f481082a0 2024-09-12T01:47:27.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4c9aa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48112220 con 0x7f2f481036e0 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 0x7f2f481984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 0x7f2f481984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49590/0 (socket says 172.21.15.122:49590) 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 -- 172.21.15.122:0/3592659682 learned_addr learned my addr 172.21.15.122:0/3592659682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f481036e0 0x7f2f48198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 -- 172.21.15.122:0/3592659682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f481036e0 msgr2=0x7f2f48198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f481036e0 0x7f2f48198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 -- 172.21.15.122:0/3592659682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f481082a0 msgr2=0x7f2f4819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f481082a0 0x7f2f4819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.356+0000 7f2f4659c640 1 -- 172.21.15.122:0/3592659682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4819d470 con 0x7f2f48102d30 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.357+0000 7f2f4659c640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 0x7f2f481984c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800cab0 tx=0x7f2f3800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.357+0000 7f2f377fe640 1 -- 172.21.15.122:0/3592659682 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38007d30 con 0x7f2f48102d30 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.357+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4819d760 con 0x7f2f48102d30 2024-09-12T01:47:27.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.357+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f481a5040 con 0x7f2f48102d30 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.359+0000 7f2f377fe640 1 -- 172.21.15.122:0/3592659682 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f380035f0 con 0x7f2f48102d30 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.359+0000 7f2f377fe640 1 -- 172.21.15.122:0/3592659682 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800f6b0 con 0x7f2f48102d30 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.360+0000 7f2f4ed83740 1 Processor -- start 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.360+0000 7f2f4ed83740 1 -- start start 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 0x560f8bed1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bedc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bedf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560f8be37280 con 0x560f8bed1b60 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f8bddeb10 con 0x560f8bed2480 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f8be33240 con 0x560f8bedcd40 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f377fe640 1 -- 172.21.15.122:0/3592659682 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2f3800f910 con 0x7f2f48102d30 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f46d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 0x560f8bed1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f46d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 0x560f8bed1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49598/0 (socket says 172.21.15.122:49598) 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f46d9d640 1 -- 172.21.15.122:0/2983706407 learned_addr learned my addr 172.21.15.122:0/2983706407 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f46d9d640 1 -- 172.21.15.122:0/2983706407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 msgr2=0x560f8bedf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bedf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f46d9d640 1 -- 172.21.15.122:0/2983706407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 msgr2=0x560f8bedc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.361+0000 7f2f4659c640 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bedc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bedc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f46d9d640 1 -- 172.21.15.122:0/2983706407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f8bdeb1f0 con 0x560f8bed1b60 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f4659c640 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bedc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 0x560f8bed1f40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009df0 tx=0x7f2f40009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac4c6682e6ba07a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f34ff9640 1 -- 172.21.15.122:0/2983706407 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40030620 con 0x560f8bed1b60 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f34ff9640 1 -- 172.21.15.122:0/2983706407 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4003f070 con 0x560f8bed1b60 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.362+0000 7f2f34ff9640 1 -- 172.21.15.122:0/2983706407 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40004380 con 0x560f8bed1b60 2024-09-12T01:47:27.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.363+0000 7f2f377fe640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2f300ab520 0x7f2f300ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.363+0000 7f2f377fe640 1 -- 172.21.15.122:0/3592659682 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2f380cf110 con 0x7f2f48102d30 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.363+0000 7f2f4ed83740 1 -- 172.21.15.122:0/2983706407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 msgr2=0x560f8bed1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.363+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 0x560f8bed1f40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009df0 tx=0x7f2f40009a20 comp rx=0 tx=0).stop 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.363+0000 7f2f4659c640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2f300ab520 0x7f2f300ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 -- 172.21.15.122:0/2983706407 shutdown_connections 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bedf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bedc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/2983706407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed1b60 0x560f8bed1f40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 -- 172.21.15.122:0/2983706407 >> 172.21.15.122:0/2983706407 conn(0x560f8bece0f0 msgr2=0x560f8bece4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 -- 172.21.15.122:0/2983706407 shutdown_connections 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4659c640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2f300ab520 0x7f2f300ad9e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f3809b760 tx=0x7f2f38097a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 -- 172.21.15.122:0/2983706407 wait complete. 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.364+0000 7f2f4ed83740 1 -- start start 2024-09-12T01:47:27.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 Processor -- start 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bedc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bed58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bed5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560f8bee3d20 con 0x560f8bed2480 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f8bdf1f10 con 0x560f8bed1b60 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f8bee3ea0 con 0x560f8bedcd40 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f4ed83740 10 client.0 _ll_register_callbacks cb 0x560f8bdbaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f45d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bed58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f45d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bed58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49602/0 (socket says 172.21.15.122:49602) 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f45d9b640 1 -- 172.21.15.122:0/1042408538 learned_addr learned my addr 172.21.15.122:0/1042408538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bedc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.365+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bed5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f45d9b640 1 -- 172.21.15.122:0/1042408538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 msgr2=0x560f8bed5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bed5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f45d9b640 1 -- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 msgr2=0x560f8bedc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bedc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f45d9b640 1 -- 172.21.15.122:0/1042408538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8bba86a0 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bedc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bed58f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c005fd0 tx=0x7f2f3c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927220013b6066b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f2dffb640 1 -- 172.21.15.122:0/1042408538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c016070 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.366+0000 7f2f2dffb640 1 -- 172.21.15.122:0/1042408538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2f3c005220 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4003f070 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f2dffb640 1 -- 172.21.15.122:0/1042408538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c01f050 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 10 client.15618 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f8beeb730 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f2dffb640 1 -- 172.21.15.122:0/1042408538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4003f070 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f2dffb640 1 -- 172.21.15.122:0/1042408538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2f3c00f040 con 0x560f8bed2480 2024-09-12T01:47:27.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 20 client.15618 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.367+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8beeca80 0x560f8beeee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.368+0000 7f2f4ed83740 4 client.15618 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.368+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f8be447f0 con 0x560f8beeca80 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.368+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8beeca80 0x560f8beeee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.368+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8beeca80 0x560f8beeee70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccca65b7c491b018 server_cookie=cfe719b4daf87311 in_seq=0 out_seq=0 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.368+0000 7f2f2dffb640 10 client.15618 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f2dffb640 1 -- 172.21.15.122:0/1042408538 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f8be447f0 con 0x560f8beeca80 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f2dffb640 10 client.15618 handle_command_reply: tid=0 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 client.15618 shutdown 2024-09-12T01:47:27.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 msgr2=0x560f8bed58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bed58f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c005fd0 tx=0x7f2f3c00e6c0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 shutdown_connections 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8beeca80 0x560f8beeee70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bed5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bed58f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/1042408538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bedc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 >> 172.21.15.122:0/1042408538 conn(0x560f8bece0f0 msgr2=0x560f8bed3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.369+0000 7f2f2dffb640 0 client.15618 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.370+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 shutdown_connections 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.370+0000 7f2f4ed83740 1 -- 172.21.15.122:0/1042408538 wait complete. 2024-09-12T01:47:27.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.370+0000 7f2f4ed83740 20 client.15618 trim_cache size 0 max 16384 2024-09-12T01:47:27.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.387+0000 7f2f4ed83740 1 Processor -- start 2024-09-12T01:47:27.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.387+0000 7f2f4ed83740 1 -- start start 2024-09-12T01:47:27.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bed1b60 0x560f8bef4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bef4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bedcd40 0x560f8bee1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28010d60 con 0x560f8bedcd40 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800e630 con 0x560f8bed2480 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800c460 con 0x560f8bed1b60 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f46d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bedcd40 0x560f8bee1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bef4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bef4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:59244/0 (socket says 172.21.15.122:59244) 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 -- 172.21.15.122:0/3609300692 learned_addr learned my addr 172.21.15.122:0/3609300692 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f46d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bedcd40 0x560f8bee1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49610/0 (socket says 172.21.15.122:49610) 2024-09-12T01:47:27.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bed1b60 0x560f8bef4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 -- 172.21.15.122:0/3609300692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bed1b60 msgr2=0x560f8bef4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bed1b60 0x560f8bef4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 -- 172.21.15.122:0/3609300692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bedcd40 msgr2=0x560f8bee1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.388+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bedcd40 0x560f8bee1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.389+0000 7f2f45d9b640 1 -- 172.21.15.122:0/3609300692 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f8bedc930 con 0x560f8bed2480 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.389+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bef4ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0119d0 tx=0x7f2f3c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863fab11e2f3bcce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.389+0000 7f2f2ffff640 1 -- 172.21.15.122:0/3609300692 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c016070 con 0x560f8bed2480 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.389+0000 7f2f2ffff640 1 -- 172.21.15.122:0/3609300692 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f3c00ebc0 con 0x560f8bed2480 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.389+0000 7f2f2ffff640 1 -- 172.21.15.122:0/3609300692 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c004cf0 con 0x560f8bed2480 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 -- 172.21.15.122:0/3609300692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 msgr2=0x560f8bef4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bef4ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0119d0 tx=0x7f2f3c011720 comp rx=0 tx=0).stop 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 -- 172.21.15.122:0/3609300692 shutdown_connections 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bedcd40 0x560f8bee1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed2480 0x560f8bef4ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/3609300692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bed1b60 0x560f8bef4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 -- 172.21.15.122:0/3609300692 >> 172.21.15.122:0/3609300692 conn(0x560f8bece0f0 msgr2=0x560f8bed3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:27.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 -- 172.21.15.122:0/3609300692 shutdown_connections 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.390+0000 7f2f4ed83740 1 -- 172.21.15.122:0/3609300692 wait complete. 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 -- start start 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 Processor -- start 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bee4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bee5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bee5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800caf0 con 0x560f8bed2480 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800df60 con 0x560f8bed1b60 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2800d1c0 con 0x560f8bedcd40 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f4ed83740 10 client.0 _ll_register_callbacks cb 0x7f2f3c005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:27.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f45d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bee5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.391+0000 7f2f45d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bee5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49622/0 (socket says 172.21.15.122:49622) 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 -- 172.21.15.122:0/4269582185 learned_addr learned my addr 172.21.15.122:0/4269582185 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bee5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bee4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 -- 172.21.15.122:0/4269582185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 msgr2=0x560f8bee5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bee5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 -- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 msgr2=0x560f8bee4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bee4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 -- 172.21.15.122:0/4269582185 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8bf567e0 con 0x560f8bed2480 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bee4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f45d9b640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bee5510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009440 tx=0x7f2f3c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c2006ff8503c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.392+0000 7f2f2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:27.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f2dffb640 1 -- 172.21.15.122:0/4269582185 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c044070 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f2dffb640 1 -- 172.21.15.122:0/4269582185 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2f3c01aa30 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f3c00ebc0 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f2dffb640 1 -- 172.21.15.122:0/4269582185 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c050050 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 10 client.15624 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f8bf57ae0 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f2dffb640 1 -- 172.21.15.122:0/4269582185 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f3c01ac30 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f2dffb640 1 -- 172.21.15.122:0/4269582185 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2f3c024440 con 0x560f8bed2480 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 20 client.15624 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8bf58140 0x560f8bf5a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 4 client.15624 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:47:27.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.393+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f2f3c00ebc0 con 0x560f8bf58140 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.394+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8bf58140 0x560f8bf5a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.394+0000 7f2f46d9d640 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8bf58140 0x560f8bf5a560 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750789d32fb6d1ed server_cookie=413b6f1a4509b506 in_seq=0 out_seq=0 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.394+0000 7f2f2dffb640 10 client.15624 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.395+0000 7f2f2dffb640 1 -- 172.21.15.122:0/4269582185 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f2f3c00ebc0 con 0x560f8bf58140 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.395+0000 7f2f2dffb640 10 client.15624 handle_command_reply: tid=0 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.395+0000 7f2f4ed83740 1 client.15624 shutdown 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 msgr2=0x560f8bee5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bee5510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009440 tx=0x7f2f3c005bb0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 shutdown_connections 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560f8bf58140 0x560f8bf5a560 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f2dffb640 0 client.15624 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f8bedcd40 0x560f8bee5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f8bed2480 0x560f8bee5510 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 --2- 172.21.15.122:0/4269582185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f8bed1b60 0x560f8bee4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 >> 172.21.15.122:0/4269582185 conn(0x560f8bece0f0 msgr2=0x560f8bed3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 shutdown_connections 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 1 -- 172.21.15.122:0/4269582185 wait complete. 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:27.399 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-09-12T01:47:27.400 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "173a7d39-12cc-48f7-83f7-46b36334ac23", 2024-09-12T01:47:27.400 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-09-12T01:47:27.400 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.396+0000 7f2f4ed83740 20 client.15624 trim_cache size 0 max 16384 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2f300ab520 msgr2=0x7f2f300ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2f300ab520 0x7f2f300ad9e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f3809b760 tx=0x7f2f38097a70 comp rx=0 tx=0).stop 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 msgr2=0x7f2f481984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 0x7f2f481984c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800cab0 tx=0x7f2f3800cf70 comp rx=0 tx=0).stop 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 shutdown_connections 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2f300ab520 0x7f2f300ad9e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f481082a0 0x7f2f4819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f481036e0 0x7f2f48198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:27.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 --2- 172.21.15.122:0/3592659682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48102d30 0x7f2f481984c0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800cab0 tx=0x7f2f3800cf70 comp rx=0 tx=0).stop 2024-09-12T01:47:27.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 >> 172.21.15.122:0/3592659682 conn(0x7f2f480fc7d0 msgr2=0x7f2f48105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:27.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 shutdown_connections 2024-09-12T01:47:27.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:27.397+0000 7f2f4c9aa640 1 -- 172.21.15.122:0/3592659682 wait complete. 2024-09-12T01:47:27.467 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:47:27.585 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:27 smithi122 ceph-mon[30925]: pgmap v364: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T01:47:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:27 smithi151 ceph-mon[35229]: pgmap v364: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T01:47:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:27 smithi183 ceph-mon[35127]: pgmap v364: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T01:47:28.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:47:28.233 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:28.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:28 smithi122 ceph-mon[30925]: scrub queued for path: / 2024-09-12T01:47:28.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:28 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:28.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:28 smithi122 ceph-mon[30925]: scrub summary: active paths [/] 2024-09-12T01:47:28.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:28 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:28.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 -- 172.21.15.122:0/2998875120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00106a70 msgr2=0x7fcb00106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:28.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/2998875120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00106a70 0x7fcb00106e50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009930 tx=0x7fcaf402a420 comp rx=0 tx=0).stop 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 -- 172.21.15.122:0/2998875120 shutdown_connections 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/2998875120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00101410 0x7fcb0010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/2998875120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb00100a70 0x7fcb00100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/2998875120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00106a70 0x7fcb00106e50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 -- 172.21.15.122:0/2998875120 >> 172.21.15.122:0/2998875120 conn(0x7fcb000fc790 msgr2=0x7fcb000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.754+0000 7fcb06f1c640 1 -- 172.21.15.122:0/2998875120 shutdown_connections 2024-09-12T01:47:28.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.755+0000 7fcb06f1c640 1 -- 172.21.15.122:0/2998875120 wait complete. 2024-09-12T01:47:28.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.756+0000 7fcb06f1c640 1 Processor -- start 2024-09-12T01:47:28.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.756+0000 7fcb06f1c640 1 -- start start 2024-09-12T01:47:28.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.756+0000 7fcb06f1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 0x7fcb00196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:28.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.756+0000 7fcb06f1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb00101410 0x7fcb00196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:28.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.756+0000 7fcb06f1c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00106a70 0x7fcb0019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:28.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.756+0000 7fcb06f1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0010fd60 con 0x7fcb00100a70 2024-09-12T01:47:28.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb06f1c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0010fbe0 con 0x7fcb00101410 2024-09-12T01:47:28.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 0x7fcb00196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:28.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 0x7fcb00196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49640/0 (socket says 172.21.15.122:49640) 2024-09-12T01:47:28.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 -- 172.21.15.122:0/3777548919 learned_addr learned my addr 172.21.15.122:0/3777548919 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:28.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcafffff640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb00101410 0x7fcb00196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb054b9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00106a70 0x7fcb0019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb06f1c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0010fee0 con 0x7fcb00106a70 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 -- 172.21.15.122:0/3777548919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00106a70 msgr2=0x7fcb0019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00106a70 0x7fcb0019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 -- 172.21.15.122:0/3777548919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb00101410 msgr2=0x7fcb00196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb00101410 0x7fcb00196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb04cb8640 1 -- 172.21.15.122:0/3777548919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0019b1b0 con 0x7fcb00100a70 2024-09-12T01:47:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.757+0000 7fcb054b9640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00106a70 0x7fcb0019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.758+0000 7fcb04cb8640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 0x7fcb00196200 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009230 tx=0x7fcaf40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.758+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4040070 con 0x7fcb00100a70 2024-09-12T01:47:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.758+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf4031c80 con 0x7fcb00100a70 2024-09-12T01:47:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.758+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0019b440 con 0x7fcb00100a70 2024-09-12T01:47:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.758+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4035020 con 0x7fcb00100a70 2024-09-12T01:47:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.758+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0019b8b0 con 0x7fcb00100a70 2024-09-12T01:47:28.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.760+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb00100ed0 con 0x7fcb00100a70 2024-09-12T01:47:28.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.762+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fcaf4043070 con 0x7fcb00100a70 2024-09-12T01:47:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.762+0000 7fcafdffb640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcaf00a4ea0 0x7fcaf00a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.763+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcaf40efc10 con 0x7fcb00100a70 2024-09-12T01:47:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.763+0000 7fcafffff640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcaf00a4ea0 0x7fcaf00a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:28.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.764+0000 7fcafffff640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcaf00a4ea0 0x7fcaf00a7360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcb000fed50 tx=0x7fcaec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:28.772+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcaf40bca10 con 0x7fcb00100a70 2024-09-12T01:47:28.789 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:28.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:28 smithi151 ceph-mon[35229]: scrub queued for path: / 2024-09-12T01:47:28.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:28 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:28.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:28 smithi151 ceph-mon[35229]: scrub summary: active paths [/] 2024-09-12T01:47:28.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:28 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:28 smithi183 ceph-mon[35127]: scrub queued for path: / 2024-09-12T01:47:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:28 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:28 smithi183 ceph-mon[35127]: scrub summary: active paths [/] 2024-09-12T01:47:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:28 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:47:29.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.187+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb00109010 con 0x7fcb00100a70 2024-09-12T01:47:29.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.189+0000 7fcafdffb640 1 -- 172.21.15.122:0/3777548919 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7fcaf40bc310 con 0x7fcb00100a70 2024-09-12T01:47:29.189 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:47:29.189 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-12T01:46:19:586759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:46:19.050885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:47:29.189 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcaf00a4ea0 msgr2=0x7fcaf00a7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcaf00a4ea0 0x7fcaf00a7360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcb000fed50 tx=0x7fcaec005f90 comp rx=0 tx=0).stop 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 msgr2=0x7fcb00196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 0x7fcb00196200 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009230 tx=0x7fcaf40045e0 comp rx=0 tx=0).stop 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 shutdown_connections 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcaf00a4ea0 0x7fcaf00a7360 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb00106a70 0x7fcb0019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:29.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb00101410 0x7fcb00196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:29.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 --2- 172.21.15.122:0/3777548919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb00100a70 0x7fcb00196200 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:29.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 >> 172.21.15.122:0/3777548919 conn(0x7fcb000fc790 msgr2=0x7fcb000fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:29.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 shutdown_connections 2024-09-12T01:47:29.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.192+0000 7fcb06f1c640 1 -- 172.21.15.122:0/3777548919 wait complete. 2024-09-12T01:47:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:29 smithi151 ceph-mon[35229]: pgmap v365: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T01:47:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:29 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3777548919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:29 smithi183 ceph-mon[35127]: pgmap v365: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T01:47:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:29 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3777548919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:29.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:29 smithi122 ceph-mon[30925]: pgmap v365: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-12T01:47:29.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:29 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3777548919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:47:30.019 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 6977f67f --release --await' 2024-09-12T01:47:30.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 6977f67f --release --await 2024-09-12T01:47:30.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 -- 172.21.15.122:0/3925617302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac102d30 msgr2=0x7f3fac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/3925617302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac102d30 0x7f3fac103110 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009930 tx=0x7f3fa002a420 comp rx=0 tx=0).stop 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 -- 172.21.15.122:0/3925617302 shutdown_connections 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/3925617302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fac1082a0 0x7f3fac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/3925617302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fac1036e0 0x7f3fac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/3925617302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac102d30 0x7f3fac103110 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 -- 172.21.15.122:0/3925617302 >> 172.21.15.122:0/3925617302 conn(0x7f3fac0fc7d0 msgr2=0x7f3fac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 -- 172.21.15.122:0/3925617302 shutdown_connections 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.320+0000 7f3fb2d16640 1 -- 172.21.15.122:0/3925617302 wait complete. 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.321+0000 7f3fb2d16640 1 Processor -- start 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.321+0000 7f3fb2d16640 1 -- start start 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.321+0000 7f3fb2d16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fac102d30 0x7f3fac100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.321+0000 7f3fb2d16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 0x7f3fac100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fb2d16640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fac1082a0 0x7f3fac101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fb2d16640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1120a0 con 0x7f3fac1036e0 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fb2d16640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac111f20 con 0x7f3fac1082a0 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fb2d16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac112220 con 0x7f3fac102d30 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 0x7f3fac100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 0x7f3fac100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49660/0 (socket says 172.21.15.122:49660) 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 -- 172.21.15.122:0/2630330721 learned_addr learned my addr 172.21.15.122:0/2630330721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 -- 172.21.15.122:0/2630330721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fac102d30 msgr2=0x7f3fac100990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:47:30.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fac102d30 0x7f3fac100990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 -- 172.21.15.122:0/2630330721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fac1082a0 msgr2=0x7f3fac101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fac1082a0 0x7f3fac101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 -- 172.21.15.122:0/2630330721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac1a6aa0 con 0x7f3fac1036e0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fabfff640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 0x7f3fac100ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800d750 tx=0x7f3f9800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fa9ffb640 1 -- 172.21.15.122:0/2630330721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98017070 con 0x7f3fac1036e0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.322+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac1a6d90 con 0x7f3fac1036e0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.323+0000 7f3fa9ffb640 1 -- 172.21.15.122:0/2630330721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f98005500 con 0x7f3fac1036e0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.323+0000 7f3fa9ffb640 1 -- 172.21.15.122:0/2630330721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98004e00 con 0x7f3fac1036e0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.323+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac1a7270 con 0x7f3fac1036e0 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 Processor -- start 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 -- start start 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a34880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 0x5627a4a3f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a41a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5627a499a2e0 con 0x5627a4a34dc0 2024-09-12T01:47:30.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5627a4941ce0 con 0x5627a4a3f680 2024-09-12T01:47:30.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5627a49962a0 con 0x5627a4a344a0 2024-09-12T01:47:30.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 0x5627a4a3f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 0x5627a4a3f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49668/0 (socket says 172.21.15.122:49668) 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/1428721529 learned_addr learned my addr 172.21.15.122:0/1428721529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a34880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/1428721529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 msgr2=0x5627a4a34880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a34880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/1428721529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 msgr2=0x5627a4a41a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a41a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.326+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/1428721529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a4926440 con 0x5627a4a34dc0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.327+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 0x5627a4a3f140 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8477afa7898060e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.327+0000 7f3fa9ffb640 1 -- 172.21.15.122:0/2630330721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3f98010460 con 0x7f3fac1036e0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.327+0000 7f3f96ffd640 1 -- 172.21.15.122:0/1428721529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa003e030 con 0x5627a4a34dc0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.327+0000 7f3f96ffd640 1 -- 172.21.15.122:0/1428721529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa0032df0 con 0x5627a4a34dc0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.327+0000 7f3f96ffd640 1 -- 172.21.15.122:0/1428721529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0035040 con 0x5627a4a34dc0 2024-09-12T01:47:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1428721529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 msgr2=0x5627a4a3f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 0x5627a4a3f140 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa00095a0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fa9ffb640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3f9c0a2c90 0x7f3f9c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1428721529 shutdown_connections 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a41a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a34dc0 0x5627a4a3f140 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa00095a0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1428721529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a34880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1428721529 >> 172.21.15.122:0/1428721529 conn(0x5627a4a30a30 msgr2=0x5627a4a30e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fa9ffb640 1 -- 172.21.15.122:0/2630330721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3f980cebb0 con 0x7f3fac1036e0 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.328+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3f9c0a2c90 0x7f3f9c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.329+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1428721529 shutdown_connections 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.329+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1428721529 wait complete. 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.329+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3f9c0a2c90 0x7f3f9c0a5150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009370 tx=0x7f3fa40092a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.329+0000 7f3fb50ef740 1 -- start start 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.329+0000 7f3fb50ef740 1 Processor -- start 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a344a0 0x5627a4a3c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5627a4a422a0 con 0x5627a4a3cef0 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5627a4955050 con 0x5627a4a344a0 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5627a4a42420 con 0x5627a4a3f680 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb50ef740 10 client.0 _ll_register_callbacks cb 0x5627a4995160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49678/0 (socket says 172.21.15.122:49678) 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 -- 172.21.15.122:0/959018117 learned_addr learned my addr 172.21.15.122:0/959018117 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a344a0 0x5627a4a3c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fabfff640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 -- 172.21.15.122:0/959018117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 msgr2=0x5627a4a3c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a3c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 -- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a344a0 msgr2=0x5627a4a3c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a344a0 0x5627a4a3c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fb12b3640 1 -- 172.21.15.122:0/959018117 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a470b3c0 con 0x5627a4a3cef0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.330+0000 7f3fabfff640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4b740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011670 tx=0x7f3fa4009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac102cb00390cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3f8bfff640 1 -- 172.21.15.122:0/959018117 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4022070 con 0x5627a4a3cef0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3f8bfff640 1 -- 172.21.15.122:0/959018117 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3fa40109a0 con 0x5627a4a3cef0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa0032df0 con 0x5627a4a3cef0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3f8bfff640 1 -- 172.21.15.122:0/959018117 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3fa4010ba0 con 0x5627a4a3cef0 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3fb50ef740 10 client.15642 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:30.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627a4a4e300 con 0x5627a4a3cef0 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.331+0000 7f3f8bfff640 1 -- 172.21.15.122:0/959018117 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa0032df0 con 0x5627a4a3cef0 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3f8bfff640 1 -- 172.21.15.122:0/959018117 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3fa40045f0 con 0x5627a4a3cef0 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb50ef740 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb50ef740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb50ef740 20 client.15642 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4a4f710 0x5627a4a51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb50ef740 4 client.15642 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627a49a7850 con 0x5627a4a4f710 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4a4f710 0x5627a4a51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3f8bfff640 10 client.15642 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.332+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4a4f710 0x5627a4a51b40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d464af2b6ac14f6d server_cookie=18562d1fa200c5de in_seq=0 out_seq=0 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3f8bfff640 1 -- 172.21.15.122:0/959018117 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627a49a7850 con 0x5627a4a4f710 2024-09-12T01:47:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3f8bfff640 10 client.15642 handle_command_reply: tid=0 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 client.15642 shutdown 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 msgr2=0x5627a4a4b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4b740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011670 tx=0x7f3fa4009440 comp rx=0 tx=0).stop 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 shutdown_connections 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4a4f710 0x5627a4a51b40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3f8bfff640 0 client.15642 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4b740 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/959018117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a344a0 0x5627a4a3c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.333+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 >> 172.21.15.122:0/959018117 conn(0x5627a4a30a30 msgr2=0x5627a4a36710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.334+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 shutdown_connections 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.334+0000 7f3fb50ef740 1 -- 172.21.15.122:0/959018117 wait complete. 2024-09-12T01:47:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.334+0000 7f3fb50ef740 20 client.15642 trim_cache size 0 max 16384 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 Processor -- start 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 -- start start 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a57460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1b2040 con 0x5627a4a3cef0 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1af910 con 0x5627a4a3f680 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fb50ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1ad740 con 0x5627a4a344a0 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fabfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fabfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49680/0 (socket says 172.21.15.122:49680) 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.351+0000 7f3fabfff640 1 -- 172.21.15.122:0/1092279907 learned_addr learned my addr 172.21.15.122:0/1092279907 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fb12b3640 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a57460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fabfff640 1 -- 172.21.15.122:0/1092279907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 msgr2=0x5627a4a4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fabfff640 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fabfff640 1 -- 172.21.15.122:0/1092279907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 msgr2=0x5627a4a57460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fabfff640 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a57460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fabfff640 1 -- 172.21.15.122:0/1092279907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a4a41de0 con 0x5627a4a3cef0 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3fabfff640 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4ba70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009610 tx=0x7f3f9809ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9080e45cba1fb801 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3f967fc640 1 -- 172.21.15.122:0/1092279907 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98020040 con 0x5627a4a3cef0 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3f967fc640 1 -- 172.21.15.122:0/1092279907 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f980b4270 con 0x5627a4a3cef0 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.352+0000 7f3f967fc640 1 -- 172.21.15.122:0/1092279907 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98097920 con 0x5627a4a3cef0 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1092279907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 msgr2=0x5627a4a4ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4ba70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009610 tx=0x7f3f9809ba70 comp rx=0 tx=0).stop 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1092279907 shutdown_connections 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3f680 0x5627a4a57460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a3cef0 0x5627a4a4ba70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/1092279907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a344a0 0x5627a4a4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1092279907 >> 172.21.15.122:0/1092279907 conn(0x5627a4a30a30 msgr2=0x5627a4a3e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "173a7d39-12cc-48f7-83f7-46b36334ac23": { 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-09-12T01:47:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "173a7d39-12cc-48f7-83f7-46b36334ac23", 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1092279907 shutdown_connections 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.353+0000 7f3fb50ef740 1 -- 172.21.15.122:0/1092279907 wait complete. 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 -- start start 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 Processor -- start 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 0x5627a4a39400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb0ab2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 0x5627a4a39400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb0ab2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 0x5627a4a39400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49694/0 (socket says 172.21.15.122:49694) 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/159922677 learned_addr learned my addr 172.21.15.122:0/159922677 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3cef0 0x5627a4a39940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a39e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1addd0 con 0x5627a4a344a0 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1af240 con 0x5627a4a3cef0 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1ae4a0 con 0x5627a4a3f680 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.354+0000 7f3fb50ef740 10 client.0 _ll_register_callbacks cb 0x7f3fa4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/159922677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 msgr2=0x5627a4a39e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a39e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fabfff640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3cef0 0x5627a4a39940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3cef0 msgr2=0x5627a4a39940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3cef0 0x5627a4a39940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb0ab2640 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a4ac1c40 con 0x5627a4a344a0 2024-09-12T01:47:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb0ab2640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 0x5627a4a39400 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37c90f040300e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3f8bfff640 1 -- 172.21.15.122:0/159922677 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0035040 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f980b4270 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3f8bfff640 1 -- 172.21.15.122:0/159922677 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3fa0004030 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fabfff640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3cef0 0x5627a4a39940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3f8bfff640 1 -- 172.21.15.122:0/159922677 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3fa005c050 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb50ef740 10 client.15648 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627a4ac2ea0 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.355+0000 7f3f8bfff640 1 -- 172.21.15.122:0/159922677 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f980b4270 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3f8bfff640 1 -- 172.21.15.122:0/159922677 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3fa0004750 con 0x5627a4a344a0 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fb50ef740 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fb50ef740 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:30.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fb50ef740 20 client.15648 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4ac31e0 0x5627a4ac55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fb50ef740 4 client.15648 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627a49a1d60 con 0x5627a4ac31e0 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fabfff640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4ac31e0 0x5627a4ac55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3f8bfff640 10 client.15648 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.356+0000 7f3fabfff640 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4ac31e0 0x5627a4ac55d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81a41fb075d3d2d server_cookie=4834e1d8eedca464 in_seq=0 out_seq=0 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3f8bfff640 1 -- 172.21.15.122:0/159922677 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5627a49a1d60 con 0x5627a4ac31e0 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3f8bfff640 10 client.15648 handle_command_reply: tid=0 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 client.15648 shutdown 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 msgr2=0x5627a4a39400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 0x5627a4a39400 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006010 tx=0x7f3fa0001e50 comp rx=0 tx=0).stop 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 shutdown_connections 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5627a4ac31e0 0x5627a4ac55d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627a4a3f680 0x5627a4a39e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3f8bfff640 0 client.15648 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627a4a3cef0 0x5627a4a39940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 --2- 172.21.15.122:0/159922677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5627a4a344a0 0x5627a4a39400 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 >> 172.21.15.122:0/159922677 conn(0x5627a4a30a30 msgr2=0x5627a4a2f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 shutdown_connections 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 1 -- 172.21.15.122:0/159922677 wait complete. 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.357+0000 7f3fb50ef740 20 client.15648 trim_cache size 0 max 16384 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3f9c0a2c90 msgr2=0x7f3f9c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3f9c0a2c90 0x7f3f9c0a5150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009370 tx=0x7f3fa40092a0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 msgr2=0x7f3fac100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 0x7f3fac100ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800d750 tx=0x7f3f9800dc10 comp rx=0 tx=0).stop 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 shutdown_connections 2024-09-12T01:47:30.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3f9c0a2c90 0x7f3f9c0a5150 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fac1082a0 0x7f3fac101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fac1036e0 0x7f3fac100ed0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 --2- 172.21.15.122:0/2630330721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fac102d30 0x7f3fac100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:30.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 >> 172.21.15.122:0/2630330721 conn(0x7f3fac0fc7d0 msgr2=0x7f3fac105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:30.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 shutdown_connections 2024-09-12T01:47:30.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:29.358+0000 7f3fb2d16640 1 -- 172.21.15.122:0/2630330721 wait complete. 2024-09-12T01:47:30.854 INFO:tasks.cephfs.filesystem:scrub status for tag:173a7d39-12cc-48f7-83f7-46b36334ac23 - {'path': '/', 'tag': '173a7d39-12cc-48f7-83f7-46b36334ac23', 'options': 'recursive,force'} 2024-09-12T01:47:30.920 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:47:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.475+0000 7f73bffc9640 1 -- 172.21.15.122:0/1252955394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b8103660 msgr2=0x7f73b810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.475+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1252955394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b8103660 0x7f73b810bb50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009950 tx=0x7f73a802a4f0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.476+0000 7f73bffc9640 1 -- 172.21.15.122:0/1252955394 shutdown_connections 2024-09-12T01:47:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.476+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1252955394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b810c170 0x7f73b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.476+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1252955394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b8103660 0x7f73b810bb50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.476+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1252955394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8102d40 0x7f73b8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.476+0000 7f73bffc9640 1 -- 172.21.15.122:0/1252955394 >> 172.21.15.122:0/1252955394 conn(0x7f73b80fc7c0 msgr2=0x7f73b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.477+0000 7f73bffc9640 1 -- 172.21.15.122:0/1252955394 shutdown_connections 2024-09-12T01:47:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.477+0000 7f73bffc9640 1 -- 172.21.15.122:0/1252955394 wait complete. 2024-09-12T01:47:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.478+0000 7f73bffc9640 1 Processor -- start 2024-09-12T01:47:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.478+0000 7f73bffc9640 1 -- start start 2024-09-12T01:47:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bffc9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 0x7f73b8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bffc9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b810c170 0x7f73b8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bffc9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b819cf60 0x7f73b819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bffc9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8111f70 con 0x7f73b819cf60 2024-09-12T01:47:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bffc9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8111df0 con 0x7f73b8102d40 2024-09-12T01:47:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bdd65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 0x7f73b8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bdd65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 0x7f73b8198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:47796/0 (socket says 172.21.15.122:47796) 2024-09-12T01:47:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bdd65640 1 -- 172.21.15.122:0/1338572910 learned_addr learned my addr 172.21.15.122:0/1338572910 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bd564640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b810c170 0x7f73b8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.479+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81120f0 con 0x7f73b810c170 2024-09-12T01:47:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73be566640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b819cf60 0x7f73b819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73bdd65640 1 -- 172.21.15.122:0/1338572910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b810c170 msgr2=0x7f73b8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73bdd65640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b810c170 0x7f73b8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73bdd65640 1 -- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b819cf60 msgr2=0x7f73b819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73bdd65640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b819cf60 0x7f73b819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73bdd65640 1 -- 172.21.15.122:0/1338572910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b819da20 con 0x7f73b8102d40 2024-09-12T01:47:31.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.480+0000 7f73be566640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b819cf60 0x7f73b819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:31.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.481+0000 7f73bdd65640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 0x7f73b8198690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e9f0 tx=0x7f73b400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:31.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.481+0000 7f73aeffd640 1 -- 172.21.15.122:0/1338572910 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b400cd80 con 0x7f73b8102d40 2024-09-12T01:47:31.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.481+0000 7f73aeffd640 1 -- 172.21.15.122:0/1338572910 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73b4005500 con 0x7f73b8102d40 2024-09-12T01:47:31.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.481+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b819dd10 con 0x7f73b8102d40 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.481+0000 7f73aeffd640 1 -- 172.21.15.122:0/1338572910 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b4010690 con 0x7f73b8102d40 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.481+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b81a5490 con 0x7f73b8102d40 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.485+0000 7f73aeffd640 1 -- 172.21.15.122:0/1338572910 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f73b40046c0 con 0x7f73b8102d40 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.485+0000 7f73c23a2740 1 Processor -- start 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 -- start start 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f12a0 0x555cf18f1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf18ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555cf1847620 con 0x555cf18ec260 2024-09-12T01:47:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555cf17eee50 con 0x555cf18f12a0 2024-09-12T01:47:31.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73c23a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555cf18435e0 con 0x555cf18f1bc0 2024-09-12T01:47:31.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73bd564640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73bd564640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18ec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45698/0 (socket says 172.21.15.122:45698) 2024-09-12T01:47:31.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73bd564640 1 -- 172.21.15.122:0/948001971 learned_addr learned my addr 172.21.15.122:0/948001971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73bdd65640 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf18ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73be566640 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f12a0 0x555cf18f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.486+0000 7f73aeffd640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73a40a2c90 0x7f73a40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73bd564640 1 -- 172.21.15.122:0/948001971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 msgr2=0x555cf18ebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73bd564640 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf18ebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73bd564640 1 -- 172.21.15.122:0/948001971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f12a0 msgr2=0x555cf18f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73bd564640 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f12a0 0x555cf18f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73bd564640 1 -- 172.21.15.122:0/948001971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf17dfd40 con 0x555cf18ec260 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73aeffd640 1 -- 172.21.15.122:0/1338572910 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f73b40cf600 con 0x7f73b8102d40 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73be566640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73a40a2c90 0x7f73a40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.487+0000 7f73bd564640 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18ec6e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009950 tx=0x7f73a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114a616b0f5a3d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.488+0000 7f73a3fff640 1 -- 172.21.15.122:0/948001971 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8040070 con 0x555cf18ec260 2024-09-12T01:47:31.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.488+0000 7f73a3fff640 1 -- 172.21.15.122:0/948001971 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a8005500 con 0x555cf18ec260 2024-09-12T01:47:31.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.488+0000 7f73a3fff640 1 -- 172.21.15.122:0/948001971 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8035040 con 0x555cf18ec260 2024-09-12T01:47:31.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 -- 172.21.15.122:0/948001971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 msgr2=0x555cf18ec6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73be566640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73a40a2c90 0x7f73a40a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f73b0005fd0 tx=0x7f73b0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18ec6e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009950 tx=0x7f73a8005fd0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 -- 172.21.15.122:0/948001971 shutdown_connections 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18ec6e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf18ebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 --2- 172.21.15.122:0/948001971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f12a0 0x555cf18f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 -- 172.21.15.122:0/948001971 >> 172.21.15.122:0/948001971 conn(0x555cf18de0a0 msgr2=0x555cf18de480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 -- 172.21.15.122:0/948001971 shutdown_connections 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 -- 172.21.15.122:0/948001971 wait complete. 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.489+0000 7f73c23a2740 1 -- start start 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.490+0000 7f73c23a2740 1 Processor -- start 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.490+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 0x555cf18f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.490+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.490+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73c23a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555cf18f8580 con 0x555cf18f12a0 2024-09-12T01:47:31.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73c23a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555cf17cff10 con 0x555cf18f1bc0 2024-09-12T01:47:31.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73c23a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555cf18f8700 con 0x555cf18ec260 2024-09-12T01:47:31.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73c23a2740 10 client.0 _ll_register_callbacks cb 0x555cf17f3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:31.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 0x555cf18f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 0x555cf18f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:56436/0 (socket says 172.21.15.122:56436) 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 -- 172.21.15.122:0/2492169883 learned_addr learned my addr 172.21.15.122:0/2492169883 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bd564640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73be566640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 -- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 msgr2=0x555cf18f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 -- 172.21.15.122:0/2492169883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 msgr2=0x555cf18f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bdd65640 1 -- 172.21.15.122:0/2492169883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf1556580 con 0x555cf18ec260 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.491+0000 7f73bd564640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 0x555cf18f5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e9c0 tx=0x7f73b409b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b966fd60cb70ac68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73a17fa640 1 -- 172.21.15.122:0/2492169883 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b409bd00 con 0x555cf18ec260 2024-09-12T01:47:31.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73a17fa640 1 -- 172.21.15.122:0/2492169883 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f73b40cd200 con 0x555cf18ec260 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a8005500 con 0x555cf18ec260 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73a17fa640 1 -- 172.21.15.122:0/2492169883 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73b40ba050 con 0x555cf18ec260 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73c23a2740 10 client.25081 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.492+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555cf1620960 con 0x555cf18ec260 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73a17fa640 1 -- 172.21.15.122:0/2492169883 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a8005500 con 0x555cf18ec260 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73a17fa640 1 -- 172.21.15.122:0/2492169883 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f73b40a2a20 con 0x555cf18ec260 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73c23a2740 10 client.25081 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73c23a2740 10 client.25081 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73c23a2740 20 client.25081 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf19055e0 0x555cf19079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73c23a2740 4 client.25081 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf1854b90 con 0x555cf19055e0 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.493+0000 7f73bd564640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf19055e0 0x555cf19079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.494+0000 7f73a17fa640 10 client.25081 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:31.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.494+0000 7f73bd564640 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf19055e0 0x555cf19079d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75fd94e6debbba9 server_cookie=5b161f56a6738f5 in_seq=0 out_seq=0 2024-09-12T01:47:31.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.494+0000 7f73a17fa640 1 -- 172.21.15.122:0/2492169883 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f73a804fe80 con 0x555cf19055e0 2024-09-12T01:47:31.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.494+0000 7f73a17fa640 10 client.25081 handle_command_reply: tid=0 2024-09-12T01:47:31.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 client.25081 shutdown 2024-09-12T01:47:31.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 msgr2=0x555cf18f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 0x555cf18f5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e9c0 tx=0x7f73b409b660 comp rx=0 tx=0).stop 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 shutdown_connections 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf19055e0 0x555cf19079d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73a17fa640 0 client.25081 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2492169883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18ec260 0x555cf18f5130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 >> 172.21.15.122:0/2492169883 conn(0x555cf18de0a0 msgr2=0x555cf18dd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 shutdown_connections 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 1 -- 172.21.15.122:0/2492169883 wait complete. 2024-09-12T01:47:31.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.495+0000 7f73c23a2740 20 client.25081 trim_cache size 0 max 16384 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 Processor -- start 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 -- start start 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18ec260 0x555cf18ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf190d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ae210 con 0x555cf18f12a0 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81abae0 con 0x555cf18ec260 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.512+0000 7f73c23a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a9910 con 0x555cf18f1bc0 2024-09-12T01:47:31.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45710/0 (socket says 172.21.15.122:45710) 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 -- 172.21.15.122:0/2452764118 learned_addr learned my addr 172.21.15.122:0/2452764118 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18ec260 0x555cf18ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73be566640 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf190d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 -- 172.21.15.122:0/2452764118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 msgr2=0x555cf190d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf190d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 -- 172.21.15.122:0/2452764118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18ec260 msgr2=0x555cf18ebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18ec260 0x555cf18ebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 -- 172.21.15.122:0/2452764118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf18df700 con 0x555cf18f12a0 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73be566640 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf190d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73bd564640 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f4090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f73a8004990 tx=0x7f73a80049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc94ad1ad7924d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73a37fe640 1 -- 172.21.15.122:0/2452764118 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8035040 con 0x555cf18f12a0 2024-09-12T01:47:31.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73a37fe640 1 -- 172.21.15.122:0/2452764118 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a8053070 con 0x555cf18f12a0 2024-09-12T01:47:31.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.513+0000 7f73a37fe640 1 -- 172.21.15.122:0/2452764118 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8040040 con 0x555cf18f12a0 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 -- 172.21.15.122:0/2452764118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 msgr2=0x555cf18f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f4090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f73a8004990 tx=0x7f73a80049c0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 -- 172.21.15.122:0/2452764118 shutdown_connections 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f1bc0 0x555cf190d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18f12a0 0x555cf18f4090 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2452764118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18ec260 0x555cf18ebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 -- 172.21.15.122:0/2452764118 >> 172.21.15.122:0/2452764118 conn(0x555cf18de0a0 msgr2=0x555cf18f5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.514+0000 7f73c23a2740 1 -- 172.21.15.122:0/2452764118 shutdown_connections 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.515+0000 7f73c23a2740 1 -- 172.21.15.122:0/2452764118 wait complete. 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.515+0000 7f73c23a2740 1 -- start start 2024-09-12T01:47:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.515+0000 7f73c23a2740 1 Processor -- start 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.515+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f12a0 0x555cf18fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73c23a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18fa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73c23a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a9fa0 con 0x555cf18ec260 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73c23a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ab410 con 0x555cf18f1bc0 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73c23a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81aa670 con 0x555cf18f12a0 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73c23a2740 10 client.0 _ll_register_callbacks cb 0x7f73b4004b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18fdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45724/0 (socket says 172.21.15.122:45724) 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 -- 172.21.15.122:0/2396966626 learned_addr learned my addr 172.21.15.122:0/2396966626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bd564640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f12a0 0x555cf18fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73be566640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 -- 172.21.15.122:0/2396966626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f12a0 msgr2=0x555cf18fe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f12a0 0x555cf18fe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 -- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 msgr2=0x555cf18fa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18fa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 -- 172.21.15.122:0/2396966626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf18fafd0 con 0x555cf18ec260 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bd564640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f12a0 0x555cf18fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73bdd65640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18fdb70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f73b40a2e30 tx=0x7f73b40ccf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6293b03a1369ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.516+0000 7f73a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73a17fa640 1 -- 172.21.15.122:0/2396966626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b40a9080 con 0x555cf18ec260 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a8053070 con 0x555cf18ec260 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73a17fa640 1 -- 172.21.15.122:0/2396966626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f73b40a4bc0 con 0x555cf18ec260 2024-09-12T01:47:31.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73a17fa640 1 -- 172.21.15.122:0/2396966626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73b40a00b0 con 0x555cf18ec260 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73c23a2740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555cf1970bd0 con 0x555cf18ec260 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.517+0000 7f73a17fa640 1 -- 172.21.15.122:0/2396966626 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a8053070 con 0x555cf18ec260 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73a17fa640 1 -- 172.21.15.122:0/2396966626 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f73b4097ae0 con 0x555cf18ec260 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73c23a2740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73c23a2740 10 client.15666 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73c23a2740 20 client.15666 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf1971280 0x555cf1973690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73c23a2740 4 client.15666 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6977f67f", "release": true, "await": true}] 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.518+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6977f67f", "release": true, "await": true}) -- 0x555cf184f0a0 con 0x555cf1971280 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.519+0000 7f73bd564640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf1971280 0x555cf1973690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.519+0000 7f73bd564640 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf1971280 0x555cf1973690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e448604f096840 server_cookie=4c548645d125f721 in_seq=0 out_seq=0 2024-09-12T01:47:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.519+0000 7f73a17fa640 10 client.15666 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73a17fa640 1 -- 172.21.15.122:0/2396966626 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555cf184f0a0 con 0x555cf1971280 2024-09-12T01:47:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73a17fa640 10 client.15666 handle_command_reply: tid=0 2024-09-12T01:47:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 client.15666 shutdown 2024-09-12T01:47:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 msgr2=0x555cf18fdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18fdb70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f73b40a2e30 tx=0x7f73b40ccf70 comp rx=0 tx=0).stop 2024-09-12T01:47:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 shutdown_connections 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555cf1971280 0x555cf1973690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cf18f1bc0 0x555cf18fa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf18f12a0 0x555cf18fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 --2- 172.21.15.122:0/2396966626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cf18ec260 0x555cf18fdb70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 >> 172.21.15.122:0/2396966626 conn(0x555cf18de0a0 msgr2=0x555cf18dcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73a17fa640 0 client.15666 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 shutdown_connections 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 1 -- 172.21.15.122:0/2396966626 wait complete. 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.521+0000 7f73c23a2740 20 client.15666 trim_cache size 0 max 16384 2024-09-12T01:47:31.526 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 26, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 4, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "6977f67f": { 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 4, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:47:31.527 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.5, 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:47:31.528 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73a40a2c90 msgr2=0x7f73a40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73a40a2c90 0x7f73a40a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f73b0005fd0 tx=0x7f73b0004ab0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 msgr2=0x7f73b8198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 0x7f73b8198690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e9f0 tx=0x7f73b400edf0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 shutdown_connections 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73a40a2c90 0x7f73a40a5150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b819cf60 0x7f73b819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.522+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b810c170 0x7f73b8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.523+0000 7f73bffc9640 1 --2- 172.21.15.122:0/1338572910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73b8102d40 0x7f73b8198690 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:31.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.523+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 >> 172.21.15.122:0/1338572910 conn(0x7f73b80fc7c0 msgr2=0x7f73b80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:31.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.523+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 shutdown_connections 2024-09-12T01:47:31.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:47:31.523+0000 7f73bffc9640 1 -- 172.21.15.122:0/1338572910 wait complete. 2024-09-12T01:47:31.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:31 smithi151 ceph-mon[35229]: pgmap v366: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T01:47:31.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:31 smithi151 ceph-mon[35229]: scrub summary: idle 2024-09-12T01:47:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:31 smithi183 ceph-mon[35127]: pgmap v366: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T01:47:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:31 smithi183 ceph-mon[35127]: scrub summary: idle 2024-09-12T01:47:31.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:31 smithi122 ceph-mon[30925]: pgmap v366: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T01:47:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:31 smithi122 ceph-mon[30925]: scrub summary: idle 2024-09-12T01:47:32.278 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6977f67f', total seconds elapsed: 29.3 2024-09-12T01:47:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 -- 172.21.15.122:0/1073931580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a100fffe0 msgr2=0x7f4a101003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/1073931580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a100fffe0 0x7f4a101003e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00aa10 tx=0x7f4a0c02b720 comp rx=0 tx=0).stop 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 -- 172.21.15.122:0/1073931580 shutdown_connections 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/1073931580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a10101b90 0x7f4a10108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/1073931580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a101011d0 0x7f4a10101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/1073931580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a100fffe0 0x7f4a101003e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 -- 172.21.15.122:0/1073931580 >> 172.21.15.122:0/1073931580 conn(0x7f4a100fb740 msgr2=0x7f4a100fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 -- 172.21.15.122:0/1073931580 shutdown_connections 2024-09-12T01:47:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.903+0000 7f4a16d9f640 1 -- 172.21.15.122:0/1073931580 wait complete. 2024-09-12T01:47:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.904+0000 7f4a16d9f640 1 Processor -- start 2024-09-12T01:47:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.904+0000 7f4a16d9f640 1 -- start start 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a16d9f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a100fffe0 0x7f4a10195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a16d9f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a101011d0 0x7f4a10195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a16d9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 0x7f4a1019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a16d9f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010b410 con 0x7f4a10101b90 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a16d9f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010b290 con 0x7f4a100fffe0 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a16d9f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010b590 con 0x7f4a101011d0 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 0x7f4a1019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 0x7f4a1019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45738/0 (socket says 172.21.15.122:45738) 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 -- 172.21.15.122:0/2694029273 learned_addr learned my addr 172.21.15.122:0/2694029273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1559c640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a101011d0 0x7f4a10195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a100fffe0 0x7f4a10195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 -- 172.21.15.122:0/2694029273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a101011d0 msgr2=0x7f4a10195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a101011d0 0x7f4a10195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 -- 172.21.15.122:0/2694029273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a100fffe0 msgr2=0x7f4a10195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a100fffe0 0x7f4a10195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a1659e640 1 -- 172.21.15.122:0/2694029273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1019d6d0 con 0x7f4a10101b90 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.905+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a100fffe0 0x7f4a10195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.906+0000 7f4a1659e640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 0x7f4a1019d060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800ea20 tx=0x7f4a0800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.906+0000 7f4a06ffd640 1 -- 172.21.15.122:0/2694029273 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08015d90 con 0x7f4a10101b90 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.906+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1019d9c0 con 0x7f4a10101b90 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.906+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1019df60 con 0x7f4a10101b90 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.906+0000 7f4a06ffd640 1 -- 172.21.15.122:0/2694029273 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0800c910 con 0x7f4a10101b90 2024-09-12T01:47:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.906+0000 7f4a06ffd640 1 -- 172.21.15.122:0/2694029273 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08023060 con 0x7f4a10101b90 2024-09-12T01:47:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.908+0000 7f4a06ffd640 1 -- 172.21.15.122:0/2694029273 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f4a0800cac0 con 0x7f4a10101b90 2024-09-12T01:47:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.909+0000 7f4a06ffd640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49fc0a2c90 0x7f49fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.909+0000 7f4a06ffd640 1 -- 172.21.15.122:0/2694029273 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4a080d37b0 con 0x7f4a10101b90 2024-09-12T01:47:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.909+0000 7f4a1ae71740 1 Processor -- start 2024-09-12T01:47:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.909+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49fc0a2c90 0x7f49fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 -- start start 2024-09-12T01:47:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139dcc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcccc0 0x562139dd1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dd4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562139d272f0 con 0x562139dcc380 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562139ccf270 con 0x562139dcccc0 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562139d232b0 con 0x562139dd1d50 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49fc0a2c90 0x7f49fc0a5150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006010 tx=0x7f4a0c030df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1659e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dd4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1659e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dd4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:56480/0 (socket says 172.21.15.122:56480) 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1659e640 1 -- 172.21.15.122:0/1910071980 learned_addr learned my addr 172.21.15.122:0/1910071980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a1559c640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcccc0 0x562139dd1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.910+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139dcc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1659e640 1 -- 172.21.15.122:0/1910071980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcccc0 msgr2=0x562139dd1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1659e640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcccc0 0x562139dd1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1659e640 1 -- 172.21.15.122:0/1910071980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 msgr2=0x562139dcc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1659e640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139dcc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1659e640 1 -- 172.21.15.122:0/1910071980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562139cce340 con 0x562139dd1d50 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1559c640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcccc0 0x562139dd1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139dcc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f4a1659e640 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dd4210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a0809c310 tx=0x7f4a0809ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a19ed7b3007b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f49fbfff640 1 -- 172.21.15.122:0/1910071980 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0809e1b0 con 0x562139dd1d50 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f49fbfff640 1 -- 172.21.15.122:0/1910071980 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a080d13b0 con 0x562139dd1d50 2024-09-12T01:47:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.911+0000 7f49fbfff640 1 -- 172.21.15.122:0/1910071980 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0809bc00 con 0x562139dd1d50 2024-09-12T01:47:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1910071980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 msgr2=0x562139dd4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dd4210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a0809c310 tx=0x7f4a0809ff70 comp rx=0 tx=0).stop 2024-09-12T01:47:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1910071980 shutdown_connections 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dd4210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcccc0 0x562139dd1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1910071980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139dcc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1910071980 >> 172.21.15.122:0/1910071980 conn(0x562139dc79f0 msgr2=0x562139dc7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1910071980 shutdown_connections 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.912+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1910071980 wait complete. 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 -- start start 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 Processor -- start 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcc380 0x562139dd6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 0x562139dd6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562139dd4cc0 con 0x562139dcccc0 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562139cb0290 con 0x562139dcc380 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562139dd4e40 con 0x562139dd1d50 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1ae71740 10 client.0 _ll_register_callbacks cb 0x562139ce1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1559c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 0x562139dd6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1559c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 0x562139dd6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45758/0 (socket says 172.21.15.122:45758) 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.913+0000 7f4a1559c640 1 -- 172.21.15.122:0/3965089578 learned_addr learned my addr 172.21.15.122:0/3965089578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1659e640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcc380 0x562139dd6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1559c640 1 -- 172.21.15.122:0/3965089578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 msgr2=0x562139dddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1559c640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1559c640 1 -- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcc380 msgr2=0x562139dd6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1559c640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcc380 0x562139dd6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1559c640 1 -- 172.21.15.122:0/3965089578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562139a3a210 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcc380 0x562139dd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1659e640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1559c640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 0x562139dd6b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a000126e0 tx=0x7f4a00012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a48c067137d11db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f49f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f49f97fa640 1 -- 172.21.15.122:0/3965089578 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00012190 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f49f97fa640 1 -- 172.21.15.122:0/3965089578 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4a0000f920 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a080d13b0 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f49f97fa640 1 -- 172.21.15.122:0/3965089578 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a0000fb20 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1ae71740 10 client.15678 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.914+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562139b06620 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f49f97fa640 1 -- 172.21.15.122:0/3965089578 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0000fcc0 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f49f97fa640 1 -- 172.21.15.122:0/3965089578 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4a0001c420 con 0x562139dcccc0 2024-09-12T01:47:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a1ae71740 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a1ae71740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a1ae71740 20 client.15678 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139de22b0 0x562139de46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a1ae71740 4 client.15678 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a080d13b0 con 0x562139de22b0 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.915+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139de22b0 0x562139de46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.916+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139de22b0 0x562139de46e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb3b95dd9cefcd4 server_cookie=35031fa7ec1ad83d in_seq=0 out_seq=0 2024-09-12T01:47:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.916+0000 7f49f97fa640 10 client.15678 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.916+0000 7f49f97fa640 1 -- 172.21.15.122:0/3965089578 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4a080d13b0 con 0x562139de22b0 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.916+0000 7f49f97fa640 10 client.15678 handle_command_reply: tid=0 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 client.15678 shutdown 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 msgr2=0x562139dd6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 0x562139dd6b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a000126e0 tx=0x7f4a00012ba0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 shutdown_connections 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139de22b0 0x562139de46e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dd1d50 0x562139dddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcccc0 0x562139dd6b70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/3965089578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dcc380 0x562139dd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 >> 172.21.15.122:0/3965089578 conn(0x562139dc79f0 msgr2=0x562139dc9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f49f97fa640 0 client.15678 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 shutdown_connections 2024-09-12T01:47:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 1 -- 172.21.15.122:0/3965089578 wait complete. 2024-09-12T01:47:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.917+0000 7f4a1ae71740 20 client.15678 trim_cache size 0 max 16384 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 Processor -- start 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 -- start start 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dcc380 0x562139dc8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 0x562139dddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139dc9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101a9620 con 0x562139dd1d50 2024-09-12T01:47:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1559c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 0x562139dddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1559c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 0x562139dddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45774/0 (socket says 172.21.15.122:45774) 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1559c640 1 -- 172.21.15.122:0/1777302740 learned_addr learned my addr 172.21.15.122:0/1777302740 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101a6ef0 con 0x562139dd6ca0 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.945+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101a4d20 con 0x562139dcc380 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dcc380 0x562139dc8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1659e640 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139dc9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1559c640 1 -- 172.21.15.122:0/1777302740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dcc380 msgr2=0x562139dc8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1559c640 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dcc380 0x562139dc8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1559c640 1 -- 172.21.15.122:0/1777302740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 msgr2=0x562139dc9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1559c640 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139dc9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1559c640 1 -- 172.21.15.122:0/1777302740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562139dc7810 con 0x562139dd1d50 2024-09-12T01:47:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dcc380 0x562139dc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f4a1559c640 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 0x562139dddb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a0000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfdd151397cfd7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f49fb7fe640 1 -- 172.21.15.122:0/1777302740 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00012190 con 0x562139dd1d50 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f49fb7fe640 1 -- 172.21.15.122:0/1777302740 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0000fcc0 con 0x562139dd1d50 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.946+0000 7f49fb7fe640 1 -- 172.21.15.122:0/1777302740 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00010b70 con 0x562139dd1d50 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 msgr2=0x562139dddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 0x562139dddb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a0000f850 comp rx=0 tx=0).stop 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 shutdown_connections 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139dc9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dd1d50 0x562139dddb70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/1777302740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139dcc380 0x562139dc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 >> 172.21.15.122:0/1777302740 conn(0x562139dc79f0 msgr2=0x562139dbe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.947+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 shutdown_connections 2024-09-12T01:47:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.948+0000 7f4a1ae71740 1 -- 172.21.15.122:0/1777302740 wait complete. 2024-09-12T01:47:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.948+0000 7f4a1ae71740 1 -- start start 2024-09-12T01:47:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.948+0000 7f4a1ae71740 1 Processor -- start 2024-09-12T01:47:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.948+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139ddb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.948+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139ddbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.948+0000 7f4a1ae71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139ddcbc0 0x562139ddd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101a53b0 con 0x562139dcc380 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101a6820 con 0x562139dd6ca0 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a1ae71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101a5a80 con 0x562139ddcbc0 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139ddb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a1ae71740 10 client.0 _ll_register_callbacks cb 0x7f4a00009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139ddb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45784/0 (socket says 172.21.15.122:45784) 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 -- 172.21.15.122:0/2542957424 learned_addr learned my addr 172.21.15.122:0/2542957424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a1659e640 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139ddcbc0 0x562139ddd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 -- 172.21.15.122:0/2542957424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139ddcbc0 msgr2=0x562139ddd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139ddcbc0 0x562139ddd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 -- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 msgr2=0x562139ddbc00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139ddbc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 -- 172.21.15.122:0/2542957424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562139ddd5b0 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f4a15d9d640 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139ddb6c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00aa10 tx=0x7f4a0c00a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8594a6efb3baf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f49f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f49f97fa640 1 -- 172.21.15.122:0/2542957424 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c059060 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.949+0000 7f49f97fa640 1 -- 172.21.15.122:0/2542957424 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4a0c005450 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0000fcc0 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f49f97fa640 1 -- 172.21.15.122:0/2542957424 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c036040 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 10 client.15684 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562139e4d840 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f49f97fa640 1 -- 172.21.15.122:0/2542957424 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0000fcc0 con 0x562139dcc380 2024-09-12T01:47:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f49f97fa640 1 -- 172.21.15.122:0/2542957424 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4a0c00b040 con 0x562139dcc380 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 20 client.15684 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139e4e1d0 0x562139e505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 4 client.15684 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562139d34a20 con 0x562139e4e1d0 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.950+0000 7f4a1559c640 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139e4e1d0 0x562139e505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.951+0000 7f49f97fa640 10 client.15684 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.951+0000 7f4a1559c640 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139e4e1d0 0x562139e505f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238729836f50b09b server_cookie=f2c8cc05ee75d6e8 in_seq=0 out_seq=0 2024-09-12T01:47:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.954+0000 7f49f97fa640 1 -- 172.21.15.122:0/2542957424 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562139d34a20 con 0x562139e4e1d0 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.954+0000 7f49f97fa640 10 client.15684 handle_command_reply: tid=0 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 client.15684 shutdown 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 msgr2=0x562139ddb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139ddb6c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00aa10 tx=0x7f4a0c00a370 comp rx=0 tx=0).stop 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 shutdown_connections 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562139e4e1d0 0x562139e505f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562139ddcbc0 0x562139ddd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562139dd6ca0 0x562139ddbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 --2- 172.21.15.122:0/2542957424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562139dcc380 0x562139ddb6c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 >> 172.21.15.122:0/2542957424 conn(0x562139dc79f0 msgr2=0x562139dc7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f49f97fa640 0 client.15684 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 shutdown_connections 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 1 -- 172.21.15.122:0/2542957424 wait complete. 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.955+0000 7f4a1ae71740 20 client.15684 trim_cache size 0 max 16384 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:47:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:47:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49fc0a2c90 msgr2=0x7f49fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49fc0a2c90 0x7f49fc0a5150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006010 tx=0x7f4a0c030df0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 msgr2=0x7f4a1019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 0x7f4a1019d060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800ea20 tx=0x7f4a0800eee0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 shutdown_connections 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49fc0a2c90 0x7f49fc0a5150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a10101b90 0x7f4a1019d060 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a101011d0 0x7f4a10195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 --2- 172.21.15.122:0/2694029273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a100fffe0 0x7f4a10195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 >> 172.21.15.122:0/2694029273 conn(0x7f4a100fb740 msgr2=0x7f4a10104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 shutdown_connections 2024-09-12T01:47:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:32.956+0000 7f4a16d9f640 1 -- 172.21.15.122:0/2694029273 wait complete. 2024-09-12T01:47:33.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:33 smithi151 ceph-mon[35229]: pgmap v367: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-12T01:47:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:33 smithi183 ceph-mon[35127]: pgmap v367: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-12T01:47:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:33 smithi122 ceph-mon[30925]: pgmap v367: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-12T01:47:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:35 smithi151 ceph-mon[35229]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-12T01:47:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:35 smithi183 ceph-mon[35127]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-12T01:47:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:35 smithi122 ceph-mon[30925]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-12T01:47:37.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:37 smithi151 ceph-mon[35229]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-12T01:47:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:37 smithi183 ceph-mon[35127]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-12T01:47:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:37 smithi122 ceph-mon[30925]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-12T01:47:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:39 smithi151 ceph-mon[35229]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:39 smithi183 ceph-mon[35127]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:39.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:39 smithi122 ceph-mon[30925]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:41.580 INFO:tasks.workunit.client.0.smithi122.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr: git switch -c 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr:Or undo this operation with: 2024-09-12T01:47:41.581 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.582 INFO:tasks.workunit.client.0.smithi122.stderr: git switch - 2024-09-12T01:47:41.582 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.582 INFO:tasks.workunit.client.0.smithi122.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T01:47:41.582 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-09-12T01:47:41.582 INFO:tasks.workunit.client.0.smithi122.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T01:47:41.586 DEBUG:teuthology.orchestra.run.smithi122:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T01:47:41.647 INFO:tasks.workunit.client.0.smithi122.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T01:47:41.650 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T01:47:41.650 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T01:47:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:41 smithi151 ceph-mon[35229]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:41 smithi183 ceph-mon[35127]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:41 smithi122 ceph-mon[30925]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:42.403 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T01:47:42.490 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T01:47:42.533 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T01:47:42.534 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T01:47:42.535 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T01:47:42.576 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T01:47:42.577 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T01:47:42.577 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T01:47:42.634 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-12T01:47:42.635 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-12T01:47:42.636 DEBUG:teuthology.orchestra.run.smithi122:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-12T01:47:42.696 INFO:tasks.workunit.client.0.smithi122.stderr:+ dbench 1 2024-09-12T01:47:42.703 INFO:tasks.workunit.client.0.smithi122.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-12T01:47:42.703 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-09-12T01:47:42.703 INFO:tasks.workunit.client.0.smithi122.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-12T01:47:42.712 INFO:tasks.workunit.client.0.smithi122.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-12T01:47:42.813 INFO:tasks.workunit.client.0.smithi122.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-12T01:47:42.813 INFO:tasks.workunit.client.0.smithi122.stdout:releasing clients 2024-09-12T01:47:43.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 3983 40.07 MB/sec warmup 1 sec latency 9.977 ms 2024-09-12T01:47:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:43 smithi151 ceph-mon[35229]: pgmap v372: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:43 smithi183 ceph-mon[35127]: pgmap v372: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:43.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:43 smithi122 ceph-mon[30925]: pgmap v372: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T01:47:44.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 7527 29.59 MB/sec warmup 2 sec latency 12.085 ms 2024-09-12T01:47:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:47:44.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:44 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:44.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:47:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:44 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:47:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:44 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:45.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 11111 26.10 MB/sec warmup 3 sec latency 15.452 ms 2024-09-12T01:47:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:45 smithi183 ceph-mon[35127]: pgmap v373: 97 pgs: 97 active+clean; 801 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-12T01:47:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:45 smithi183 ceph-mon[35127]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T01:47:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:47:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:47:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:45.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:45 smithi122 ceph-mon[30925]: pgmap v373: 97 pgs: 97 active+clean; 801 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-12T01:47:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:45 smithi122 ceph-mon[30925]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T01:47:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:47:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:47:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 -- 172.21.15.122:0/3282452063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 msgr2=0x7fc3e8108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3282452063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e8108e90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009900 tx=0x7fc3d802a410 comp rx=0 tx=0).stop 2024-09-12T01:47:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 -- 172.21.15.122:0/3282452063 shutdown_connections 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3282452063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e8108e90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3282452063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8101c40 0x7fc3e81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3282452063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8100a50 0x7fc3e8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 -- 172.21.15.122:0/3282452063 >> 172.21.15.122:0/3282452063 conn(0x7fc3e80fc1b0 msgr2=0x7fc3e80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 -- 172.21.15.122:0/3282452063 shutdown_connections 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.048+0000 7fc3ed447640 1 -- 172.21.15.122:0/3282452063 wait complete. 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 Processor -- start 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 -- start start 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8100a50 0x7fc3e8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8101c40 0x7fc3e8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e810c750 con 0x7fc3e8102600 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e810c5d0 con 0x7fc3e8100a50 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3ed447640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e810c8d0 con 0x7fc3e8101c40 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3e6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8100a50 0x7fc3e8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50226/0 (socket says 172.21.15.122:50226) 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8101c40 0x7fc3e8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.049+0000 7fc3e77fe640 1 -- 172.21.15.122:0/3501354450 learned_addr learned my addr 172.21.15.122:0/3501354450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e67fc640 1 -- 172.21.15.122:0/3501354450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8100a50 msgr2=0x7fc3e8197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8100a50 0x7fc3e8197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e67fc640 1 -- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 msgr2=0x7fc3e819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e819f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e67fc640 1 -- 172.21.15.122:0/3501354450 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819fa70 con 0x7fc3e8101c40 2024-09-12T01:47:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8100a50 0x7fc3e8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8101c40 0x7fc3e8198390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b3f0 tx=0x7fc3d000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3dffff640 1 -- 172.21.15.122:0/3501354450 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0015020 con 0x7fc3e8101c40 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819fd60 con 0x7fc3e8101c40 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3dffff640 1 -- 172.21.15.122:0/3501354450 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d0011d10 con 0x7fc3e8101c40 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3dffff640 1 -- 172.21.15.122:0/3501354450 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000f2e0 con 0x7fc3e8101c40 2024-09-12T01:47:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.050+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a02a0 con 0x7fc3e8101c40 2024-09-12T01:47:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.052+0000 7fc3dffff640 1 -- 172.21.15.122:0/3501354450 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc3d0018020 con 0x7fc3e8101c40 2024-09-12T01:47:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.052+0000 7fc3f01dd740 1 Processor -- start 2024-09-12T01:47:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.052+0000 7fc3f01dd740 1 -- start start 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddeada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadded830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55abadd492f0 con 0x55abadde62e0 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abadcf1270 con 0x55abadde59a0 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55abadd452b0 con 0x55abaddeb370 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddeada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52874/0 (socket says 172.21.15.122:52874) 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e77fe640 1 -- 172.21.15.122:0/2921058519 learned_addr learned my addr 172.21.15.122:0/2921058519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3dffff640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3d40a2c30 0x7fc3d40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadded830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3d40a2c30 0x7fc3d40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e6ffd640 1 -- 172.21.15.122:0/2921058519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 msgr2=0x55abadded830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadded830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e6ffd640 1 -- 172.21.15.122:0/2921058519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 msgr2=0x55abadde5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e6ffd640 1 -- 172.21.15.122:0/2921058519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abadcf0340 con 0x55abadde62e0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.053+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.054+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddeada0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8101aa0 tx=0x7fc3e0011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6604a253dca0bba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.054+0000 7fc3dd7fa640 1 -- 172.21.15.122:0/2921058519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0003c30 con 0x55abadde62e0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.054+0000 7fc3dffff640 1 -- 172.21.15.122:0/3501354450 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d00ce7e0 con 0x7fc3e8101c40 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.054+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3d40a2c30 0x7fc3d40a50f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8006010 tx=0x7fc3d802a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.054+0000 7fc3dd7fa640 1 -- 172.21.15.122:0/2921058519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e000fe50 con 0x55abadde62e0 2024-09-12T01:47:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.054+0000 7fc3dd7fa640 1 -- 172.21.15.122:0/2921058519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00193e0 con 0x55abadde62e0 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- 172.21.15.122:0/2921058519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 msgr2=0x55abaddeada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddeada0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8101aa0 tx=0x7fc3e0011c70 comp rx=0 tx=0).stop 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- 172.21.15.122:0/2921058519 shutdown_connections 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadded830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddeada0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/2921058519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- 172.21.15.122:0/2921058519 >> 172.21.15.122:0/2921058519 conn(0x55abadde1010 msgr2=0x55abadde1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- 172.21.15.122:0/2921058519 shutdown_connections 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- 172.21.15.122:0/2921058519 wait complete. 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- start start 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 Processor -- start 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddf44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abaddeb370 0x55abaddfb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55abaddf2750 con 0x55abadde62e0 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abadcd2290 con 0x55abaddeb370 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.055+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55abaddf28d0 con 0x55abadde59a0 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3f01dd740 10 client.0 _ll_register_callbacks cb 0x55abadd03140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddf44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41890/0 (socket says 172.21.15.122:41890) 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddf44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50250/0 (socket says 172.21.15.122:50250) 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e6ffd640 1 -- 172.21.15.122:0/713560979 learned_addr learned my addr 172.21.15.122:0/713560979 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abaddeb370 0x55abaddfb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 -- 172.21.15.122:0/713560979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 msgr2=0x55abaddf3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 -- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abaddeb370 msgr2=0x55abaddfb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abaddeb370 0x55abaddfb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 -- 172.21.15.122:0/713560979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abada5c210 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddf44b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3d009ad00 tx=0x7fc3d0010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3224be3d4bf92d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/713560979 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0010eb0 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/713560979 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d00976e0 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e000fe50 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/713560979 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3d009b700 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3f01dd740 10 client.15696 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abadb28620 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.056+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/713560979 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e000fe50 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/713560979 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc3d009a450 con 0x55abadde62e0 2024-09-12T01:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3f01dd740 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3f01dd740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3f01dd740 20 client.15696 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abaddffc80 0x55abade02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3f01dd740 4 client.15696 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abadd56a20 con 0x55abaddffc80 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abaddffc80 0x55abade02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abaddffc80 0x55abade02090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c51ad575b523d45 server_cookie=8d07b26634861fb0 in_seq=0 out_seq=0 2024-09-12T01:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.057+0000 7fc3ce7fc640 10 client.15696 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/713560979 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abadd56a20 con 0x55abaddffc80 2024-09-12T01:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3ce7fc640 10 client.15696 handle_command_reply: tid=0 2024-09-12T01:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 client.15696 shutdown 2024-09-12T01:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 msgr2=0x55abaddf44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddf44b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3d009ad00 tx=0x7fc3d0010470 comp rx=0 tx=0).stop 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 shutdown_connections 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abaddffc80 0x55abade02090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abaddeb370 0x55abaddfb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddf44b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3ce7fc640 0 client.15696 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/713560979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.059+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 >> 172.21.15.122:0/713560979 conn(0x55abadde1010 msgr2=0x55abadde2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.060+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 shutdown_connections 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.060+0000 7fc3f01dd740 1 -- 172.21.15.122:0/713560979 wait complete. 2024-09-12T01:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.060+0000 7fc3f01dd740 20 client.15696 trim_cache size 0 max 16384 2024-09-12T01:47:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.075+0000 7fc3f01dd740 1 Processor -- start 2024-09-12T01:47:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.075+0000 7fc3f01dd740 1 -- start start 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadde2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81a97d0 con 0x55abadde62e0 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81a70a0 con 0x55abadde59a0 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81a4ed0 con 0x55abaddeb370 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadde2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddfb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50262/0 (socket says 172.21.15.122:50262) 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e67fc640 1 -- 172.21.15.122:0/1953064243 learned_addr learned my addr 172.21.15.122:0/1953064243 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 -- 172.21.15.122:0/1953064243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 msgr2=0x55abadde2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 -- 172.21.15.122:0/1953064243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 msgr2=0x55abaddfb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddfb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 -- 172.21.15.122:0/1953064243 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abadde67c0 con 0x55abaddeb370 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e67fc640 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddfb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadde2940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009a20 tx=0x7fc3d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a05e022c76571ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3dcff9640 1 -- 172.21.15.122:0/1953064243 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8051070 con 0x55abaddeb370 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3dcff9640 1 -- 172.21.15.122:0/1953064243 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d80055d0 con 0x55abaddeb370 2024-09-12T01:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.076+0000 7fc3dcff9640 1 -- 172.21.15.122:0/1953064243 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8035040 con 0x55abaddeb370 2024-09-12T01:47:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 -- 172.21.15.122:0/1953064243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 msgr2=0x55abadde2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadde2940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009a20 tx=0x7fc3d80046c0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 -- 172.21.15.122:0/1953064243 shutdown_connections 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abaddeb370 0x55abadde2940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abadde62e0 0x55abaddfb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/1953064243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde59a0 0x55abadde2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 -- 172.21.15.122:0/1953064243 >> 172.21.15.122:0/1953064243 conn(0x55abadde1010 msgr2=0x55abadde01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 -- 172.21.15.122:0/1953064243 shutdown_connections 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.077+0000 7fc3f01dd740 1 -- 172.21.15.122:0/1953064243 wait complete. 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 -- start start 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 Processor -- start 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde62e0 0x55abaddf9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abaddeb370 0x55abaddfa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81a5560 con 0x55abaddeb370 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81a69d0 con 0x55abadde62e0 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81a5c30 con 0x55abadde59a0 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3f01dd740 10 client.0 _ll_register_callbacks cb 0x7fc3d0011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abaddeb370 0x55abaddfa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde62e0 0x55abaddf9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41900/0 (socket says 172.21.15.122:41900) 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde62e0 0x55abaddf9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52908/0 (socket says 172.21.15.122:52908) 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e6ffd640 1 -- 172.21.15.122:0/93545355 learned_addr learned my addr 172.21.15.122:0/93545355 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e77fe640 1 -- 172.21.15.122:0/93545355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 msgr2=0x55abaddf9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e77fe640 1 -- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde62e0 msgr2=0x55abaddf9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde62e0 0x55abaddf9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.078+0000 7fc3e77fe640 1 -- 172.21.15.122:0/93545355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abaddfac70 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3e77fe640 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abaddeb370 0x55abaddfa660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8004730 tx=0x7fc3d8009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1302289a9967214f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/93545355 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8035070 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/93545355 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc3d80035f0 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d80055d0 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/93545355 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3d805c050 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 10 client.15702 fetch_fsmap learned FSMap version 26 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abade6af10 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/93545355 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d80055d0 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/93545355 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc3d804c640 con 0x55abaddeb370 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 20 client.15702 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abade6b820 0x55abade6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 4 client.15702 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abadd50d70 con 0x55abade6b820 2024-09-12T01:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.079+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abade6b820 0x55abade6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.080+0000 7fc3e6ffd640 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abade6b820 0x55abade6dc40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e68b23d7da6d35 server_cookie=7b5fc2ac2a4e5229 in_seq=0 out_seq=0 2024-09-12T01:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.080+0000 7fc3ce7fc640 10 client.15702 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:45 smithi151 ceph-mon[35229]: pgmap v373: 97 pgs: 97 active+clean; 801 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-12T01:47:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:45 smithi151 ceph-mon[35229]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T01:47:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:47:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:47:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3ce7fc640 1 -- 172.21.15.122:0/93545355 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abadd50d70 con 0x55abade6b820 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3ce7fc640 10 client.15702 handle_command_reply: tid=0 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 client.15702 shutdown 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abaddeb370 msgr2=0x55abaddfa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abaddeb370 0x55abaddfa660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8004730 tx=0x7fc3d8009600 comp rx=0 tx=0).stop 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 shutdown_connections 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55abade6b820 0x55abade6dc40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55abaddeb370 0x55abaddfa660 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3ce7fc640 0 client.15702 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abadde62e0 0x55abaddf9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 --2- 172.21.15.122:0/93545355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abadde59a0 0x55abaddf9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 >> 172.21.15.122:0/93545355 conn(0x55abadde1010 msgr2=0x55abadde01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 shutdown_connections 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.143+0000 7fc3f01dd740 1 -- 172.21.15.122:0/93545355 wait complete. 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3f01dd740 20 client.15702 trim_cache size 0 max 16384 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:47:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:47:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3d40a2c30 msgr2=0x7fc3d40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3d40a2c30 0x7fc3d40a50f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8006010 tx=0x7fc3d802a410 comp rx=0 tx=0).stop 2024-09-12T01:47:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8101c40 msgr2=0x7fc3e8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8101c40 0x7fc3e8198390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b3f0 tx=0x7fc3d000b8b0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 shutdown_connections 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3d40a2c30 0x7fc3d40a50f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8102600 0x7fc3e819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3e8101c40 0x7fc3e8198390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 --2- 172.21.15.122:0/3501354450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3e8100a50 0x7fc3e8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 >> 172.21.15.122:0/3501354450 conn(0x7fc3e80fc1b0 msgr2=0x7fc3e806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 shutdown_connections 2024-09-12T01:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:46.144+0000 7fc3ed447640 1 -- 172.21.15.122:0/3501354450 wait complete. 2024-09-12T01:47:46.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 14407 24.28 MB/sec warmup 4 sec latency 47.368 ms 2024-09-12T01:47:47.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17072 21.74 MB/sec warmup 5 sec latency 210.453 ms 2024-09-12T01:47:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:47 smithi183 ceph-mon[35127]: pgmap v374: 97 pgs: 97 active+clean; 4.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 354 KiB/s wr, 63 op/s 2024-09-12T01:47:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:47 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:47:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:47 smithi122 ceph-mon[30925]: pgmap v374: 97 pgs: 97 active+clean; 4.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 354 KiB/s wr, 63 op/s 2024-09-12T01:47:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:47 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:47:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:47 smithi151 ceph-mon[35229]: pgmap v374: 97 pgs: 97 active+clean; 4.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 354 KiB/s wr, 63 op/s 2024-09-12T01:47:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:47 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:47:48.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17072 18.11 MB/sec warmup 6 sec latency 1210.530 ms 2024-09-12T01:47:49.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17072 15.53 MB/sec warmup 7 sec latency 2210.621 ms 2024-09-12T01:47:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:49 smithi183 ceph-mon[35127]: pgmap v375: 97 pgs: 97 active+clean; 6.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 616 KiB/s wr, 103 op/s 2024-09-12T01:47:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:49 smithi122 ceph-mon[30925]: pgmap v375: 97 pgs: 97 active+clean; 6.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 616 KiB/s wr, 103 op/s 2024-09-12T01:47:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:49 smithi151 ceph-mon[35229]: pgmap v375: 97 pgs: 97 active+clean; 6.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 616 KiB/s wr, 103 op/s 2024-09-12T01:47:50.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17072 13.58 MB/sec warmup 8 sec latency 3210.718 ms 2024-09-12T01:47:51.813 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17072 12.08 MB/sec warmup 9 sec latency 4210.802 ms 2024-09-12T01:47:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:51 smithi183 ceph-mon[35127]: pgmap v376: 97 pgs: 97 active+clean; 7.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 737 KiB/s wr, 127 op/s 2024-09-12T01:47:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:51 smithi122 ceph-mon[30925]: pgmap v376: 97 pgs: 97 active+clean; 7.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 737 KiB/s wr, 127 op/s 2024-09-12T01:47:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:51 smithi151 ceph-mon[35229]: pgmap v376: 97 pgs: 97 active+clean; 7.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 737 KiB/s wr, 127 op/s 2024-09-12T01:47:52.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17123 11.00 MB/sec warmup 10 sec latency 4325.554 ms 2024-09-12T01:47:53.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17123 10.00 MB/sec warmup 11 sec latency 1851.650 ms 2024-09-12T01:47:53.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:53 smithi151 ceph-mon[35229]: pgmap v377: 97 pgs: 97 active+clean; 8.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 858 KiB/s wr, 156 op/s 2024-09-12T01:47:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:53 smithi183 ceph-mon[35127]: pgmap v377: 97 pgs: 97 active+clean; 8.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 858 KiB/s wr, 156 op/s 2024-09-12T01:47:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:53 smithi122 ceph-mon[30925]: pgmap v377: 97 pgs: 97 active+clean; 8.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 858 KiB/s wr, 156 op/s 2024-09-12T01:47:54.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17123 9.16 MB/sec warmup 12 sec latency 2851.681 ms 2024-09-12T01:47:55.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17123 8.46 MB/sec warmup 13 sec latency 3851.753 ms 2024-09-12T01:47:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:55 smithi183 ceph-mon[35127]: pgmap v378: 97 pgs: 97 active+clean; 8.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 867 KiB/s wr, 158 op/s 2024-09-12T01:47:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:55 smithi122 ceph-mon[30925]: pgmap v378: 97 pgs: 97 active+clean; 8.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 867 KiB/s wr, 158 op/s 2024-09-12T01:47:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:55 smithi151 ceph-mon[35229]: pgmap v378: 97 pgs: 97 active+clean; 8.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 867 KiB/s wr, 158 op/s 2024-09-12T01:47:56.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 17123 7.85 MB/sec warmup 14 sec latency 4851.795 ms 2024-09-12T01:47:57.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 19463 8.12 MB/sec warmup 15 sec latency 4959.891 ms 2024-09-12T01:47:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:57 smithi183 ceph-mon[35127]: pgmap v379: 97 pgs: 97 active+clean; 8.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 858 KiB/s wr, 159 op/s 2024-09-12T01:47:57.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:57 smithi122 ceph-mon[30925]: pgmap v379: 97 pgs: 97 active+clean; 8.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 858 KiB/s wr, 159 op/s 2024-09-12T01:47:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:57 smithi151 ceph-mon[35229]: pgmap v379: 97 pgs: 97 active+clean; 8.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 858 KiB/s wr, 159 op/s 2024-09-12T01:47:58.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 20711 8.07 MB/sec warmup 16 sec latency 731.273 ms 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 -- 172.21.15.122:0/665029113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f040ffe20 msgr2=0x7f4f04100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 --2- 172.21.15.122:0/665029113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f040ffe20 0x7f4f04100220 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009900 tx=0x7f4ef402a410 comp rx=0 tx=0).stop 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 -- 172.21.15.122:0/665029113 shutdown_connections 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 --2- 172.21.15.122:0/665029113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f041037b0 0x7f4f0410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 --2- 172.21.15.122:0/665029113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f04102df0 0x7f4f04103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 --2- 172.21.15.122:0/665029113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f040ffe20 0x7f4f04100220 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.242+0000 7f4f08c17640 1 -- 172.21.15.122:0/665029113 >> 172.21.15.122:0/665029113 conn(0x7f4f040fb980 msgr2=0x7f4f040fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 -- 172.21.15.122:0/665029113 shutdown_connections 2024-09-12T01:47:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 -- 172.21.15.122:0/665029113 wait complete. 2024-09-12T01:47:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 Processor -- start 2024-09-12T01:47:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 -- start start 2024-09-12T01:47:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f040ffe20 0x7f4f04197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f04102df0 0x7f4f041982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.243+0000 7f4f08c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 0x7f4f0419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f08c17640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0410d840 con 0x7f4f041037b0 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f08c17640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0410d6c0 con 0x7f4f040ffe20 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f08c17640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0410d9c0 con 0x7f4f04102df0 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 0x7f4f0419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 0x7f4f0419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42182/0 (socket says 172.21.15.122:42182) 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 -- 172.21.15.122:0/3476706554 learned_addr learned my addr 172.21.15.122:0/3476706554 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f0259c640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f040ffe20 0x7f4f04197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f04102df0 0x7f4f041982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 -- 172.21.15.122:0/3476706554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f04102df0 msgr2=0x7f4f041982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f04102df0 0x7f4f041982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 -- 172.21.15.122:0/3476706554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f040ffe20 msgr2=0x7f4f04197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f040ffe20 0x7f4f04197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 -- 172.21.15.122:0/3476706554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0419f980 con 0x7f4f041037b0 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f0259c640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f040ffe20 0x7f4f04197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f02d9d640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 0x7f4f0419f310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00eaa0 tx=0x7f4efc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4ef37fe640 1 -- 172.21.15.122:0/3476706554 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00cdb0 con 0x7f4f041037b0 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4ef37fe640 1 -- 172.21.15.122:0/3476706554 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4efc005020 con 0x7f4f041037b0 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4ef37fe640 1 -- 172.21.15.122:0/3476706554 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0116d0 con 0x7f4f041037b0 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0419fc70 con 0x7f4f041037b0 2024-09-12T01:47:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.244+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041a0170 con 0x7f4f041037b0 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.247+0000 7f4ef37fe640 1 -- 172.21.15.122:0/3476706554 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4efc0054d0 con 0x7f4f041037b0 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.247+0000 7f4f0ba89740 1 Processor -- start 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 -- start start 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d4cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f183cab2f0 con 0x55f183d4d370 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f183c53270 con 0x55f183d482e0 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f183ca72b0 con 0x55f183d479a0 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d4f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42196/0 (socket says 172.21.15.122:42196) 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 -- 172.21.15.122:0/36158302 learned_addr learned my addr 172.21.15.122:0/36158302 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f0259c640 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d4cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4ef37fe640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4eec0a2c30 0x7f4eec0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 -- 172.21.15.122:0/36158302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 msgr2=0x55f183d47da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d47da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 -- 172.21.15.122:0/36158302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 msgr2=0x55f183d4cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d4cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.248+0000 7f4f01d9b640 1 -- 172.21.15.122:0/36158302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f183c52340 con 0x55f183d4d370 2024-09-12T01:47:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.249+0000 7f4f02d9d640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4eec0a2c30 0x7f4eec0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.249+0000 7f4ef37fe640 1 -- 172.21.15.122:0/3476706554 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4efc0cf310 con 0x7f4f041037b0 2024-09-12T01:47:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.249+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d4f830 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8013930 tx=0x7f4ef800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3752d0d5604bae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.249+0000 7f4ef0ff9640 1 -- 172.21.15.122:0/36158302 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800e950 con 0x55f183d4d370 2024-09-12T01:47:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.249+0000 7f4ef0ff9640 1 -- 172.21.15.122:0/36158302 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef8011440 con 0x55f183d4d370 2024-09-12T01:47:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.249+0000 7f4ef0ff9640 1 -- 172.21.15.122:0/36158302 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8010690 con 0x55f183d4d370 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- 172.21.15.122:0/36158302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 msgr2=0x55f183d4f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f02d9d640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4eec0a2c30 0x7f4eec0a50f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00cfd0 tx=0x7f4efc091050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d4f830 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8013930 tx=0x7f4ef800b460 comp rx=0 tx=0).stop 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- 172.21.15.122:0/36158302 shutdown_connections 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d4f830 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d4cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/36158302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d47da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- 172.21.15.122:0/36158302 >> 172.21.15.122:0/36158302 conn(0x55f183d43010 msgr2=0x55f183d43410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- 172.21.15.122:0/36158302 shutdown_connections 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- 172.21.15.122:0/36158302 wait complete. 2024-09-12T01:47:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- start start 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 Processor -- start 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d479a0 0x55f183d51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d482e0 0x55f183d52100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d59140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f183d50340 con 0x55f183d4d370 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f183c34290 con 0x55f183d479a0 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.250+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f183d504c0 con 0x55f183d482e0 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f0ba89740 10 client.0 _ll_register_callbacks cb 0x55f183c65140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d59140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d59140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42212/0 (socket says 172.21.15.122:42212) 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 -- 172.21.15.122:0/4126442813 learned_addr learned my addr 172.21.15.122:0/4126442813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d479a0 0x55f183d51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f0259c640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d482e0 0x55f183d52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 -- 172.21.15.122:0/4126442813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d482e0 msgr2=0x55f183d52100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d482e0 0x55f183d52100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 -- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d479a0 msgr2=0x55f183d51bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d479a0 0x55f183d51bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 -- 172.21.15.122:0/4126442813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1839be210 con 0x55f183d4d370 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d59140 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80057a0 tx=0x7f4ef8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39da2df7de97f054 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/4126442813 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80115f0 con 0x55f183d4d370 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/4126442813 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4ef8011790 con 0x55f183d4d370 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef8011440 con 0x55f183d4d370 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/4126442813 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ef800e720 con 0x55f183d4d370 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f0ba89740 10 client.15714 fetch_fsmap learned FSMap version 27 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.251+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f183a8a620 con 0x55f183d4d370 2024-09-12T01:47:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/4126442813 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef8011440 con 0x55f183d4d370 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/4126442813 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef802b020 con 0x55f183d4d370 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0ba89740 10 client.15714 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0ba89740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0ba89740 20 client.15714 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d5d5a0 0x55f183d5f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0ba89740 4 client.15714 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f183cb8a20 con 0x55f183d5d5a0 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.252+0000 7f4f0259c640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d5d5a0 0x55f183d5f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4ee9ffb640 10 client.15714 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0259c640 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d5d5a0 0x55f183d5f9d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e368188680a7a5c7 server_cookie=534b75dab1203cb2 in_seq=0 out_seq=0 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/4126442813 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f183cb8a20 con 0x55f183d5d5a0 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4ee9ffb640 10 client.15714 handle_command_reply: tid=0 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 client.15714 shutdown 2024-09-12T01:47:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 msgr2=0x55f183d59140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d59140 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80057a0 tx=0x7f4ef8011b00 comp rx=0 tx=0).stop 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 shutdown_connections 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d5d5a0 0x55f183d5f9d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d4d370 0x55f183d59140 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d482e0 0x55f183d52100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/4126442813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d479a0 0x55f183d51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4ee9ffb640 0 client.15714 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.253+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 >> 172.21.15.122:0/4126442813 conn(0x55f183d43010 msgr2=0x55f183d44820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.254+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 shutdown_connections 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.254+0000 7f4f0ba89740 1 -- 172.21.15.122:0/4126442813 wait complete. 2024-09-12T01:47:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.254+0000 7f4f0ba89740 20 client.15714 trim_cache size 0 max 16384 2024-09-12T01:47:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 Processor -- start 2024-09-12T01:47:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 -- start start 2024-09-12T01:47:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d43010 0x55f183d421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 0x55f183d59100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d44400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041ab8e0 con 0x55f183d479a0 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041a91b0 con 0x55f183d482e0 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041a6fe0 con 0x55f183d43010 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 0x55f183d59100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 0x55f183d59100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42220/0 (socket says 172.21.15.122:42220) 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 -- 172.21.15.122:0/2033682026 learned_addr learned my addr 172.21.15.122:0/2033682026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d43010 0x55f183d421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 -- 172.21.15.122:0/2033682026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d43010 msgr2=0x55f183d421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d43010 0x55f183d421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 -- 172.21.15.122:0/2033682026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 msgr2=0x55f183d44400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d44400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 -- 172.21.15.122:0/2033682026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f183d522a0 con 0x55f183d479a0 2024-09-12T01:47:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d43010 0x55f183d421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.268+0000 7f4f0259c640 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 0x55f183d59100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4006010 tx=0x7f4ef4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234d584fc88cdcfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4eebfff640 1 -- 172.21.15.122:0/2033682026 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4035040 con 0x55f183d479a0 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4eebfff640 1 -- 172.21.15.122:0/2033682026 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef40046c0 con 0x55f183d479a0 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4eebfff640 1 -- 172.21.15.122:0/2033682026 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40319a0 con 0x55f183d479a0 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 -- 172.21.15.122:0/2033682026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 msgr2=0x55f183d59100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 0x55f183d59100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4006010 tx=0x7f4ef4004610 comp rx=0 tx=0).stop 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 -- 172.21.15.122:0/2033682026 shutdown_connections 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d44400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d479a0 0x55f183d59100 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/2033682026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d43010 0x55f183d421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 -- 172.21.15.122:0/2033682026 >> 172.21.15.122:0/2033682026 conn(0x55f183d48da0 msgr2=0x55f183d4cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 -- 172.21.15.122:0/2033682026 shutdown_connections 2024-09-12T01:47:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 -- 172.21.15.122:0/2033682026 wait complete. 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 -- start start 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.269+0000 7f4f0ba89740 1 Processor -- start 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 0x55f183d566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d57b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041a7670 con 0x55f183d43010 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041a8ae0 con 0x55f183d482e0 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041a7d40 con 0x55f183d479a0 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 10 client.0 _ll_register_callbacks cb 0x7f4ef4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 0x55f183d566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 0x55f183d566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42232/0 (socket says 172.21.15.122:42232) 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 -- 172.21.15.122:0/361854405 learned_addr learned my addr 172.21.15.122:0/361854405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0259c640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d57b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 -- 172.21.15.122:0/361854405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 msgr2=0x55f183d56c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d56c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 -- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 msgr2=0x55f183d57b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d57b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 -- 172.21.15.122:0/361854405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f183d58190 con 0x55f183d43010 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f01d9b640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 0x55f183d566f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4ef802d7c0 tx=0x7f4ef800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1ac7abbfee2bee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/361854405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef802dce0 con 0x55f183d43010 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/361854405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4ef8011440 con 0x55f183d43010 2024-09-12T01:47:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.270+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef40046c0 con 0x55f183d43010 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/361854405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ef8011640 con 0x55f183d43010 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 10 client.15720 fetch_fsmap learned FSMap version 27 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f183d59410 con 0x55f183d43010 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/361854405 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef40046c0 con 0x55f183d43010 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/361854405 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef802b020 con 0x55f183d43010 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 10 client.15720 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 20 client.15720 populate_metadata read hostname 'smithi122' 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d4d370 0x55f183dca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 4 client.15720 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.271+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f183cb2d70 con 0x55f183d4d370 2024-09-12T01:47:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.272+0000 7f4f0259c640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d4d370 0x55f183dca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:47:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.274+0000 7f4f0259c640 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d4d370 0x55f183dca9c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593f5b29b7f83a49 server_cookie=bdb6aca5d129a9c6 in_seq=0 out_seq=0 2024-09-12T01:47:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.274+0000 7f4ee9ffb640 10 client.15720 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4ee9ffb640 1 -- 172.21.15.122:0/361854405 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f183cb2d70 con 0x55f183d4d370 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4ee9ffb640 10 client.15720 handle_command_reply: tid=0 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 client.15720 shutdown 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 msgr2=0x55f183d566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 0x55f183d566f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4ef802d7c0 tx=0x7f4ef800b460 comp rx=0 tx=0).stop 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 shutdown_connections 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f183d4d370 0x55f183dca9c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f183d482e0 0x55f183d57b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4ee9ffb640 0 client.15720 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:47:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f183d479a0 0x55f183d56c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 --2- 172.21.15.122:0/361854405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f183d43010 0x55f183d566f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 >> 172.21.15.122:0/361854405 conn(0x55f183d48da0 msgr2=0x55f183d421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 shutdown_connections 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.290+0000 7f4f0ba89740 1 -- 172.21.15.122:0/361854405 wait complete. 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f0ba89740 20 client.15720 trim_cache size 0 max 16384 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:47:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:47:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4eec0a2c30 msgr2=0x7f4eec0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4eec0a2c30 0x7f4eec0a50f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00cfd0 tx=0x7f4efc091050 comp rx=0 tx=0).stop 2024-09-12T01:47:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 msgr2=0x7f4f0419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 0x7f4f0419f310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00eaa0 tx=0x7f4efc00ef60 comp rx=0 tx=0).stop 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 shutdown_connections 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4eec0a2c30 0x7f4eec0a50f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f041037b0 0x7f4f0419f310 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f04102df0 0x7f4f041982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 --2- 172.21.15.122:0/3476706554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f040ffe20 0x7f4f04197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 >> 172.21.15.122:0/3476706554 conn(0x7f4f040fb980 msgr2=0x7f4f04106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 shutdown_connections 2024-09-12T01:47:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:47:59.291+0000 7f4f08c17640 1 -- 172.21.15.122:0/3476706554 wait complete. 2024-09-12T01:47:59.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 20711 7.60 MB/sec warmup 17 sec latency 1731.361 ms 2024-09-12T01:47:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:47:59 smithi183 ceph-mon[35127]: pgmap v380: 97 pgs: 97 active+clean; 9.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 590 KiB/s wr, 112 op/s 2024-09-12T01:47:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:47:59 smithi122 ceph-mon[30925]: pgmap v380: 97 pgs: 97 active+clean; 9.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 590 KiB/s wr, 112 op/s 2024-09-12T01:48:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:47:59 smithi151 ceph-mon[35229]: pgmap v380: 97 pgs: 97 active+clean; 9.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 590 KiB/s wr, 112 op/s 2024-09-12T01:48:00.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 20711 7.17 MB/sec warmup 18 sec latency 2731.471 ms 2024-09-12T01:48:00.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:48:01.110 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:01.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 -- 172.21.15.122:0/1769737698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641036e0 msgr2=0x7f6264107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 --2- 172.21.15.122:0/1769737698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641036e0 0x7f6264107b90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6258009930 tx=0x7f625802a420 comp rx=0 tx=0).stop 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 -- 172.21.15.122:0/1769737698 shutdown_connections 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 --2- 172.21.15.122:0/1769737698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62641082a0 0x7f626410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 --2- 172.21.15.122:0/1769737698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641036e0 0x7f6264107b90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 --2- 172.21.15.122:0/1769737698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 0x7f6264103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 -- 172.21.15.122:0/1769737698 >> 172.21.15.122:0/1769737698 conn(0x7f62640fc7d0 msgr2=0x7f62640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 -- 172.21.15.122:0/1769737698 shutdown_connections 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 -- 172.21.15.122:0/1769737698 wait complete. 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.595+0000 7f6269710640 1 Processor -- start 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 -- start start 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 0x7f6264198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62641036e0 0x7f6264198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 0x7f626419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62641120a0 con 0x7f62641082a0 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6264111f20 con 0x7f6264102d30 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6269710640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6264112220 con 0x7f62641036e0 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 0x7f626419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 0x7f626419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46002/0 (socket says 172.21.15.122:46002) 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 -- 172.21.15.122:0/400136052 learned_addr learned my addr 172.21.15.122:0/400136052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62627fc640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62641036e0 0x7f6264198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6262ffd640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 0x7f6264198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 -- 172.21.15.122:0/400136052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62641036e0 msgr2=0x7f6264198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62641036e0 0x7f6264198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 -- 172.21.15.122:0/400136052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 msgr2=0x7f6264198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 0x7f6264198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f62637fe640 1 -- 172.21.15.122:0/400136052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626419d5f0 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.596+0000 7f6262ffd640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 0x7f6264198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.597+0000 7f62637fe640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 0x7f626419cf10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d750 tx=0x7f625c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.597+0000 7f624bfff640 1 -- 172.21.15.122:0/400136052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c017070 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.597+0000 7f624bfff640 1 -- 172.21.15.122:0/400136052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c005500 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.597+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626419d8e0 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.597+0000 7f624bfff640 1 -- 172.21.15.122:0/400136052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c004d90 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.597+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641a51c0 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.599+0000 7f624bfff640 1 -- 172.21.15.122:0/400136052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f625c004f30 con 0x7f62641082a0 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.599+0000 7f626bae9740 1 Processor -- start 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.599+0000 7f626bae9740 1 -- start start 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f624bfff640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f624c0a2b80 0x7f624c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f626bae9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef24a0 0x55b87fef2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f626bae9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fefd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f626bae9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87feffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f626bae9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b87fe582e0 con 0x55b87fefd680 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f626bae9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b87fdffce0 con 0x55b87fef2dc0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f626bae9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b87fe542a0 con 0x55b87fef24a0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62627fc640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f624c0a2b80 0x7f624c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f624bfff640 1 -- 172.21.15.122:0/400136052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f625c0ce9d0 con 0x7f62641082a0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87feffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87feffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46004/0 (socket says 172.21.15.122:46004) 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 -- 172.21.15.122:0/2776045816 learned_addr learned my addr 172.21.15.122:0/2776045816 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f6262ffd640 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef24a0 0x55b87fef2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62627fc640 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fefd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 -- 172.21.15.122:0/2776045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef24a0 msgr2=0x55b87fef2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef24a0 0x55b87fef2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 -- 172.21.15.122:0/2776045816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 msgr2=0x55b87fefd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fefd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 -- 172.21.15.122:0/2776045816 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87fde4440 con 0x55b87fefd680 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62627fc640 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fefd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62627fc640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f624c0a2b80 0x7f624c0a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6258005940 tx=0x7f6258005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62637fe640 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87feffa70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f625c009610 tx=0x7f625c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83041cd9f9f636f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.600+0000 7f62497fa640 1 -- 172.21.15.122:0/2776045816 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c020070 con 0x55b87fefd680 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f62497fa640 1 -- 172.21.15.122:0/2776045816 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c09b1f0 con 0x55b87fefd680 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f62497fa640 1 -- 172.21.15.122:0/2776045816 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0bb0b0 con 0x55b87fefd680 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 -- 172.21.15.122:0/2776045816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 msgr2=0x55b87feffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87feffa70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f625c009610 tx=0x7f625c000bc0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 -- 172.21.15.122:0/2776045816 shutdown_connections 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87feffa70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fefd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 --2- 172.21.15.122:0/2776045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef24a0 0x55b87fef2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 -- 172.21.15.122:0/2776045816 >> 172.21.15.122:0/2776045816 conn(0x55b87feeea30 msgr2=0x55b87feeee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 -- 172.21.15.122:0/2776045816 shutdown_connections 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.601+0000 7f626bae9740 1 -- 172.21.15.122:0/2776045816 wait complete. 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 -- start start 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 Processor -- start 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87fefd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87fefa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87fefa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b87ff00250 con 0x55b87fefd680 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b87fe13050 con 0x55b87fef24a0 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b87ff003d0 con 0x55b87fef2dc0 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f626bae9740 10 client.0 _ll_register_callbacks cb 0x55b87fe53160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87fefa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87fefa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46020/0 (socket says 172.21.15.122:46020) 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 -- 172.21.15.122:0/3588017206 learned_addr learned my addr 172.21.15.122:0/3588017206 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62627fc640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87fefa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87fefd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 -- 172.21.15.122:0/3588017206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 msgr2=0x55b87fefa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87fefa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 -- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 msgr2=0x55b87fefd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87fefd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f62637fe640 1 -- 172.21.15.122:0/3588017206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87fbc93c0 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.602+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87fefd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f62637fe640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87fefa8e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f625c0a3930 tx=0x7f625c005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fac1678b740d94e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f6246ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f6246ff5640 1 -- 172.21.15.122:0/3588017206 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c014070 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f6246ff5640 1 -- 172.21.15.122:0/3588017206 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f625c09f160 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625c09b1f0 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f6246ff5640 1 -- 172.21.15.122:0/3588017206 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f625c020050 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 10 client.15732 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b87ff0bbe0 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f6246ff5640 1 -- 172.21.15.122:0/3588017206 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c09b1f0 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f6246ff5640 1 -- 172.21.15.122:0/3588017206 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f625c050b40 con 0x55b87fefd680 2024-09-12T01:48:01.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 20 client.15732 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff0cf30 0x55b87ff0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 4 client.15732 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.603+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b87fe65850 con 0x55b87ff0cf30 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.604+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff0cf30 0x55b87ff0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.604+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff0cf30 0x55b87ff0f320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b21ec763423fc server_cookie=8b9d0f41ba38d22 in_seq=0 out_seq=0 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.604+0000 7f6246ff5640 10 client.15732 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f6246ff5640 1 -- 172.21.15.122:0/3588017206 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b87fe65850 con 0x55b87ff0cf30 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f6246ff5640 10 client.15732 handle_command_reply: tid=0 2024-09-12T01:48:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 client.15732 shutdown 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 msgr2=0x55b87fefa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87fefa8e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f625c0a3930 tx=0x7f625c005390 comp rx=0 tx=0).stop 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 shutdown_connections 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff0cf30 0x55b87ff0f320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87fefa8e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87fefa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 --2- 172.21.15.122:0/3588017206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87fefd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f6246ff5640 0 client.15732 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 >> 172.21.15.122:0/3588017206 conn(0x55b87feeea30 msgr2=0x55b87fef4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 shutdown_connections 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 1 -- 172.21.15.122:0/3588017206 wait complete. 2024-09-12T01:48:01.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.605+0000 7f626bae9740 20 client.15732 trim_cache size 0 max 16384 2024-09-12T01:48:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 Processor -- start 2024-09-12T01:48:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 -- start start 2024-09-12T01:48:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 0x55b87fefc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fef7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fefd680 0x55b87fef7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62641affc0 con 0x55b87fef24a0 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62641ad890 con 0x55b87fef2dc0 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f626bae9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62641ab6c0 con 0x55b87fefd680 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 0x55b87fefc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 0x55b87fefc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46024/0 (socket says 172.21.15.122:46024) 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 -- 172.21.15.122:0/1913982929 learned_addr learned my addr 172.21.15.122:0/1913982929 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 -- 172.21.15.122:0/1913982929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fefd680 msgr2=0x55b87fef7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fefd680 0x55b87fef7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 -- 172.21.15.122:0/1913982929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 msgr2=0x55b87fef7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fef7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 -- 172.21.15.122:0/1913982929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87fef8840 con 0x55b87fef24a0 2024-09-12T01:48:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6262ffd640 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 0x55b87fefc260 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55b87ff004a0 tx=0x7f6254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb03f4ed15e0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.622+0000 7f6248ff9640 1 -- 172.21.15.122:0/1913982929 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254010550 con 0x55b87fef24a0 2024-09-12T01:48:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f6248ff9640 1 -- 172.21.15.122:0/1913982929 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6254010bd0 con 0x55b87fef24a0 2024-09-12T01:48:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f6248ff9640 1 -- 172.21.15.122:0/1913982929 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254011c90 con 0x55b87fef24a0 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 -- 172.21.15.122:0/1913982929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 msgr2=0x55b87fefc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 0x55b87fefc260 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55b87ff004a0 tx=0x7f6254004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 -- 172.21.15.122:0/1913982929 shutdown_connections 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fefd680 0x55b87fef7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef2dc0 0x55b87fef7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 --2- 172.21.15.122:0/1913982929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fef24a0 0x55b87fefc260 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 -- 172.21.15.122:0/1913982929 >> 172.21.15.122:0/1913982929 conn(0x55b87feeea30 msgr2=0x55b87fefbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 -- 172.21.15.122:0/1913982929 shutdown_connections 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 -- 172.21.15.122:0/1913982929 wait complete. 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.623+0000 7f626bae9740 1 -- start start 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 Processor -- start 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87ff109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87ff10ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87ff708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62641abd50 con 0x55b87fefd680 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62641ad1c0 con 0x55b87fef24a0 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62641ac420 con 0x55b87fef2dc0 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f626bae9740 10 client.0 _ll_register_callbacks cb 0x7f62540099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87ff708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87ff708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46040/0 (socket says 172.21.15.122:46040) 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 -- 172.21.15.122:0/3121584714 learned_addr learned my addr 172.21.15.122:0/3121584714 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62627fc640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87ff10ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87ff109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 -- 172.21.15.122:0/3121584714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 msgr2=0x55b87ff10ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87ff10ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 -- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 msgr2=0x55b87ff109a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87ff109a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.624+0000 7f62637fe640 1 -- 172.21.15.122:0/3121584714 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87ff70fa0 con 0x55b87fefd680 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f62637fe640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87ff708b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f625c097bc0 tx=0x7f625c0bb2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564c85781a6c5bce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6246ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6246ff5640 1 -- 172.21.15.122:0/3121584714 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c014070 con 0x55b87fefd680 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6246ff5640 1 -- 172.21.15.122:0/3121584714 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f625c096f40 con 0x55b87fefd680 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254010bd0 con 0x55b87fefd680 2024-09-12T01:48:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6246ff5640 1 -- 172.21.15.122:0/3121584714 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f625c0056b0 con 0x55b87fefd680 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 10 client.15738 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b87ff70fa0 con 0x55b87fefd680 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6246ff5640 1 -- 172.21.15.122:0/3121584714 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6254010bd0 con 0x55b87fefd680 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6246ff5640 1 -- 172.21.15.122:0/3121584714 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f625c097140 con 0x55b87fefd680 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 20 client.15738 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff786c0 0x55b87ff7aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 4 client.15738 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b87fe5fd60 con 0x55b87ff786c0 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.625+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff786c0 0x55b87ff7aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f6246ff5640 10 client.15738 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f6262ffd640 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff786c0 0x55b87ff7aae0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae728a6409efed09 server_cookie=1345c76847367321 in_seq=0 out_seq=0 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f6246ff5640 1 -- 172.21.15.122:0/3121584714 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b87fe5fd60 con 0x55b87ff786c0 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f6246ff5640 10 client.15738 handle_command_reply: tid=0 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 client.15738 shutdown 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 msgr2=0x55b87ff708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87ff708b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f625c097bc0 tx=0x7f625c0bb2e0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 shutdown_connections 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b87ff786c0 0x55b87ff7aae0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b87fefd680 0x55b87ff708b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b87fef2dc0 0x55b87ff10ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f6246ff5640 0 client.15738 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 --2- 172.21.15.122:0/3121584714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b87fef24a0 0x55b87ff109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 >> 172.21.15.122:0/3121584714 conn(0x55b87feeea30 msgr2=0x55b87fef4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 shutdown_connections 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 1 -- 172.21.15.122:0/3121584714 wait complete. 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.626+0000 7f626bae9740 20 client.15738 trim_cache size 0 max 16384 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-09-12T01:48:01.633 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:48:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f624c0a2b80 msgr2=0x7f624c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f624c0a2b80 0x7f624c0a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6258005940 tx=0x7f6258005b50 comp rx=0 tx=0).stop 2024-09-12T01:48:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 msgr2=0x7f626419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 0x7f626419cf10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d750 tx=0x7f625c00dc10 comp rx=0 tx=0).stop 2024-09-12T01:48:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 shutdown_connections 2024-09-12T01:48:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f624c0a2b80 0x7f624c0a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62641082a0 0x7f626419cf10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62641036e0 0x7f6264198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 --2- 172.21.15.122:0/400136052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6264102d30 0x7f6264198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 >> 172.21.15.122:0/400136052 conn(0x7f62640fc7d0 msgr2=0x7f6264106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 shutdown_connections 2024-09-12T01:48:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:01.627+0000 7f6269710640 1 -- 172.21.15.122:0/400136052 wait complete. 2024-09-12T01:48:01.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 20711 6.80 MB/sec warmup 19 sec latency 3731.575 ms 2024-09-12T01:48:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:01 smithi183 ceph-mon[35127]: pgmap v381: 97 pgs: 97 active+clean; 9.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 357 KiB/s wr, 76 op/s 2024-09-12T01:48:01.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:01 smithi122 ceph-mon[30925]: pgmap v381: 97 pgs: 97 active+clean; 9.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 357 KiB/s wr, 76 op/s 2024-09-12T01:48:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:01 smithi151 ceph-mon[35229]: pgmap v381: 97 pgs: 97 active+clean; 9.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 357 KiB/s wr, 76 op/s 2024-09-12T01:48:02.441 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:48:02.441 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:48:02.700 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:02.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 23384 7.07 MB/sec warmup 20 sec latency 3839.826 ms 2024-09-12T01:48:03.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 -- 172.21.15.122:0/3106468338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c103660 msgr2=0x7f343c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:03.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 --2- 172.21.15.122:0/3106468338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c103660 0x7f343c10bb50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3430009900 tx=0x7f343002a480 comp rx=0 tx=0).stop 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 -- 172.21.15.122:0/3106468338 shutdown_connections 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 --2- 172.21.15.122:0/3106468338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c10c170 0x7f343c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 --2- 172.21.15.122:0/3106468338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c103660 0x7f343c10bb50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 --2- 172.21.15.122:0/3106468338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f343c102d40 0x7f343c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 -- 172.21.15.122:0/3106468338 >> 172.21.15.122:0/3106468338 conn(0x7f343c0fc7a0 msgr2=0x7f343c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 -- 172.21.15.122:0/3106468338 shutdown_connections 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.165+0000 7f3440f3f640 1 -- 172.21.15.122:0/3106468338 wait complete. 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 Processor -- start 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 -- start start 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 0x7f343c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c103660 0x7f343c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:03.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f343c10c170 0x7f343c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f343c111f70 con 0x7f343c102d40 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f343c111df0 con 0x7f343c103660 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3440f3f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1120f0 con 0x7f343c10c170 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f343a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 0x7f343c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f343a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 0x7f343c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46054/0 (socket says 172.21.15.122:46054) 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f3439d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c103660 0x7f343c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f343a59c640 1 -- 172.21.15.122:0/860578510 learned_addr learned my addr 172.21.15.122:0/860578510 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.166+0000 7f343ad9d640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f343c10c170 0x7f343c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f343a59c640 1 -- 172.21.15.122:0/860578510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f343c10c170 msgr2=0x7f343c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f343a59c640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f343c10c170 0x7f343c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f343a59c640 1 -- 172.21.15.122:0/860578510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c103660 msgr2=0x7f343c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f343a59c640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c103660 0x7f343c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f343a59c640 1 -- 172.21.15.122:0/860578510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c19d420 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f3439d9b640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c103660 0x7f343c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f343a59c640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 0x7f343c198470 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d770 tx=0x7f342c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c017070 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f342c005500 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c19d710 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c004e00 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.167+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c1a4ff0 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.168+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c06b070 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.169+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f342c004fa0 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.170+0000 7f342b7fe640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34240a2b80 0x7f34240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.170+0000 7f3439d9b640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34240a2b80 0x7f34240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.170+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f342c0cee40 con 0x7f343c102d40 2024-09-12T01:48:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.171+0000 7f3439d9b640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34240a2b80 0x7f34240a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3430009230 tx=0x7f34300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:03.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.173+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f342c09b750 con 0x7f343c102d40 2024-09-12T01:48:03.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.374+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f343c1071e0 con 0x7f343c102d40 2024-09-12T01:48:03.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.375+0000 7f342b7fe640 1 -- 172.21.15.122:0/860578510 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11933 (secure 0 0 0) 0x7f342c09fa50 con 0x7f343c102d40 2024-09-12T01:48:03.375 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:48:03.375 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"btime":"2024-09-12T01:47:46:618717+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:47:46.181402+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T01:48:03.375 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-09-12T01:48:03.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34240a2b80 msgr2=0x7f34240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:03.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34240a2b80 0x7f34240a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3430009230 tx=0x7f34300058d0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 msgr2=0x7f343c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:03.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 0x7f343c198470 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d770 tx=0x7f342c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:48:03.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 shutdown_connections 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f34240a2b80 0x7f34240a5040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f343c10c170 0x7f343c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343c103660 0x7f343c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 --2- 172.21.15.122:0/860578510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343c102d40 0x7f343c198470 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 >> 172.21.15.122:0/860578510 conn(0x7f343c0fc7a0 msgr2=0x7f343c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 shutdown_connections 2024-09-12T01:48:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:03.377+0000 7f3440f3f640 1 -- 172.21.15.122:0/860578510 wait complete. 2024-09-12T01:48:03.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24203 7.00 MB/sec warmup 21 sec latency 759.216 ms 2024-09-12T01:48:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:03 smithi183 ceph-mon[35127]: pgmap v382: 97 pgs: 97 active+clean; 9.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 259 KiB/s wr, 57 op/s 2024-09-12T01:48:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:03 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/860578510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:03 smithi122 ceph-mon[30925]: pgmap v382: 97 pgs: 97 active+clean; 9.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 259 KiB/s wr, 57 op/s 2024-09-12T01:48:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:03 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/860578510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:04.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:48:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:03 smithi151 ceph-mon[35229]: pgmap v382: 97 pgs: 97 active+clean; 9.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 259 KiB/s wr, 57 op/s 2024-09-12T01:48:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:03 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/860578510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:04.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24203 6.68 MB/sec warmup 22 sec latency 1759.240 ms 2024-09-12T01:48:04.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.813+0000 7f8c58ae8640 1 -- 172.21.15.122:0/3059076324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54108290 msgr2=0x7f8c5410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.813+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/3059076324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54108290 0x7f8c5410ec30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009930 tx=0x7f8c4c02a420 comp rx=0 tx=0).stop 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 -- 172.21.15.122:0/3059076324 shutdown_connections 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/3059076324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54108290 0x7f8c5410ec30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/3059076324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54103740 0x7f8c54107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/3059076324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54102d90 0x7f8c54103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 -- 172.21.15.122:0/3059076324 >> 172.21.15.122:0/3059076324 conn(0x7f8c540fc7d0 msgr2=0x7f8c540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 -- 172.21.15.122:0/3059076324 shutdown_connections 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 -- 172.21.15.122:0/3059076324 wait complete. 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 Processor -- start 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 -- start start 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.814+0000 7f8c58ae8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 0x7f8c541986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54103740 0x7f8c54198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54108290 0x7f8c5419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54112150 con 0x7f8c54102d90 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54111fd0 con 0x7f8c54108290 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541122d0 con 0x7f8c54103740 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 0x7f8c541986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 0x7f8c541986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46070/0 (socket says 172.21.15.122:46070) 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 -- 172.21.15.122:0/4040675006 learned_addr learned my addr 172.21.15.122:0/4040675006 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54108290 0x7f8c5419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54103740 0x7f8c54198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 -- 172.21.15.122:0/4040675006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54103740 msgr2=0x7f8c54198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54103740 0x7f8c54198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 -- 172.21.15.122:0/4040675006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54108290 msgr2=0x7f8c5419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54108290 0x7f8c5419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 -- 172.21.15.122:0/4040675006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5419d660 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54108290 0x7f8c5419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c5259c640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 0x7f8c541986b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400ba50 tx=0x7f8c4400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c437fe640 1 -- 172.21.15.122:0/4040675006 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44019070 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5419d950 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c437fe640 1 -- 172.21.15.122:0/4040675006 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4400cbe0 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c437fe640 1 -- 172.21.15.122:0/4040675006 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c440076e0 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.815+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a5230 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.817+0000 7f8c437fe640 1 -- 172.21.15.122:0/4040675006 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8c4401c070 con 0x7f8c54102d90 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 Processor -- start 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 -- start start 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c437fe640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c3c0a2b80 0x7f8c3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb602b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb602ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cb5f912f0 con 0x562cb602b5d0 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cb5f38cf0 con 0x562cb602cb40 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cb5f8d2b0 con 0x562cb603a780 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c3c0a2b80 0x7f8c3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c437fe640 1 -- 172.21.15.122:0/4040675006 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8c440ce710 con 0x7f8c54102d90 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.818+0000 7f8c51d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb602ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb602ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46086/0 (socket says 172.21.15.122:46086) 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 -- 172.21.15.122:0/1206060859 learned_addr learned my addr 172.21.15.122:0/1206060859 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c5259c640 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb602b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 -- 172.21.15.122:0/1206060859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 msgr2=0x562cb603cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 -- 172.21.15.122:0/1206060859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 msgr2=0x562cb602b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb602b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 -- 172.21.15.122:0/1206060859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb5f1d3a0 con 0x562cb602b5d0 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c3c0a2b80 0x7f8c3c0a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005ab0 tx=0x7f8c4c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb602ba10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800a8b0 tx=0x7f8c4800ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381137c404842777 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c40ff9640 1 -- 172.21.15.122:0/1206060859 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48015070 con 0x562cb602b5d0 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c40ff9640 1 -- 172.21.15.122:0/1206060859 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c480113f0 con 0x562cb602b5d0 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.819+0000 7f8c40ff9640 1 -- 172.21.15.122:0/1206060859 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480185a0 con 0x562cb602b5d0 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1206060859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 msgr2=0x562cb602ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb602ba10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800a8b0 tx=0x7f8c4800ad70 comp rx=0 tx=0).stop 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1206060859 shutdown_connections 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb602ba10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1206060859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb602b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1206060859 >> 172.21.15.122:0/1206060859 conn(0x562cb5ff4310 msgr2=0x562cb6027c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1206060859 shutdown_connections 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1206060859 wait complete. 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.820+0000 7f8c5aec1740 1 -- start start 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 Processor -- start 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602b5d0 0x562cb603a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb60373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 0x562cb6037940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cb603d480 con 0x562cb603a780 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cb5f4c060 con 0x562cb602cb40 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cb603d600 con 0x562cb602b5d0 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5aec1740 10 client.0 _ll_register_callbacks cb 0x562cb5c9f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 0x562cb6037940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 0x562cb6037940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46092/0 (socket says 172.21.15.122:46092) 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 -- 172.21.15.122:0/3592107451 learned_addr learned my addr 172.21.15.122:0/3592107451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c5259c640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602b5d0 0x562cb603a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb60373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 -- 172.21.15.122:0/3592107451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602b5d0 msgr2=0x562cb603a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602b5d0 0x562cb603a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 -- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 msgr2=0x562cb60373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb60373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 -- 172.21.15.122:0/3592107451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb5d023c0 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb60373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 0x562cb6037940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009930 tx=0x7f8c4c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f49a5631af43f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.821+0000 7f8c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c39ffb640 1 -- 172.21.15.122:0/3592107451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c035050 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c39ffb640 1 -- 172.21.15.122:0/3592107451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8c4c049470 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c480113f0 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c39ffb640 1 -- 172.21.15.122:0/3592107451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c049670 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 10 client.15756 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562cb6048eb0 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c39ffb640 1 -- 172.21.15.122:0/3592107451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c480113f0 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c39ffb640 1 -- 172.21.15.122:0/3592107451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8c4c03d9a0 con 0x562cb603a780 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 10 client.15756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:48:04.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 20 client.15756 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb604a3c0 0x562cb604c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 4 client.15756 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cb5f9e860 con 0x562cb604a3c0 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.822+0000 7f8c5259c640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb604a3c0 0x562cb604c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.823+0000 7f8c39ffb640 10 client.15756 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.823+0000 7f8c5259c640 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb604a3c0 0x562cb604c850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa14d6d8f3fbbd6 server_cookie=16b0a41cf30aa04f in_seq=0 out_seq=0 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.823+0000 7f8c39ffb640 1 -- 172.21.15.122:0/3592107451 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cb5f9e860 con 0x562cb604a3c0 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.823+0000 7f8c39ffb640 10 client.15756 handle_command_reply: tid=0 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 client.15756 shutdown 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 msgr2=0x562cb6037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 0x562cb6037940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009930 tx=0x7f8c4c009260 comp rx=0 tx=0).stop 2024-09-12T01:48:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 shutdown_connections 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb604a3c0 0x562cb604c850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb603a780 0x562cb6037940 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602cb40 0x562cb60373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c39ffb640 0 client.15756 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/3592107451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602b5d0 0x562cb603a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 >> 172.21.15.122:0/3592107451 conn(0x562cb5ff4310 msgr2=0x562cb602d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 shutdown_connections 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 1 -- 172.21.15.122:0/3592107451 wait complete. 2024-09-12T01:48:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.824+0000 7f8c5aec1740 20 client.15756 trim_cache size 0 max 16384 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 Processor -- start 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 -- start start 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb6027e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602cb40 0x562cb6046720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb603a780 0x562cb6052120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541add30 con 0x562cb602b5d0 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541ab600 con 0x562cb603a780 2024-09-12T01:48:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541a9430 con 0x562cb602cb40 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb6027e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb6027e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46100/0 (socket says 172.21.15.122:46100) 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 -- 172.21.15.122:0/1680082493 learned_addr learned my addr 172.21.15.122:0/1680082493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602cb40 0x562cb6046720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 -- 172.21.15.122:0/1680082493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602cb40 msgr2=0x562cb6046720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602cb40 0x562cb6046720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 -- 172.21.15.122:0/1680082493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb603a780 msgr2=0x562cb6052120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb603a780 0x562cb6052120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb603a780 0x562cb6052120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 -- 172.21.15.122:0/1680082493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb602b2d0 con 0x562cb602b5d0 2024-09-12T01:48:04.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c5259c640 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb6027e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c440a36b0 tx=0x7f8c4400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df449234a0c07cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.837+0000 7f8c3bfff640 1 -- 172.21.15.122:0/1680082493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c440b5070 con 0x562cb602b5d0 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c3bfff640 1 -- 172.21.15.122:0/1680082493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4409f320 con 0x562cb602b5d0 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c3bfff640 1 -- 172.21.15.122:0/1680082493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44099300 con 0x562cb602b5d0 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1680082493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 msgr2=0x562cb6027e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb6027e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c440a36b0 tx=0x7f8c4400e040 comp rx=0 tx=0).stop 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1680082493 shutdown_connections 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb603a780 0x562cb6052120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb602cb40 0x562cb6046720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/1680082493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602b5d0 0x562cb6027e10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1680082493 >> 172.21.15.122:0/1680082493 conn(0x562cb5ff4310 msgr2=0x562cb6027700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1680082493 shutdown_connections 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 -- 172.21.15.122:0/1680082493 wait complete. 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.838+0000 7f8c5aec1740 1 -- start start 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 Processor -- start 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602b5d0 0x562cb603e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 0x562cb603ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541a9ac0 con 0x562cb602cb40 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541aaf30 con 0x562cb602b5d0 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541aa190 con 0x562cb603a780 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5aec1740 10 client.0 _ll_register_callbacks cb 0x7f8c4c041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 0x562cb603ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 0x562cb603ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46104/0 (socket says 172.21.15.122:46104) 2024-09-12T01:48:04.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 -- 172.21.15.122:0/366540434 learned_addr learned my addr 172.21.15.122:0/366540434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5259c640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602b5d0 0x562cb603e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c52d9d640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 -- 172.21.15.122:0/366540434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 msgr2=0x562cb603f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 -- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602b5d0 msgr2=0x562cb603e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602b5d0 0x562cb603e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 -- 172.21.15.122:0/366540434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb60b42f0 con 0x562cb602cb40 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c5259c640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602b5d0 0x562cb603e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c51d9b640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 0x562cb603ed10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800a8b0 tx=0x7f8c4800cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd618b9ae8cc136 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.839+0000 7f8c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c39ffb640 1 -- 172.21.15.122:0/366540434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48029060 con 0x562cb602cb40 2024-09-12T01:48:04.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c39ffb640 1 -- 172.21.15.122:0/366540434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8c48024470 con 0x562cb602cb40 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4409f320 con 0x562cb602cb40 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c39ffb640 1 -- 172.21.15.122:0/366540434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8c48015040 con 0x562cb602cb40 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 10 client.15762 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562cb60b55b0 con 0x562cb602cb40 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c39ffb640 1 -- 172.21.15.122:0/366540434 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4409f320 con 0x562cb602cb40 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c39ffb640 1 -- 172.21.15.122:0/366540434 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8c4802e070 con 0x562cb602cb40 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 10 client.15762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 20 client.15762 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb60b59f0 0x562cb60b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 4 client.15762 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cb5f98d70 con 0x562cb60b59f0 2024-09-12T01:48:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.840+0000 7f8c5259c640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb60b59f0 0x562cb60b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5259c640 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb60b59f0 0x562cb60b7de0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f27687c6bdc1df5e server_cookie=dc3a793f14f5fd3f in_seq=0 out_seq=0 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c39ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c39ffb640 1 -- 172.21.15.122:0/366540434 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cb5f98d70 con 0x562cb60b59f0 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c39ffb640 10 client.15762 handle_command_reply: tid=0 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 client.15762 shutdown 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 msgr2=0x562cb603ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 0x562cb603ed10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800a8b0 tx=0x7f8c4800cfd0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 shutdown_connections 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562cb60b59f0 0x562cb60b7de0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cb603a780 0x562cb603f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cb602cb40 0x562cb603ed10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 --2- 172.21.15.122:0/366540434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cb602b5d0 0x562cb603e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 >> 172.21.15.122:0/366540434 conn(0x562cb5ff4310 msgr2=0x562cb6038c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c39ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 shutdown_connections 2024-09-12T01:48:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 1 -- 172.21.15.122:0/366540434 wait complete. 2024-09-12T01:48:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.841+0000 7f8c5aec1740 20 client.15762 trim_cache size 0 max 16384 2024-09-12T01:48:04.852 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c3c0a2b80 msgr2=0x7f8c3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c3c0a2b80 0x7f8c3c0a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005ab0 tx=0x7f8c4c009680 comp rx=0 tx=0).stop 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 msgr2=0x7f8c541986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 0x7f8c541986b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400ba50 tx=0x7f8c4400bf10 comp rx=0 tx=0).stop 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 shutdown_connections 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8c3c0a2b80 0x7f8c3c0a5040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c54108290 0x7f8c5419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54103740 0x7f8c54198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 --2- 172.21.15.122:0/4040675006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c54102d90 0x7f8c541986b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 >> 172.21.15.122:0/4040675006 conn(0x7f8c540fc7d0 msgr2=0x7f8c54105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 shutdown_connections 2024-09-12T01:48:04.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:04.842+0000 7f8c58ae8640 1 -- 172.21.15.122:0/4040675006 wait complete. 2024-09-12T01:48:05.524 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:48:05.771 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:05.814 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24203 6.39 MB/sec warmup 23 sec latency 2759.346 ms 2024-09-12T01:48:05.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:05 smithi122 ceph-mon[30925]: pgmap v383: 97 pgs: 97 active+clean; 11 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 265 KiB/s wr, 53 op/s 2024-09-12T01:48:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:05 smithi183 ceph-mon[35127]: pgmap v383: 97 pgs: 97 active+clean; 11 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 265 KiB/s wr, 53 op/s 2024-09-12T01:48:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:05 smithi151 ceph-mon[35229]: pgmap v383: 97 pgs: 97 active+clean; 11 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 265 KiB/s wr, 53 op/s 2024-09-12T01:48:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 -- 172.21.15.122:0/1295510049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 msgr2=0x7f49881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 --2- 172.21.15.122:0/1295510049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f49881029b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497402a480 comp rx=0 tx=0).stop 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 -- 172.21.15.122:0/1295510049 shutdown_connections 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 --2- 172.21.15.122:0/1295510049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4988102ef0 0x7f498810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 --2- 172.21.15.122:0/1295510049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f49881029b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 --2- 172.21.15.122:0/1295510049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988100e20 0x7f4988101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 -- 172.21.15.122:0/1295510049 >> 172.21.15.122:0/1295510049 conn(0x7f49880fc7d0 msgr2=0x7f49880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 -- 172.21.15.122:0/1295510049 shutdown_connections 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.229+0000 7f498ef22640 1 -- 172.21.15.122:0/1295510049 wait complete. 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 Processor -- start 2024-09-12T01:48:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 -- start start 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4988100e20 0x7f4988197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f4988197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988102ef0 0x7f498819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4988110f80 con 0x7f4988102570 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4988110e00 con 0x7f4988100e20 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ef22640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4988111100 con 0x7f4988102ef0 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f4988197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f4988197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46126/0 (socket says 172.21.15.122:46126) 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 -- 172.21.15.122:0/961893043 learned_addr learned my addr 172.21.15.122:0/961893043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498ccbe640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4988100e20 0x7f4988197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498d4bf640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988102ef0 0x7f498819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 -- 172.21.15.122:0/961893043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988102ef0 msgr2=0x7f498819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988102ef0 0x7f498819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 -- 172.21.15.122:0/961893043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4988100e20 msgr2=0x7f4988197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4988100e20 0x7f4988197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 -- 172.21.15.122:0/961893043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498819c3d0 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f498d4bf640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988102ef0 0x7f498819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.230+0000 7f4987fff640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f4988197960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f4974003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.231+0000 7f4985ffb640 1 -- 172.21.15.122:0/961893043 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497403f070 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.231+0000 7f4985ffb640 1 -- 172.21.15.122:0/961893043 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4974031c30 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.231+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498819c660 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.231+0000 7f4985ffb640 1 -- 172.21.15.122:0/961893043 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974035030 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.231+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a3fa0 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.233+0000 7f4985ffb640 1 -- 172.21.15.122:0/961893043 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4974042070 con 0x7f4988102570 2024-09-12T01:48:06.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.233+0000 7f49912fb740 1 Processor -- start 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.233+0000 7f49912fb740 1 -- start start 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.233+0000 7f49912fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.233+0000 7f49912fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.233+0000 7f49912fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f49912fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564a0041e2f0 con 0x564a004b85d0 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f49912fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a003c5cf0 con 0x564a004b9b40 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f49912fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a0041a2b0 con 0x564a004c5580 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f4985ffb640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49780a2b80 0x7f49780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46138/0 (socket says 172.21.15.122:46138) 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 -- 172.21.15.122:0/3603829268 learned_addr learned my addr 172.21.15.122:0/3603829268 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498d4bf640 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f4987fff640 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498d4bf640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49780a2b80 0x7f49780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f4985ffb640 1 -- 172.21.15.122:0/961893043 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f49740ef8e0 con 0x7f4988102570 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 -- 172.21.15.122:0/3603829268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 msgr2=0x564a004c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 -- 172.21.15.122:0/3603829268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 msgr2=0x564a004b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 -- 172.21.15.122:0/3603829268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a003aa3a0 con 0x564a004b85d0 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498d4bf640 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498d4bf640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49780a2b80 0x7f49780a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4980005c40 tx=0x7f4980009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.234+0000 7f498ccbe640 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004b8a10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f497c00ac10 tx=0x7f497c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ba57ff80e68ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f4972ffd640 1 -- 172.21.15.122:0/3603829268 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c01d070 con 0x564a004b85d0 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f4972ffd640 1 -- 172.21.15.122:0/3603829268 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497c018b20 con 0x564a004b85d0 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f4972ffd640 1 -- 172.21.15.122:0/3603829268 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c010ba0 con 0x564a004b85d0 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 -- 172.21.15.122:0/3603829268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 msgr2=0x564a004b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004b8a10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f497c00ac10 tx=0x7f497c00f790 comp rx=0 tx=0).stop 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 -- 172.21.15.122:0/3603829268 shutdown_connections 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004b8a10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 --2- 172.21.15.122:0/3603829268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 -- 172.21.15.122:0/3603829268 >> 172.21.15.122:0/3603829268 conn(0x564a00481310 msgr2=0x564a004b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 -- 172.21.15.122:0/3603829268 shutdown_connections 2024-09-12T01:48:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.235+0000 7f49912fb740 1 -- 172.21.15.122:0/3603829268 wait complete. 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 -- start start 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 Processor -- start 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564a004c8280 con 0x564a004b85d0 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a003d9060 con 0x564a004b9b40 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a004c8400 con 0x564a004c5580 2024-09-12T01:48:06.276 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f49912fb740 10 client.0 _ll_register_callbacks cb 0x564a0012c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f498ccbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f498ccbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46152/0 (socket says 172.21.15.122:46152) 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f498ccbe640 1 -- 172.21.15.122:0/1442722698 learned_addr learned my addr 172.21.15.122:0/1442722698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f4987fff640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.236+0000 7f498d4bf640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f498ccbe640 1 -- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 msgr2=0x564a004c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f498ccbe640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f498ccbe640 1 -- 172.21.15.122:0/1442722698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 msgr2=0x564a004c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f498ccbe640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f498ccbe640 1 -- 172.21.15.122:0/1442722698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0018f3c0 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f498ccbe640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004c4f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f497c010720 tx=0x7f497c00a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544fc635317e090c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f4967fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f4967fff640 1 -- 172.21.15.122:0/1442722698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c01d050 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f4967fff640 1 -- 172.21.15.122:0/1442722698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f497c010bb0 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f497c018b20 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f4967fff640 1 -- 172.21.15.122:0/1442722698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f497c010db0 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f49912fb740 10 client.15774 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a004d3d50 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.237+0000 7f4967fff640 1 -- 172.21.15.122:0/1442722698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497c018b20 con 0x564a004b85d0 2024-09-12T01:48:06.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f4967fff640 1 -- 172.21.15.122:0/1442722698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f497c02d020 con 0x564a004b85d0 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f49912fb740 10 client.15774 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f49912fb740 10 client.15774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f49912fb740 20 client.15774 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f49912fb740 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a004d51b0 0x564a004d75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f49912fb740 4 client.15774 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0042b860 con 0x564a004d51b0 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f4987fff640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a004d51b0 0x564a004d75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f4967fff640 10 client.15774 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.238+0000 7f4987fff640 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a004d51b0 0x564a004d75a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c4e86b85da3b985 server_cookie=9884df8e14d0b109 in_seq=0 out_seq=0 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f4967fff640 1 -- 172.21.15.122:0/1442722698 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a0042b860 con 0x564a004d51b0 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f4967fff640 10 client.15774 handle_command_reply: tid=0 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 client.15774 shutdown 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 msgr2=0x564a004c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004c4f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f497c010720 tx=0x7f497c00a910 comp rx=0 tx=0).stop 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 shutdown_connections 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a004d51b0 0x564a004d75a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004c5580 0x564a004c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b9b40 0x564a004c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 --2- 172.21.15.122:0/1442722698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b85d0 0x564a004c4f90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f4967fff640 0 client.15774 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.239+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 >> 172.21.15.122:0/1442722698 conn(0x564a00481310 msgr2=0x564a004ba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.240+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 shutdown_connections 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.240+0000 7f49912fb740 1 -- 172.21.15.122:0/1442722698 wait complete. 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.240+0000 7f49912fb740 20 client.15774 trim_cache size 0 max 16384 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.252+0000 7f49912fb740 1 Processor -- start 2024-09-12T01:48:06.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.252+0000 7f49912fb740 1 -- start start 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49912fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b85d0 0x564a004d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49912fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 0x564a004dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49912fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004c5580 0x564a004dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49912fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49881adbe0 con 0x564a004b9b40 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49912fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49881ab4b0 con 0x564a004c5580 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49912fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49881a92e0 con 0x564a004b85d0 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 0x564a004dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 0x564a004dcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46154/0 (socket says 172.21.15.122:46154) 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 -- 172.21.15.122:0/3755345626 learned_addr learned my addr 172.21.15.122:0/3755345626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f498d4bf640 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004c5580 0x564a004dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 -- 172.21.15.122:0/3755345626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b85d0 msgr2=0x564a004d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b85d0 0x564a004d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 -- 172.21.15.122:0/3755345626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004c5580 msgr2=0x564a004dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004c5580 0x564a004dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 -- 172.21.15.122:0/3755345626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a004b5960 con 0x564a004b9b40 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f498d4bf640 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004c5580 0x564a004dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f4987fff640 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 0x564a004dcbd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f49740ed3a0 tx=0x7f4974036b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55a95be86d32fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49727fc640 1 -- 172.21.15.122:0/3755345626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974049040 con 0x564a004b9b40 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49727fc640 1 -- 172.21.15.122:0/3755345626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740bc100 con 0x564a004b9b40 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.253+0000 7f49727fc640 1 -- 172.21.15.122:0/3755345626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740dc0b0 con 0x564a004b9b40 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 -- 172.21.15.122:0/3755345626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 msgr2=0x564a004dcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 0x564a004dcbd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f49740ed3a0 tx=0x7f4974036b60 comp rx=0 tx=0).stop 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 -- 172.21.15.122:0/3755345626 shutdown_connections 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004c5580 0x564a004dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004b9b40 0x564a004dcbd0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 --2- 172.21.15.122:0/3755345626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b85d0 0x564a004d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 -- 172.21.15.122:0/3755345626 >> 172.21.15.122:0/3755345626 conn(0x564a00481310 msgr2=0x564a004c1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 -- 172.21.15.122:0/3755345626 shutdown_connections 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 -- 172.21.15.122:0/3755345626 wait complete. 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 -- start start 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.254+0000 7f49912fb740 1 Processor -- start 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b85d0 0x564a004bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b9b40 0x564a004bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 0x564a004bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49881a9970 con 0x564a004c5580 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49881aade0 con 0x564a004b85d0 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49881aa040 con 0x564a004b9b40 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f49912fb740 10 client.0 _ll_register_callbacks cb 0x7f497c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 0x564a004bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 0x564a004bfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46166/0 (socket says 172.21.15.122:46166) 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 -- 172.21.15.122:0/1379964774 learned_addr learned my addr 172.21.15.122:0/1379964774 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498ccbe640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b85d0 0x564a004bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f4987fff640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b9b40 0x564a004bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 -- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b9b40 msgr2=0x564a004bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b9b40 0x564a004bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 -- 172.21.15.122:0/1379964774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b85d0 msgr2=0x564a004bf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b85d0 0x564a004bf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 -- 172.21.15.122:0/1379964774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0053eba0 con 0x564a004c5580 2024-09-12T01:48:06.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f498d4bf640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 0x564a004bfc30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4980011f30 tx=0x7f4980004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6525fa4fac4709d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.255+0000 7f4967fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f4967fff640 1 -- 172.21.15.122:0/1379964774 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980027070 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f4967fff640 1 -- 172.21.15.122:0/1379964774 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4980013d10 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49740bc100 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f4967fff640 1 -- 172.21.15.122:0/1379964774 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f498002c460 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 10 client.15780 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a0053fd90 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f4967fff640 1 -- 172.21.15.122:0/1379964774 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740bc100 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f4967fff640 1 -- 172.21.15.122:0/1379964774 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f498000f420 con 0x564a004c5580 2024-09-12T01:48:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 10 client.15780 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 20 client.15780 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a00540510 0x564a00542930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 4 client.15780 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a00425d70 con 0x564a00540510 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.256+0000 7f498ccbe640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a00540510 0x564a00542930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f4967fff640 10 client.15780 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f498ccbe640 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a00540510 0x564a00542930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6e1158a4b198670 server_cookie=7165deabe8661e48 in_seq=0 out_seq=0 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f4967fff640 1 -- 172.21.15.122:0/1379964774 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a00425d70 con 0x564a00540510 2024-09-12T01:48:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f4967fff640 10 client.15780 handle_command_reply: tid=0 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 client.15780 shutdown 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 msgr2=0x564a004bfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 0x564a004bfc30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4980011f30 tx=0x7f4980004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 shutdown_connections 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x564a00540510 0x564a00542930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a004c5580 0x564a004bfc30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a004b9b40 0x564a004bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 --2- 172.21.15.122:0/1379964774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a004b85d0 0x564a004bf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f4967fff640 0 client.15780 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 >> 172.21.15.122:0/1379964774 conn(0x564a00481310 msgr2=0x564a004ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:06.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 shutdown_connections 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 1 -- 172.21.15.122:0/1379964774 wait complete. 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.257+0000 7f49912fb740 20 client.15780 trim_cache size 0 max 16384 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49780a2b80 msgr2=0x7f49780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49780a2b80 0x7f49780a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4980005c40 tx=0x7f4980009200 comp rx=0 tx=0).stop 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 msgr2=0x7f4988197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f4988197960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f4974003c10 comp rx=0 tx=0).stop 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 shutdown_connections 2024-09-12T01:48:06.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49780a2b80 0x7f49780a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4988102ef0 0x7f498819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4988102570 0x7f4988197960 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 --2- 172.21.15.122:0/961893043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4988100e20 0x7f4988197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 >> 172.21.15.122:0/961893043 conn(0x7f49880fc7d0 msgr2=0x7f49880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 shutdown_connections 2024-09-12T01:48:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:06.258+0000 7f498ef22640 1 -- 172.21.15.122:0/961893043 wait complete. 2024-09-12T01:48:06.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24203 6.12 MB/sec warmup 24 sec latency 3759.387 ms 2024-09-12T01:48:06.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:48:07.213 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 -- 172.21.15.122:0/2996446986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102570 msgr2=0x7fa4e41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2996446986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102570 0x7fa4e41029b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009900 tx=0x7fa4e002a480 comp rx=0 tx=0).stop 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 -- 172.21.15.122:0/2996446986 shutdown_connections 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2996446986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4102ef0 0x7fa4e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2996446986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102570 0x7fa4e41029b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2996446986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4100e20 0x7fa4e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 -- 172.21.15.122:0/2996446986 >> 172.21.15.122:0/2996446986 conn(0x7fa4e40fc7d0 msgr2=0x7fa4e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 -- 172.21.15.122:0/2996446986 shutdown_connections 2024-09-12T01:48:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.675+0000 7fa4eb923640 1 -- 172.21.15.122:0/2996446986 wait complete. 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 Processor -- start 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 -- start start 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4100e20 0x7fa4e4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4102570 0x7fa4e4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 0x7fa4e419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4110fa0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4110e20 con 0x7fa4e4100e20 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4eb923640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4111120 con 0x7fa4e4102570 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 0x7fa4e419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 0x7fa4e419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46190/0 (socket says 172.21.15.122:46190) 2024-09-12T01:48:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 -- 172.21.15.122:0/2163866726 learned_addr learned my addr 172.21.15.122:0/2163866726 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4102570 0x7fa4e4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4100e20 0x7fa4e4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 -- 172.21.15.122:0/2163866726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4102570 msgr2=0x7fa4e4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4102570 0x7fa4e4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 -- 172.21.15.122:0/2163866726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4100e20 msgr2=0x7fa4e4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4100e20 0x7fa4e4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 -- 172.21.15.122:0/2163866726 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e419d4e0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4102570 0x7fa4e4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4100e20 0x7fa4e4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.676+0000 7fa4e9ec0640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 0x7fa4e419ce00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00cce0 tx=0x7fa4dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.677+0000 7fa4d67fc640 1 -- 172.21.15.122:0/2163866726 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00f450 con 0x7fa4e4102ef0 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.677+0000 7fa4d67fc640 1 -- 172.21.15.122:0/2163866726 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc00fad0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.677+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e419d7d0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.677+0000 7fa4d67fc640 1 -- 172.21.15.122:0/2163866726 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc018c10 con 0x7fa4e4102ef0 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.677+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e41a50b0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.679+0000 7fa4d67fc640 1 -- 172.21.15.122:0/2163866726 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa4dc00f5f0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.679+0000 7fa4edcfc740 1 Processor -- start 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.679+0000 7fa4edcfc740 1 -- start start 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f4149090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f41495d0 0x5653f4149a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f4154370 0x5653f4156760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653f40af2f0 con 0x5653f414ab40 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5653f4056cf0 con 0x5653f4154370 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653f40ab2b0 con 0x5653f41495d0 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4d67fc640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa4d00a2b80 0x7fa4d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f4149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f4149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46206/0 (socket says 172.21.15.122:46206) 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/4119051608 learned_addr learned my addr 172.21.15.122:0/4119051608 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e9ec0640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa4d00a2b80 0x7fa4d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f41495d0 0x5653f4149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/4119051608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f41495d0 msgr2=0x5653f4149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f41495d0 0x5653f4149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/4119051608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f4154370 msgr2=0x5653f4156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f4154370 0x5653f4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/4119051608 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f403b3a0 con 0x5653f414ab40 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4d67fc640 1 -- 172.21.15.122:0/2163866726 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa4dc0cf7d0 con 0x7fa4e4102ef0 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f4149090 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005940 tx=0x7fa4e0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e60b63cc54317e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4e9ec0640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa4d00a2b80 0x7fa4d00a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc096ad0 tx=0x7fa4dc0c6bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.680+0000 7fa4cf7fe640 1 -- 172.21.15.122:0/4119051608 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0041070 con 0x5653f414ab40 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4cf7fe640 1 -- 172.21.15.122:0/4119051608 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0004490 con 0x5653f414ab40 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4cf7fe640 1 -- 172.21.15.122:0/4119051608 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0035070 con 0x5653f414ab40 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 -- 172.21.15.122:0/4119051608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 msgr2=0x5653f4149090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f4149090 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005940 tx=0x7fa4e0005e50 comp rx=0 tx=0).stop 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 -- 172.21.15.122:0/4119051608 shutdown_connections 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f4154370 0x5653f4156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f41495d0 0x5653f4149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/4119051608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f4149090 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 -- 172.21.15.122:0/4119051608 >> 172.21.15.122:0/4119051608 conn(0x5653f4112310 msgr2=0x5653f4145c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 -- 172.21.15.122:0/4119051608 shutdown_connections 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.681+0000 7fa4edcfc740 1 -- 172.21.15.122:0/4119051608 wait complete. 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 -- start start 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 Processor -- start 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 0x5653f4153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f414ab40 0x5653f4150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 0x5653f41515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653f4157010 con 0x5653f41495d0 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5653f406a060 con 0x5653f414ab40 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653f4157190 con 0x5653f4154370 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4edcfc740 10 client.0 _ll_register_callbacks cb 0x5653f3dbd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4e8ebe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 0x5653f4153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4e8ebe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 0x5653f4153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46214/0 (socket says 172.21.15.122:46214) 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/239600022 learned_addr learned my addr 172.21.15.122:0/239600022 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f414ab40 0x5653f4150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.682+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/239600022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 msgr2=0x5653f41515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 0x5653f41515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f414ab40 msgr2=0x5653f4150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f414ab40 0x5653f4150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/239600022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f3e203c0 con 0x5653f41495d0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 0x5653f4153d30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0031a40 tx=0x7fa4e0031d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d495dc9d233986ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4ccff9640 1 -- 172.21.15.122:0/239600022 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0035070 con 0x5653f41495d0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4ccff9640 1 -- 172.21.15.122:0/239600022 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa4e0046660 con 0x5653f41495d0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e0004490 con 0x5653f41495d0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4ccff9640 1 -- 172.21.15.122:0/239600022 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4e0041040 con 0x5653f41495d0 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 10 client.15792 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653f4162a70 con 0x5653f41495d0 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4ccff9640 1 -- 172.21.15.122:0/239600022 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0004490 con 0x5653f41495d0 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4ccff9640 1 -- 172.21.15.122:0/239600022 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4e0052030 con 0x5653f41495d0 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 20 client.15792 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.683+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f4163dc0 0x5653f41661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.684+0000 7fa4edcfc740 4 client.15792 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.684+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f40bc860 con 0x5653f4163dc0 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.684+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f4163dc0 0x5653f41661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.684+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f4163dc0 0x5653f41661b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14f81d4506db5ef server_cookie=d3719a25c0bb6a5d in_seq=0 out_seq=0 2024-09-12T01:48:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.684+0000 7fa4ccff9640 10 client.15792 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4ccff9640 1 -- 172.21.15.122:0/239600022 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653f40bc860 con 0x5653f4163dc0 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4ccff9640 10 client.15792 handle_command_reply: tid=0 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 client.15792 shutdown 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 msgr2=0x5653f4153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 0x5653f4153d30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0031a40 tx=0x7fa4e0031d10 comp rx=0 tx=0).stop 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 shutdown_connections 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f4163dc0 0x5653f41661b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 0x5653f41515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f414ab40 0x5653f4150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4ccff9640 0 client.15792 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/239600022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f41495d0 0x5653f4153d30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 >> 172.21.15.122:0/239600022 conn(0x5653f4112310 msgr2=0x5653f414b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 shutdown_connections 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 1 -- 172.21.15.122:0/239600022 wait complete. 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.685+0000 7fa4edcfc740 20 client.15792 trim_cache size 0 max 16384 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 Processor -- start 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 -- start start 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f41602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f41508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 0x5653f416b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41afdb0 con 0x5653f414ab40 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ad680 con 0x5653f41495d0 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ab4b0 con 0x5653f4154370 2024-09-12T01:48:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f41508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f41508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46230/0 (socket says 172.21.15.122:46230) 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 -- 172.21.15.122:0/2144000132 learned_addr learned my addr 172.21.15.122:0/2144000132 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f41602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 -- 172.21.15.122:0/2144000132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 msgr2=0x5653f416b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 0x5653f416b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 -- 172.21.15.122:0/2144000132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 msgr2=0x5653f41602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f41602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.698+0000 7fa4e96bf640 1 -- 172.21.15.122:0/2144000132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f4149d50 con 0x5653f414ab40 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f41508a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x5653f4150870 tx=0x7fa4d800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70180bf6b51453fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:07.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4ceffd640 1 -- 172.21.15.122:0/2144000132 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8015930 con 0x5653f414ab40 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4ceffd640 1 -- 172.21.15.122:0/2144000132 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d801e030 con 0x5653f414ab40 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4ceffd640 1 -- 172.21.15.122:0/2144000132 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80133a0 con 0x5653f414ab40 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 -- 172.21.15.122:0/2144000132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 msgr2=0x5653f41508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f41508a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x5653f4150870 tx=0x7fa4d800cf70 comp rx=0 tx=0).stop 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 -- 172.21.15.122:0/2144000132 shutdown_connections 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4154370 0x5653f416b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f414ab40 0x5653f41508a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/2144000132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f41602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 -- 172.21.15.122:0/2144000132 >> 172.21.15.122:0/2144000132 conn(0x5653f4112310 msgr2=0x5653f414f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 -- 172.21.15.122:0/2144000132 shutdown_connections 2024-09-12T01:48:07.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.699+0000 7fa4edcfc740 1 -- 172.21.15.122:0/2144000132 wait complete. 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 -- start start 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 Processor -- start 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f415c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f414ab40 0x5653f415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 0x5653f4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41abb40 con 0x5653f4154370 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41acfb0 con 0x5653f41495d0 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ac210 con 0x5653f414ab40 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4edcfc740 10 client.0 _ll_register_callbacks cb 0x7fa4e0004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4e8ebe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 0x5653f4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4e8ebe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 0x5653f4157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46238/0 (socket says 172.21.15.122:46238) 2024-09-12T01:48:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/31875372 learned_addr learned my addr 172.21.15.122:0/31875372 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f415c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.700+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f414ab40 0x5653f415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/31875372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f414ab40 msgr2=0x5653f415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f414ab40 0x5653f415cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 msgr2=0x5653f415c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f415c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4e8ebe640 1 -- 172.21.15.122:0/31875372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f4158320 con 0x5653f4154370 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4e8ebe640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 0x5653f4157af0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0031f70 tx=0x7fa4e0031d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e90fa2a65eff78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4ccff9640 1 -- 172.21.15.122:0/31875372 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0035040 con 0x5653f4154370 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4ccff9640 1 -- 172.21.15.122:0/31875372 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa4e003b630 con 0x5653f4154370 2024-09-12T01:48:07.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d801e030 con 0x5653f4154370 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4ccff9640 1 -- 172.21.15.122:0/31875372 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4e003b830 con 0x5653f4154370 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4edcfc740 10 client.15798 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.701+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653f41ce310 con 0x5653f4154370 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4ccff9640 1 -- 172.21.15.122:0/31875372 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d801e030 con 0x5653f4154370 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4ccff9640 1 -- 172.21.15.122:0/31875372 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4e0052040 con 0x5653f4154370 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4edcfc740 10 client.15798 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4edcfc740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4edcfc740 20 client.15798 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f41ce980 0x5653f41d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4edcfc740 4 client.15798 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:48:07.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653f40b6d70 con 0x5653f41ce980 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f41ce980 0x5653f41d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4ccff9640 10 client.15798 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.702+0000 7fa4e96bf640 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f41ce980 0x5653f41d0d70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295ca7dddeafcde6 server_cookie=b3c681588b9472ad in_seq=0 out_seq=0 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4ccff9640 1 -- 172.21.15.122:0/31875372 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653f40b6d70 con 0x5653f41ce980 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4ccff9640 10 client.15798 handle_command_reply: tid=0 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 client.15798 shutdown 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 msgr2=0x5653f4157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 0x5653f4157af0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0031f70 tx=0x7fa4e0031d10 comp rx=0 tx=0).stop 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 shutdown_connections 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5653f41ce980 0x5653f41d0d70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653f4154370 0x5653f4157af0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f414ab40 0x5653f415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4ccff9640 0 client.15798 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 --2- 172.21.15.122:0/31875372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653f41495d0 0x5653f415c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 >> 172.21.15.122:0/31875372 conn(0x5653f4112310 msgr2=0x5653f41444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 shutdown_connections 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 1 -- 172.21.15.122:0/31875372 wait complete. 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4edcfc740 20 client.15798 trim_cache size 0 max 16384 2024-09-12T01:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:48:07.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa4d00a2b80 msgr2=0x7fa4d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa4d00a2b80 0x7fa4d00a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc096ad0 tx=0x7fa4dc0c6bc0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 msgr2=0x7fa4e419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.703+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 0x7fa4e419ce00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00cce0 tx=0x7fa4dc00a620 comp rx=0 tx=0).stop 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 shutdown_connections 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa4d00a2b80 0x7fa4d00a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e4102ef0 0x7fa4e419ce00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e4102570 0x7fa4e4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 --2- 172.21.15.122:0/2163866726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4100e20 0x7fa4e4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 >> 172.21.15.122:0/2163866726 conn(0x7fa4e40fc7d0 msgr2=0x7fa4e406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 shutdown_connections 2024-09-12T01:48:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:07.704+0000 7fa4eb923640 1 -- 172.21.15.122:0/2163866726 wait complete. 2024-09-12T01:48:07.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24254 5.93 MB/sec warmup 25 sec latency 3864.116 ms 2024-09-12T01:48:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:07 smithi183 ceph-mon[35127]: pgmap v384: 97 pgs: 97 active+clean; 11 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 282 KiB/s wr, 54 op/s 2024-09-12T01:48:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:07 smithi122 ceph-mon[30925]: pgmap v384: 97 pgs: 97 active+clean; 11 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 282 KiB/s wr, 54 op/s 2024-09-12T01:48:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:07 smithi151 ceph-mon[35229]: pgmap v384: 97 pgs: 97 active+clean; 11 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 282 KiB/s wr, 54 op/s 2024-09-12T01:48:08.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24254 5.70 MB/sec warmup 26 sec latency 1856.876 ms 2024-09-12T01:48:09.421 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:48:09.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:48:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:09.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:09 smithi122 ceph-mon[30925]: pgmap v385: 97 pgs: 97 active+clean; 12 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 349 KiB/s wr, 61 op/s 2024-09-12T01:48:09.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 24532 5.52 MB/sec warmup 27 sec latency 2047.851 ms 2024-09-12T01:48:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:09 smithi183 ceph-mon[35127]: pgmap v385: 97 pgs: 97 active+clean; 12 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 349 KiB/s wr, 61 op/s 2024-09-12T01:48:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:09 smithi151 ceph-mon[35229]: pgmap v385: 97 pgs: 97 active+clean; 12 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 349 KiB/s wr, 61 op/s 2024-09-12T01:48:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2323022487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 msgr2=0x7fb7c410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2323022487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c410bb50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c002a480 comp rx=0 tx=0).stop 2024-09-12T01:48:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2323022487 shutdown_connections 2024-09-12T01:48:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2323022487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c410c170 0x7fb7c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2323022487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c410bb50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2323022487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c4102d40 0x7fb7c4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2323022487 >> 172.21.15.122:0/2323022487 conn(0x7fb7c40fc7a0 msgr2=0x7fb7c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2323022487 shutdown_connections 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2323022487 wait complete. 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 Processor -- start 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.135+0000 7fb7cbeb0640 1 -- start start 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4102d40 0x7fb7c4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c410c170 0x7fb7c419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4112030 con 0x7fb7c4103660 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4111eb0 con 0x7fb7c410c170 2024-09-12T01:48:10.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41121b0 con 0x7fb7c4102d40 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c4198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46254/0 (socket says 172.21.15.122:46254) 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 -- 172.21.15.122:0/2115050589 learned_addr learned my addr 172.21.15.122:0/2115050589 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4102d40 0x7fb7c4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c410c170 0x7fb7c419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 -- 172.21.15.122:0/2115050589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4102d40 msgr2=0x7fb7c4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4102d40 0x7fb7c4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 -- 172.21.15.122:0/2115050589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c410c170 msgr2=0x7fb7c419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c410c170 0x7fb7c419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 -- 172.21.15.122:0/2115050589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c419d530 con 0x7fb7c4103660 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c410c170 0x7fb7c419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4102d40 0x7fb7c4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7c944b640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c4198ac0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005ab0 tx=0x7fb7c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7b6ffd640 1 -- 172.21.15.122:0/2115050589 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c003f030 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c419d7c0 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7b6ffd640 1 -- 172.21.15.122:0/2115050589 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c0003ce0 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.136+0000 7fb7b6ffd640 1 -- 172.21.15.122:0/2115050589 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0035070 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.137+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c41a5100 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.139+0000 7fb7b6ffd640 1 -- 172.21.15.122:0/2115050589 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb7c0042030 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.139+0000 7fb7ce289740 1 Processor -- start 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.139+0000 7fb7b6ffd640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7b00a2b80 0x7fb7b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.139+0000 7fb7b6ffd640 1 -- 172.21.15.122:0/2115050589 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb7c00ef950 con 0x7fb7c4103660 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.139+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7b00a2b80 0x7fb7b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.139+0000 7fb7ce289740 1 -- start start 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abfeb60 0x56091abfef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abff480 0x56091ac09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 0x56091ac0c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ce289740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56091ab64280 con 0x56091ac09d40 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ce289740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56091ab0bb10 con 0x56091abfeb60 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ce289740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56091ab60240 con 0x56091abff480 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 0x56091ac0c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7b00a2b80 0x7fb7b00a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007e30 tx=0x7fb7bc007340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 0x56091ac0c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46260/0 (socket says 172.21.15.122:46260) 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 -- 172.21.15.122:0/3630418165 learned_addr learned my addr 172.21.15.122:0/3630418165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7c944b640 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abff480 0x56091ac09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abfeb60 0x56091abfef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 -- 172.21.15.122:0/3630418165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abff480 msgr2=0x56091ac09800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abff480 0x56091ac09800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 -- 172.21.15.122:0/3630418165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abfeb60 msgr2=0x56091abfef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abfeb60 0x56091abfef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 -- 172.21.15.122:0/3630418165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091ab181f0 con 0x56091ac09d40 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.140+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 0x56091ac0c130 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007a70 tx=0x7fb7bc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b763a9dd05e9de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7affff640 1 -- 172.21.15.122:0/3630418165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc01c440 con 0x56091ac09d40 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7affff640 1 -- 172.21.15.122:0/3630418165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7bc021070 con 0x56091ac09d40 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7affff640 1 -- 172.21.15.122:0/3630418165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc01b900 con 0x56091ac09d40 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 -- 172.21.15.122:0/3630418165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 msgr2=0x56091ac0c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 0x56091ac0c130 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007a70 tx=0x7fb7bc00f000 comp rx=0 tx=0).stop 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 -- 172.21.15.122:0/3630418165 shutdown_connections 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091ac09d40 0x56091ac0c130 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abff480 0x56091ac09800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3630418165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abfeb60 0x56091abfef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 -- 172.21.15.122:0/3630418165 >> 172.21.15.122:0/3630418165 conn(0x56091abfb0f0 msgr2=0x56091abfb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 -- 172.21.15.122:0/3630418165 shutdown_connections 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.141+0000 7fb7ce289740 1 -- 172.21.15.122:0/3630418165 wait complete. 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 -- start start 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 Processor -- start 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abfeb60 0x56091ac06c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 0x56091ac07140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091ac09d40 0x56091ac07680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56091ac0c910 con 0x56091abff480 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56091ab1ef10 con 0x56091ac09d40 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56091ac0ca90 con 0x56091abfeb60 2024-09-12T01:48:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ce289740 10 client.0 _ll_register_callbacks cb 0x56091aae7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7c944b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 0x56091ac07140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7c944b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 0x56091ac07140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46272/0 (socket says 172.21.15.122:46272) 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7c944b640 1 -- 172.21.15.122:0/2625693304 learned_addr learned my addr 172.21.15.122:0/2625693304 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091ac09d40 0x56091ac07680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.142+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abfeb60 0x56091ac06c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7c944b640 1 -- 172.21.15.122:0/2625693304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abfeb60 msgr2=0x56091ac06c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7c944b640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abfeb60 0x56091ac06c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7c944b640 1 -- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091ac09d40 msgr2=0x56091ac07680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7c944b640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091ac09d40 0x56091ac07680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7c944b640 1 -- 172.21.15.122:0/2625693304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091a8d56a0 con 0x56091abff480 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7c944b640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 0x56091ac07140 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00ed450 tx=0x7fb7c00bbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b7a0fda07a0ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/2625693304 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00d5060 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/2625693304 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb7c003b460 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7bc021070 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/2625693304 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7c0049070 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ce289740 10 client.15810 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56091ac184f0 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/2625693304 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7bc021070 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/2625693304 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c00b8820 con 0x56091abff480 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ce289740 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ce289740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.143+0000 7fb7ce289740 20 client.15810 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.144+0000 7fb7ce289740 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac198e0 0x56091ac1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.144+0000 7fb7ce289740 4 client.15810 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.144+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56091ab717f0 con 0x56091ac198e0 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.144+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac198e0 0x56091ac1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.144+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac198e0 0x56091ac1bd00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42acf61ec7f20b4e server_cookie=25c298eb7e81609b in_seq=0 out_seq=0 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.144+0000 7fb7ad7fa640 10 client.15810 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/2625693304 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56091ab717f0 con 0x56091ac198e0 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ad7fa640 10 client.15810 handle_command_reply: tid=0 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 client.15810 shutdown 2024-09-12T01:48:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 msgr2=0x56091ac07140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 0x56091ac07140 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00ed450 tx=0x7fb7c00bbdb0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 shutdown_connections 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac198e0 0x56091ac1bd00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091ac09d40 0x56091ac07680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abff480 0x56091ac07140 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 --2- 172.21.15.122:0/2625693304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091abfeb60 0x56091ac06c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 >> 172.21.15.122:0/2625693304 conn(0x56091abfb0f0 msgr2=0x56091abfa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ad7fa640 0 client.15810 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 shutdown_connections 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 1 -- 172.21.15.122:0/2625693304 wait complete. 2024-09-12T01:48:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.145+0000 7fb7ce289740 20 client.15810 trim_cache size 0 max 16384 2024-09-12T01:48:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 Processor -- start 2024-09-12T01:48:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 -- start start 2024-09-12T01:48:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac08a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41aff30 con 0x56091abfeb60 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41ad800 con 0x56091abff480 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7ce289740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41ab630 con 0x56091ac09d40 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7c9c4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac08a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7c9c4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac08a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46276/0 (socket says 172.21.15.122:46276) 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.162+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/1476277675 learned_addr learned my addr 172.21.15.122:0/1476277675 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c944b640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/1476277675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 msgr2=0x56091ac21b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac21b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/1476277675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 msgr2=0x56091ac215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/1476277675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091ac051b0 con 0x56091abfeb60 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c944b640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac21b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac08a60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8007990 tx=0x7fb7b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82145e490624ed13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7af7fe640 1 -- 172.21.15.122:0/1476277675 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8016b50 con 0x56091abfeb60 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7af7fe640 1 -- 172.21.15.122:0/1476277675 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8016cf0 con 0x56091abfeb60 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.163+0000 7fb7af7fe640 1 -- 172.21.15.122:0/1476277675 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8014480 con 0x56091abfeb60 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 -- 172.21.15.122:0/1476277675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 msgr2=0x56091ac08a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac08a60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8007990 tx=0x7fb7b8008040 comp rx=0 tx=0).stop 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 -- 172.21.15.122:0/1476277675 shutdown_connections 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac21b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 --2- 172.21.15.122:0/1476277675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac08a60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 -- 172.21.15.122:0/1476277675 >> 172.21.15.122:0/1476277675 conn(0x56091abfb0f0 msgr2=0x56091ac08350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 -- 172.21.15.122:0/1476277675 shutdown_connections 2024-09-12T01:48:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 -- 172.21.15.122:0/1476277675 wait complete. 2024-09-12T01:48:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 -- start start 2024-09-12T01:48:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.164+0000 7fb7ce289740 1 Processor -- start 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac11ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41abcc0 con 0x56091abfeb60 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41ad130 con 0x56091abff480 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41ac390 con 0x56091ac09d40 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ce289740 10 client.0 _ll_register_callbacks cb 0x7fb7c0003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac11ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac11ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46284/0 (socket says 172.21.15.122:46284) 2024-09-12T01:48:10.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/3723903334 learned_addr learned my addr 172.21.15.122:0/3723903334 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c944b640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ca44d640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/3723903334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 msgr2=0x56091ac0ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac0ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 msgr2=0x56091ac12220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac12220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 -- 172.21.15.122:0/3723903334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091ac0f1f0 con 0x56091abfeb60 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7c9c4c640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac11ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80075d0 tx=0x7fb7b801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b12d22fb51a39f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.165+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/3723903334 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80258f0 con 0x56091abfeb60 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/3723903334 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb7b8025a90 con 0x56091abfeb60 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b8016cf0 con 0x56091abfeb60 2024-09-12T01:48:10.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/3723903334 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7b8025c90 con 0x56091abfeb60 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 10 client.15816 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56091ac84d60 con 0x56091abfeb60 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/3723903334 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b8016cf0 con 0x56091abfeb60 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/3723903334 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb7b80395c0 con 0x56091abfeb60 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 20 client.15816 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac85400 0x56091ac87820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 4 client.15816 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56091ab6bd00 con 0x56091ac85400 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.166+0000 7fb7c944b640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac85400 0x56091ac87820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ad7fa640 10 client.15816 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:10.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7c944b640 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac85400 0x56091ac87820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6082988aba19fb server_cookie=1e4a285d3417e440 in_seq=0 out_seq=0 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ad7fa640 1 -- 172.21.15.122:0/3723903334 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56091ab6bd00 con 0x56091ac85400 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ad7fa640 10 client.15816 handle_command_reply: tid=0 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 client.15816 shutdown 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 msgr2=0x56091ac11ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac11ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80075d0 tx=0x7fb7b801ce50 comp rx=0 tx=0).stop 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 shutdown_connections 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56091ac85400 0x56091ac87820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56091ac09d40 0x56091ac0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56091abff480 0x56091ac12220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 --2- 172.21.15.122:0/3723903334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56091abfeb60 0x56091ac11ce0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ad7fa640 0 client.15816 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.167+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 >> 172.21.15.122:0/3723903334 conn(0x56091abfb0f0 msgr2=0x56091ac08a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 shutdown_connections 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7ce289740 1 -- 172.21.15.122:0/3723903334 wait complete. 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7ce289740 20 client.15816 trim_cache size 0 max 16384 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "28c02e45-7f8a-4b57-bcef-5f6c3fc7c2f9", 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-09-12T01:48:10.177 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7b00a2b80 msgr2=0x7fb7b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7b00a2b80 0x7fb7b00a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007e30 tx=0x7fb7bc007340 comp rx=0 tx=0).stop 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 msgr2=0x7fb7c4198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c4198ac0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0005ab0 tx=0x7fb7c00042f0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 shutdown_connections 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7b00a2b80 0x7fb7b00a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7c410c170 0x7fb7c419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c4103660 0x7fb7c4198ac0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 --2- 172.21.15.122:0/2115050589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4102d40 0x7fb7c4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:10.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 >> 172.21.15.122:0/2115050589 conn(0x7fb7c40fc7a0 msgr2=0x7fb7c40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:10.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 shutdown_connections 2024-09-12T01:48:10.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:10.168+0000 7fb7cbeb0640 1 -- 172.21.15.122:0/2115050589 wait complete. 2024-09-12T01:48:10.616 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.172+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000009 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ auth{0=1,1=1} v189 ap=1 f(v0 m2024-09-12T01:47:43.020066+0000 9=0+9) n(v4 rc2024-09-12T01:48:03.011565+0000 b15948518 78=68+10) (inest mix dirty) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@2},l=15135 | dirtyscattered=1 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x56004e8d7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000009:[<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000009:[<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:47:43.020066+0000 9=0+9)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:03.011565+0000 b15948518 78=68+10)","memory_value.dirstat":"f(v0 m2024-09-12T01:47:43.020066+0000 9=0+9)","memory_value.rstat":"n(v4 rc2024-09-12T01:48:03.011565+0000 b15948518 78=68+10)","error_str":""},"return_code":0} 2024-09-12T01:48:10.616 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.173+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.001 auth v159 ap=1 s=1048576 n(v0 rc2024-09-12T01:47:42.838070+0000 b1048576 1=1+0) caps={15135=pAsLsXsFsc/-@1},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea08b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000204:[<0x10000000008/filler.001 v6>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000204:[<0x10000000008/filler.001 v159>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.173+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.004 auth v167 ap=1 s=1048576 n(v0 rc2024-09-12T01:47:42.851070+0000 b1048576 1=1+0) caps={15135=pAsLsXsFsc/-@1},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea09600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000207:[<0x10000000008/filler.004 v12>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000207:[<0x10000000008/filler.004 v167>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.173+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.002 auth v163 ap=1 s=1048576 n(v0 rc2024-09-12T01:47:42.843070+0000 b1048576 1=1+0) caps={15135=pAsLsXsFsc/-@1},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea09080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000205:[<0x10000000008/filler.002 v8>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000205:[<0x10000000008/filler.002 v163>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.174+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.003 auth v165 ap=1 s=1048576 n(v0 rc2024-09-12T01:47:42.847070+0000 b1048576 1=1+0) caps={15135=pAsLsXsFsc/-@1},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea09b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000206:[<0x10000000008/filler.003 v10>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000206:[<0x10000000008/filler.003 v165>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.174+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000a [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.000 auth v161 ap=1 s=1048576 n(v0 rc2024-09-12T01:47:42.834070+0000 b1048576 1=1+0) caps={15135=pAsLsXsFsc/-@1},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea0a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000000a:[<0x10000000008/filler.000 v4>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000000a:[<0x10000000008/filler.000 v161>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.175+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{0=1} v4956 ap=1 f(v0 m2024-09-12T01:48:02.405580+0000 11=11+0) n(v0 rc2024-09-12T01:48:02.405580+0000 b2861056 12=11+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@266},l=15135 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56004ea0a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:02.405580+0000 11=11+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.405580+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:02.405580+0000 11=11+0)","memory_value.rstat":"n(v0 rc2024-09-12T01:48:02.405580+0000 b2861056 12=11+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.175+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{0=1} v4930 ap=1 f(v0 m2024-09-12T01:48:03.011565+0000 8=8+0) n(v1 rc2024-09-12T01:48:03.011565+0000 b1283356 9=8+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@355},l=15135 | dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56004ea0ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v2328>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v4930>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:03.011565+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:03.011565+0000 b1283356 9=8+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:03.011565+0000 8=8+0)","memory_value.rstat":"n(v1 rc2024-09-12T01:48:03.011565+0000 b1283356 9=8+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.175+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{0=1} v4964 ap=1 f(v0 m2024-09-12T01:48:02.681573+0000 5=5+0) n(v1 rc2024-09-12T01:48:02.681573+0000 b814326 6=5+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@248},l=15135 | dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56004ea0b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v2304>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v4964>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:02.681573+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.681573+0000 b814326 6=5+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:02.681573+0000 5=5+0)","memory_value.rstat":"n(v1 rc2024-09-12T01:48:02.681573+0000 b814326 6=5+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.177+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000240 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/ auth{0=1} v4962 ap=1 f(v0 m2024-09-12T01:47:43.028066+0000 3=3+0) n(v0 rc2024-09-12T01:48:02.832569+0000 b3244032 4=3+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@8},l=15135 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x56004ea0b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000240:[<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000240:[<0x10000000009/SEED v4962>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:47:43.028066+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.832569+0000 b3244032 4=3+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:47:43.028066+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-09-12T01:48:02.832569+0000 b3244032 4=3+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.177+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{0=1,1=1} v4958 ap=1 f(v0 m2024-09-12T01:48:02.831569+0000 24=24+0) n(v1 rc2024-09-12T01:48:02.831569+0000 b5230239 25=24+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@414},l=15135 | dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x56004e8d7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:02.831569+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.831569+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:02.831569+0000 24=24+0)","memory_value.rstat":"n(v1 rc2024-09-12T01:48:02.831569+0000 b5230239 25=24+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.617 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.177+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/ auth{0=1} v4952 ap=1 f(v0 m2024-09-12T01:48:02.429579+0000 3=3+0) n(v0 rc2024-09-12T01:48:02.429579+0000 b780288 4=3+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@603},l=15135 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56004ea2e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v2368>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v4952>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:02.429579+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.429579+0000 b780288 4=3+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:02.429579+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-09-12T01:48:02.429579+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.618 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.178+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{0=1} v4950 ap=1 f(v0 m2024-09-12T01:48:02.601575+0000 7=7+0) n(v0 rc2024-09-12T01:48:02.601575+0000 b1462582 8=7+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@260},l=15135 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56004ea2e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v2500>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v4950>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:02.601575+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.601575+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:02.601575+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-09-12T01:48:02.601575+0000 b1462582 8=7+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.618 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.178+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{0=1} v4942 ap=1 f(v0 m2024-09-12T01:48:02.442579+0000 1=1+0) n(v0 rc2024-09-12T01:48:02.442579+0000 b260096 2=1+1) (inest mix) (ifile excl) caps={15135=pAsLsXsFsx/pAsLsXsFsx@111},l=15135 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x56004ea2eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v2388>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v4942>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:02.442579+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:02.442579+0000 b260096 2=1+1)","memory_value.dirstat":"f(v0 m2024-09-12T01:48:02.442579+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-12T01:48:02.442579+0000 b260096 2=1+1)","error_str":""},"return_code":0} 2024-09-12T01:48:10.618 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.186+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000008 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/ auth{0=1,1=1} v62 ap=2 RANDEPHEMERALPIN f(v0 m2024-09-12T01:47:42.850070+0000 6=5+1) n(v5 rc2024-09-12T01:48:03.011565+0000 b21191398 84=73+11) (isnap sync r=1) (inest sync->mix g=0,1 dirty) caps={15135=pAsLsXsFs/pAsLsXsFsx@4} | dirtyscattered=1 lock=1 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 randepin 0x56004e8d6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000008:[<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000008:[<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:47:42.850070+0000 6=5+1)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:48:03.011565+0000 b21191398 84=73+11)","memory_value.dirstat":"f(v0 m2024-09-12T01:47:42.850070+0000 6=5+1)","memory_value.rstat":"n(v5 rc2024-09-12T01:48:03.011565+0000 b21191398 84=73+11)","error_str":""},"return_code":0} 2024-09-12T01:48:10.618 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.186+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth v418 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.921068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea2f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v418>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.186+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth v410 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.907069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea2f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v410>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.187+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth v408 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.903069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea2fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v408>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.188+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth v412 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.925068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@58},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea30100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v412>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.188+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth v422 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.917068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@55},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea30680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v422>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.189+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth v402 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.896069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea30c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v402>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.189+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth v420 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.910069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v420>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.189+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth v406 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.163586+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@55},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea31700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v406>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.621 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.189+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth v404 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.914068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea38000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v404>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.190+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth v416 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.900069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@55},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v416>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.190+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth v414 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.169586+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@55},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v2352>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v414>,<0x10000000009/COREL v4956>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.191+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth v735 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.850569+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@73},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea39080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v2328>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v735>,<0x10000000009/EXCEL v4930>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.191+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth v739 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.962566+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@72},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea39b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v2328>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v739>,<0x10000000009/EXCEL v4930>,<0x10000000008/~dmtmp v192>,<0x10000000007/client0 v62>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.192+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{0=1,1=1} v4990 ap=3 RANDEPHEMERALPIN f(v62 m2024-09-12T01:48:09.108413+0000 9=9+0) n(v3 rc2024-09-12T01:48:09.108413+0000 b2083964 10=9+1) (inest sync->mix g=0,1) (ifile mix) caps={15135=pAsLsXs/pAsLsXsFsx@127} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x56004ea08000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v2366>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v4986>,<0x10000000008/~dmtmp v189>,<0x10000000007/client0 v59>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T01:48:09.094413+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T01:47:58.080688+0000 b12543 7=6+1)","memory_value.dirstat":"f(v62 m2024-09-12T01:48:09.108413+0000 9=9+0)","memory_value.rstat":"n(v3 rc2024-09-12T01:48:09.108413+0000 b2083964 10=9+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.193+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC auth v487 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.857070+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea48000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v4>,<0x10000000009/WORD v2304>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v487>,<0x10000000009/WORD v4964>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.193+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC auth v485 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.861070+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea48580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v6>,<0x10000000009/WORD v2304>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v485>,<0x10000000009/WORD v4964>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.193+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC auth v483 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.853070+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@52},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea49080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v2>,<0x10000000009/WORD v2304>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v483>,<0x10000000009/WORD v4964>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.193+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL auth v10 ap=1 s=327680 n(v0 rc2024-09-12T01:48:02.832569+0000 b327680 1=1+0) caps={15135=pAsLsXsFsc/-@3},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea49b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v10>,<0x10000000009/SEED v4962>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.193+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL auth v12 ap=1 s=1048576 n(v0 rc2024-09-12T01:48:02.832569+0000 b1048576 1=1+0) caps={15135=pAsLsXsFsc/-@3},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea4a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v12>,<0x10000000009/SEED v4962>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.622 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.193+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL auth v8 ap=1 s=1867776 n(v0 rc2024-09-12T01:48:02.832569+0000 b1867776 1=1+0) caps={15135=pAsLsXsFsc/-@3},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea4a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v8>,<0x10000000009/SEED v4962>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.194+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022b [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX auth v785 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.814570+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea4ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000022b:[<0x10000000229/COURSES.PX v4>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000022b:[<0x10000000229/COURSES.PX v785>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.194+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022c [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04 auth v765 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.814570+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea4b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000022c:[<0x10000000229/COURSES.X04 v6>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000022c:[<0x10000000229/COURSES.X04 v765>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.194+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000231 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0 auth v787 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.972067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000231:[<0x10000000229/STUDENTS.YG0 v16>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000231:[<0x10000000229/STUDENTS.YG0 v787>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.194+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000237 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX auth v763 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.994066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000237:[<0x10000000229/ENROLL.PX v28>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000237:[<0x10000000229/ENROLL.PX v763>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.194+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022f [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX auth v761 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.814570+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000022f:[<0x10000000229/STUDENTS.PX v12>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000022f:[<0x10000000229/STUDENTS.PX v761>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.194+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023b [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV auth v799 ap=1 s=260096 n(v0 rc2024-09-12T01:47:43.008066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023b:[<0x10000000229/STUDENTS.TV v36>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000023b:[<0x10000000229/STUDENTS.TV v799>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.195+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023c [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL auth v801 ap=1 s=260096 n(v0 rc2024-09-12T01:47:43.011066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea53b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023c:[<0x10000000229/STUDENTS.FSL v38>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000023c:[<0x10000000229/STUDENTS.FSL v801>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.195+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000233 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1 auth v777 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.979067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000233:[<0x10000000229/STUDENTS.YG1 v20>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000233:[<0x10000000229/STUDENTS.YG1 v777>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.195+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000239 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03 auth v797 ap=1 s=260096 n(v0 rc2024-09-12T01:47:43.000066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea54680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000239:[<0x10000000229/ENROLL.Y03 v32>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000239:[<0x10000000229/ENROLL.Y03 v797>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.195+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023d [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL auth v775 ap=1 s=260096 n(v0 rc2024-09-12T01:47:43.015066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea54c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023d:[<0x10000000229/COURSES.FSL v40>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000023d:[<0x10000000229/COURSES.FSL v775>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.195+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000234 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL auth v789 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.983067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea55180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000234:[<0x10000000229/STUDENTS.VAL v22>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000234:[<0x10000000229/STUDENTS.VAL v789>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.623 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.195+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022a [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB auth v771 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.814570+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea55700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000022a:[<0x10000000229/COURSES.DB v2>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000022a:[<0x10000000229/COURSES.DB v771>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.196+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022d [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04 auth v795 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.814570+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000022d:[<0x10000000229/COURSES.Y04 v8>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000022d:[<0x10000000229/COURSES.Y04 v795>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.196+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000238 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03 auth v793 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.997066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000238:[<0x10000000229/ENROLL.X03 v30>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000238:[<0x10000000229/ENROLL.X03 v793>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.196+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000235 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB auth v773 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.987067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000235:[<0x10000000229/STUDENTS.MB v24>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000235:[<0x10000000229/STUDENTS.MB v773>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.196+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023a [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL auth v767 ap=1 s=260096 n(v0 rc2024-09-12T01:47:43.004066+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023a:[<0x10000000229/ENROLL.VAL v34>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000023a:[<0x10000000229/ENROLL.VAL v767>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.196+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000236 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB auth v779 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.990067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000236:[<0x10000000229/ENROLL.DB v26>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000236:[<0x10000000229/ENROLL.DB v779>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.196+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000232 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1 auth v783 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.975067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000232:[<0x10000000229/STUDENTS.XG1 v18>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000232:[<0x10000000229/STUDENTS.XG1 v783>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.197+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000230 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0 auth v759 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.968067+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000230:[<0x10000000229/STUDENTS.XG0 v14>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000230:[<0x10000000229/STUDENTS.XG0 v759>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.197+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022e [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB auth v769 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.814570+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@91},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000022e:[<0x10000000229/STUDENTS.DB v10>,<0x10000000009/PARADOX v2378>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000022e:[<0x10000000229/STUDENTS.DB v769>,<0x10000000009/PARADOX v4958>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.197+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v898 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.937068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@115},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea5f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v2368>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000226:[<0x10000000225/T1.XLS v898>,<0x10000000009/PM v4952>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.197+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v902 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.941068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@115},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v2368>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000227:[<0x10000000225/EVENTS.DOC v902>,<0x10000000009/PM v4952>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.197+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v900 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.945068+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@115},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v2368>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000228:[<0x10000000225/MOVED.DOC v900>,<0x10000000009/PM v4952>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.624 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.198+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT auth v842 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.459579+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@60},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v2500>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v842>,<0x10000000009/PWRPNT v4950>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.625 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.198+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP auth v840 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.875069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@63},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v2500>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v840>,<0x10000000009/PWRPNT v4950>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.625 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.198+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT auth v844 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.886069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@60},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v2500>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v844>,<0x10000000009/PWRPNT v4950>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.625 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.199+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT auth v846 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.548576+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@60},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v2500>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v846>,<0x10000000009/PWRPNT v4950>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.625 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.199+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA auth v838 ap=1 s=260096 n(v0 rc2024-09-12T01:47:42.878069+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@63},l=15135(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v2500>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v838>,<0x10000000009/PWRPNT v4950>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.625 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T01:48:10.199+0000 7fd7a4ee1640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [2,head] /volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB auth v242 ap=1 s=260096 n(v0 rc2024-09-12T01:48:02.079588+0000 b260096 1=1+0) caps={15135=pAsLsXsFsc/-@25},l=15135(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56004ea6d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v94>,<0x10000000009/ACCESS v2388>,<0x10000000008/~dmtmp v36>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v12>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v28>,<0x10000000001/sv_1 v51>,<0x10000000000/qa v41>,<0x1/volumes v40>]//[]","memoryvalue":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v242>,<0x10000000009/ACCESS v4942>,<0x10000000008/~dmtmp v196>,<0x10000000007/client0 v64>,<0x10000000006/clients v27>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v24>,<0x100000001fa/7eec84f0-36f8-4977-b35c-6cefbfc555ff v43>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T01:48:10.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 25854 5.56 MB/sec warmup 28 sec latency 1083.694 ms 2024-09-12T01:48:10.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:10 smithi183 ceph-mon[35127]: scrub queued for path: / 2024-09-12T01:48:10.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:10 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:10.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:10 smithi183 ceph-mon[35127]: scrub summary: active paths [/] 2024-09-12T01:48:10.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:10 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:10 smithi122 ceph-mon[30925]: scrub queued for path: / 2024-09-12T01:48:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:10 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:10 smithi122 ceph-mon[30925]: scrub summary: active paths [/] 2024-09-12T01:48:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:10 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-mon[35229]: scrub queued for path: / 2024-09-12T01:48:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-mon[35229]: scrub summary: active paths [/] 2024-09-12T01:48:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:10 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 28966 6.00 MB/sec warmup 29 sec latency 11.166 ms 2024-09-12T01:48:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: scrub summary: active paths [/] 2024-09-12T01:48:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.001) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.004) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.002) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.003) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000000a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.000) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: scrub summary: active paths [/] 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:11 smithi183 ceph-mon[35127]: pgmap v386: 97 pgs: 97 active+clean; 17 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 757 KiB/s wr, 57 op/s 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: scrub summary: active paths [/] 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.001) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.004) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.002) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.003) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000000a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.000) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: scrub summary: active paths [/] 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.991 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:11.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:11.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:11 smithi122 ceph-mon[30925]: pgmap v386: 97 pgs: 97 active+clean; 17 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 757 KiB/s wr, 57 op/s 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: scrub summary: active paths [/] 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.001) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.004) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.002) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.003) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000000a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.000) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: scrub summary: active paths [/] 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000008 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000021b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000021a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000021e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000218 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000021c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000021d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000219 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000022c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000237 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000022f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000023c (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.120 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000023d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000022a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000238 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000230 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000022e (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.121 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.h log and `damage ls` output for details 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: scrub summary: idle+waiting paths [/] 2024-09-12T01:48:12.122 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:11 smithi151 ceph-mon[35229]: pgmap v386: 97 pgs: 97 active+clean; 17 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 757 KiB/s wr, 57 op/s 2024-09-12T01:48:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 -- 172.21.15.122:0/1746661215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb034102640 msgr2=0x7fb034104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 --2- 172.21.15.122:0/1746661215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb034102640 0x7fb034104a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb024009900 tx=0x7fb02402a410 comp rx=0 tx=0).stop 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 -- 172.21.15.122:0/1746661215 shutdown_connections 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 --2- 172.21.15.122:0/1746661215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0341078a0 0x7fb034109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 --2- 172.21.15.122:0/1746661215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb034104f70 0x7fb034107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 --2- 172.21.15.122:0/1746661215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb034102640 0x7fb034104a30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 -- 172.21.15.122:0/1746661215 >> 172.21.15.122:0/1746661215 conn(0x7fb0340fc1e0 msgr2=0x7fb0340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 -- 172.21.15.122:0/1746661215 shutdown_connections 2024-09-12T01:48:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.387+0000 7fb0388f1640 1 -- 172.21.15.122:0/1746661215 wait complete. 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 Processor -- start 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 -- start start 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb034102640 0x7fb034197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 0x7fb034198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 0x7fb03419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410d820 con 0x7fb0341078a0 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410d6a0 con 0x7fb034104f70 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb0388f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410d9a0 con 0x7fb034102640 2024-09-12T01:48:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb032d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 0x7fb03419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb031d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 0x7fb034198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb03259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb034102640 0x7fb034197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb032d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 0x7fb03419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44920/0 (socket says 172.21.15.122:44920) 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb032d9d640 1 -- 172.21.15.122:0/3370668822 learned_addr learned my addr 172.21.15.122:0/3370668822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.388+0000 7fb031d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 0x7fb034198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:41662/0 (socket says 172.21.15.122:41662) 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb031d9b640 1 -- 172.21.15.122:0/3370668822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb034102640 msgr2=0x7fb034197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb031d9b640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb034102640 0x7fb034197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb031d9b640 1 -- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 msgr2=0x7fb03419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb031d9b640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 0x7fb03419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb031d9b640 1 -- 172.21.15.122:0/3370668822 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03419fa70 con 0x7fb034104f70 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb032d9d640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 0x7fb03419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb031d9b640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 0x7fb034198390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9c0 tx=0x7fb02800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb0237fe640 1 -- 172.21.15.122:0/3370668822 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02800ccc0 con 0x7fb034104f70 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03419fd60 con 0x7fb034104f70 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb0237fe640 1 -- 172.21.15.122:0/3370668822 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb028005020 con 0x7fb034104f70 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb0237fe640 1 -- 172.21.15.122:0/3370668822 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028004a00 con 0x7fb034104f70 2024-09-12T01:48:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.389+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0341a0250 con 0x7fb034104f70 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.391+0000 7fb0237fe640 1 -- 172.21.15.122:0/3370668822 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb0280051d0 con 0x7fb034104f70 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 Processor -- start 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 -- start start 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aaf3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56402aa4f2f0 con 0x56402aaf1370 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56402a9f7270 con 0x56402aaeb9a0 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03b68f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56402aa4b2b0 con 0x56402aaec2e0 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb031d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aaf3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb032d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb032d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:41664/0 (socket says 172.21.15.122:41664) 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb032d9d640 1 -- 172.21.15.122:0/1012224210 learned_addr learned my addr 172.21.15.122:0/1012224210 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb03259c640 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb0237fe640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb01c0a2c30 0x7fb01c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.392+0000 7fb032d9d640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb01c0a2c30 0x7fb01c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb03259c640 1 -- 172.21.15.122:0/1012224210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 msgr2=0x56402aaebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb03259c640 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb03259c640 1 -- 172.21.15.122:0/1012224210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 msgr2=0x56402aaf3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb0237fe640 1 -- 172.21.15.122:0/3370668822 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb0280cef70 con 0x7fb034104f70 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb03259c640 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aaf3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb03259c640 1 -- 172.21.15.122:0/1012224210 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402a9f6340 con 0x56402aaec2e0 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb031d9b640 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aaf3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb03259c640 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf0da0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb024031440 tx=0x7fb024005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da355af160f94ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb032d9d640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb01c0a2c30 0x7fb01c0a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009fd0 tx=0x7fb02c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb020ff9640 1 -- 172.21.15.122:0/1012224210 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024040070 con 0x56402aaec2e0 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb020ff9640 1 -- 172.21.15.122:0/1012224210 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024004030 con 0x56402aaec2e0 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.393+0000 7fb020ff9640 1 -- 172.21.15.122:0/1012224210 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024035040 con 0x56402aaec2e0 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- 172.21.15.122:0/1012224210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 msgr2=0x56402aaf0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf0da0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb024031440 tx=0x7fb024005e00 comp rx=0 tx=0).stop 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- 172.21.15.122:0/1012224210 shutdown_connections 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aaf3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf0da0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1012224210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- 172.21.15.122:0/1012224210 >> 172.21.15.122:0/1012224210 conn(0x56402aae7010 msgr2=0x56402aae7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- 172.21.15.122:0/1012224210 shutdown_connections 2024-09-12T01:48:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- 172.21.15.122:0/1012224210 wait complete. 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- start start 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 Processor -- start 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aafd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56402aaf42e0 con 0x56402aaf1370 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56402a9d8290 con 0x56402aaeb9a0 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.394+0000 7fb03b68f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56402aaf4460 con 0x56402aaec2e0 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb03b68f740 10 client.0 _ll_register_callbacks cb 0x56402aa09140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aafd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aafd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44940/0 (socket says 172.21.15.122:44940) 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 -- 172.21.15.122:0/2313787680 learned_addr learned my addr 172.21.15.122:0/2313787680 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb03259c640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb031d9b640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 -- 172.21.15.122:0/2313787680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 msgr2=0x56402aaf60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 -- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 msgr2=0x56402aaf5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 -- 172.21.15.122:0/2313787680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402a762210 con 0x56402aaf1370 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb031d9b640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb03259c640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb032d9d640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aafd0f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009a10 tx=0x7fb02c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd24e2e9cccac8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb019ffb640 1 -- 172.21.15.122:0/2313787680 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c025070 con 0x56402aaf1370 2024-09-12T01:48:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb019ffb640 1 -- 172.21.15.122:0/2313787680 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb02c004d20 con 0x56402aaf1370 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.395+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb024004030 con 0x56402aaf1370 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb019ffb640 1 -- 172.21.15.122:0/2313787680 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb02c030050 con 0x56402aaf1370 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 10 client.15828 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56402a82e620 con 0x56402aaf1370 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb019ffb640 1 -- 172.21.15.122:0/2313787680 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024004030 con 0x56402aaf1370 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb019ffb640 1 -- 172.21.15.122:0/2313787680 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb02c01f040 con 0x56402aaf1370 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 10 client.15828 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 20 client.15828 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab018d0 0x56402ab03ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 4 client.15828 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56402aa5ca20 con 0x56402ab018d0 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.396+0000 7fb03259c640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab018d0 0x56402ab03ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.397+0000 7fb019ffb640 10 client.15828 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.397+0000 7fb03259c640 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab018d0 0x56402ab03ce0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b354f0d84e95eb server_cookie=66a4dafb07b76f16 in_seq=0 out_seq=0 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.397+0000 7fb019ffb640 1 -- 172.21.15.122:0/2313787680 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56402aa5ca20 con 0x56402ab018d0 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.397+0000 7fb019ffb640 10 client.15828 handle_command_reply: tid=0 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 client.15828 shutdown 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 msgr2=0x56402aafd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aafd0f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009a10 tx=0x7fb02c004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 shutdown_connections 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab018d0 0x56402ab03ce0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaf1370 0x56402aafd0f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aaf60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb019ffb640 0 client.15828 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 --2- 172.21.15.122:0/2313787680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 >> 172.21.15.122:0/2313787680 conn(0x56402aae7010 msgr2=0x56402aae8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 shutdown_connections 2024-09-12T01:48:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 1 -- 172.21.15.122:0/2313787680 wait complete. 2024-09-12T01:48:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.398+0000 7fb03b68f740 20 client.15828 trim_cache size 0 max 16384 2024-09-12T01:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 Processor -- start 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 -- start start 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 0x56402aaebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aafd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaf1370 0x56402aaf5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341ab850 con 0x56402aaeb9a0 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a9120 con 0x56402aaf1370 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03b68f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a6f50 con 0x56402aaec2e0 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 0x56402aaebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 0x56402aaebd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44950/0 (socket says 172.21.15.122:44950) 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb03259c640 1 -- 172.21.15.122:0/1940609056 learned_addr learned my addr 172.21.15.122:0/1940609056 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb031d9b640 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aafd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.413+0000 7fb032d9d640 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaf1370 0x56402aaf5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb03259c640 1 -- 172.21.15.122:0/1940609056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 msgr2=0x56402aafd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb03259c640 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aafd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb03259c640 1 -- 172.21.15.122:0/1940609056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaf1370 msgr2=0x56402aaf5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb03259c640 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaf1370 0x56402aaf5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb03259c640 1 -- 172.21.15.122:0/1940609056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402aae7ca0 con 0x56402aaeb9a0 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb032d9d640 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaf1370 0x56402aaf5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb03259c640 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 0x56402aaebd80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb024031440 tx=0x7fb0240315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c6474843cbbde7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb01bfff640 1 -- 172.21.15.122:0/1940609056 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024040050 con 0x56402aaeb9a0 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb01bfff640 1 -- 172.21.15.122:0/1940609056 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024032e50 con 0x56402aaeb9a0 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.414+0000 7fb01bfff640 1 -- 172.21.15.122:0/1940609056 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024035040 con 0x56402aaeb9a0 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- 172.21.15.122:0/1940609056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 msgr2=0x56402aaebd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 0x56402aaebd80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb024031440 tx=0x7fb0240315d0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- 172.21.15.122:0/1940609056 shutdown_connections 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaf1370 0x56402aaf5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaec2e0 0x56402aafd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- 172.21.15.122:0/1940609056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaeb9a0 0x56402aaebd80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- 172.21.15.122:0/1940609056 >> 172.21.15.122:0/1940609056 conn(0x56402aae7010 msgr2=0x56402aaed520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- 172.21.15.122:0/1940609056 shutdown_connections 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- 172.21.15.122:0/1940609056 wait complete. 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- start start 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 Processor -- start 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 0x56402aafa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaf1370 0x56402ab6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a75e0 con 0x56402aaec2e0 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a8a50 con 0x56402aaeb9a0 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a7cb0 con 0x56402aaf1370 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03b68f740 10 client.0 _ll_register_callbacks cb 0x7fb02c00fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb031d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 0x56402aafa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb031d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 0x56402aafa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44956/0 (socket says 172.21.15.122:44956) 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb031d9b640 1 -- 172.21.15.122:0/934165627 learned_addr learned my addr 172.21.15.122:0/934165627 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb03259c640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.415+0000 7fb032d9d640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaf1370 0x56402ab6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb031d9b640 1 -- 172.21.15.122:0/934165627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaf1370 msgr2=0x56402ab6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb031d9b640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaf1370 0x56402ab6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb031d9b640 1 -- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 msgr2=0x56402aaf9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb031d9b640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb031d9b640 1 -- 172.21.15.122:0/934165627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402ab6aaf0 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03259c640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb031d9b640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 0x56402aafa4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb0280981a0 tx=0x7fb02809b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29fcf2129332b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb019ffb640 1 -- 172.21.15.122:0/934165627 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02801c040 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb019ffb640 1 -- 172.21.15.122:0/934165627 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb02809ba30 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb024032e50 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb019ffb640 1 -- 172.21.15.122:0/934165627 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb02809fb80 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 10 client.15834 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56402ab6bea0 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb019ffb640 1 -- 172.21.15.122:0/934165627 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024032e50 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb019ffb640 1 -- 172.21.15.122:0/934165627 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb028004ba0 con 0x56402aaec2e0 2024-09-12T01:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 20 client.15834 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab6c890 0x56402ab6ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 4 client.15834 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.416+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56402aa56d70 con 0x56402ab6c890 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.417+0000 7fb03259c640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab6c890 0x56402ab6ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.417+0000 7fb019ffb640 10 client.15834 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.417+0000 7fb03259c640 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab6c890 0x56402ab6ecb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f7a8d314fc24c5 server_cookie=eda5616552a8ddbe in_seq=0 out_seq=0 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb019ffb640 1 -- 172.21.15.122:0/934165627 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56402aa56d70 con 0x56402ab6c890 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb019ffb640 10 client.15834 handle_command_reply: tid=0 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 client.15834 shutdown 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 msgr2=0x56402aafa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 0x56402aafa4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb0280981a0 tx=0x7fb02809b660 comp rx=0 tx=0).stop 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 shutdown_connections 2024-09-12T01:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56402ab6c890 0x56402ab6ecb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56402aaf1370 0x56402ab6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56402aaec2e0 0x56402aafa4b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb019ffb640 0 client.15834 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 --2- 172.21.15.122:0/934165627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56402aaeb9a0 0x56402aaf9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 >> 172.21.15.122:0/934165627 conn(0x56402aae7010 msgr2=0x56402aaf5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 shutdown_connections 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.429+0000 7fb03b68f740 1 -- 172.21.15.122:0/934165627 wait complete. 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb03b68f740 20 client.15834 trim_cache size 0 max 16384 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb01c0a2c30 msgr2=0x7fb01c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb01c0a2c30 0x7fb01c0a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009fd0 tx=0x7fb02c004ab0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 msgr2=0x7fb034198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 0x7fb034198390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9c0 tx=0x7fb02800ee80 comp rx=0 tx=0).stop 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 shutdown_connections 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb01c0a2c30 0x7fb01c0a50f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0341078a0 0x7fb03419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb034104f70 0x7fb034198390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 --2- 172.21.15.122:0/3370668822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb034102640 0x7fb034197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 >> 172.21.15.122:0/3370668822 conn(0x7fb0340fc1e0 msgr2=0x7fb034102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 shutdown_connections 2024-09-12T01:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:12.430+0000 7fb0388f1640 1 -- 172.21.15.122:0/3370668822 wait complete. 2024-09-12T01:48:12.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 32203 6.43 MB/sec warmup 30 sec latency 16.483 ms 2024-09-12T01:48:13.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 35461 6.79 MB/sec warmup 31 sec latency 12.016 ms 2024-09-12T01:48:13.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:48:13.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:13 smithi122 ceph-mon[30925]: pgmap v387: 97 pgs: 97 active+clean; 25 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 2.0 MiB/s wr, 80 op/s 2024-09-12T01:48:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:13 smithi183 ceph-mon[35127]: pgmap v387: 97 pgs: 97 active+clean; 25 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 2.0 MiB/s wr, 80 op/s 2024-09-12T01:48:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:13 smithi151 ceph-mon[35229]: pgmap v387: 97 pgs: 97 active+clean; 25 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 2.0 MiB/s wr, 80 op/s 2024-09-12T01:48:14.184 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:14.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.744+0000 7f7d8f537640 1 -- 172.21.15.122:0/103944111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 msgr2=0x7f7d881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.744+0000 7f7d8f537640 1 --2- 172.21.15.122:0/103944111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d881029b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009900 tx=0x7f7d7802a480 comp rx=0 tx=0).stop 2024-09-12T01:48:14.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.744+0000 7f7d8f537640 1 -- 172.21.15.122:0/103944111 shutdown_connections 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 --2- 172.21.15.122:0/103944111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88102ef0 0x7f7d8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 --2- 172.21.15.122:0/103944111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d881029b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 --2- 172.21.15.122:0/103944111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88100e20 0x7f7d88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 -- 172.21.15.122:0/103944111 >> 172.21.15.122:0/103944111 conn(0x7f7d880fc7d0 msgr2=0x7f7d880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 -- 172.21.15.122:0/103944111 shutdown_connections 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 -- 172.21.15.122:0/103944111 wait complete. 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 Processor -- start 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.745+0000 7f7d8f537640 1 -- start start 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88100e20 0x7f7d88198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d88198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88102ef0 0x7f7d8819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88110f50 con 0x7f7d88102570 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88110dd0 con 0x7f7d88100e20 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881110d0 con 0x7f7d88102ef0 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d88198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d88198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44974/0 (socket says 172.21.15.122:44974) 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 -- 172.21.15.122:0/3612031844 learned_addr learned my addr 172.21.15.122:0/3612031844 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88100e20 0x7f7d88198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88102ef0 0x7f7d8819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 -- 172.21.15.122:0/3612031844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88102ef0 msgr2=0x7f7d8819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88102ef0 0x7f7d8819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 -- 172.21.15.122:0/3612031844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88100e20 msgr2=0x7f7d88198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88100e20 0x7f7d88198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 -- 172.21.15.122:0/3612031844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8819d5b0 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d88198b40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7d78006010 tx=0x7f7d78003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d7e7fc640 1 -- 172.21.15.122:0/3612031844 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7803f070 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.746+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8819d840 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.747+0000 7f7d7e7fc640 1 -- 172.21.15.122:0/3612031844 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d78031c30 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.747+0000 7f7d7e7fc640 1 -- 172.21.15.122:0/3612031844 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78035030 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.747+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d881a5180 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.749+0000 7f7d7e7fc640 1 -- 172.21.15.122:0/3612031844 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7d78042070 con 0x7f7d88102570 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.749+0000 7f7d91910740 1 Processor -- start 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.749+0000 7f7d91910740 1 -- start start 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.749+0000 7f7d91910740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d504880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.749+0000 7f7d91910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d91910740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d511a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d91910740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bd1d46a2e0 con 0x55bd1d50f680 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d91910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bd1d411ce0 con 0x55bd1d504dc0 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d91910740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd1d4662a0 con 0x55bd1d5044a0 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d511a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:41718/0 (socket says 172.21.15.122:41718) 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/2967352038 learned_addr learned my addr 172.21.15.122:0/2967352038 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d511a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44978/0 (socket says 172.21.15.122:44978) 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d504880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d7e7fc640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/2967352038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 msgr2=0x55bd1d504880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d504880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/2967352038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 msgr2=0x55bd1d511a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d511a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/2967352038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1d3f6440 con 0x55bd1d504dc0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d7e7fc640 1 -- 172.21.15.122:0/3612031844 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7d780ef750 con 0x7f7d88102570 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.750+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50f140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400bcd0 tx=0x7f7d840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f948062369f9c299 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.751+0000 7f7d737fe640 1 -- 172.21.15.122:0/2967352038 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84005220 con 0x55bd1d504dc0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.751+0000 7f7d737fe640 1 -- 172.21.15.122:0/2967352038 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d84004c80 con 0x55bd1d504dc0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.751+0000 7f7d737fe640 1 -- 172.21.15.122:0/2967352038 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840117a0 con 0x55bd1d504dc0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.751+0000 7f7d91910740 1 -- 172.21.15.122:0/2967352038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 msgr2=0x55bd1d50f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.751+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7d740a2b80 0x7f7d740a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005fd0 tx=0x7f7d80004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.751+0000 7f7d91910740 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50f140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400bcd0 tx=0x7f7d840107a0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- 172.21.15.122:0/2967352038 shutdown_connections 2024-09-12T01:48:14.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d511a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50f140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 --2- 172.21.15.122:0/2967352038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d504880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- 172.21.15.122:0/2967352038 >> 172.21.15.122:0/2967352038 conn(0x55bd1d500a30 msgr2=0x55bd1d500e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- 172.21.15.122:0/2967352038 shutdown_connections 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- 172.21.15.122:0/2967352038 wait complete. 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- start start 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 Processor -- start 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d50eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d50c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bd1d516660 con 0x55bd1d50f680 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bd1d425050 con 0x55bd1d504dc0 2024-09-12T01:48:14.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.752+0000 7f7d91910740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd1d5167e0 con 0x55bd1d5044a0 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d91910740 10 client.0 _ll_register_callbacks cb 0x55bd1d465160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d50c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d50c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44982/0 (socket says 172.21.15.122:44982) 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8d2d3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d50eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3400671864 learned_addr learned my addr 172.21.15.122:0/3400671864 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3400671864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 msgr2=0x55bd1d50eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d50eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 msgr2=0x55bd1d50c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3400671864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1d1db3c0 con 0x55bd1d50f680 2024-09-12T01:48:14.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d50c800 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7d80013410 tx=0x7f7d80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf1774c8ddb6033 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3400671864 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800108e0 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3400671864 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7d80010a80 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d84004c80 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.753+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3400671864 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d80030050 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 10 client.15846 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd1d522090 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3400671864 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d84004c80 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3400671864 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7d80020400 con 0x55bd1d50f680 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:14.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 20 client.15846 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d5235a0 0x55bd1d525a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 4 client.15846 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd1d477850 con 0x55bd1d5235a0 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d5235a0 0x55bd1d525a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d70ff9640 10 client.15846 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.754+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d5235a0 0x55bd1d525a10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0ed4ce9d32f47d server_cookie=c5b184187bec13db in_seq=0 out_seq=0 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3400671864 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd1d477850 con 0x55bd1d5235a0 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d70ff9640 10 client.15846 handle_command_reply: tid=0 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 client.15846 shutdown 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 msgr2=0x55bd1d50c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d50c800 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7d80013410 tx=0x7f7d80004ab0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 shutdown_connections 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d5235a0 0x55bd1d525a10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d50c800 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 --2- 172.21.15.122:0/3400671864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d50eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d70ff9640 0 client.15846 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.755+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 >> 172.21.15.122:0/3400671864 conn(0x55bd1d500a30 msgr2=0x55bd1d506710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.756+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 shutdown_connections 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.756+0000 7f7d91910740 1 -- 172.21.15.122:0/3400671864 wait complete. 2024-09-12T01:48:14.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.756+0000 7f7d91910740 20 client.15846 trim_cache size 0 max 16384 2024-09-12T01:48:14.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.774+0000 7f7d91910740 1 Processor -- start 2024-09-12T01:48:14.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.774+0000 7f7d91910740 1 -- start start 2024-09-12T01:48:14.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d91910740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d51f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d91910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d91910740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d52b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d91910740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881add00 con 0x55bd1d50f680 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d91910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881ab5d0 con 0x55bd1d504dc0 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d91910740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881a9420 con 0x55bd1d5044a0 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d52b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d52b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44998/0 (socket says 172.21.15.122:44998) 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3028038596 learned_addr learned my addr 172.21.15.122:0/3028038596 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d51f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3028038596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 msgr2=0x55bd1d51f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d51f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3028038596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 msgr2=0x55bd1d50ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 -- 172.21.15.122:0/3028038596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1d50b250 con 0x55bd1d50f680 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d8dad4640 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d52b2e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d80020820 tx=0x7f7d80005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c00d8e7473dea44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.775+0000 7f7d72ffd640 1 -- 172.21.15.122:0/3028038596 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8003a070 con 0x55bd1d50f680 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d72ffd640 1 -- 172.21.15.122:0/3028038596 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d80025020 con 0x55bd1d50f680 2024-09-12T01:48:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d72ffd640 1 -- 172.21.15.122:0/3028038596 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80024020 con 0x55bd1d50f680 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 -- 172.21.15.122:0/3028038596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 msgr2=0x55bd1d52b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d52b2e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d80020820 tx=0x7f7d80005cb0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 -- 172.21.15.122:0/3028038596 shutdown_connections 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d50f680 0x55bd1d52b2e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d504dc0 0x55bd1d50ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 --2- 172.21.15.122:0/3028038596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d51f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.776+0000 7f7d91910740 1 -- 172.21.15.122:0/3028038596 >> 172.21.15.122:0/3028038596 conn(0x55bd1d500a30 msgr2=0x55bd1d50e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 -- 172.21.15.122:0/3028038596 shutdown_connections 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 -- 172.21.15.122:0/3028038596 wait complete. 2024-09-12T01:48:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 -- start start 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 Processor -- start 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d517760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d504dc0 0x55bd1d517ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d50f680 0x55bd1d5181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881a9a90 con 0x55bd1d504dc0 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881aaf00 con 0x55bd1d50f680 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.777+0000 7f7d91910740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d881aa160 con 0x55bd1d5044a0 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d91910740 10 client.0 _ll_register_callbacks cb 0x7f7d80009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d504dc0 0x55bd1d517ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d517760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d50f680 0x55bd1d5181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8cad2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d504dc0 0x55bd1d517ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45006/0 (socket says 172.21.15.122:45006) 2024-09-12T01:48:14.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d517760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:54704/0 (socket says 172.21.15.122:54704) 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8cad2640 1 -- 172.21.15.122:0/3613699766 learned_addr learned my addr 172.21.15.122:0/3613699766 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d50f680 msgr2=0x55bd1d5181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d50f680 0x55bd1d5181e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/3613699766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d504dc0 msgr2=0x55bd1d517ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d504dc0 0x55bd1d517ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 -- 172.21.15.122:0/3613699766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1d50f360 con 0x55bd1d5044a0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d8d2d3640 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d517760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400bca0 tx=0x7f7d84005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b2be0efced57b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3613699766 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84028860 con 0x55bd1d5044a0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d80025020 con 0x55bd1d5044a0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.778+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3613699766 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7d84028a00 con 0x55bd1d5044a0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3613699766 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d84028c00 con 0x55bd1d5044a0 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 10 client.25231 fetch_fsmap learned FSMap version 27 2024-09-12T01:48:14.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd1d58e3a0 con 0x55bd1d5044a0 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3613699766 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d80025020 con 0x55bd1d5044a0 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3613699766 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7d8401b430 con 0x55bd1d5044a0 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 10 client.25231 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 20 client.25231 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d58e840 0x55bd1d590c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 4 client.25231 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd1d471d60 con 0x55bd1d58e840 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d58e840 0x55bd1d590c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d8cad2640 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d58e840 0x55bd1d590c60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519e3b140ff9aa19 server_cookie=93f03d9e32fc4e9c in_seq=0 out_seq=0 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.779+0000 7f7d70ff9640 10 client.25231 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d70ff9640 1 -- 172.21.15.122:0/3613699766 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bd1d471d60 con 0x55bd1d58e840 2024-09-12T01:48:14.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d70ff9640 10 client.25231 handle_command_reply: tid=0 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 client.25231 shutdown 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 msgr2=0x55bd1d517760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d517760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400bca0 tx=0x7f7d84005850 comp rx=0 tx=0).stop 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 shutdown_connections 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bd1d58e840 0x55bd1d590c60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bd1d50f680 0x55bd1d5181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd1d504dc0 0x55bd1d517ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 --2- 172.21.15.122:0/3613699766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd1d5044a0 0x55bd1d517760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d70ff9640 0 client.25231 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 >> 172.21.15.122:0/3613699766 conn(0x55bd1d500a30 msgr2=0x55bd1d50e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 shutdown_connections 2024-09-12T01:48:14.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 1 -- 172.21.15.122:0/3613699766 wait complete. 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.780+0000 7f7d91910740 20 client.25231 trim_cache size 0 max 16384 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: "28c02e45-7f8a-4b57-bcef-5f6c3fc7c2f9": { 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "28c02e45-7f8a-4b57-bcef-5f6c3fc7c2f9", 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-09-12T01:48:14.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:14.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:14.787 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:48:14.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7d740a2b80 msgr2=0x7f7d740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7d740a2b80 0x7f7d740a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005fd0 tx=0x7f7d80004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 msgr2=0x7f7d88198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d88198b40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7d78006010 tx=0x7f7d78003c10 comp rx=0 tx=0).stop 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 shutdown_connections 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d88102ef0 0x7f7d8819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d88102570 0x7f7d88198b40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 --2- 172.21.15.122:0/3612031844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d88100e20 0x7f7d88198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 >> 172.21.15.122:0/3612031844 conn(0x7f7d880fc7d0 msgr2=0x7f7d880fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 shutdown_connections 2024-09-12T01:48:14.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:14.781+0000 7f7d8f537640 1 -- 172.21.15.122:0/3612031844 wait complete. 2024-09-12T01:48:14.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 38006 6.87 MB/sec warmup 32 sec latency 34.202 ms 2024-09-12T01:48:15.736 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:15 smithi122 ceph-mon[30925]: pgmap v388: 97 pgs: 97 active+clean; 36 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.8 MiB/s wr, 163 op/s 2024-09-12T01:48:15.765 INFO:tasks.cephfs.filesystem:scrub status for tag:28c02e45-7f8a-4b57-bcef-5f6c3fc7c2f9 - {'path': '/', 'tag': '28c02e45-7f8a-4b57-bcef-5f6c3fc7c2f9', 'options': 'recursive,force'} 2024-09-12T01:48:15.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 40459 7.16 MB/sec warmup 33 sec latency 14.459 ms 2024-09-12T01:48:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:15 smithi183 ceph-mon[35127]: pgmap v388: 97 pgs: 97 active+clean; 36 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.8 MiB/s wr, 163 op/s 2024-09-12T01:48:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:15 smithi151 ceph-mon[35229]: pgmap v388: 97 pgs: 97 active+clean; 36 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.8 MiB/s wr, 163 op/s 2024-09-12T01:48:16.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 43217 7.37 MB/sec warmup 34 sec latency 13.905 ms 2024-09-12T01:48:17.779 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T01:48:17.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:48:17.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 46540 7.70 MB/sec warmup 35 sec latency 13.762 ms 2024-09-12T01:48:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:17 smithi183 ceph-mon[35127]: pgmap v389: 97 pgs: 97 active+clean; 30 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 160 op/s 2024-09-12T01:48:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:17 smithi122 ceph-mon[30925]: pgmap v389: 97 pgs: 97 active+clean; 30 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 160 op/s 2024-09-12T01:48:18.048 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:17 smithi151 ceph-mon[35229]: pgmap v389: 97 pgs: 97 active+clean; 30 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 160 op/s 2024-09-12T01:48:18.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 -- 172.21.15.122:0/2462420920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc108290 msgr2=0x7f00cc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 --2- 172.21.15.122:0/2462420920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc108290 0x7f00cc10ec30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009cd0 tx=0x7f00c402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 -- 172.21.15.122:0/2462420920 shutdown_connections 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 --2- 172.21.15.122:0/2462420920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc108290 0x7f00cc10ec30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 --2- 172.21.15.122:0/2462420920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00cc103740 0x7f00cc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 --2- 172.21.15.122:0/2462420920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00cc102d90 0x7f00cc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 -- 172.21.15.122:0/2462420920 >> 172.21.15.122:0/2462420920 conn(0x7f00cc0fc7d0 msgr2=0x7f00cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 -- 172.21.15.122:0/2462420920 shutdown_connections 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 -- 172.21.15.122:0/2462420920 wait complete. 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 Processor -- start 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.540+0000 7f00d27a1640 1 -- start start 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 0x7f00cc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:18.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00cc103740 0x7f00cc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00cc108290 0x7f00cc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc112090 con 0x7f00cc102d90 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc111f10 con 0x7f00cc108290 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc112210 con 0x7f00cc103740 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 0x7f00cc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 0x7f00cc198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45010/0 (socket says 172.21.15.122:45010) 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d0d3e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00cc108290 0x7f00cc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 -- 172.21.15.122:0/697355800 learned_addr learned my addr 172.21.15.122:0/697355800 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cb7fe640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00cc103740 0x7f00cc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 -- 172.21.15.122:0/697355800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00cc103740 msgr2=0x7f00cc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00cc103740 0x7f00cc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 -- 172.21.15.122:0/697355800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00cc108290 msgr2=0x7f00cc19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:18.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00cc108290 0x7f00cc19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 -- 172.21.15.122:0/697355800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc19d500 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00cbfff640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 0x7f00cc198550 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00e9f0 tx=0x7f00bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00cd80 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00bc010910 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc018990 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.541+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc19d7f0 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.542+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a50d0 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.543+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc103170 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.544+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f00bc010ac0 con 0x7f00cc102d90 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.544+0000 7f00c97fa640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f00b40a2b80 0x7f00b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.544+0000 7f00cb7fe640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f00b40a2b80 0x7f00b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.544+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f00bc0cf400 con 0x7f00cc102d90 2024-09-12T01:48:18.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.545+0000 7f00cb7fe640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f00b40a2b80 0x7f00b40a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0feda0 tx=0x7f00c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:18.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.549+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00bc09c360 con 0x7f00cc102d90 2024-09-12T01:48:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T01:48:18.665+0000 7f42afbd7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T01:48:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:18 smithi122 ceph-mon[30925]: pgmap v390: 97 pgs: 97 active+clean; 34 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-09-12T01:48:18.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.775+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00cc10a2a0 con 0x7f00cc102d90 2024-09-12T01:48:18.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.776+0000 7f00c97fa640 1 -- 172.21.15.122:0/697355800 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11934 (secure 0 0 0) 0x7f00bc09bc60 con 0x7f00cc102d90 2024-09-12T01:48:18.776 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:48:18.776 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"btime":"2024-09-12T01:48:18:666269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:48:17.839016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:48:18.776 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f00b40a2b80 msgr2=0x7f00b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f00b40a2b80 0x7f00b40a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0feda0 tx=0x7f00c0005f90 comp rx=0 tx=0).stop 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 msgr2=0x7f00cc198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 0x7f00cc198550 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00e9f0 tx=0x7f00bc00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 shutdown_connections 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f00b40a2b80 0x7f00b40a5040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00cc108290 0x7f00cc19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00cc103740 0x7f00cc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 --2- 172.21.15.122:0/697355800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc102d90 0x7f00cc198550 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:18.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 >> 172.21.15.122:0/697355800 conn(0x7f00cc0fc7d0 msgr2=0x7f00cc106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:18.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 shutdown_connections 2024-09-12T01:48:18.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:18.778+0000 7f00d27a1640 1 -- 172.21.15.122:0/697355800 wait complete. 2024-09-12T01:48:18.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 49653 7.97 MB/sec warmup 36 sec latency 33.271 ms 2024-09-12T01:48:18.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:18 smithi183 ceph-mon[35127]: pgmap v390: 97 pgs: 97 active+clean; 34 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-09-12T01:48:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:18 smithi151 ceph-mon[35229]: pgmap v390: 97 pgs: 97 active+clean; 34 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-09-12T01:48:19.510 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T01:48:19.511 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T01:48:19.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:19 smithi122 ceph-mon[30925]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T01:48:19.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:19 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:active 2024-09-12T01:48:19.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:19 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:48:19.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:19 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/697355800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:19.814 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:19.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 52665 8.14 MB/sec warmup 37 sec latency 10.075 ms 2024-09-12T01:48:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:19 smithi183 ceph-mon[35127]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T01:48:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:19 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:active 2024-09-12T01:48:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:19 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:48:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:19 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/697355800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:19 smithi151 ceph-mon[35229]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T01:48:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:19 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] up:active 2024-09-12T01:48:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:19 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:48:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:19 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/697355800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 -- 172.21.15.122:0/597379355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c108300 msgr2=0x7f306c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 --2- 172.21.15.122:0/597379355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c108300 0x7f306c10ec70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3064009cd0 tx=0x7f306402a8d0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 -- 172.21.15.122:0/597379355 shutdown_connections 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 --2- 172.21.15.122:0/597379355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c108300 0x7f306c10ec70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 --2- 172.21.15.122:0/597379355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 0x7f306c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 --2- 172.21.15.122:0/597379355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f306c102dd0 0x7f306c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 -- 172.21.15.122:0/597379355 >> 172.21.15.122:0/597379355 conn(0x7f306c0fc7d0 msgr2=0x7f306c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 -- 172.21.15.122:0/597379355 shutdown_connections 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.303+0000 7f3070ce8640 1 -- 172.21.15.122:0/597379355 wait complete. 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 Processor -- start 2024-09-12T01:48:20.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 -- start start 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 0x7f306c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 0x7f306c100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f306c108300 0x7f306c1014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1120d0 con 0x7f306c102dd0 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f306c111f50 con 0x7f306c103780 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.304+0000 7f3070ce8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f306c112250 con 0x7f306c108300 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 0x7f306c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 0x7f306c100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53906/0 (socket says 172.21.15.122:53906) 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 -- 172.21.15.122:0/3213119782 learned_addr learned my addr 172.21.15.122:0/3213119782 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f3069d9b640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 0x7f306c100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306ad9d640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f306c108300 0x7f306c1014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 -- 172.21.15.122:0/3213119782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f306c108300 msgr2=0x7f306c1014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f306c108300 0x7f306c1014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 -- 172.21.15.122:0/3213119782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 msgr2=0x7f306c100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 0x7f306c100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 -- 172.21.15.122:0/3213119782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c1a6a10 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f3069d9b640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 0x7f306c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f306a59c640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 0x7f306c100a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f305c00d950 tx=0x7f305c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f305b7fe640 1 -- 172.21.15.122:0/3213119782 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c017070 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f305b7fe640 1 -- 172.21.15.122:0/3213119782 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c009c30 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c1a6d00 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f305b7fe640 1 -- 172.21.15.122:0/3213119782 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c00f6b0 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.305+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1a7230 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.307+0000 7f305b7fe640 1 -- 172.21.15.122:0/3213119782 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f305c0109d0 con 0x7f306c102dd0 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 Processor -- start 2024-09-12T01:48:20.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 -- start start 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f305b7fe640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30540a2c00 0x7f30540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cd8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6c352a0 con 0x55e5d6ce3150 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6bdcb10 con 0x55e5d6cd8890 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f30730c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6c31260 con 0x55e5d6cd7f70 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f306ad9d640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30540a2c00 0x7f30540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.308+0000 7f305b7fe640 1 -- 172.21.15.122:0/3213119782 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f305c0ceed0 con 0x7f306c102dd0 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:42164/0 (socket says 172.21.15.122:42164) 2024-09-12T01:48:20.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 -- 172.21.15.122:0/3296386926 learned_addr learned my addr 172.21.15.122:0/3296386926 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f306a59c640 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cd8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 -- 172.21.15.122:0/3296386926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 msgr2=0x55e5d6cd8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cd8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 -- 172.21.15.122:0/3296386926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 msgr2=0x55e5d6ce5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 -- 172.21.15.122:0/3296386926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d69a6b90 con 0x55e5d6cd8890 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f306ad9d640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30540a2c00 0x7f30540a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3064006010 tx=0x7f3064009a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3069d9b640 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce2c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3060009fd0 tx=0x7f30600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1997f74c2d085a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3058ff9640 1 -- 172.21.15.122:0/3296386926 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060004230 con 0x55e5d6cd8890 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3058ff9640 1 -- 172.21.15.122:0/3296386926 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30600043d0 con 0x55e5d6cd8890 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.309+0000 7f3058ff9640 1 -- 172.21.15.122:0/3296386926 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306000faa0 con 0x55e5d6cd8890 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 -- 172.21.15.122:0/3296386926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 msgr2=0x55e5d6ce2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce2c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3060009fd0 tx=0x7f30600118d0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 -- 172.21.15.122:0/3296386926 shutdown_connections 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce2c10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 --2- 172.21.15.122:0/3296386926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cd8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 -- 172.21.15.122:0/3296386926 >> 172.21.15.122:0/3296386926 conn(0x55e5d6cd4500 msgr2=0x55e5d6cd48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 -- 172.21.15.122:0/3296386926 shutdown_connections 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.310+0000 7f30730c1740 1 -- 172.21.15.122:0/3296386926 wait complete. 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 -- start start 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 Processor -- start 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cdff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6ce5d20 con 0x55e5d6ce3150 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6bbd6e0 con 0x55e5d6cd7f70 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6ce5ea0 con 0x55e5d6cd8890 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f30730c1740 10 client.0 _ll_register_callbacks cb 0x55e5d6be1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f306a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cdff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53936/0 (socket says 172.21.15.122:53936) 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f306ad9d640 1 -- 172.21.15.122:0/2121856898 learned_addr learned my addr 172.21.15.122:0/2121856898 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f3069d9b640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f306ad9d640 1 -- 172.21.15.122:0/2121856898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 msgr2=0x55e5d6ce04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.311+0000 7f306ad9d640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f306ad9d640 1 -- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 msgr2=0x55e5d6cdff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f306ad9d640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cdff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f306ad9d640 1 -- 172.21.15.122:0/2121856898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d6cef190 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f306a59c640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cdff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f306ad9d640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce0a00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3064005fc0 tx=0x7f306402a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856ec9b3433f3b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f3051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f3051ffb640 1 -- 172.21.15.122:0/2121856898 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306404f070 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f3051ffb640 1 -- 172.21.15.122:0/2121856898 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3064004040 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30600043d0 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f3051ffb640 1 -- 172.21.15.122:0/2121856898 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3064003e40 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f30730c1740 10 client.15870 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5d6cf1af0 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.312+0000 7f3051ffb640 1 -- 172.21.15.122:0/2121856898 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30600043d0 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f3051ffb640 1 -- 172.21.15.122:0/2121856898 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3064030590 con 0x55e5d6ce3150 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f30730c1740 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f30730c1740 10 client.15870 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:48:20.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f30730c1740 20 client.15870 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:20.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f30730c1740 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6cf2fb0 0x55e5d6cf5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f30730c1740 4 client.15870 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:20.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d6c429d0 con 0x55e5d6cf2fb0 2024-09-12T01:48:20.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f306a59c640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6cf2fb0 0x55e5d6cf5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f306a59c640 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6cf2fb0 0x55e5d6cf5420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4b0dd481c72024 server_cookie=cea38dc36cd56976 in_seq=0 out_seq=0 2024-09-12T01:48:20.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.313+0000 7f3051ffb640 10 client.15870 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:20.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f3051ffb640 1 -- 172.21.15.122:0/2121856898 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5d6c429d0 con 0x55e5d6cf2fb0 2024-09-12T01:48:20.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f3051ffb640 10 client.15870 handle_command_reply: tid=0 2024-09-12T01:48:20.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 client.15870 shutdown 2024-09-12T01:48:20.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 msgr2=0x55e5d6ce0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce0a00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3064005fc0 tx=0x7f306402a8d0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 shutdown_connections 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6cf2fb0 0x55e5d6cf5420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce0a00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 --2- 172.21.15.122:0/2121856898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cdff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f3051ffb640 0 client.15870 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 >> 172.21.15.122:0/2121856898 conn(0x55e5d6cd4500 msgr2=0x55e5d6ccae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 shutdown_connections 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 1 -- 172.21.15.122:0/2121856898 wait complete. 2024-09-12T01:48:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.314+0000 7f30730c1740 20 client.15870 trim_cache size 0 max 16384 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 Processor -- start 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 -- start start 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cda9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6cfacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1adca0 con 0x55e5d6ce3150 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1ab570 con 0x55e5d6cd7f70 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f30730c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1a93a0 con 0x55e5d6cd8890 2024-09-12T01:48:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6cfacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6cfacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53938/0 (socket says 172.21.15.122:53938) 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f306ad9d640 1 -- 172.21.15.122:0/3717216723 learned_addr learned my addr 172.21.15.122:0/3717216723 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f306a59c640 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cda9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.332+0000 7f3069d9b640 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306ad9d640 1 -- 172.21.15.122:0/3717216723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 msgr2=0x55e5d6ce1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306ad9d640 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306ad9d640 1 -- 172.21.15.122:0/3717216723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 msgr2=0x55e5d6cda9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306ad9d640 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cda9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306ad9d640 1 -- 172.21.15.122:0/3717216723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d6cd4330 con 0x55e5d6ce3150 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306a59c640 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cda9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f306ad9d640 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6cfacf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3064009430 tx=0x7f30640318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d160f37f244f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f3053fff640 1 -- 172.21.15.122:0/3717216723 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306404f050 con 0x55e5d6ce3150 2024-09-12T01:48:20.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f3053fff640 1 -- 172.21.15.122:0/3717216723 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064063070 con 0x55e5d6ce3150 2024-09-12T01:48:20.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.333+0000 7f3053fff640 1 -- 172.21.15.122:0/3717216723 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064004860 con 0x55e5d6ce3150 2024-09-12T01:48:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- 172.21.15.122:0/3717216723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 msgr2=0x55e5d6cfacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6cfacf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3064009430 tx=0x7f30640318f0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- 172.21.15.122:0/3717216723 shutdown_connections 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6ce3150 0x55e5d6cfacf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- 172.21.15.122:0/3717216723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6cd7f70 0x55e5d6cda9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- 172.21.15.122:0/3717216723 >> 172.21.15.122:0/3717216723 conn(0x55e5d6cd4500 msgr2=0x55e5d6ccae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- 172.21.15.122:0/3717216723 shutdown_connections 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- 172.21.15.122:0/3717216723 wait complete. 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- start start 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 Processor -- start 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6ceb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1a9a30 con 0x55e5d6cd8890 2024-09-12T01:48:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1aaea0 con 0x55e5d6ce3150 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1aa100 con 0x55e5d6cd7f70 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f30730c1740 10 client.0 _ll_register_callbacks cb 0x7f3064031d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:42180/0 (socket says 172.21.15.122:42180) 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53942/0 (socket says 172.21.15.122:53942) 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.334+0000 7f306ad9d640 1 -- 172.21.15.122:0/1302100950 learned_addr learned my addr 172.21.15.122:0/1302100950 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306a59c640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6ceb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306ad9d640 1 -- 172.21.15.122:0/1302100950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 msgr2=0x55e5d6ceb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306ad9d640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6ceb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306ad9d640 1 -- 172.21.15.122:0/1302100950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 msgr2=0x55e5d6ce80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306ad9d640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306ad9d640 1 -- 172.21.15.122:0/1302100950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d6cc7260 con 0x55e5d6ce3150 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f3069d9b640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f306ad9d640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce8600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3064004930 tx=0x7f306402a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8239b5bf32954 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f3051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f3051ffb640 1 -- 172.21.15.122:0/1302100950 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306406c070 con 0x55e5d6ce3150 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3064063070 con 0x55e5d6ce3150 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f3051ffb640 1 -- 172.21.15.122:0/1302100950 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3064004000 con 0x55e5d6ce3150 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f3051ffb640 1 -- 172.21.15.122:0/1302100950 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3064078050 con 0x55e5d6ce3150 2024-09-12T01:48:20.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f30730c1740 10 client.25256 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5d6d5e910 con 0x55e5d6ce3150 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.335+0000 7f3051ffb640 1 -- 172.21.15.122:0/1302100950 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064063070 con 0x55e5d6ce3150 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f3051ffb640 1 -- 172.21.15.122:0/1302100950 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3064068460 con 0x55e5d6ce3150 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f30730c1740 10 client.25256 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f30730c1740 10 client.25256 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f30730c1740 20 client.25256 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f30730c1740 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6d5f020 0x55e5d6d61440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f30730c1740 4 client.25256 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55e5d6c3cd20 con 0x55e5d6d5f020 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f306a59c640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6d5f020 0x55e5d6d61440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f3051ffb640 10 client.25256 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.336+0000 7f306a59c640 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6d5f020 0x55e5d6d61440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5137b971fe024adc server_cookie=acdff97b156acd12 in_seq=0 out_seq=0 2024-09-12T01:48:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f3051ffb640 1 -- 172.21.15.122:0/1302100950 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e5d6c3cd20 con 0x55e5d6d5f020 2024-09-12T01:48:20.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f3051ffb640 10 client.25256 handle_command_reply: tid=0 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 client.25256 shutdown 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 msgr2=0x55e5d6ce8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce8600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3064004930 tx=0x7f306402a8d0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 shutdown_connections 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5d6d5f020 0x55e5d6d61440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d6ce3150 0x55e5d6ce8600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5d6cd8890 0x55e5d6ce80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f3051ffb640 0 client.25256 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 --2- 172.21.15.122:0/1302100950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5d6cd7f70 0x55e5d6ceb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 >> 172.21.15.122:0/1302100950 conn(0x55e5d6cd4500 msgr2=0x55e5d6cdff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 shutdown_connections 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 1 -- 172.21.15.122:0/1302100950 wait complete. 2024-09-12T01:48:20.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.430+0000 7f30730c1740 20 client.25256 trim_cache size 0 max 16384 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 28, 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 6, 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "7f4985a4": { 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 6, 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:48:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 133.4, 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:48:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:20.434 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:48:20.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30540a2c00 msgr2=0x7f30540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30540a2c00 0x7f30540a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3064006010 tx=0x7f3064009a10 comp rx=0 tx=0).stop 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 msgr2=0x7f306c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 0x7f306c100a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f305c00d950 tx=0x7f305c00de10 comp rx=0 tx=0).stop 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 shutdown_connections 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f30540a2c00 0x7f30540a50c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f306c108300 0x7f306c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f306c103780 0x7f306c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 --2- 172.21.15.122:0/3213119782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f306c102dd0 0x7f306c100a30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 >> 172.21.15.122:0/3213119782 conn(0x7f306c0fc7d0 msgr2=0x7f306c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 shutdown_connections 2024-09-12T01:48:20.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:20.431+0000 7f3070ce8640 1 -- 172.21.15.122:0/3213119782 wait complete. 2024-09-12T01:48:20.815 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 8.10 MB/sec warmup 38 sec latency 395.083 ms 2024-09-12T01:48:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:20 smithi183 ceph-mon[35127]: pgmap v391: 97 pgs: 97 active+clean; 35 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 242 op/s 2024-09-12T01:48:20.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:20 smithi122 ceph-mon[30925]: pgmap v391: 97 pgs: 97 active+clean; 35 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 242 op/s 2024-09-12T01:48:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:20 smithi151 ceph-mon[35229]: pgmap v391: 97 pgs: 97 active+clean; 35 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 242 op/s 2024-09-12T01:48:21.143 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7f4985a4', quiesce took 3.4 seconds. Will release after: 10.0 2024-09-12T01:48:21.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 7.89 MB/sec warmup 39 sec latency 1395.146 ms 2024-09-12T01:48:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:21 smithi183 ceph-mon[35127]: scrub summary: idle 2024-09-12T01:48:21.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:21 smithi122 ceph-mon[30925]: scrub summary: idle 2024-09-12T01:48:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:21 smithi151 ceph-mon[35229]: scrub summary: idle 2024-09-12T01:48:22.764 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:22 smithi122 ceph-mon[30925]: scrub summary: idle 2024-09-12T01:48:22.764 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:22 smithi122 ceph-mon[30925]: scrub summary: idle 2024-09-12T01:48:22.764 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:22 smithi122 ceph-mon[30925]: pgmap v392: 97 pgs: 97 active+clean; 35 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 246 op/s 2024-09-12T01:48:22.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 7.70 MB/sec warmup 40 sec latency 2395.203 ms 2024-09-12T01:48:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:22 smithi183 ceph-mon[35127]: scrub summary: idle 2024-09-12T01:48:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:22 smithi183 ceph-mon[35127]: scrub summary: idle 2024-09-12T01:48:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:22 smithi183 ceph-mon[35127]: pgmap v392: 97 pgs: 97 active+clean; 35 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 246 op/s 2024-09-12T01:48:23.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:22 smithi151 ceph-mon[35229]: scrub summary: idle 2024-09-12T01:48:23.017 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:22 smithi151 ceph-mon[35229]: scrub summary: idle 2024-09-12T01:48:23.017 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:22 smithi151 ceph-mon[35229]: pgmap v392: 97 pgs: 97 active+clean; 35 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 246 op/s 2024-09-12T01:48:23.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 7.51 MB/sec warmup 41 sec latency 3395.279 ms 2024-09-12T01:48:24.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 7.33 MB/sec warmup 42 sec latency 4395.320 ms 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 -- 172.21.15.122:0/2188572712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4103710 msgr2=0x7fadc4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 --2- 172.21.15.122:0/2188572712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4103710 0x7fadc4109fa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009ca0 tx=0x7fadbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 -- 172.21.15.122:0/2188572712 shutdown_connections 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 --2- 172.21.15.122:0/2188572712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4103710 0x7fadc4109fa0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 --2- 172.21.15.122:0/2188572712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadc4102d50 0x7fadc41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 --2- 172.21.15.122:0/2188572712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc4101b60 0x7fadc4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 -- 172.21.15.122:0/2188572712 >> 172.21.15.122:0/2188572712 conn(0x7fadc40fd2e0 msgr2=0x7fadc40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 -- 172.21.15.122:0/2188572712 shutdown_connections 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 -- 172.21.15.122:0/2188572712 wait complete. 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.526+0000 7fadca053640 1 Processor -- start 2024-09-12T01:48:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 -- start start 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc4101b60 0x7fadc4075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 0x7fadc40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadc4103710 0x7fadc4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadc410d7e0 con 0x7fadc4102d50 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadc410d660 con 0x7fadc4101b60 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadca053640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadc410d960 con 0x7fadc4103710 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 0x7fadc40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 0x7fadc40745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53962/0 (socket says 172.21.15.122:53962) 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1855292376 learned_addr learned my addr 172.21.15.122:0/1855292376 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc4101b60 0x7fadc4075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadc4103710 0x7fadc4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1855292376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadc4103710 msgr2=0x7fadc4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadc4103710 0x7fadc4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1855292376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc4101b60 msgr2=0x7fadc4075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc4101b60 0x7fadc4075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1855292376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc40753b0 con 0x7fadc4102d50 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.527+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 0x7fadc40745e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d770 tx=0x7fadb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.528+0000 7fadc0ff9640 1 -- 172.21.15.122:0/1855292376 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8017070 con 0x7fadc4102d50 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.528+0000 7fadc0ff9640 1 -- 172.21.15.122:0/1855292376 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb8005500 con 0x7fadc4102d50 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.528+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41a1ec0 con 0x7fadc4102d50 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.528+0000 7fadc0ff9640 1 -- 172.21.15.122:0/1855292376 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8004e00 con 0x7fadc4102d50 2024-09-12T01:48:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.528+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a22d0 con 0x7fadc4102d50 2024-09-12T01:48:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.530+0000 7fadc0ff9640 1 -- 172.21.15.122:0/1855292376 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fadb8005000 con 0x7fadc4102d50 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.530+0000 7fadccde8740 1 Processor -- start 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.530+0000 7fadccde8740 1 -- start start 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadccde8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af746da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadccde8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7472e0 0x55f1af74bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadccde8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af74e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadccde8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f1af6aa2f0 con 0x55f1af74c370 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadccde8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f1af652270 con 0x55f1af7472e0 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadccde8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f1af6a62b0 con 0x55f1af7469a0 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc0ff9640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fadac0a2c30 0x7fadac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af74e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af74e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53964/0 (socket says 172.21.15.122:53964) 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1539438655 learned_addr learned my addr 172.21.15.122:0/1539438655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7472e0 0x55f1af74bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af746da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fadac0a2c30 0x7fadac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc0ff9640 1 -- 172.21.15.122:0/1855292376 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fadb80ceac0 con 0x7fadc4102d50 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 -- 172.21.15.122:0/1539438655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 msgr2=0x55f1af746da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af746da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 -- 172.21.15.122:0/1539438655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 msgr2=0x55f1af74e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af74e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 -- 172.21.15.122:0/1539438655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1af651340 con 0x55f1af7472e0 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af74e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fadac0a2c30 0x7fadac0a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadbc02ade0 tx=0x7fadbc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.531+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7472e0 0x55f1af74bda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadb400bbb0 tx=0x7fadb400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550502ee23f4009a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fada9ffb640 1 -- 172.21.15.122:0/1539438655 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4015070 con 0x55f1af7472e0 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fada9ffb640 1 -- 172.21.15.122:0/1539438655 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4005020 con 0x55f1af7472e0 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fada9ffb640 1 -- 172.21.15.122:0/1539438655 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4004a00 con 0x55f1af7472e0 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 -- 172.21.15.122:0/1539438655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7472e0 msgr2=0x55f1af74bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7472e0 0x55f1af74bda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadb400bbb0 tx=0x7fadb400f730 comp rx=0 tx=0).stop 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 -- 172.21.15.122:0/1539438655 shutdown_connections 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af74e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7472e0 0x55f1af74bda0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 --2- 172.21.15.122:0/1539438655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af746da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 -- 172.21.15.122:0/1539438655 >> 172.21.15.122:0/1539438655 conn(0x55f1af742010 msgr2=0x55f1af742410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 -- 172.21.15.122:0/1539438655 shutdown_connections 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.532+0000 7fadccde8740 1 -- 172.21.15.122:0/1539438655 wait complete. 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 -- start start 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 Processor -- start 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7469a0 0x55f1af750bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af751110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af7580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f1af74f340 con 0x55f1af74c370 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f1af633290 con 0x55f1af7469a0 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f1af74f4c0 con 0x55f1af7472e0 2024-09-12T01:48:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadccde8740 10 client.0 _ll_register_callbacks cb 0x55f1af664140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af7580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af7580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53970/0 (socket says 172.21.15.122:53970) 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadc3fff640 1 -- 172.21.15.122:0/3482201313 learned_addr learned my addr 172.21.15.122:0/3482201313 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadc37fe640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7469a0 0x55f1af750bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.533+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af751110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc3fff640 1 -- 172.21.15.122:0/3482201313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 msgr2=0x55f1af751110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc3fff640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af751110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc3fff640 1 -- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7469a0 msgr2=0x55f1af750bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc3fff640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7469a0 0x55f1af750bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc3fff640 1 -- 172.21.15.122:0/3482201313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1af3bd210 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc37fe640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7469a0 0x55f1af750bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadc3fff640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af7580e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fadbc035000 tx=0x7fadbc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37184e865d364231 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fada2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fada2ffd640 1 -- 172.21.15.122:0/3482201313 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc050070 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fada2ffd640 1 -- 172.21.15.122:0/3482201313 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fadbc0046b0 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb4005020 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fada2ffd640 1 -- 172.21.15.122:0/3482201313 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fadbc05c050 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadccde8740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1af489620 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.534+0000 7fada2ffd640 1 -- 172.21.15.122:0/3482201313 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4005020 con 0x55f1af74c370 2024-09-12T01:48:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fada2ffd640 1 -- 172.21.15.122:0/3482201313 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fadbc0305c0 con 0x55f1af74c370 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadccde8740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadccde8740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadccde8740 20 client.15888 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadccde8740 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af75c820 0x55f1af75eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadccde8740 4 client.15888 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1af6b7a20 con 0x55f1af75c820 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadc37fe640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af75c820 0x55f1af75eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fadc37fe640 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af75c820 0x55f1af75eca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c53c2f6cd166eb server_cookie=41e4dd6da9785457 in_seq=0 out_seq=0 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.535+0000 7fada2ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fada2ffd640 1 -- 172.21.15.122:0/3482201313 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1af6b7a20 con 0x55f1af75c820 2024-09-12T01:48:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fada2ffd640 10 client.15888 handle_command_reply: tid=0 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 client.15888 shutdown 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 msgr2=0x55f1af7580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af7580e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fadbc035000 tx=0x7fadbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 shutdown_connections 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af75c820 0x55f1af75eca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af74c370 0x55f1af7580e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af751110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fada2ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 --2- 172.21.15.122:0/3482201313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af7469a0 0x55f1af750bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 >> 172.21.15.122:0/3482201313 conn(0x55f1af742010 msgr2=0x55f1af743820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.536+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 shutdown_connections 2024-09-12T01:48:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.537+0000 7fadccde8740 1 -- 172.21.15.122:0/3482201313 wait complete. 2024-09-12T01:48:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.537+0000 7fadccde8740 20 client.15888 trim_cache size 0 max 16384 2024-09-12T01:48:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 Processor -- start 2024-09-12T01:48:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 -- start start 2024-09-12T01:48:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af742ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 0x55f1af758000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af743400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41adad0 con 0x55f1af7472e0 2024-09-12T01:48:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ab3a0 con 0x55f1af74c370 2024-09-12T01:48:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadccde8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a91d0 con 0x55f1af7469a0 2024-09-12T01:48:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 0x55f1af758000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 0x55f1af758000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53976/0 (socket says 172.21.15.122:53976) 2024-09-12T01:48:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1649472511 learned_addr learned my addr 172.21.15.122:0/1649472511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af742ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1649472511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 msgr2=0x55f1af742ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af742ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1649472511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 msgr2=0x55f1af743400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af743400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 -- 172.21.15.122:0/1649472511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1af741e30 con 0x55f1af7472e0 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.551+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 0x55f1af758000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fadb809b020 tx=0x7fadb8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b16dab9e6bdca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fada97fa640 1 -- 172.21.15.122:0/1649472511 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020040 con 0x55f1af7472e0 2024-09-12T01:48:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fada97fa640 1 -- 172.21.15.122:0/1649472511 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb80a3f90 con 0x55f1af7472e0 2024-09-12T01:48:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fada97fa640 1 -- 172.21.15.122:0/1649472511 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb809a770 con 0x55f1af7472e0 2024-09-12T01:48:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 -- 172.21.15.122:0/1649472511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 msgr2=0x55f1af758000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 0x55f1af758000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fadb809b020 tx=0x7fadb8009c30 comp rx=0 tx=0).stop 2024-09-12T01:48:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 -- 172.21.15.122:0/1649472511 shutdown_connections 2024-09-12T01:48:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af743400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7472e0 0x55f1af758000 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 --2- 172.21.15.122:0/1649472511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7469a0 0x55f1af742ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 -- 172.21.15.122:0/1649472511 >> 172.21.15.122:0/1649472511 conn(0x55f1af742010 msgr2=0x55f1af750bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 -- 172.21.15.122:0/1649472511 shutdown_connections 2024-09-12T01:48:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 -- 172.21.15.122:0/1649472511 wait complete. 2024-09-12T01:48:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.552+0000 7fadccde8740 1 -- start start 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 Processor -- start 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 0x55f1af7557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af755d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af756d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a9860 con 0x55f1af7469a0 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41aacd0 con 0x55f1af74c370 2024-09-12T01:48:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41a9f30 con 0x55f1af7472e0 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadccde8740 10 client.0 _ll_register_callbacks cb 0x7fadbc031cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 0x55f1af7557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 0x55f1af7557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53992/0 (socket says 172.21.15.122:53992) 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 -- 172.21.15.122:0/1272112530 learned_addr learned my addr 172.21.15.122:0/1272112530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af756d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af755d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 -- 172.21.15.122:0/1272112530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 msgr2=0x55f1af755d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af755d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 -- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 msgr2=0x55f1af756d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af756d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.553+0000 7fadc37fe640 1 -- 172.21.15.122:0/1272112530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1af757330 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadc37fe640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 0x55f1af7557d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fadb400bbb0 tx=0x7fadb400b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffbaf02e9507882 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fada2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fada2ffd640 1 -- 172.21.15.122:0/1272112530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb402b020 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fada2ffd640 1 -- 172.21.15.122:0/1272112530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fadb4005240 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb80a3f90 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fada2ffd640 1 -- 172.21.15.122:0/1272112530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fadb4015040 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1af7c76c0 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fada2ffd640 1 -- 172.21.15.122:0/1272112530 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb80a3f90 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fada2ffd640 1 -- 172.21.15.122:0/1272112530 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fadb40274e0 con 0x55f1af7469a0 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 20 client.15894 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af7c7df0 0x55f1af7ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 4 client.15894 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.554+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1af6b1d70 con 0x55f1af7c7df0 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.555+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af7c7df0 0x55f1af7ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.555+0000 7fadc2ffd640 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af7c7df0 0x55f1af7ca210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb71e62957ee68f server_cookie=7f35c5eb49d4834e in_seq=0 out_seq=0 2024-09-12T01:48:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.555+0000 7fada2ffd640 10 client.15894 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.565+0000 7fada2ffd640 1 -- 172.21.15.122:0/1272112530 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1af6b1d70 con 0x55f1af7c7df0 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.565+0000 7fada2ffd640 10 client.15894 handle_command_reply: tid=0 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.565+0000 7fadccde8740 1 client.15894 shutdown 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.565+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 msgr2=0x55f1af7557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.565+0000 7fadccde8740 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 0x55f1af7557d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fadb400bbb0 tx=0x7fadb400b3a0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.565+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 shutdown_connections 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f1af7c7df0 0x55f1af7ca210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f1af74c370 0x55f1af756d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fada2ffd640 0 client.15894 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f1af7472e0 0x55f1af755d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 --2- 172.21.15.122:0/1272112530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f1af7469a0 0x55f1af7557d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 >> 172.21.15.122:0/1272112530 conn(0x55f1af742010 msgr2=0x55f1af7409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 shutdown_connections 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 1 -- 172.21.15.122:0/1272112530 wait complete. 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadccde8740 20 client.15894 trim_cache size 0 max 16384 2024-09-12T01:48:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:48:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:48:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:48:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:48:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fadac0a2c30 msgr2=0x7fadac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fadac0a2c30 0x7fadac0a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadbc02ade0 tx=0x7fadbc0095c0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 msgr2=0x7fadc40745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 0x7fadc40745e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d770 tx=0x7fadb800dc30 comp rx=0 tx=0).stop 2024-09-12T01:48:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 shutdown_connections 2024-09-12T01:48:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fadac0a2c30 0x7fadac0a50f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadc4103710 0x7fadc4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4102d50 0x7fadc40745e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 --2- 172.21.15.122:0/1855292376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc4101b60 0x7fadc4075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 >> 172.21.15.122:0/1855292376 conn(0x7fadc40fd2e0 msgr2=0x7fadc4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 shutdown_connections 2024-09-12T01:48:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:25.566+0000 7fadca053640 1 -- 172.21.15.122:0/1855292376 wait complete. 2024-09-12T01:48:25.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 7.16 MB/sec warmup 43 sec latency 5395.380 ms 2024-09-12T01:48:25.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:25 smithi151 ceph-mon[35229]: pgmap v393: 97 pgs: 97 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-12T01:48:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:25 smithi183 ceph-mon[35127]: pgmap v393: 97 pgs: 97 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-12T01:48:25.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:25 smithi122 ceph-mon[30925]: pgmap v393: 97 pgs: 97 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-12T01:48:26.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 7.00 MB/sec warmup 44 sec latency 6395.461 ms 2024-09-12T01:48:27.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 6.84 MB/sec warmup 45 sec latency 7395.560 ms 2024-09-12T01:48:27.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:27 smithi151 ceph-mon[35229]: pgmap v394: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T01:48:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:27 smithi183 ceph-mon[35127]: pgmap v394: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T01:48:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:27 smithi122 ceph-mon[30925]: pgmap v394: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T01:48:28.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 6.69 MB/sec warmup 46 sec latency 8395.657 ms 2024-09-12T01:48:29.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 6.55 MB/sec warmup 47 sec latency 9395.775 ms 2024-09-12T01:48:29.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:29 smithi151 ceph-mon[35229]: pgmap v395: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-09-12T01:48:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:29 smithi183 ceph-mon[35127]: pgmap v395: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-09-12T01:48:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:29 smithi122 ceph-mon[30925]: pgmap v395: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-09-12T01:48:30.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 6.41 MB/sec warmup 48 sec latency 10395.855 ms 2024-09-12T01:48:31.144 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:48:31.399 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:31.771 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:31 smithi122 ceph-mon[30925]: pgmap v396: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 522 KiB/s wr, 87 op/s 2024-09-12T01:48:31.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 6.28 MB/sec warmup 49 sec latency 11395.913 ms 2024-09-12T01:48:31.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 -- 172.21.15.122:0/4189872285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 msgr2=0x7fc3501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:31.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 --2- 172.21.15.122:0/4189872285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501021c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009900 tx=0x7fc34c02a410 comp rx=0 tx=0).stop 2024-09-12T01:48:31.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 -- 172.21.15.122:0/4189872285 shutdown_connections 2024-09-12T01:48:31.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 --2- 172.21.15.122:0/4189872285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 0x7fc35010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:31.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 --2- 172.21.15.122:0/4189872285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc350102790 0x7fc3501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:31.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 --2- 172.21.15.122:0/4189872285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501021c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 -- 172.21.15.122:0/4189872285 >> 172.21.15.122:0/4189872285 conn(0x7fc3500fd8c0 msgr2=0x7fc3500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 -- 172.21.15.122:0/4189872285 shutdown_connections 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.851+0000 7fc357619640 1 -- 172.21.15.122:0/4189872285 wait complete. 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 Processor -- start 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 -- start start 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc350102790 0x7fc3501a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 0x7fc35019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc350112010 con 0x7fc350101de0 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc350111e90 con 0x7fc350106f10 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc357619640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc350112190 con 0x7fc350102790 2024-09-12T01:48:31.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41074/0 (socket says 172.21.15.122:41074) 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 -- 172.21.15.122:0/1136846182 learned_addr learned my addr 172.21.15.122:0/1136846182 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc355bb6640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 0x7fc35019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 -- 172.21.15.122:0/1136846182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc350102790 msgr2=0x7fc3501a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc350102790 0x7fc3501a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 -- 172.21.15.122:0/1136846182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 msgr2=0x7fc35019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 0x7fc35019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc3553b5640 1 -- 172.21.15.122:0/1136846182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35019ad10 con 0x7fc350101de0 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.852+0000 7fc355bb6640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 0x7fc35019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.853+0000 7fc3553b5640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501a0420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc34c005ab0 tx=0x7fc34c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:31.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.853+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c03f030 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.853+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35019afa0 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.853+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c003ce0 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.853+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c035070 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.853+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501a72f0 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.854+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc350102240 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.855+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc34c042030 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.856+0000 7fc3467fc640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc33c0a2c00 0x7fc33c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.856+0000 7fc354bb4640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc33c0a2c00 0x7fc33c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.856+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc34c0efc70 con 0x7fc350101de0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.856+0000 7fc354bb4640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc33c0a2c00 0x7fc33c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc3500ff810 tx=0x7fc340006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:31.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:31.859+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc34c0bcb90 con 0x7fc350101de0 2024-09-12T01:48:31.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:31 smithi151 ceph-mon[35229]: pgmap v396: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 522 KiB/s wr, 87 op/s 2024-09-12T01:48:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:31 smithi183 ceph-mon[35127]: pgmap v396: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 522 KiB/s wr, 87 op/s 2024-09-12T01:48:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.061+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc35019bb10 con 0x7fc350101de0 2024-09-12T01:48:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.062+0000 7fc3467fc640 1 -- 172.21.15.122:0/1136846182 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11934 (secure 0 0 0) 0x7fc34c0bc490 con 0x7fc350101de0 2024-09-12T01:48:32.062 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:48:32.063 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"btime":"2024-09-12T01:48:18:666269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:48:17.839016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:48:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.064+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc33c0a2c00 msgr2=0x7fc33c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.064+0000 7fc357619640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc33c0a2c00 0x7fc33c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc3500ff810 tx=0x7fc340006c90 comp rx=0 tx=0).stop 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.064+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 msgr2=0x7fc3501a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.064+0000 7fc357619640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501a0420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc34c005ab0 tx=0x7fc34c0042f0 comp rx=0 tx=0).stop 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 shutdown_connections 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc33c0a2c00 0x7fc33c0a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc350106f10 0x7fc35019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc350102790 0x7fc3501a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 --2- 172.21.15.122:0/1136846182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc350101de0 0x7fc3501a0420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:32.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 >> 172.21.15.122:0/1136846182 conn(0x7fc3500fd8c0 msgr2=0x7fc3500fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:32.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 shutdown_connections 2024-09-12T01:48:32.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:32.065+0000 7fc357619640 1 -- 172.21.15.122:0/1136846182 wait complete. 2024-09-12T01:48:32.777 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 7f4985a4 --release --await' 2024-09-12T01:48:32.778 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 7f4985a4 --release --await 2024-09-12T01:48:32.801 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:32 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1136846182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:32.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53792 6.16 MB/sec warmup 50 sec latency 12395.966 ms 2024-09-12T01:48:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:32 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1136846182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:32 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1136846182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:33.020 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:33.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 -- 172.21.15.122:0/1160329258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724101de0 msgr2=0x7fa7241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 --2- 172.21.15.122:0/1160329258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724101de0 0x7fa7241021c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa714009900 tx=0x7fa71402a410 comp rx=0 tx=0).stop 2024-09-12T01:48:33.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 -- 172.21.15.122:0/1160329258 shutdown_connections 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 --2- 172.21.15.122:0/1160329258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa724106f10 0x7fa72410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 --2- 172.21.15.122:0/1160329258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa724102790 0x7fa7241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 --2- 172.21.15.122:0/1160329258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724101de0 0x7fa7241021c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 -- 172.21.15.122:0/1160329258 >> 172.21.15.122:0/1160329258 conn(0x7fa7240fd8c0 msgr2=0x7fa7240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 -- 172.21.15.122:0/1160329258 shutdown_connections 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.492+0000 7fa729d25640 1 -- 172.21.15.122:0/1160329258 wait complete. 2024-09-12T01:48:33.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 Processor -- start 2024-09-12T01:48:33.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 -- start start 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa724101de0 0x7fa724198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 0x7fa724198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa724106f10 0x7fa72419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa724112010 con 0x7fa724102790 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa724111e90 con 0x7fa724101de0 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa729d25640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa724112190 con 0x7fa724106f10 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 0x7fa724198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 0x7fa724198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41086/0 (socket says 172.21.15.122:41086) 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa722ffd640 1 -- 172.21.15.122:0/3896460539 learned_addr learned my addr 172.21.15.122:0/3896460539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa7237fe640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa724101de0 0x7fa724198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.493+0000 7fa723fff640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa724106f10 0x7fa72419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa722ffd640 1 -- 172.21.15.122:0/3896460539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa724106f10 msgr2=0x7fa72419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa722ffd640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa724106f10 0x7fa72419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa722ffd640 1 -- 172.21.15.122:0/3896460539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa724101de0 msgr2=0x7fa724198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa722ffd640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa724101de0 0x7fa724198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa722ffd640 1 -- 172.21.15.122:0/3896460539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72419d630 con 0x7fa724102790 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa722ffd640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 0x7fa724198bc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa71800b4a0 tx=0x7fa71800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa720ff9640 1 -- 172.21.15.122:0/3896460539 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718014070 con 0x7fa724102790 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa720ff9640 1 -- 172.21.15.122:0/3896460539 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa718010af0 con 0x7fa724102790 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72419d920 con 0x7fa724102790 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa720ff9640 1 -- 172.21.15.122:0/3896460539 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718017d10 con 0x7fa724102790 2024-09-12T01:48:33.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.494+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7241a5200 con 0x7fa724102790 2024-09-12T01:48:33.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.496+0000 7fa720ff9640 1 -- 172.21.15.122:0/3896460539 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa718017070 con 0x7fa724102790 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 Processor -- start 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 -- start start 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa720ff9640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa70c0a2b80 0x7fa70c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x558229470440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x55822947ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x55822947d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582293d5620 con 0x558229470060 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55822937ce50 con 0x558229470980 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582293d15e0 con 0x55822947b240 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa723fff640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa70c0a2b80 0x7fa70c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa720ff9640 1 -- 172.21.15.122:0/3896460539 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa7180ce480 con 0x7fa724102790 2024-09-12T01:48:33.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x558229470440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa723fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x55822947d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x558229470440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41088/0 (socket says 172.21.15.122:41088) 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 -- 172.21.15.122:0/1966308308 learned_addr learned my addr 172.21.15.122:0/1966308308 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa722ffd640 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x55822947ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 -- 172.21.15.122:0/1966308308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 msgr2=0x55822947d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x55822947d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 -- 172.21.15.122:0/1966308308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 msgr2=0x55822947ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x55822947ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa723fff640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa70c0a2b80 0x7fa70c0a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009fd0 tx=0x7fa71c009aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa7237fe640 1 -- 172.21.15.122:0/1966308308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822936dd40 con 0x558229470060 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.497+0000 7fa722ffd640 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x55822947ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.498+0000 7fa7237fe640 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x558229470440 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa714009900 tx=0x7fa7140316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe2da9e8d7560fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:33.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.498+0000 7fa709ffb640 1 -- 172.21.15.122:0/1966308308 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714040070 con 0x558229470060 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.498+0000 7fa709ffb640 1 -- 172.21.15.122:0/1966308308 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa714005020 con 0x558229470060 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.498+0000 7fa709ffb640 1 -- 172.21.15.122:0/1966308308 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714035040 con 0x558229470060 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1966308308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 msgr2=0x558229470440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x558229470440 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa714009900 tx=0x7fa7140316e0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1966308308 shutdown_connections 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x55822947d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x55822947ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1966308308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x558229470440 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1966308308 >> 172.21.15.122:0/1966308308 conn(0x55822946c5f0 msgr2=0x55822946c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1966308308 shutdown_connections 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1966308308 wait complete. 2024-09-12T01:48:33.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 -- start start 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 Processor -- start 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.499+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558229470060 0x5582294780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 0x558229478b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55822947de10 con 0x55822947b240 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55822935df10 con 0x558229470980 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55822947df90 con 0x558229470060 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 10 client.0 _ll_register_callbacks cb 0x558229381db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 0x558229478b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 0x558229478b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41096/0 (socket says 172.21.15.122:41096) 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 -- 172.21.15.122:0/2865375983 learned_addr learned my addr 172.21.15.122:0/2865375983 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa7237fe640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558229470060 0x5582294780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa722ffd640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 -- 172.21.15.122:0/2865375983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558229470060 msgr2=0x5582294780b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558229470060 0x5582294780b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 -- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 msgr2=0x5582294785f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294785f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 -- 172.21.15.122:0/2865375983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582290e4580 con 0x55822947b240 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa722ffd640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa723fff640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 0x558229478b30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa71c01c6c0 tx=0x7fa71c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8c9c05e07ceb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa702ffd640 1 -- 172.21.15.122:0/2865375983 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c02a060 con 0x55822947b240 2024-09-12T01:48:33.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa702ffd640 1 -- 172.21.15.122:0/2865375983 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa71c01a5e0 con 0x55822947b240 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.500+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa714005020 con 0x55822947b240 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa702ffd640 1 -- 172.21.15.122:0/2865375983 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa71c01a7e0 con 0x55822947b240 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 10 client.15912 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582291ae960 con 0x55822947b240 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa702ffd640 1 -- 172.21.15.122:0/2865375983 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa714005020 con 0x55822947b240 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa702ffd640 1 -- 172.21.15.122:0/2865375983 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa71c02f070 con 0x55822947b240 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 10 client.15912 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 10 client.15912 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 20 client.15912 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55822948aca0 0x55822948d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 4 client.15912 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582293e2b90 con 0x55822948aca0 2024-09-12T01:48:33.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.501+0000 7fa7237fe640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55822948aca0 0x55822948d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.502+0000 7fa7237fe640 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55822948aca0 0x55822948d140 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53f83c0e4cba67d server_cookie=52d18dd7ac29a257 in_seq=0 out_seq=0 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.502+0000 7fa702ffd640 10 client.15912 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.502+0000 7fa702ffd640 1 -- 172.21.15.122:0/2865375983 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582293e2b90 con 0x55822948aca0 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.502+0000 7fa702ffd640 10 client.15912 handle_command_reply: tid=0 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.502+0000 7fa72c0fe740 1 client.15912 shutdown 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 msgr2=0x558229478b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 0x558229478b30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa71c01c6c0 tx=0x7fa71c004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 shutdown_connections 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55822948aca0 0x55822948d140 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55822947b240 0x558229478b30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/2865375983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558229470060 0x5582294780b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa702ffd640 0 client.15912 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:33.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 >> 172.21.15.122:0/2865375983 conn(0x55822946c5f0 msgr2=0x55822946b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:33.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 shutdown_connections 2024-09-12T01:48:33.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 1 -- 172.21.15.122:0/2865375983 wait complete. 2024-09-12T01:48:33.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.503+0000 7fa72c0fe740 20 client.15912 trim_cache size 0 max 16384 2024-09-12T01:48:33.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 Processor -- start 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 -- start start 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470060 0x558229472ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 0x55822946b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294835c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241affb0 con 0x558229470980 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241ad880 con 0x558229470060 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241ab6b0 con 0x55822947b240 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 0x55822946b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 0x55822946b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41108/0 (socket says 172.21.15.122:41108) 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 -- 172.21.15.122:0/1144536868 learned_addr learned my addr 172.21.15.122:0/1144536868 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa723fff640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa7237fe640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470060 0x558229472ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 -- 172.21.15.122:0/1144536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 msgr2=0x5582294835c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294835c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 -- 172.21.15.122:0/1144536868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470060 msgr2=0x558229472ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470060 0x558229472ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 -- 172.21.15.122:0/1144536868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822946abb0 con 0x558229470980 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa723fff640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294835c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa7237fe640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470060 0x558229472ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.520+0000 7fa722ffd640 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 0x55822946b640 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa71800be60 tx=0x7fa7180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7613b6f03886880 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa7097fa640 1 -- 172.21.15.122:0/1144536868 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71801e070 con 0x558229470980 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa7097fa640 1 -- 172.21.15.122:0/1144536868 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa718098fa0 con 0x558229470980 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa7097fa640 1 -- 172.21.15.122:0/1144536868 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71809a0f0 con 0x558229470980 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1144536868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 msgr2=0x55822946b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 0x55822946b640 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa71800be60 tx=0x7fa7180046c0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1144536868 shutdown_connections 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294835c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470980 0x55822946b640 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/1144536868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470060 0x558229472ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1144536868 >> 172.21.15.122:0/1144536868 conn(0x55822946c5f0 msgr2=0x55822946af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1144536868 shutdown_connections 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.521+0000 7fa72c0fe740 1 -- 172.21.15.122:0/1144536868 wait complete. 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 -- start start 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 Processor -- start 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x5582294f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241abd40 con 0x558229470060 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241ad1b0 con 0x558229470980 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241ac410 con 0x55822947b240 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa72c0fe740 10 client.0 _ll_register_callbacks cb 0x7fa71c007ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x5582294f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa7237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x5582294f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41114/0 (socket says 172.21.15.122:41114) 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa7237fe640 1 -- 172.21.15.122:0/758450412 learned_addr learned my addr 172.21.15.122:0/758450412 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa723fff640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.522+0000 7fa722ffd640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa7237fe640 1 -- 172.21.15.122:0/758450412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 msgr2=0x5582294ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa7237fe640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa7237fe640 1 -- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 msgr2=0x5582294f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa7237fe640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa7237fe640 1 -- 172.21.15.122:0/758450412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582294ee090 con 0x558229470060 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa723fff640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa722ffd640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa7237fe640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x5582294f2250 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa714004620 tx=0x7fa7140045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b4bcbdecd7253c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa702ffd640 1 -- 172.21.15.122:0/758450412 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714054030 con 0x558229470060 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa702ffd640 1 -- 172.21.15.122:0/758450412 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa714005020 con 0x558229470060 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa718098fa0 con 0x558229470060 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa702ffd640 1 -- 172.21.15.122:0/758450412 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa714040040 con 0x558229470060 2024-09-12T01:48:33.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa72c0fe740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582294faba0 con 0x558229470060 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa702ffd640 1 -- 172.21.15.122:0/758450412 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa718098fa0 con 0x558229470060 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa702ffd640 1 -- 172.21.15.122:0/758450412 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa71403c650 con 0x558229470060 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa72c0fe740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa72c0fe740 10 client.15918 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.523+0000 7fa72c0fe740 20 client.15918 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.524+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582294fafa0 0x5582294fd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.524+0000 7fa72c0fe740 4 client.15918 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7f4985a4", "release": true, "await": true}] 2024-09-12T01:48:33.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.524+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7f4985a4", "release": true, "await": true}) -- 0x5582293dd0a0 con 0x5582294fafa0 2024-09-12T01:48:33.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.524+0000 7fa722ffd640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582294fafa0 0x5582294fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:33.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.524+0000 7fa722ffd640 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582294fafa0 0x5582294fd390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480751c490f680c8 server_cookie=f3fde128719e9b9e in_seq=0 out_seq=0 2024-09-12T01:48:33.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.524+0000 7fa702ffd640 10 client.15918 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:33.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa702ffd640 1 -- 172.21.15.122:0/758450412 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5582293dd0a0 con 0x5582294fafa0 2024-09-12T01:48:33.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa702ffd640 10 client.15918 handle_command_reply: tid=0 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 client.15918 shutdown 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 msgr2=0x5582294f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x5582294f2250 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa714004620 tx=0x7fa7140045b0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 shutdown_connections 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582294fafa0 0x5582294fd390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55822947b240 0x5582294ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558229470980 0x5582294f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 --2- 172.21.15.122:0/758450412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558229470060 0x5582294f2250 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 >> 172.21.15.122:0/758450412 conn(0x55822946c5f0 msgr2=0x558229478070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa702ffd640 0 client.15918 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 shutdown_connections 2024-09-12T01:48:33.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 1 -- 172.21.15.122:0/758450412 wait complete. 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.534+0000 7fa72c0fe740 20 client.15918 trim_cache size 0 max 16384 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 28, 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 8, 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout: "7f4985a4": { 2024-09-12T01:48:33.536 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 8, 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 133.4, 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:48:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:33.538 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa70c0a2b80 msgr2=0x7fa70c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa70c0a2b80 0x7fa70c0a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009fd0 tx=0x7fa71c009aa0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 msgr2=0x7fa724198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 0x7fa724198bc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa71800b4a0 tx=0x7fa71800b960 comp rx=0 tx=0).stop 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 shutdown_connections 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa70c0a2b80 0x7fa70c0a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa724106f10 0x7fa72419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa724102790 0x7fa724198bc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 --2- 172.21.15.122:0/3896460539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa724101de0 0x7fa724198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:33.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 >> 172.21.15.122:0/3896460539 conn(0x7fa7240fd8c0 msgr2=0x7fa7240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:33.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 shutdown_connections 2024-09-12T01:48:33.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:33.535+0000 7fa729d25640 1 -- 172.21.15.122:0/3896460539 wait complete. 2024-09-12T01:48:33.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:33 smithi122 ceph-mon[30925]: pgmap v397: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 372 KiB/s wr, 69 op/s 2024-09-12T01:48:33.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 54520 6.08 MB/sec warmup 51 sec latency 13115.775 ms 2024-09-12T01:48:33.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:33 smithi151 ceph-mon[35229]: pgmap v397: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 372 KiB/s wr, 69 op/s 2024-09-12T01:48:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:33 smithi183 ceph-mon[35127]: pgmap v397: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 372 KiB/s wr, 69 op/s 2024-09-12T01:48:34.298 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7f4985a4', total seconds elapsed: 16.5 2024-09-12T01:48:34.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 57572 6.29 MB/sec warmup 52 sec latency 10.211 ms 2024-09-12T01:48:35.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 60770 6.53 MB/sec warmup 53 sec latency 7.356 ms 2024-09-12T01:48:35.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:35 smithi151 ceph-mon[35229]: pgmap v398: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 301 KiB/s wr, 62 op/s 2024-09-12T01:48:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:35 smithi183 ceph-mon[35127]: pgmap v398: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 301 KiB/s wr, 62 op/s 2024-09-12T01:48:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:35 smithi122 ceph-mon[30925]: pgmap v398: 97 pgs: 97 active+clean; 38 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 301 KiB/s wr, 62 op/s 2024-09-12T01:48:36.816 INFO:tasks.workunit.client.0.smithi122.stdout: 1 64087 6.76 MB/sec warmup 54 sec latency 12.492 ms 2024-09-12T01:48:37.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 66892 6.87 MB/sec warmup 55 sec latency 10.429 ms 2024-09-12T01:48:37.868 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:37 smithi151 ceph-mon[35229]: pgmap v399: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 65 KiB/s wr, 17 op/s 2024-09-12T01:48:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:37 smithi183 ceph-mon[35127]: pgmap v399: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 65 KiB/s wr, 17 op/s 2024-09-12T01:48:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:37 smithi122 ceph-mon[30925]: pgmap v399: 97 pgs: 97 active+clean; 38 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 65 KiB/s wr, 17 op/s 2024-09-12T01:48:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.654+0000 7fcdd37fe640 1 -- 172.21.15.122:0/3449037995 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc02f310 con 0x7fcdd4102600 2024-09-12T01:48:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 -- 172.21.15.122:0/3449037995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 msgr2=0x7fcdd4108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 --2- 172.21.15.122:0/3449037995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd4108e90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009ca0 tx=0x7fcdcc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 -- 172.21.15.122:0/3449037995 shutdown_connections 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 --2- 172.21.15.122:0/3449037995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd4108e90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 --2- 172.21.15.122:0/3449037995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdd4101c40 0x7fcdd41020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 --2- 172.21.15.122:0/3449037995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdd4100a50 0x7fcdd4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 -- 172.21.15.122:0/3449037995 >> 172.21.15.122:0/3449037995 conn(0x7fcdd40fc1b0 msgr2=0x7fcdd40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 -- 172.21.15.122:0/3449037995 shutdown_connections 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.655+0000 7fcddaf11640 1 -- 172.21.15.122:0/3449037995 wait complete. 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 Processor -- start 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 -- start start 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdd4100a50 0x7fcdd406ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdd4101c40 0x7fcdd419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410c6b0 con 0x7fcdd4102600 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410c530 con 0x7fcdd4101c40 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcddaf11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410c830 con 0x7fcdd4100a50 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41140/0 (socket says 172.21.15.122:41140) 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 -- 172.21.15.122:0/2315602574 learned_addr learned my addr 172.21.15.122:0/2315602574 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdd4101c40 0x7fcdd419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdd4100a50 0x7fcdd406ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 -- 172.21.15.122:0/2315602574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdd4100a50 msgr2=0x7fcdd406ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdd4100a50 0x7fcdd406ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 -- 172.21.15.122:0/2315602574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdd4101c40 msgr2=0x7fcdd419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdd4101c40 0x7fcdd419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 -- 172.21.15.122:0/2315602574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd419f9a0 con 0x7fcdd4102600 2024-09-12T01:48:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.656+0000 7fcdd94ae640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd419f090 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc031ae0 tx=0x7fcdcc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.657+0000 7fcdd1ffb640 1 -- 172.21.15.122:0/2315602574 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc003b70 con 0x7fcdd4102600 2024-09-12T01:48:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.657+0000 7fcdd1ffb640 1 -- 172.21.15.122:0/2315602574 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc003d10 con 0x7fcdd4102600 2024-09-12T01:48:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.657+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd419fbd0 con 0x7fcdd4102600 2024-09-12T01:48:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.657+0000 7fcdd1ffb640 1 -- 172.21.15.122:0/2315602574 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc02fc40 con 0x7fcdd4102600 2024-09-12T01:48:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.657+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd41a0120 con 0x7fcdd4102600 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.659+0000 7fcdd1ffb640 1 -- 172.21.15.122:0/2315602574 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcdcc004050 con 0x7fcdd4102600 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.659+0000 7fcdddca7740 1 Processor -- start 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.659+0000 7fcdddca7740 1 -- start start 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 0x558a848e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e92e0 0x558a848edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848f0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdddca7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a8484c2f0 con 0x558a848e89a0 2024-09-12T01:48:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdddca7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a847f4270 con 0x558a848e92e0 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdddca7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a848482b0 con 0x558a848ee370 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 0x558a848e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e92e0 0x558a848edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 0x558a848e8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41142/0 (socket says 172.21.15.122:41142) 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd3fff640 1 -- 172.21.15.122:0/2930914224 learned_addr learned my addr 172.21.15.122:0/2930914224 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd1ffb640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcdc40a2c30 0x7fcdc40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd94ae640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcdc40a2c30 0x7fcdc40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2930914224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 msgr2=0x558a848f0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848f0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2930914224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 msgr2=0x558a848e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 0x558a848e8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2930914224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a847f3340 con 0x558a848e92e0 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 0x558a848e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd1ffb640 1 -- 172.21.15.122:0/2315602574 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcdcc0f00c0 con 0x7fcdd4102600 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.660+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e92e0 0x558a848edda0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009bf0 tx=0x7fcdc8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16976844821f8a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdd94ae640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcdc40a2c30 0x7fcdc40a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0b8ea0 tx=0x7fcdcc08e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdbeffd640 1 -- 172.21.15.122:0/2930914224 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8009da0 con 0x558a848e92e0 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdbeffd640 1 -- 172.21.15.122:0/2930914224 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc8005020 con 0x558a848e92e0 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdbeffd640 1 -- 172.21.15.122:0/2930914224 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8004a90 con 0x558a848e92e0 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 -- 172.21.15.122:0/2930914224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e92e0 msgr2=0x558a848edda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e92e0 0x558a848edda0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009bf0 tx=0x7fcdc8011810 comp rx=0 tx=0).stop 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 -- 172.21.15.122:0/2930914224 shutdown_connections 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e92e0 0x558a848edda0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2930914224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e89a0 0x558a848e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 -- 172.21.15.122:0/2930914224 >> 172.21.15.122:0/2930914224 conn(0x558a848e4010 msgr2=0x558a848e4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 -- 172.21.15.122:0/2930914224 shutdown_connections 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.661+0000 7fcdddca7740 1 -- 172.21.15.122:0/2930914224 wait complete. 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 -- start start 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 Processor -- start 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a848fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a848f1340 con 0x558a848e92e0 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a847d5290 con 0x558a848ee370 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a848f14c0 con 0x558a848e89a0 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdddca7740 10 client.0 _ll_register_callbacks cb 0x558a84806140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41156/0 (socket says 172.21.15.122:41156) 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdd3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2758893343 learned_addr learned my addr 172.21.15.122:0/2758893343 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.662+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a848fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2758893343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 msgr2=0x558a848f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 msgr2=0x558a848fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a848fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd8cad640 1 -- 172.21.15.122:0/2758893343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8455f210 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f3140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcdc801b040 tx=0x7fcdc8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a48c6c896c76c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdb3fff640 1 -- 172.21.15.122:0/2758893343 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8029020 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdb3fff640 1 -- 172.21.15.122:0/2758893343 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcdc8025470 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc8005020 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdb3fff640 1 -- 172.21.15.122:0/2758893343 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdc8009da0 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdddca7740 10 client.15930 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a8462b620 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdb3fff640 1 -- 172.21.15.122:0/2758893343 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc8005020 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.663+0000 7fcdb3fff640 1 -- 172.21.15.122:0/2758893343 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcdc80194e0 con 0x558a848e92e0 2024-09-12T01:48:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdddca7740 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdddca7740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdddca7740 20 client.15930 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a848fe6b0 0x558a84900aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdddca7740 4 client.15930 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a84859a20 con 0x558a848fe6b0 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a848fe6b0 0x558a84900aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdb3fff640 10 client.15930 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.664+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a848fe6b0 0x558a84900aa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6602ba376215a9c7 server_cookie=767e018ebb3a6220 in_seq=0 out_seq=0 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdb3fff640 1 -- 172.21.15.122:0/2758893343 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a84859a20 con 0x558a848fe6b0 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdb3fff640 10 client.15930 handle_command_reply: tid=0 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 client.15930 shutdown 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 msgr2=0x558a848f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f3140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcdc801b040 tx=0x7fcdc8004ac0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 shutdown_connections 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a848fe6b0 0x558a84900aa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a848fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f3140 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 --2- 172.21.15.122:0/2758893343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdb3fff640 0 client.15930 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 >> 172.21.15.122:0/2758893343 conn(0x558a848e4010 msgr2=0x558a848e5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 shutdown_connections 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 1 -- 172.21.15.122:0/2758893343 wait complete. 2024-09-12T01:48:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.665+0000 7fcdddca7740 20 client.15930 trim_cache size 0 max 16384 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 Processor -- start 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 -- start start 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e89a0 0x558a848e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41ab830 con 0x558a848e92e0 2024-09-12T01:48:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a9100 con 0x558a848e89a0 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdddca7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a6f30 con 0x558a848ee370 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848fa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41158/0 (socket says 172.21.15.122:41158) 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3251709459 learned_addr learned my addr 172.21.15.122:0/3251709459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e89a0 0x558a848e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3251709459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 msgr2=0x558a848e30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848e30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3251709459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e89a0 msgr2=0x558a848e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e89a0 0x558a848e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3251709459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a84901120 con 0x558a848e92e0 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848fa0d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8019af0 tx=0x7fcdc8009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c105ba97ba6311 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.680+0000 7fcdbe7fc640 1 -- 172.21.15.122:0/3251709459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8029040 con 0x558a848e92e0 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdbe7fc640 1 -- 172.21.15.122:0/3251709459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc8004ec0 con 0x558a848e92e0 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdbe7fc640 1 -- 172.21.15.122:0/3251709459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80196b0 con 0x558a848e92e0 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 -- 172.21.15.122:0/3251709459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 msgr2=0x558a848fa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848fa0d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8019af0 tx=0x7fcdc8009a70 comp rx=0 tx=0).stop 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 -- 172.21.15.122:0/3251709459 shutdown_connections 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848ee370 0x558a848e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848fa0d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3251709459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848e89a0 0x558a848e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 -- 172.21.15.122:0/3251709459 >> 172.21.15.122:0/3251709459 conn(0x558a848e4010 msgr2=0x558a848e29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 -- 172.21.15.122:0/3251709459 shutdown_connections 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 -- 172.21.15.122:0/3251709459 wait complete. 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.681+0000 7fcdddca7740 1 -- start start 2024-09-12T01:48:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 Processor -- start 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a84967370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a75c0 con 0x558a848e92e0 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a8a30 con 0x558a848ee370 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41a7c90 con 0x558a848e89a0 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdddca7740 10 client.0 _ll_register_callbacks cb 0x7fcdc0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41172/0 (socket says 172.21.15.122:41172) 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3070710283 learned_addr learned my addr 172.21.15.122:0/3070710283 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a84967370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3070710283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 msgr2=0x558a848f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 msgr2=0x558a84967370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a84967370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 -- 172.21.15.122:0/3070710283 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a849678e0 con 0x558a848e92e0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a84967370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdd8cad640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f7620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009380 tx=0x7fcdc8009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9d66794ef07fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.682+0000 7fcdb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdb3fff640 1 -- 172.21.15.122:0/3070710283 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8025880 con 0x558a848e92e0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdb3fff640 1 -- 172.21.15.122:0/3070710283 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fcdc8025a20 con 0x558a848e92e0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc8004ec0 con 0x558a848e92e0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdb3fff640 1 -- 172.21.15.122:0/3070710283 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdc8004ac0 con 0x558a848e92e0 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a84968bd0 con 0x558a848e92e0 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdb3fff640 1 -- 172.21.15.122:0/3070710283 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc8004ec0 con 0x558a848e92e0 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdb3fff640 1 -- 172.21.15.122:0/3070710283 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcdc8037070 con 0x558a848e92e0 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 20 client.15936 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a84969370 0x558a8496b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 4 client.15936 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a84853d70 con 0x558a84969370 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.683+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a84969370 0x558a8496b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.684+0000 7fcdb3fff640 10 client.15936 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.684+0000 7fcdd3fff640 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a84969370 0x558a8496b790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ade77021b3718a0 server_cookie=62681fa8ba4e0d64 in_seq=0 out_seq=0 2024-09-12T01:48:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdb3fff640 1 -- 172.21.15.122:0/3070710283 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a84853d70 con 0x558a84969370 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdb3fff640 10 client.15936 handle_command_reply: tid=0 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 client.15936 shutdown 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 msgr2=0x558a848f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f7620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009380 tx=0x7fcdc8009f40 comp rx=0 tx=0).stop 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 shutdown_connections 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558a84969370 0x558a8496b790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a848ee370 0x558a84967370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a848e92e0 0x558a848f7620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdb3fff640 0 client.15936 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 --2- 172.21.15.122:0/3070710283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a848e89a0 0x558a848f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.690+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 >> 172.21.15.122:0/3070710283 conn(0x558a848e4010 msgr2=0x558a848e31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 shutdown_connections 2024-09-12T01:48:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcdddca7740 1 -- 172.21.15.122:0/3070710283 wait complete. 2024-09-12T01:48:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcdddca7740 20 client.15936 trim_cache size 0 max 16384 2024-09-12T01:48:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:48:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:48:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:48:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcdc40a2c30 msgr2=0x7fcdc40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcdc40a2c30 0x7fcdc40a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0b8ea0 tx=0x7fcdcc08e040 comp rx=0 tx=0).stop 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 msgr2=0x7fcdd419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd419f090 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc031ae0 tx=0x7fcdcc031ee0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 shutdown_connections 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcdc40a2c30 0x7fcdc40a50f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdd4102600 0x7fcdd419f090 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdd4101c40 0x7fcdd419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 --2- 172.21.15.122:0/2315602574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdd4100a50 0x7fcdd406ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 >> 172.21.15.122:0/2315602574 conn(0x7fcdd40fc1b0 msgr2=0x7fcdd4104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 shutdown_connections 2024-09-12T01:48:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:38.691+0000 7fcddaf11640 1 -- 172.21.15.122:0/2315602574 wait complete. 2024-09-12T01:48:38.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 69849 6.99 MB/sec warmup 56 sec latency 9.574 ms 2024-09-12T01:48:39.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 72492 7.17 MB/sec warmup 57 sec latency 11.244 ms 2024-09-12T01:48:39.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:39 smithi151 ceph-mon[35229]: pgmap v400: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 311 KiB/s wr, 43 op/s 2024-09-12T01:48:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:39 smithi183 ceph-mon[35127]: pgmap v400: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 311 KiB/s wr, 43 op/s 2024-09-12T01:48:39.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:39 smithi122 ceph-mon[30925]: pgmap v400: 97 pgs: 97 active+clean; 41 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 311 KiB/s wr, 43 op/s 2024-09-12T01:48:40.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 75351 7.29 MB/sec warmup 58 sec latency 10.251 ms 2024-09-12T01:48:41.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78621 7.49 MB/sec warmup 59 sec latency 18.475 ms 2024-09-12T01:48:41.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:41 smithi151 ceph-mon[35229]: pgmap v401: 97 pgs: 97 active+clean; 46 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 787 KiB/s wr, 125 op/s 2024-09-12T01:48:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:41 smithi183 ceph-mon[35127]: pgmap v401: 97 pgs: 97 active+clean; 46 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 787 KiB/s wr, 125 op/s 2024-09-12T01:48:41.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:41 smithi122 ceph-mon[30925]: pgmap v401: 97 pgs: 97 active+clean; 46 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 787 KiB/s wr, 125 op/s 2024-09-12T01:48:42.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 81879 7.68 MB/sec warmup 60 sec latency 15.222 ms 2024-09-12T01:48:43.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 84842 7.77 MB/sec warmup 61 sec latency 10.914 ms 2024-09-12T01:48:43.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:43 smithi151 ceph-mon[35229]: pgmap v402: 97 pgs: 97 active+clean; 47 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 900 KiB/s wr, 135 op/s 2024-09-12T01:48:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:43 smithi183 ceph-mon[35127]: pgmap v402: 97 pgs: 97 active+clean; 47 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 900 KiB/s wr, 135 op/s 2024-09-12T01:48:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:43 smithi122 ceph-mon[30925]: pgmap v402: 97 pgs: 97 active+clean; 47 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 900 KiB/s wr, 135 op/s 2024-09-12T01:48:44.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 87951 7.92 MB/sec warmup 62 sec latency 10.270 ms 2024-09-12T01:48:45.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:48:45.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 89905 7.95 MB/sec warmup 63 sec latency 74.222 ms 2024-09-12T01:48:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:45 smithi151 ceph-mon[35229]: pgmap v403: 97 pgs: 97 active+clean; 48 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-12T01:48:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:48:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:45 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:45 smithi183 ceph-mon[35127]: pgmap v403: 97 pgs: 97 active+clean; 48 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-12T01:48:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:48:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:45 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:45 smithi122 ceph-mon[30925]: pgmap v403: 97 pgs: 97 active+clean; 48 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-09-12T01:48:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:48:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:45 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:46.037 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:46.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 -- 172.21.15.122:0/1255104377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184100e20 msgr2=0x7fb184101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/1255104377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184100e20 0x7fb184101200 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb18002a480 comp rx=0 tx=0).stop 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 -- 172.21.15.122:0/1255104377 shutdown_connections 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/1255104377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 0x7fb18410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/1255104377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184102570 0x7fb1841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/1255104377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184100e20 0x7fb184101200 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 -- 172.21.15.122:0/1255104377 >> 172.21.15.122:0/1255104377 conn(0x7fb1840fc7d0 msgr2=0x7fb1840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 -- 172.21.15.122:0/1255104377 shutdown_connections 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.528+0000 7fb18bb5d640 1 -- 172.21.15.122:0/1255104377 wait complete. 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 Processor -- start 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 -- start start 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184100e20 0x7fb184198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 0x7fb184198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 0x7fb18419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb184110f80 con 0x7fb184102570 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb184110e00 con 0x7fb184102ef0 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18bb5d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb184111100 con 0x7fb184100e20 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb1890f8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 0x7fb184198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb1890f8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 0x7fb184198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35688/0 (socket says 172.21.15.122:35688) 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb1890f8640 1 -- 172.21.15.122:0/3872571460 learned_addr learned my addr 172.21.15.122:0/3872571460 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184100e20 0x7fb184198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.529+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 0x7fb18419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1890f8640 1 -- 172.21.15.122:0/3872571460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184100e20 msgr2=0x7fb184198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184100e20 0x7fb184198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1890f8640 1 -- 172.21.15.122:0/3872571460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 msgr2=0x7fb18419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 0x7fb18419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1890f8640 1 -- 172.21.15.122:0/3872571460 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18419d4b0 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 0x7fb18419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184100e20 0x7fb184198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 0x7fb184198a40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb17400d770 tx=0x7fb17400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb17affd640 1 -- 172.21.15.122:0/3872571460 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174017070 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18419d7a0 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841a5080 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb17affd640 1 -- 172.21.15.122:0/3872571460 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb174005500 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.530+0000 7fb17affd640 1 -- 172.21.15.122:0/3872571460 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174020060 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.532+0000 7fb17affd640 1 -- 172.21.15.122:0/3872571460 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb174009c30 con 0x7fb184102570 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.532+0000 7fb18df36740 1 Processor -- start 2024-09-12T01:48:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 -- start start 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c6b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5584a6bd12e0 con 0x5584a6c76680 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5584a6b78ce0 con 0x5584a6c6bdc0 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18df36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584a6bcd2a0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb1898f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb1890f8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c6b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb1898f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c76140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52318/0 (socket says 172.21.15.122:52318) 2024-09-12T01:48:46.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb1898f9640 1 -- 172.21.15.122:0/848276618 learned_addr learned my addr 172.21.15.122:0/848276618 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.533+0000 7fb17affd640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1700a2b80 0x7fb1700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb1898f9640 1 -- 172.21.15.122:0/848276618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 msgr2=0x5584a6c6b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb1898f9640 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c6b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb1898f9640 1 -- 172.21.15.122:0/848276618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 msgr2=0x5584a6c78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb1898f9640 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb17affd640 1 -- 172.21.15.122:0/3872571460 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb1740ce9a0 con 0x7fb184102570 2024-09-12T01:48:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb1898f9640 1 -- 172.21.15.122:0/848276618 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a6b5d440 con 0x5584a6c6bdc0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c6b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1700a2b80 0x7fb1700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb1898f9640 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c76140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb18002a9a0 tx=0x7fb180005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ad5616ad2bd0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb16ffff640 1 -- 172.21.15.122:0/848276618 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180035030 con 0x5584a6c6bdc0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb16ffff640 1 -- 172.21.15.122:0/848276618 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb180044070 con 0x5584a6c6bdc0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.534+0000 7fb16ffff640 1 -- 172.21.15.122:0/848276618 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180031540 con 0x5584a6c6bdc0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 -- 172.21.15.122:0/848276618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 msgr2=0x5584a6c76140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1700a2b80 0x7fb1700a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0049d0 tx=0x7fb17c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c76140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb18002a9a0 tx=0x7fb180005e80 comp rx=0 tx=0).stop 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 -- 172.21.15.122:0/848276618 shutdown_connections 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c76140 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 --2- 172.21.15.122:0/848276618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c6b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 -- 172.21.15.122:0/848276618 >> 172.21.15.122:0/848276618 conn(0x5584a6c67a30 msgr2=0x5584a6c67e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 -- 172.21.15.122:0/848276618 shutdown_connections 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 -- 172.21.15.122:0/848276618 wait complete. 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 -- start start 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.535+0000 7fb18df36740 1 Processor -- start 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c75ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c6bdc0 0x5584a6c73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c76680 0x5584a6c736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5584a6c79250 con 0x5584a6c6bdc0 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5584a6b8c050 con 0x5584a6c76680 2024-09-12T01:48:46.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584a6c793d0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18df36740 10 client.0 _ll_register_callbacks cb 0x5584a6bcc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c76680 0x5584a6c736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c75ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1890f8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c6bdc0 0x5584a6c73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c76680 0x5584a6c736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52320/0 (socket says 172.21.15.122:52320) 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c75ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53356/0 (socket says 172.21.15.122:53356) 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb18a0fa640 1 -- 172.21.15.122:0/3743477342 learned_addr learned my addr 172.21.15.122:0/3743477342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 -- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c76680 msgr2=0x5584a6c736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c76680 0x5584a6c736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 -- 172.21.15.122:0/3743477342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c6bdc0 msgr2=0x5584a6c73180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c6bdc0 0x5584a6c73180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 -- 172.21.15.122:0/3743477342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a69423c0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c75ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb180000c00 tx=0x7fb18002ac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f4fef08b498ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.536+0000 7fb16d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3743477342 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180051030 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb180044070 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3743477342 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb1800038c0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3743477342 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb180035070 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 10 client.25312 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584a6c792a0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3743477342 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb180044070 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3743477342 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb180004750 con 0x5584a6c6b4a0 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 10 client.25312 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 10 client.25312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 20 client.25312 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6c88120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 4 client.25312 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.537+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584a6bde850 con 0x5584a6c85d30 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.538+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6c88120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.538+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6c88120 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8e15b3762f8c84 server_cookie=ed22096c76ad043d in_seq=0 out_seq=0 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.538+0000 7fb16d7fa640 10 client.25312 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3743477342 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584a6bde850 con 0x5584a6c85d30 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb16d7fa640 10 client.25312 handle_command_reply: tid=0 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 client.25312 shutdown 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 msgr2=0x5584a6c75ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c75ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb180000c00 tx=0x7fb18002ac10 comp rx=0 tx=0).stop 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 shutdown_connections 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6c88120 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c76680 0x5584a6c736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb16d7fa640 0 client.25312 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c6bdc0 0x5584a6c73180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 --2- 172.21.15.122:0/3743477342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c75ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 >> 172.21.15.122:0/3743477342 conn(0x5584a6c67a30 msgr2=0x5584a6c6d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 shutdown_connections 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 1 -- 172.21.15.122:0/3743477342 wait complete. 2024-09-12T01:48:46.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.539+0000 7fb18df36740 20 client.25312 trim_cache size 0 max 16384 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 Processor -- start 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 -- start start 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6b4a0 0x5584a6c66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6bdc0 0x5584a6c8d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb168010d10 con 0x5584a6c76680 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800e5e0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18df36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800c410 con 0x5584a6c6bdc0 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c8deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35710/0 (socket says 172.21.15.122:35710) 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 -- 172.21.15.122:0/3480010036 learned_addr learned my addr 172.21.15.122:0/3480010036 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6b4a0 0x5584a6c66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6bdc0 0x5584a6c8d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 -- 172.21.15.122:0/3480010036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6bdc0 msgr2=0x5584a6c8d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6bdc0 0x5584a6c8d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 -- 172.21.15.122:0/3480010036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6b4a0 msgr2=0x5584a6c66a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6b4a0 0x5584a6c66a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.557+0000 7fb18a0fa640 1 -- 172.21.15.122:0/3480010036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a6c67860 con 0x5584a6c76680 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c8deb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb17c012480 tx=0x7fb17c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbba2ac927c4e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb16f7fe640 1 -- 172.21.15.122:0/3480010036 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c01e190 con 0x5584a6c76680 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb16f7fe640 1 -- 172.21.15.122:0/3480010036 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17c01e4f0 con 0x5584a6c76680 2024-09-12T01:48:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb16f7fe640 1 -- 172.21.15.122:0/3480010036 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c005280 con 0x5584a6c76680 2024-09-12T01:48:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 -- 172.21.15.122:0/3480010036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 msgr2=0x5584a6c8deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c8deb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb17c012480 tx=0x7fb17c019000 comp rx=0 tx=0).stop 2024-09-12T01:48:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 -- 172.21.15.122:0/3480010036 shutdown_connections 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c8deb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6bdc0 0x5584a6c8d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 --2- 172.21.15.122:0/3480010036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6b4a0 0x5584a6c66a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.558+0000 7fb18df36740 1 -- 172.21.15.122:0/3480010036 >> 172.21.15.122:0/3480010036 conn(0x5584a6c67a30 msgr2=0x5584a6c66350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 -- 172.21.15.122:0/3480010036 shutdown_connections 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 -- 172.21.15.122:0/3480010036 wait complete. 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 -- start start 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 Processor -- start 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c7eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800caa0 con 0x5584a6c76680 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800df10 con 0x5584a6c6bdc0 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.559+0000 7fb18df36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800d170 con 0x5584a6c6b4a0 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb18df36740 10 client.0 _ll_register_callbacks cb 0x7fb1740d1f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c7eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c7eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53374/0 (socket says 172.21.15.122:53374) 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb18a0fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c7c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35718/0 (socket says 172.21.15.122:35718) 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1890f8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 -- 172.21.15.122:0/3268615633 learned_addr learned my addr 172.21.15.122:0/3268615633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 -- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 msgr2=0x5584a6c7c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c7c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 -- 172.21.15.122:0/3268615633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 msgr2=0x5584a6c7c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c7c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 -- 172.21.15.122:0/3268615633 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a6c7cff0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb18a0fa640 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb1898f9640 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c7eea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb18002a9a0 tx=0x7fb180003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41f937d0999b1e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb16d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3268615633 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180051040 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb17c01e4f0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.560+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3268615633 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb180040c60 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3268615633 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1800043f0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 10 client.25315 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584a6cf0a00 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3268615633 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17c01e4f0 con 0x5584a6c6b4a0 2024-09-12T01:48:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3268615633 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb180031660 con 0x5584a6c6b4a0 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 10 client.25315 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 10 client.25315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 20 client.25315 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6cf30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 4 client.25315 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584a6c73a50 con 0x5584a6c85d30 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6cf30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb1890f8640 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6cf30d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b793900d960dc6 server_cookie=53de67978d4dd4eb in_seq=0 out_seq=0 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.561+0000 7fb16d7fa640 10 client.25315 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb16d7fa640 1 -- 172.21.15.122:0/3268615633 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584a6c73a50 con 0x5584a6c85d30 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb16d7fa640 10 client.25315 handle_command_reply: tid=0 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 client.25315 shutdown 2024-09-12T01:48:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 msgr2=0x5584a6c7eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c7eea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb18002a9a0 tx=0x7fb180003c80 comp rx=0 tx=0).stop 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 shutdown_connections 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5584a6c85d30 0x5584a6cf30d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584a6c76680 0x5584a6c7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a6c6bdc0 0x5584a6c7c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 --2- 172.21.15.122:0/3268615633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584a6c6b4a0 0x5584a6c7eea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb16d7fa640 0 client.25315 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 >> 172.21.15.122:0/3268615633 conn(0x5584a6c67a30 msgr2=0x5584a6c66b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 shutdown_connections 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 1 -- 172.21.15.122:0/3268615633 wait complete. 2024-09-12T01:48:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18df36740 20 client.25315 trim_cache size 0 max 16384 2024-09-12T01:48:46.569 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:48:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-09-12T01:48:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-09-12T01:48:46.569 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:48:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1700a2b80 msgr2=0x7fb1700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1700a2b80 0x7fb1700a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0049d0 tx=0x7fb17c004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 msgr2=0x7fb184198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.562+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 0x7fb184198a40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb17400d770 tx=0x7fb17400dc30 comp rx=0 tx=0).stop 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 shutdown_connections 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1700a2b80 0x7fb1700a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb184102ef0 0x7fb18419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb184102570 0x7fb184198a40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 --2- 172.21.15.122:0/3872571460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb184100e20 0x7fb184198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 >> 172.21.15.122:0/3872571460 conn(0x7fb1840fc7d0 msgr2=0x7fb18406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 shutdown_connections 2024-09-12T01:48:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:46.563+0000 7fb18bb5d640 1 -- 172.21.15.122:0/3872571460 wait complete. 2024-09-12T01:48:46.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:48:46.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:48:46.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:46.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 92659 8.10 MB/sec warmup 64 sec latency 22.841 ms 2024-09-12T01:48:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:48:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:48:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:48:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:48:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:48:47.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:48:47.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:48:47.658 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:47.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 95953 8.25 MB/sec warmup 65 sec latency 11.115 ms 2024-09-12T01:48:47.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:47 smithi151 ceph-mon[35229]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-09-12T01:48:47.940 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:47 smithi122 ceph-mon[30925]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-09-12T01:48:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:47 smithi183 ceph-mon[35127]: pgmap v404: 97 pgs: 97 active+clean; 54 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 -- 172.21.15.122:0/3405931599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 msgr2=0x7fa3dc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/3405931599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc10d930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009cd0 tx=0x7fa3d402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 -- 172.21.15.122:0/3405931599 shutdown_connections 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/3405931599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc10d930 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/3405931599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3dc100a60 0x7fa3dc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/3405931599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3dc106a60 0x7fa3dc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 -- 172.21.15.122:0/3405931599 >> 172.21.15.122:0/3405931599 conn(0x7fa3dc0fc780 msgr2=0x7fa3dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 -- 172.21.15.122:0/3405931599 shutdown_connections 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.152+0000 7fa3e38b2640 1 -- 172.21.15.122:0/3405931599 wait complete. 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 Processor -- start 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 -- start start 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3dc100a60 0x7fa3dc197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3dc106a60 0x7fa3dc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc110e80 con 0x7fa3dc101400 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc110d00 con 0x7fa3dc106a60 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e38b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc111000 con 0x7fa3dc100a60 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc1978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35738/0 (socket says 172.21.15.122:35738) 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 -- 172.21.15.122:0/411881205 learned_addr learned my addr 172.21.15.122:0/411881205 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e1e4f640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3dc106a60 0x7fa3dc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e164e640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3dc100a60 0x7fa3dc197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 -- 172.21.15.122:0/411881205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3dc100a60 msgr2=0x7fa3dc197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3dc100a60 0x7fa3dc197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 -- 172.21.15.122:0/411881205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3dc106a60 msgr2=0x7fa3dc19bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3dc106a60 0x7fa3dc19bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.153+0000 7fa3e0e4d640 1 -- 172.21.15.122:0/411881205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc19c340 con 0x7fa3dc101400 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.154+0000 7fa3e0e4d640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc1978d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b4a0 tx=0x7fa3cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.154+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc014070 con 0x7fa3dc101400 2024-09-12T01:48:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.154+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc010af0 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.154+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc19c630 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.154+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc017d10 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.154+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc1a3f10 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.155+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc100ec0 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.156+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa3cc017070 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.156+0000 7fa3d27fc640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa3c80a2c00 0x7fa3c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.157+0000 7fa3e164e640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa3c80a2c00 0x7fa3c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.157+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa3cc0ce330 con 0x7fa3dc101400 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.157+0000 7fa3e164e640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa3c80a2c00 0x7fa3c80a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009bd0 tx=0x7fa3d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:48.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.161+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3cc09ac00 con 0x7fa3dc101400 2024-09-12T01:48:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.383+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3dc02d7c0 con 0x7fa3dc101400 2024-09-12T01:48:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.384+0000 7fa3d27fc640 1 -- 172.21.15.122:0/411881205 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11934 (secure 0 0 0) 0x7fa3cc09ef00 con 0x7fa3dc101400 2024-09-12T01:48:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:48:48.385 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"btime":"2024-09-12T01:48:18:666269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:48:17.839016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:48:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-09-12T01:48:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa3c80a2c00 msgr2=0x7fa3c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa3c80a2c00 0x7fa3c80a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009bd0 tx=0x7fa3d8005f90 comp rx=0 tx=0).stop 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 msgr2=0x7fa3dc1978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc1978d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b4a0 tx=0x7fa3cc00b960 comp rx=0 tx=0).stop 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 shutdown_connections 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa3c80a2c00 0x7fa3c80a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3dc106a60 0x7fa3dc19bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3dc101400 0x7fa3dc1978d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 --2- 172.21.15.122:0/411881205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3dc100a60 0x7fa3dc197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.386+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 >> 172.21.15.122:0/411881205 conn(0x7fa3dc0fc780 msgr2=0x7fa3dc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.387+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 shutdown_connections 2024-09-12T01:48:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:48.387+0000 7fa3e38b2640 1 -- 172.21.15.122:0/411881205 wait complete. 2024-09-12T01:48:48.699 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:48 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/411881205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:48.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 98737 8.33 MB/sec warmup 66 sec latency 41.718 ms 2024-09-12T01:48:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:48 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/411881205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:48 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/411881205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:48:49.105 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:48:49.362 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:49.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:49 smithi122 ceph-mon[30925]: pgmap v405: 97 pgs: 97 active+clean; 53 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-12T01:48:49.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 101137 8.38 MB/sec warmup 67 sec latency 34.015 ms 2024-09-12T01:48:49.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:49 smithi151 ceph-mon[35229]: pgmap v405: 97 pgs: 97 active+clean; 53 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-12T01:48:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.859+0000 7fd602e7e640 1 -- 172.21.15.122:0/3264173697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc102d30 msgr2=0x7fd5fc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.859+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3264173697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc102d30 0x7fd5fc103110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009930 tx=0x7fd5f802a420 comp rx=0 tx=0).stop 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.859+0000 7fd602e7e640 1 -- 172.21.15.122:0/3264173697 shutdown_connections 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.859+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3264173697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc1082a0 0x7fd5fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.859+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3264173697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.859+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3264173697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc102d30 0x7fd5fc103110 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- 172.21.15.122:0/3264173697 >> 172.21.15.122:0/3264173697 conn(0x7fd5fc0fc7d0 msgr2=0x7fd5fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- 172.21.15.122:0/3264173697 shutdown_connections 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- 172.21.15.122:0/3264173697 wait complete. 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 Processor -- start 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- start start 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc102d30 0x7fd5fc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1082a0 0x7fd5fc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc112100 con 0x7fd5fc1036e0 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc111f80 con 0x7fd5fc1082a0 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd602e7e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc112280 con 0x7fd5fc102d30 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35768/0 (socket says 172.21.15.122:35768) 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd5f3fff640 1 -- 172.21.15.122:0/3625291562 learned_addr learned my addr 172.21.15.122:0/3625291562 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd60141b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1082a0 0x7fd5fc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.860+0000 7fd600c1a640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc102d30 0x7fd5fc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f3fff640 1 -- 172.21.15.122:0/3625291562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc102d30 msgr2=0x7fd5fc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc102d30 0x7fd5fc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f3fff640 1 -- 172.21.15.122:0/3625291562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1082a0 msgr2=0x7fd5fc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1082a0 0x7fd5fc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f3fff640 1 -- 172.21.15.122:0/3625291562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc1a6ac0 con 0x7fd5fc1036e0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd60141b640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1082a0 0x7fd5fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc100ed0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00e9f0 tx=0x7fd5ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f1ffb640 1 -- 172.21.15.122:0/3625291562 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00ccc0 con 0x7fd5fc1036e0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f1ffb640 1 -- 172.21.15.122:0/3625291562 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ec010910 con 0x7fd5fc1036e0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd5f1ffb640 1 -- 172.21.15.122:0/3625291562 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec018920 con 0x7fd5fc1036e0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1a6db0 con 0x7fd5fc1036e0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.861+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a7270 con 0x7fd5fc1036e0 2024-09-12T01:48:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd5f1ffb640 1 -- 172.21.15.122:0/3625291562 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd5ec010ac0 con 0x7fd5fc1036e0 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 Processor -- start 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 -- start start 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 0x562481181090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481181a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56248118c370 0x56248118e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5624810e72f0 con 0x562481182b40 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56248108ecf0 con 0x56248118c370 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd605257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624810e32b0 con 0x5624811815d0 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd5f1ffb640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e40a2c00 0x7fd5e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd60141b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 0x562481181090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd60141b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 0x562481181090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35774/0 (socket says 172.21.15.122:35774) 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd60141b640 1 -- 172.21.15.122:0/3749959904 learned_addr learned my addr 172.21.15.122:0/3749959904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:49.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56248118c370 0x56248118e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.864+0000 7fd600c1a640 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481181a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e40a2c00 0x7fd5e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 -- 172.21.15.122:0/3749959904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 msgr2=0x562481181a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481181a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 -- 172.21.15.122:0/3749959904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56248118c370 msgr2=0x56248118e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56248118c370 0x56248118e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 -- 172.21.15.122:0/3749959904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624810733a0 con 0x562481182b40 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd600c1a640 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481181a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd5f1ffb640 1 -- 172.21.15.122:0/3625291562 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd5ec0cf2a0 con 0x7fd5fc1036e0 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 0x562481181090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400adc0 tx=0x7fd5f40155c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155ca67b95f9a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd5e2ffd640 1 -- 172.21.15.122:0/3749959904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f401d030 con 0x562481182b40 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd5e2ffd640 1 -- 172.21.15.122:0/3749959904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f400a900 con 0x562481182b40 2024-09-12T01:48:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd5e2ffd640 1 -- 172.21.15.122:0/3749959904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40213c0 con 0x562481182b40 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.865+0000 7fd60141b640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e40a2c00 0x7fd5e40a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4020000 tx=0x7fd5f401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 -- 172.21.15.122:0/3749959904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 msgr2=0x562481181090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 0x562481181090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400adc0 tx=0x7fd5f40155c0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 -- 172.21.15.122:0/3749959904 shutdown_connections 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56248118c370 0x56248118e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481181a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 --2- 172.21.15.122:0/3749959904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562481182b40 0x562481181090 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 -- 172.21.15.122:0/3749959904 >> 172.21.15.122:0/3749959904 conn(0x56248114a310 msgr2=0x56248117dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 -- 172.21.15.122:0/3749959904 shutdown_connections 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 -- 172.21.15.122:0/3749959904 wait complete. 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 -- start start 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.866+0000 7fd605257740 1 Processor -- start 2024-09-12T01:48:49.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 0x56248118bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481189010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56248118c370 0x562481189600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56248118f070 con 0x5624811815d0 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624810a2060 con 0x562481182b40 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56248118f1f0 con 0x56248118c370 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd605257740 10 client.0 _ll_register_callbacks cb 0x562480df58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 0x56248118bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481189010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 0x56248118bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35788/0 (socket says 172.21.15.122:35788) 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 -- 172.21.15.122:0/4228750123 learned_addr learned my addr 172.21.15.122:0/4228750123 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd60141b640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56248118c370 0x562481189600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 -- 172.21.15.122:0/4228750123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56248118c370 msgr2=0x562481189600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56248118c370 0x562481189600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 -- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 msgr2=0x562481189010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481189010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 -- 172.21.15.122:0/4228750123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562480e583c0 con 0x5624811815d0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481189010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 0x56248118bd60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd5f802a940 tx=0x7fd5f8031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbe8fe992bc1fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.867+0000 7fd5dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4228750123 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8035040 con 0x5624811815d0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4228750123 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f8004240 con 0x5624811815d0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f400a900 con 0x5624811815d0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4228750123 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5f8004440 con 0x5624811815d0 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 10 client.15972 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:49.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56248119ab60 con 0x5624811815d0 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4228750123 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f400a900 con 0x5624811815d0 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4228750123 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd5f80035f0 con 0x5624811815d0 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 20 client.15972 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x56248119bfc0 0x56248119e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 4 client.15972 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624810f4860 con 0x56248119bfc0 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.868+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x56248119bfc0 0x56248119e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.869+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x56248119bfc0 0x56248119e3b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83c10fe3e0f254f3 server_cookie=7d6a30fe97c66f9a in_seq=0 out_seq=0 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.869+0000 7fd5dbfff640 10 client.15972 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:49.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.869+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4228750123 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624810f4860 con 0x56248119bfc0 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd5dbfff640 10 client.15972 handle_command_reply: tid=0 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 client.15972 shutdown 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 msgr2=0x56248118bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 0x56248118bd60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd5f802a940 tx=0x7fd5f8031920 comp rx=0 tx=0).stop 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 shutdown_connections 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x56248119bfc0 0x56248119e3b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56248118c370 0x562481189600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481189010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 --2- 172.21.15.122:0/4228750123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5624811815d0 0x56248118bd60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd5dbfff640 0 client.15972 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 >> 172.21.15.122:0/4228750123 conn(0x56248114a310 msgr2=0x562481183960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:49.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 shutdown_connections 2024-09-12T01:48:49.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 1 -- 172.21.15.122:0/4228750123 wait complete. 2024-09-12T01:48:49.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.870+0000 7fd605257740 20 client.15972 trim_cache size 0 max 16384 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 Processor -- start 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 -- start start 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x56248117dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481184120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481184660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1b2030 con 0x56248118c370 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1af900 con 0x562481182b40 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd605257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ad730 con 0x5624811815d0 2024-09-12T01:48:49.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd60141b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481184660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481184120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd60141b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481184660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35794/0 (socket says 172.21.15.122:35794) 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481184120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52408/0 (socket says 172.21.15.122:52408) 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.883+0000 7fd60141b640 1 -- 172.21.15.122:0/298025809 learned_addr learned my addr 172.21.15.122:0/298025809 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd600c1a640 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x56248117dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd60141b640 1 -- 172.21.15.122:0/298025809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 msgr2=0x56248117dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd60141b640 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x56248117dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd60141b640 1 -- 172.21.15.122:0/298025809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 msgr2=0x562481184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd60141b640 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481184120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd60141b640 1 -- 172.21.15.122:0/298025809 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56248118c0d0 con 0x56248118c370 2024-09-12T01:48:49.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481184120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd600c1a640 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x56248117dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd60141b640 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481184660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009440 tx=0x7fd5f40096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58072c141da25aac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd5e27fc640 1 -- 172.21.15.122:0/298025809 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4030070 con 0x56248118c370 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd5e27fc640 1 -- 172.21.15.122:0/298025809 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f4026070 con 0x56248118c370 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.884+0000 7fd5e27fc640 1 -- 172.21.15.122:0/298025809 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f401d040 con 0x56248118c370 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 -- 172.21.15.122:0/298025809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 msgr2=0x562481184660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481184660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009440 tx=0x7fd5f40096b0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 -- 172.21.15.122:0/298025809 shutdown_connections 2024-09-12T01:48:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481184660 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481184120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 --2- 172.21.15.122:0/298025809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x56248117dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 -- 172.21.15.122:0/298025809 >> 172.21.15.122:0/298025809 conn(0x56248114a310 msgr2=0x56248117cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 -- 172.21.15.122:0/298025809 shutdown_connections 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 -- 172.21.15.122:0/298025809 wait complete. 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 -- start start 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.885+0000 7fd605257740 1 Processor -- start 2024-09-12T01:48:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481186050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481186590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481186ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1addc0 con 0x56248118c370 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1af230 con 0x562481182b40 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ae490 con 0x5624811815d0 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd605257740 10 client.0 _ll_register_callbacks cb 0x7fd5f8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481186ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481186ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35808/0 (socket says 172.21.15.122:35808) 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 -- 172.21.15.122:0/4124606974 learned_addr learned my addr 172.21.15.122:0/4124606974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481186590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481186050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 -- 172.21.15.122:0/4124606974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 msgr2=0x562481186050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481186050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 -- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 msgr2=0x562481186590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481186590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd60141b640 1 -- 172.21.15.122:0/4124606974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562481205c00 con 0x56248118c370 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481186050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.886+0000 7fd5f3fff640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481186590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd60141b640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481186ad0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400ad90 tx=0x7fd5f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ef759fb66947ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd5dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4124606974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f403b060 con 0x56248118c370 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4124606974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f4021690 con 0x56248118c370 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f4026070 con 0x56248118c370 2024-09-12T01:48:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4124606974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5f4030070 con 0x56248118c370 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562481206ec0 con 0x56248118c370 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4124606974 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f4026070 con 0x56248118c370 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4124606974 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd5f4036740 con 0x56248118c370 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 20 client.15978 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.887+0000 7fd605257740 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562481207390 0x562481209780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.888+0000 7fd605257740 4 client.15978 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.888+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 --> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624810eed70 con 0x562481207390 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.888+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562481207390 0x562481209780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.888+0000 7fd5dbfff640 10 client.15978 ms_handle_connect on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.888+0000 7fd600c1a640 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562481207390 0x562481209780 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=368c524e92aeb171 server_cookie=230f95bf1c22ad79 in_seq=0 out_seq=0 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd5dbfff640 1 -- 172.21.15.122:0/4124606974 <== mds.2 v2:172.21.15.151:6836/3984907767 1 ==== command_reply(tid 0: 0 ) ==== 8+0+14560 (crc 0 0 0) 0x5624810eed70 con 0x562481207390 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd5dbfff640 10 client.15978 handle_command_reply: tid=0 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 client.15978 shutdown 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 msgr2=0x562481186ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481186ad0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400ad90 tx=0x7fd5f4004a40 comp rx=0 tx=0).stop 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 shutdown_connections 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:6836/3984907767,v1:172.21.15.151:6837/3984907767] conn(0x562481207390 0x562481209780 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56248118c370 0x562481186ad0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562481182b40 0x562481186590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 --2- 172.21.15.122:0/4124606974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624811815d0 0x562481186050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd5dbfff640 0 client.15978 ms_handle_reset on v2:172.21.15.151:6836/3984907767 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 >> 172.21.15.122:0/4124606974 conn(0x56248114a310 msgr2=0x56248118a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 shutdown_connections 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 1 -- 172.21.15.122:0/4124606974 wait complete. 2024-09-12T01:48:49.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.889+0000 7fd605257740 20 client.15978 trim_cache size 0 max 16384 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 49878873, 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628345, 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03" 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 110090539, 2024-09-12T01:48:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628337, 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0" 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 328977311, 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628309, 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL" 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 400223667, 2024-09-12T01:48:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628332, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04" 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 431945318, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628333, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04" 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 458145652, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628295, 2024-09-12T01:48:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.004" 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 475304002, 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628311, 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 573335918, 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628335, 2024-09-12T01:48:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX" 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 605755074, 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628321, 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORDPRO" 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 606885819, 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628297, 2024-09-12T01:48:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC" 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 692289559, 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628348, 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL" 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 801928416, 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628316, 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2024-09-12T01:48:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1024362100, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628339, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1" 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1033846540, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628299, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC" 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1105167294, 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628304, 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1131112544, 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628346, 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL" 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1229489712, 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628318, 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1259429202, 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628341, 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB" 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1291118371, 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628355, 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL" 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1303562831, 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628329, 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX" 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1346304330, 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628325, 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM" 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1392191144, 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628347, 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV" 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1394214234, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628308, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1459630752, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628317, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1654266283, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628327, 2024-09-12T01:48:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1690769850, 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511627785, 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp" 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1744402955, 2024-09-12T01:48:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628306, 2024-09-12T01:48:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-09-12T01:48:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1857508100, 2024-09-12T01:48:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628314, 2024-09-12T01:48:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2024-09-12T01:48:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 1859171131, 2024-09-12T01:48:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628303, 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT" 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2061731507, 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511627784, 2024-09-12T01:48:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0" 2024-09-12T01:48:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2114237310, 2024-09-12T01:48:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628338, 2024-09-12T01:48:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1" 2024-09-12T01:48:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2165299386, 2024-09-12T01:48:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628294, 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.003" 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2280732680, 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628298, 2024-09-12T01:48:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC" 2024-09-12T01:48:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2283962765, 2024-09-12T01:48:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628320, 2024-09-12T01:48:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2024-09-12T01:48:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2420400403, 2024-09-12T01:48:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628343, 2024-09-12T01:48:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX" 2024-09-12T01:48:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2437247993, 2024-09-12T01:48:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628319, 2024-09-12T01:48:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2024-09-12T01:48:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2448760793, 2024-09-12T01:48:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628340, 2024-09-12T01:48:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL" 2024-09-12T01:48:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2502401533, 2024-09-12T01:48:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511627786, 2024-09-12T01:48:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.000" 2024-09-12T01:48:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2813821176, 2024-09-12T01:48:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628342, 2024-09-12T01:48:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB" 2024-09-12T01:48:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2836932630, 2024-09-12T01:48:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628334, 2024-09-12T01:48:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB" 2024-09-12T01:48:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2932813215, 2024-09-12T01:48:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628354, 2024-09-12T01:48:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL" 2024-09-12T01:48:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2991007203, 2024-09-12T01:48:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628312, 2024-09-12T01:48:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2024-09-12T01:48:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2991690014, 2024-09-12T01:48:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628328, 2024-09-12T01:48:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2024-09-12T01:48:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3018225338, 2024-09-12T01:48:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628353, 2024-09-12T01:48:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL" 2024-09-12T01:48:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3055812580, 2024-09-12T01:48:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628310, 2024-09-12T01:48:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2024-09-12T01:48:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3085745597, 2024-09-12T01:48:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628315, 2024-09-12T01:48:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2024-09-12T01:48:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3118380721, 2024-09-12T01:48:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628296, 2024-09-12T01:48:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD" 2024-09-12T01:48:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3134662022, 2024-09-12T01:48:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628292, 2024-09-12T01:48:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.001" 2024-09-12T01:48:49.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.943 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.943 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3140997507, 2024-09-12T01:48:49.943 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628351, 2024-09-12T01:48:49.943 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB" 2024-09-12T01:48:49.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.944 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.944 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3184517587, 2024-09-12T01:48:49.944 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628350, 2024-09-12T01:48:49.944 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS" 2024-09-12T01:48:49.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.945 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.945 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3247996233, 2024-09-12T01:48:49.945 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628305, 2024-09-12T01:48:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP" 2024-09-12T01:48:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3276761257, 2024-09-12T01:48:49.946 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628302, 2024-09-12T01:48:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2024-09-12T01:48:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3452356296, 2024-09-12T01:48:49.947 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628336, 2024-09-12T01:48:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0" 2024-09-12T01:48:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.948 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3536023104, 2024-09-12T01:48:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628331, 2024-09-12T01:48:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX" 2024-09-12T01:48:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.949 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3585474328, 2024-09-12T01:48:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628307, 2024-09-12T01:48:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT" 2024-09-12T01:48:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.950 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3598582488, 2024-09-12T01:48:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628330, 2024-09-12T01:48:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB" 2024-09-12T01:48:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.951 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3769615942, 2024-09-12T01:48:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628313, 2024-09-12T01:48:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2024-09-12T01:48:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3781786739, 2024-09-12T01:48:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628349, 2024-09-12T01:48:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL" 2024-09-12T01:48:49.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3794561998, 2024-09-12T01:48:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628300, 2024-09-12T01:48:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL" 2024-09-12T01:48:49.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3957869387, 2024-09-12T01:48:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628293, 2024-09-12T01:48:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.002" 2024-09-12T01:48:49.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3994436319, 2024-09-12T01:48:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628344, 2024-09-12T01:48:49.956 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03" 2024-09-12T01:48:49.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.957 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.957 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4010718180, 2024-09-12T01:48:49.958 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628301, 2024-09-12T01:48:49.958 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-09-12T01:48:49.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.958 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.959 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4062411078, 2024-09-12T01:48:49.959 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628352, 2024-09-12T01:48:49.960 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED" 2024-09-12T01:48:49.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:48:49.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T01:48:49.960 INFO:teuthology.orchestra.run.smithi122.stdout: "damage_type": "backtrace", 2024-09-12T01:48:49.960 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4133343826, 2024-09-12T01:48:49.960 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1099511628326, 2024-09-12T01:48:49.961 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2024-09-12T01:48:49.962 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:48:49.962 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-12T01:48:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e40a2c00 msgr2=0x7fd5e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e40a2c00 0x7fd5e40a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4020000 tx=0x7fd5f401e000 comp rx=0 tx=0).stop 2024-09-12T01:48:49.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 msgr2=0x7fd5fc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:49.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc100ed0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00e9f0 tx=0x7fd5ec00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 shutdown_connections 2024-09-12T01:48:49.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e40a2c00 0x7fd5e40a50c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5fc1082a0 0x7fd5fc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5fc1036e0 0x7fd5fc100ed0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 --2- 172.21.15.122:0/3625291562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5fc102d30 0x7fd5fc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 >> 172.21.15.122:0/3625291562 conn(0x7fd5fc0fc7d0 msgr2=0x7fd5fc105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 shutdown_connections 2024-09-12T01:48:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:49.890+0000 7fd602e7e640 1 -- 172.21.15.122:0/3625291562 wait complete. 2024-09-12T01:48:49.965 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:49 smithi183 ceph-mon[35127]: pgmap v405: 97 pgs: 97 active+clean; 53 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-12T01:48:50.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 103498 8.46 MB/sec warmup 68 sec latency 72.274 ms 2024-09-12T01:48:50.918 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:48:51.195 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- 172.21.15.122:0/1163884765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00102600 msgr2=0x7f7e00108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- 172.21.15.122:0/1163884765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00102600 0x7f7e00108e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ca0 tx=0x7f7df802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- 172.21.15.122:0/1163884765 shutdown_connections 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- 172.21.15.122:0/1163884765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00102600 0x7f7e00108e90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- 172.21.15.122:0/1163884765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 0x7f7e001020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- 172.21.15.122:0/1163884765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e00100a50 0x7f7e00100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- 172.21.15.122:0/1163884765 >> 172.21.15.122:0/1163884765 conn(0x7f7e000fc1b0 msgr2=0x7f7e000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- 172.21.15.122:0/1163884765 shutdown_connections 2024-09-12T01:48:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- 172.21.15.122:0/1163884765 wait complete. 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 Processor -- start 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- start start 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 0x7f7e001049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 0x7f7e00103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e00102600 0x7f7e00103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010c6b0 con 0x7f7e00101c40 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010c530 con 0x7f7e00100a50 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f7e073b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0010c830 con 0x7f7e00102600 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 0x7f7e001049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 0x7f7e001049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36014/0 (socket says 172.21.15.122:36014) 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 -- 172.21.15.122:0/691669267 learned_addr learned my addr 172.21.15.122:0/691669267 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e04952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 0x7f7e00103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05954640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e00102600 0x7f7e00103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 -- 172.21.15.122:0/691669267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e00102600 msgr2=0x7f7e00103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e00102600 0x7f7e00103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 -- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 msgr2=0x7f7e00103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 0x7f7e00103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 -- 172.21.15.122:0/691669267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e00103df0 con 0x7f7e00100a50 2024-09-12T01:48:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e04952640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 0x7f7e00103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f7e05153640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 0x7f7e001049e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00e9c0 tx=0x7f7dfc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 -- 172.21.15.122:0/4077135880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102570 msgr2=0x7f446c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 --2- 172.21.15.122:0/4077135880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102570 0x7f446c1029b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4458009900 tx=0x7f445802a480 comp rx=0 tx=0).stop 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 -- 172.21.15.122:0/4077135880 shutdown_connections 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 --2- 172.21.15.122:0/4077135880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c102ef0 0x7f446c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 --2- 172.21.15.122:0/4077135880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102570 0x7f446c1029b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 --2- 172.21.15.122:0/4077135880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c100e20 0x7f446c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 -- 172.21.15.122:0/4077135880 >> 172.21.15.122:0/4077135880 conn(0x7f446c0fc7d0 msgr2=0x7f446c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 -- 172.21.15.122:0/4077135880 shutdown_connections 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.780+0000 7f4473009640 1 -- 172.21.15.122:0/4077135880 wait complete. 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 Processor -- start 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 -- start start 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c100e20 0x7f446c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 0x7f446c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102ef0 0x7f446c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c110f50 con 0x7f446c102ef0 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f446c110dd0 con 0x7f446c102570 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4473009640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1110d0 con 0x7f446c100e20 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f446bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 0x7f446c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4470da5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c100e20 0x7f446c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f446bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 0x7f446c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:35988/0 (socket says 172.21.15.122:35988) 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f4470da5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c100e20 0x7f446c1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57294/0 (socket says 172.21.15.122:57294) 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.781+0000 7f446bfff640 1 -- 172.21.15.122:0/467956113 learned_addr learned my addr 172.21.15.122:0/467956113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f44715a6640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102ef0 0x7f446c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f446bfff640 1 -- 172.21.15.122:0/467956113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c100e20 msgr2=0x7f446c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f446bfff640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c100e20 0x7f446c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f446bfff640 1 -- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102ef0 msgr2=0x7f446c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f446bfff640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102ef0 0x7f446c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f446bfff640 1 -- 172.21.15.122:0/467956113 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c19d4a0 con 0x7f446c102570 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f446bfff640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 0x7f446c198a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4458006010 tx=0x7f4458003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f4469ffb640 1 -- 172.21.15.122:0/467956113 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445803f070 con 0x7f446c102570 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.782+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c19d730 con 0x7f446c102570 2024-09-12T01:48:51.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.783+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c1a5070 con 0x7f446c102570 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.783+0000 7f4469ffb640 1 -- 172.21.15.122:0/467956113 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4458031c30 con 0x7f446c102570 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.783+0000 7f4469ffb640 1 -- 172.21.15.122:0/467956113 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458035030 con 0x7f446c102570 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.785+0000 7f4469ffb640 1 -- 172.21.15.122:0/467956113 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4458042070 con 0x7f446c102570 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f4469ffb640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f445c0a2c80 0x7f445c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 Processor -- start 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 -- start start 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f4470da5640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f445c0a2c80 0x7f445c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f4469ffb640 1 -- 172.21.15.122:0/467956113 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f44580ef660 con 0x7f446c102570 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f6ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f82f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563a18ed32f0 con 0x563a18f6e840 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a18e7acf0 con 0x563a18f6db80 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44753e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a18ecf2b0 con 0x563a18f6d120 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f4470da5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f6ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44715a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f82f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f446bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f4470da5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f6ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53128/0 (socket says 172.21.15.122:53128) 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44715a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f82f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36020/0 (socket says 172.21.15.122:36020) 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f446bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f6d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57334/0 (socket says 172.21.15.122:57334) 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.786+0000 7f44715a6640 1 -- 172.21.15.122:0/2280194541 learned_addr learned my addr 172.21.15.122:0/2280194541 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f4470da5640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f445c0a2c80 0x7f445c0a5140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4460009c20 tx=0x7f4460011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f446bfff640 1 -- 172.21.15.122:0/2280194541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 msgr2=0x563a18f82f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f7df67fc640 1 -- 172.21.15.122:0/691669267 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00cd80 con 0x7f7e00100a50 2024-09-12T01:48:51.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f446bfff640 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f82f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f446bfff640 1 -- 172.21.15.122:0/2280194541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 msgr2=0x563a18f6ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f446bfff640 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f6ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f446bfff640 1 -- 172.21.15.122:0/2280194541 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a18e5f3a0 con 0x563a18f6d120 2024-09-12T01:48:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f7df67fc640 1 -- 172.21.15.122:0/691669267 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dfc0035f0 con 0x7f7e00100a50 2024-09-12T01:48:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f7df67fc640 1 -- 172.21.15.122:0/691669267 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc010690 con 0x7f7e00100a50 2024-09-12T01:48:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e001a1e80 con 0x7f7e00100a50 2024-09-12T01:48:51.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f446bfff640 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f6d560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44580ed0f0 tx=0x7f44580bbdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9ff14caab79b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e001a22e0 con 0x7f7e00100a50 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.787+0000 7f4456ffd640 1 -- 172.21.15.122:0/2280194541 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458049050 con 0x563a18f6d120 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.788+0000 7f4456ffd640 1 -- 172.21.15.122:0/2280194541 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44580c0230 con 0x563a18f6d120 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.788+0000 7f4456ffd640 1 -- 172.21.15.122:0/2280194541 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445803b580 con 0x563a18f6d120 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 -- 172.21.15.122:0/2280194541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 msgr2=0x563a18f6d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f6d560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44580ed0f0 tx=0x7f44580bbdc0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 -- 172.21.15.122:0/2280194541 shutdown_connections 2024-09-12T01:48:51.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f82f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f6d560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 --2- 172.21.15.122:0/2280194541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f6ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 -- 172.21.15.122:0/2280194541 >> 172.21.15.122:0/2280194541 conn(0x563a18f36310 msgr2=0x563a18f69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 -- 172.21.15.122:0/2280194541 shutdown_connections 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 -- 172.21.15.122:0/2280194541 wait complete. 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 -- start start 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.789+0000 7f44753e2740 1 Processor -- start 2024-09-12T01:48:51.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f82e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f80120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563a18f836f0 con 0x563a18f6e840 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a18e8e060 con 0x563a18f6db80 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a18f83870 con 0x563a18f6d120 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f80120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f4470da5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f82e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 Processor -- start 2024-09-12T01:48:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 -- start start 2024-09-12T01:48:51.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f80120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36036/0 (socket says 172.21.15.122:36036) 2024-09-12T01:48:51.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f4470da5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f82e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57338/0 (socket says 172.21.15.122:57338) 2024-09-12T01:48:51.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 -- 172.21.15.122:0/2761726191 learned_addr learned my addr 172.21.15.122:0/2761726191 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44753e2740 10 client.0 _ll_register_callbacks cb 0x563a18be18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:51.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f44715a6640 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e793dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 -- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 msgr2=0x563a18f82e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f82e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 -- 172.21.15.122:0/2761726191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 msgr2=0x563a18f806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793e2e0 0x5572e7942da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e7945830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5572e78a12f0 con 0x5572e7943370 2024-09-12T01:48:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572e7849270 con 0x5572e793e2e0 2024-09-12T01:48:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572e789d2b0 con 0x5572e793d9a0 2024-09-12T01:48:51.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.790+0000 7f446bfff640 1 -- 172.21.15.122:0/2761726191 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a18c443c0 con 0x563a18f6db80 2024-09-12T01:48:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e7945830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e7945830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53150/0 (socket says 172.21.15.122:53150) 2024-09-12T01:48:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 -- 172.21.15.122:0/1821997685 learned_addr learned my addr 172.21.15.122:0/1821997685 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e05954640 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e793dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f446bfff640 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f80120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44580ed170 tx=0x7f44580c4af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a47e3a7a272175 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f444bfff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:48:51.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f444bfff640 1 -- 172.21.15.122:0/2761726191 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44580d8030 con 0x563a18f6db80 2024-09-12T01:48:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 -- 172.21.15.122:0/1821997685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 msgr2=0x5572e793dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e793dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f444bfff640 1 -- 172.21.15.122:0/2761726191 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f44580b85a0 con 0x563a18f6db80 2024-09-12T01:48:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 -- 172.21.15.122:0/1821997685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793e2e0 msgr2=0x5572e7942da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:48:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793e2e0 0x5572e7942da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 -- 172.21.15.122:0/1821997685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e7848340 con 0x5572e7943370 2024-09-12T01:48:51.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44580c0230 con 0x563a18f6db80 2024-09-12T01:48:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7df67fc640 1 -- 172.21.15.122:0/691669267 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7dfc0037a0 con 0x7f7e00100a50 2024-09-12T01:48:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7e04952640 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e7945830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7df000bcb0 tx=0x7f7df000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f7e345a6f59302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f444bfff640 1 -- 172.21.15.122:0/2761726191 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4458049020 con 0x563a18f6db80 2024-09-12T01:48:51.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f44753e2740 10 client.25361 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:51.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563a18f836f0 con 0x563a18f6db80 2024-09-12T01:48:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7deb7fe640 1 -- 172.21.15.122:0/1821997685 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0011440 con 0x5572e7943370 2024-09-12T01:48:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7deb7fe640 1 -- 172.21.15.122:0/1821997685 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0011ac0 con 0x5572e7943370 2024-09-12T01:48:51.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f444bfff640 1 -- 172.21.15.122:0/2761726191 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44580c0230 con 0x563a18f6db80 2024-09-12T01:48:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.791+0000 7f7deb7fe640 1 -- 172.21.15.122:0/1821997685 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df001bb50 con 0x5572e7943370 2024-09-12T01:48:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7df67fc640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7dec0a2cb0 0x7f7dec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f444bfff640 1 -- 172.21.15.122:0/2761726191 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44580b7c80 con 0x563a18f6db80 2024-09-12T01:48:51.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f44753e2740 10 client.25361 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:51.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f44753e2740 10 client.25361 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:48:51.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f44753e2740 20 client.25361 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:51.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f44753e2740 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18f90260 0x563a18f92650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f44753e2740 4 client.25361 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:51.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a18ee0860 con 0x563a18f90260 2024-09-12T01:48:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1821997685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 msgr2=0x5572e7945830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e7945830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7df000bcb0 tx=0x7f7df000f730 comp rx=0 tx=0).stop 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e05954640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7dec0a2cb0 0x7f7dec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1821997685 shutdown_connections 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e7945830 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793e2e0 0x5572e7942da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1821997685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e793dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1821997685 >> 172.21.15.122:0/1821997685 conn(0x5572e7939010 msgr2=0x5572e7939410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f7df67fc640 1 -- 172.21.15.122:0/691669267 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7dfc0cf4f0 con 0x7f7e00100a50 2024-09-12T01:48:51.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.792+0000 7f4470da5640 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18f90260 0x563a18f92650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1821997685 shutdown_connections 2024-09-12T01:48:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1821997685 wait complete. 2024-09-12T01:48:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e05954640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7dec0a2cb0 0x7f7dec0a5170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005e30 tx=0x7f7df8031990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e0a22d740 1 -- start start 2024-09-12T01:48:51.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f444bfff640 10 client.25361 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:51.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f4470da5640 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18f90260 0x563a18f92650 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bda3da3daa16db0 server_cookie=638ca892c0ed6034 in_seq=0 out_seq=0 2024-09-12T01:48:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e0a22d740 1 Processor -- start 2024-09-12T01:48:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e7947c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e793e2e0 0x5572e79481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e7943370 0x5572e794f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5572e79463e0 con 0x5572e793e2e0 2024-09-12T01:48:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572e782a290 con 0x5572e7943370 2024-09-12T01:48:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572e7946560 con 0x5572e793d9a0 2024-09-12T01:48:51.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f444bfff640 1 -- 172.21.15.122:0/2761726191 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a18ee0860 con 0x563a18f90260 2024-09-12T01:48:51.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.793+0000 7f444bfff640 10 client.25361 handle_command_reply: tid=0 2024-09-12T01:48:51.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 client.25361 shutdown 2024-09-12T01:48:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e0a22d740 10 client.0 _ll_register_callbacks cb 0x5572e785b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e05153640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e7947c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e793e2e0 0x5572e79481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e05153640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e7947c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57356/0 (socket says 172.21.15.122:57356) 2024-09-12T01:48:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e05153640 1 -- 172.21.15.122:0/2795635059 learned_addr learned my addr 172.21.15.122:0/2795635059 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e05954640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e7943370 0x5572e794f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 msgr2=0x563a18f80120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f80120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44580ed170 tx=0x7f44580c4af0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 shutdown_connections 2024-09-12T01:48:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18f90260 0x563a18f92650 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6db80 0x563a18f80120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 -- 172.21.15.122:0/2795635059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 msgr2=0x5572e7947c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e7947c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 -- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e7943370 msgr2=0x5572e794f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e7943370 0x5572e794f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 -- 172.21.15.122:0/2795635059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e75b4210 con 0x5572e793e2e0 2024-09-12T01:48:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e05954640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e7943370 0x5572e794f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:51.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 --2- 172.21.15.122:0/2761726191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6d120 0x563a18f82e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 >> 172.21.15.122:0/2761726191 conn(0x563a18f36310 msgr2=0x563a18f68d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7e04952640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e793e2e0 0x5572e79481d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7df000fc30 tx=0x7f7df000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a3873099753b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f444bfff640 0 client.25361 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f7de8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:51.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 shutdown_connections 2024-09-12T01:48:51.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.794+0000 7f44753e2740 1 -- 172.21.15.122:0/2761726191 wait complete. 2024-09-12T01:48:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7de8ff9640 1 -- 172.21.15.122:0/2795635059 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df002a060 con 0x5572e793e2e0 2024-09-12T01:48:51.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f44753e2740 20 client.25361 trim_cache size 0 max 16384 2024-09-12T01:48:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df0011ac0 con 0x5572e793e2e0 2024-09-12T01:48:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7de8ff9640 1 -- 172.21.15.122:0/2795635059 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7df0004250 con 0x5572e793e2e0 2024-09-12T01:48:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7de8ff9640 1 -- 172.21.15.122:0/2795635059 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7df0004450 con 0x5572e793e2e0 2024-09-12T01:48:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 10 client.16002 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572e7680620 con 0x5572e793e2e0 2024-09-12T01:48:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7de8ff9640 1 -- 172.21.15.122:0/2795635059 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0011ac0 con 0x5572e793e2e0 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7de8ff9640 1 -- 172.21.15.122:0/2795635059 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7df002f070 con 0x5572e793e2e0 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 20 client.16002 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79535d0 0x5572e79559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 4 client.16002 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.795+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e78aea20 con 0x5572e79535d0 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.796+0000 7f7e05153640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79535d0 0x5572e79559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.796+0000 7f7de8ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.796+0000 7f7e05153640 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79535d0 0x5572e79559c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127ed4693f5bb79f server_cookie=633a9a072bc3c90d in_seq=0 out_seq=0 2024-09-12T01:48:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7de8ff9640 1 -- 172.21.15.122:0/2795635059 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572e78aea20 con 0x5572e79535d0 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7de8ff9640 10 client.16002 handle_command_reply: tid=0 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 client.16002 shutdown 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e793e2e0 msgr2=0x5572e79481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e793e2e0 0x5572e79481d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7df000fc30 tx=0x7f7df000bfd0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 shutdown_connections 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79535d0 0x5572e79559c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7de8ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e7943370 0x5572e794f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e793e2e0 0x5572e79481d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/2795635059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e7947c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 >> 172.21.15.122:0/2795635059 conn(0x5572e7939010 msgr2=0x5572e793a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 shutdown_connections 2024-09-12T01:48:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 1 -- 172.21.15.122:0/2795635059 wait complete. 2024-09-12T01:48:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.797+0000 7f7e0a22d740 20 client.16002 trim_cache size 0 max 16384 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 Processor -- start 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 -- start start 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 0x563a18f800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f73e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6e840 0x563a18f6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1adc00 con 0x563a18f6d120 2024-09-12T01:48:51.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1ab4d0 con 0x563a18f6e840 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.808+0000 7f44753e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a9300 con 0x563a18f6db80 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 0x563a18f800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 0x563a18f800f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53160/0 (socket says 172.21.15.122:53160) 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 -- 172.21.15.122:0/114586141 learned_addr learned my addr 172.21.15.122:0/114586141 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f446bfff640 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f73e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 -- 172.21.15.122:0/114586141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 msgr2=0x563a18f73e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f73e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 -- 172.21.15.122:0/114586141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6e840 msgr2=0x563a18f6fc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6e840 0x563a18f6fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 -- 172.21.15.122:0/114586141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a18f7f0d0 con 0x563a18f6d120 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f4470da5640 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 0x563a18f800f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f446001d6d0 tx=0x7f446001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f120ffab9687e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f44567fc640 1 -- 172.21.15.122:0/114586141 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460017040 con 0x563a18f6d120 2024-09-12T01:48:51.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f44567fc640 1 -- 172.21.15.122:0/114586141 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4460019050 con 0x563a18f6d120 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.809+0000 7f44567fc640 1 -- 172.21.15.122:0/114586141 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460028690 con 0x563a18f6d120 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 -- 172.21.15.122:0/114586141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 msgr2=0x563a18f800f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 0x563a18f800f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f446001d6d0 tx=0x7f446001c000 comp rx=0 tx=0).stop 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 -- 172.21.15.122:0/114586141 shutdown_connections 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6e840 0x563a18f6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f73e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 --2- 172.21.15.122:0/114586141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6d120 0x563a18f800f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 -- 172.21.15.122:0/114586141 >> 172.21.15.122:0/114586141 conn(0x563a18f36310 msgr2=0x563a18f7f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 -- 172.21.15.122:0/114586141 shutdown_connections 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 -- 172.21.15.122:0/114586141 wait complete. 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.810+0000 7f44753e2740 1 -- start start 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 Processor -- start 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6d120 0x563a18f97410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f97950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a9990 con 0x563a18f6e840 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1aae00 con 0x563a18f6d120 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1aa060 con 0x563a18f6db80 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44753e2740 10 client.0 _ll_register_callbacks cb 0x7f44580d6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f97950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f97950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57376/0 (socket says 172.21.15.122:57376) 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f44715a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 -- 172.21.15.122:0/2649533511 learned_addr learned my addr 172.21.15.122:0/2649533511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f4470da5640 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6d120 0x563a18f97410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 -- 172.21.15.122:0/2649533511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6d120 msgr2=0x563a18f97410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6d120 0x563a18f97410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 -- 172.21.15.122:0/2649533511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 msgr2=0x563a18f87ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f87ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.811+0000 7f446bfff640 1 -- 172.21.15.122:0/2649533511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a18f88100 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f446bfff640 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f97950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44580b8900 tx=0x7f44580bb630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4f98daa03a9df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f444bfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f444bfff640 1 -- 172.21.15.122:0/2649533511 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44580d8070 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f444bfff640 1 -- 172.21.15.122:0/2649533511 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f44580c4860 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4460019050 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f444bfff640 1 -- 172.21.15.122:0/2649533511 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44580bb890 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f44753e2740 10 client.25363 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563a18f88100 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f444bfff640 1 -- 172.21.15.122:0/2649533511 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4460019050 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.812+0000 7f444bfff640 1 -- 172.21.15.122:0/2649533511 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44580daac0 con 0x563a18f6db80 2024-09-12T01:48:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 10 client.25363 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 10 client.25363 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 20 client.25363 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18ffae50 0x563a18ffd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 4 client.25363 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 --> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a18edad70 con 0x563a18ffae50 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f4470da5640 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18ffae50 0x563a18ffd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f444bfff640 10 client.25363 ms_handle_connect on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f4470da5640 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18ffae50 0x563a18ffd240 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3021167cd8132bcd server_cookie=913a5a3f570ef55e in_seq=0 out_seq=0 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f444bfff640 1 -- 172.21.15.122:0/2649533511 <== mds.1 v2:172.21.15.183:6836/3258783936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a18edad70 con 0x563a18ffae50 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f444bfff640 10 client.25363 handle_command_reply: tid=0 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 1 client.25363 shutdown 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 msgr2=0x563a18f97950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.813+0000 7f44753e2740 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f97950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44580b8900 tx=0x7f44580bb630 comp rx=0 tx=0).stop 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 shutdown_connections 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:6836/3258783936,v1:172.21.15.183:6837/3258783936] conn(0x563a18ffae50 0x563a18ffd240 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a18f6e840 0x563a18f87ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a18f6db80 0x563a18f97950 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 --2- 172.21.15.122:0/2649533511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a18f6d120 0x563a18f97410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f444bfff640 0 client.25363 ms_handle_reset on v2:172.21.15.183:6836/3258783936 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 >> 172.21.15.122:0/2649533511 conn(0x563a18f36310 msgr2=0x563a18f684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 shutdown_connections 2024-09-12T01:48:51.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 1 -- 172.21.15.122:0/2649533511 wait complete. 2024-09-12T01:48:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 Processor -- start 2024-09-12T01:48:51.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f44753e2740 20 client.25363 trim_cache size 0 max 16384 2024-09-12T01:48:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 -- start start 2024-09-12T01:48:51.837 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:48:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 0x5572e793dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e79380b0 0x5572e793a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001adb50 con 0x5572e7943370 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001ab420 con 0x5572e793d9a0 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001a9250 con 0x5572e79380b0 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e04952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e05153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 0x5572e793dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e05153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 0x5572e793dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36064/0 (socket says 172.21.15.122:36064) 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e04952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53166/0 (socket says 172.21.15.122:53166) 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f7e05954640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e79380b0 0x5572e793a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 -- 172.21.15.122:0/346487672 learned_addr learned my addr 172.21.15.122:0/346487672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f445c0a2c80 msgr2=0x7f445c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.814+0000 7f4473009640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f445c0a2c80 0x7f445c0a5140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4460009c20 tx=0x7f4460011810 comp rx=0 tx=0).stop 2024-09-12T01:48:51.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 msgr2=0x7f446c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 0x7f446c198a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4458006010 tx=0x7f4458003c10 comp rx=0 tx=0).stop 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 shutdown_connections 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f445c0a2c80 0x7f445c0a5140 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c102ef0 0x7f446c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f446c102570 0x7f446c198a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 --2- 172.21.15.122:0/467956113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f446c100e20 0x7f446c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 >> 172.21.15.122:0/467956113 conn(0x7f446c0fc7d0 msgr2=0x7f446c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 shutdown_connections 2024-09-12T01:48:51.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f4473009640 1 -- 172.21.15.122:0/467956113 wait complete. 2024-09-12T01:48:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 -- 172.21.15.122:0/346487672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e79380b0 msgr2=0x5572e793a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e79380b0 0x5572e793a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 -- 172.21.15.122:0/346487672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 msgr2=0x5572e794f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 -- 172.21.15.122:0/346487672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e7938e30 con 0x5572e793d9a0 2024-09-12T01:48:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7e05153640 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 0x5572e793dd80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc09bca0 tx=0x7f7dfc09e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca80f2cf8589beb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7deaffd640 1 -- 172.21.15.122:0/346487672 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0a00c0 con 0x5572e793d9a0 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7deaffd640 1 -- 172.21.15.122:0/346487672 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5572e78aea20 con 0x5572e793d9a0 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.815+0000 7f7deaffd640 1 -- 172.21.15.122:0/346487672 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0cf0d0 con 0x5572e793d9a0 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 -- 172.21.15.122:0/346487672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 msgr2=0x5572e793dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 0x5572e793dd80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc09bca0 tx=0x7f7dfc09e040 comp rx=0 tx=0).stop 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 -- 172.21.15.122:0/346487672 shutdown_connections 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e79380b0 0x5572e793a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/346487672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e793d9a0 0x5572e793dd80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 -- 172.21.15.122:0/346487672 >> 172.21.15.122:0/346487672 conn(0x5572e7939010 msgr2=0x5572e79379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 -- 172.21.15.122:0/346487672 shutdown_connections 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 -- 172.21.15.122:0/346487672 wait complete. 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.816+0000 7f7e0a22d740 1 -- start start 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 Processor -- start 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 0x5572e794cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e794d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001a98e0 con 0x5572e7943370 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001aad50 con 0x5572e79380b0 2024-09-12T01:48:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e001a9fb0 con 0x5572e793d9a0 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e0a22d740 10 client.0 _ll_register_callbacks cb 0x7f7df0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05954640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 0x5572e794cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 0x5572e794cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36078/0 (socket says 172.21.15.122:36078) 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05954640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53180/0 (socket says 172.21.15.122:53180) 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 -- 172.21.15.122:0/1083480804 learned_addr learned my addr 172.21.15.122:0/1083480804 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e04952640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e794d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 -- 172.21.15.122:0/1083480804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 msgr2=0x5572e794d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e794d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 -- 172.21.15.122:0/1083480804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 msgr2=0x5572e794e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05153640 1 -- 172.21.15.122:0/1083480804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e794e840 con 0x5572e79380b0 2024-09-12T01:48:51.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 106616 8.60 MB/sec warmup 69 sec latency 21.137 ms 2024-09-12T01:48:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e05954640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.817+0000 7f7e04952640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e794d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e05153640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 0x5572e794cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0986e0 tx=0x7f7dfc003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417db9b0445e06a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7de8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7de8ff9640 1 -- 172.21.15.122:0/1083480804 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc07e070 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7de8ff9640 1 -- 172.21.15.122:0/1083480804 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7dfc0cf2d0 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572e78aea20 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7de8ff9640 1 -- 172.21.15.122:0/1083480804 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7dfc0a9050 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 10 client.25373 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572e79be540 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7de8ff9640 1 -- 172.21.15.122:0/1083480804 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5572e78aea20 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7de8ff9640 1 -- 172.21.15.122:0/1083480804 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7dfc09bfb0 con 0x5572e79380b0 2024-09-12T01:48:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 10 client.25373 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 20 client.25373 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79beee0 0x5572e79c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 4 client.25373 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.818+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572e78a8d70 con 0x5572e79beee0 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.819+0000 7f7e04952640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79beee0 0x5572e79c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.819+0000 7f7de8ff9640 10 client.25373 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.819+0000 7f7e04952640 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79beee0 0x5572e79c12d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f79853d285012c0 server_cookie=b3efcb56b86111e9 in_seq=0 out_seq=0 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.826+0000 7f7de8ff9640 1 -- 172.21.15.122:0/1083480804 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572e78a8d70 con 0x5572e79beee0 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.826+0000 7f7de8ff9640 10 client.25373 handle_command_reply: tid=0 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.826+0000 7f7e0a22d740 1 client.25373 shutdown 2024-09-12T01:48:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.826+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 msgr2=0x5572e794cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.826+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 0x5572e794cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0986e0 tx=0x7f7dfc003cd0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 shutdown_connections 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5572e79beee0 0x5572e79c12d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e7943370 0x5572e794e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7de8ff9640 0 client.25373 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572e793d9a0 0x5572e794d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 --2- 172.21.15.122:0/1083480804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572e79380b0 0x5572e794cce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 >> 172.21.15.122:0/1083480804 conn(0x5572e7939010 msgr2=0x5572e79393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 shutdown_connections 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 1 -- 172.21.15.122:0/1083480804 wait complete. 2024-09-12T01:48:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e0a22d740 20 client.25373 trim_cache size 0 max 16384 2024-09-12T01:48:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:48:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:48:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:48:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7dec0a2cb0 msgr2=0x7f7dec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7dec0a2cb0 0x7f7dec0a5170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7df8005e30 tx=0x7f7df8031990 comp rx=0 tx=0).stop 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 msgr2=0x7f7e001049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 0x7f7e001049e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00e9c0 tx=0x7f7dfc00ee80 comp rx=0 tx=0).stop 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 shutdown_connections 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7dec0a2cb0 0x7f7dec0a5170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e00102600 0x7f7e00103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00101c40 0x7f7e00103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 --2- 172.21.15.122:0/691669267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e00100a50 0x7f7e001049e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.827+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 >> 172.21.15.122:0/691669267 conn(0x7f7e000fc1b0 msgr2=0x7f7e000fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.828+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 shutdown_connections 2024-09-12T01:48:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:48:51.828+0000 7f7e073b7640 1 -- 172.21.15.122:0/691669267 wait complete. 2024-09-12T01:48:51.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:51 smithi151 ceph-mon[35229]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-09-12T01:48:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:51 smithi183 ceph-mon[35127]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-09-12T01:48:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:51 smithi122 ceph-mon[30925]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-09-12T01:48:52.520 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:48:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:52.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 109440 8.67 MB/sec warmup 70 sec latency 9.962 ms 2024-09-12T01:48:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 -- 172.21.15.122:0/99872085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c641036e0 msgr2=0x7f6c64107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/99872085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c641036e0 0x7f6c64107b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009930 tx=0x7f6c5402a420 comp rx=0 tx=0).stop 2024-09-12T01:48:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 -- 172.21.15.122:0/99872085 shutdown_connections 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/99872085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641082a0 0x7f6c6410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/99872085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c641036e0 0x7f6c64107b90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/99872085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c64102d30 0x7f6c64103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 -- 172.21.15.122:0/99872085 >> 172.21.15.122:0/99872085 conn(0x7f6c640fc7d0 msgr2=0x7f6c640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 -- 172.21.15.122:0/99872085 shutdown_connections 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.254+0000 7f6c6b26f640 1 -- 172.21.15.122:0/99872085 wait complete. 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 Processor -- start 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 -- start start 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 0x7f6c64100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641036e0 0x7f6c64100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c641082a0 0x7f6c64101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641120a0 con 0x7f6c64102d30 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64111f20 con 0x7f6c641082a0 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6b26f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64112220 con 0x7f6c641036e0 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6900b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 0x7f6c64100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.255+0000 7f6c6900b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 0x7f6c64100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53206/0 (socket says 172.21.15.122:53206) 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 -- 172.21.15.122:0/400909717 learned_addr learned my addr 172.21.15.122:0/400909717 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6980c640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c641082a0 0x7f6c64101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6880a640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641036e0 0x7f6c64100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 -- 172.21.15.122:0/400909717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641036e0 msgr2=0x7f6c64100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641036e0 0x7f6c64100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 -- 172.21.15.122:0/400909717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c641082a0 msgr2=0x7f6c64101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c641082a0 0x7f6c64101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 -- 172.21.15.122:0/400909717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c641a6aa0 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6880a640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641036e0 0x7f6c64100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6900b640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 0x7f6c64100990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e9f0 tx=0x7f6c6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c5a7fc640 1 -- 172.21.15.122:0/400909717 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000cd80 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c5a7fc640 1 -- 172.21.15.122:0/400909717 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c60010910 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c5a7fc640 1 -- 172.21.15.122:0/400909717 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60018990 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c641a6d90 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.256+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a7240 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.259+0000 7f6c6d648740 1 Processor -- start 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.259+0000 7f6c5a7fc640 1 -- 172.21.15.122:0/400909717 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6c60010ac0 con 0x7f6c64102d30 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.259+0000 7f6c6d648740 1 -- start start 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.259+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 0x559cf50ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.259+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50ee5d0 0x559cf50eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6d648740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cf50542f0 con 0x559cf50ee5d0 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6d648740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559cf4ffbcf0 con 0x559cf50efb40 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6d648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559cf50502b0 con 0x559cf50f9370 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 0x559cf50ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 0x559cf50ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36112/0 (socket says 172.21.15.122:36112) 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 -- 172.21.15.122:0/2108174358 learned_addr learned my addr 172.21.15.122:0/2108174358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6900b640 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50ee5d0 0x559cf50eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6880a640 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 -- 172.21.15.122:0/2108174358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 msgr2=0x559cf50fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 -- 172.21.15.122:0/2108174358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50ee5d0 msgr2=0x559cf50eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50ee5d0 0x559cf50eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 -- 172.21.15.122:0/2108174358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf4fe03a0 con 0x559cf50efb40 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c5a7fc640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6c500a2b80 0x7f6c500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6980c640 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 0x559cf50ee090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00ae10 tx=0x7f6c5c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb52274e4fdfa797 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c6900b640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6c500a2b80 0x7f6c500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c4f7fe640 1 -- 172.21.15.122:0/2108174358 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c003b70 con 0x559cf50efb40 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.260+0000 7f6c5a7fc640 1 -- 172.21.15.122:0/400909717 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6c600cf3e0 con 0x7f6c64102d30 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c4f7fe640 1 -- 172.21.15.122:0/2108174358 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c003d10 con 0x559cf50efb40 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c4f7fe640 1 -- 172.21.15.122:0/2108174358 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c010b00 con 0x559cf50efb40 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 -- 172.21.15.122:0/2108174358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 msgr2=0x559cf50ee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 0x559cf50ee090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00ae10 tx=0x7f6c5c00f790 comp rx=0 tx=0).stop 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6900b640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6c500a2b80 0x7f6c500a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6c600a4380 tx=0x7f6c600d2e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 -- 172.21.15.122:0/2108174358 shutdown_connections 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50ee5d0 0x559cf50eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 --2- 172.21.15.122:0/2108174358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50efb40 0x559cf50ee090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 -- 172.21.15.122:0/2108174358 >> 172.21.15.122:0/2108174358 conn(0x559cf50b7310 msgr2=0x559cf50eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 -- 172.21.15.122:0/2108174358 shutdown_connections 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.261+0000 7f6c6d648740 1 -- 172.21.15.122:0/2108174358 wait complete. 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 -- start start 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 Processor -- start 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf50f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 0x559cf50f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cf50fc070 con 0x559cf50efb40 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559cf500f060 con 0x559cf50ee5d0 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559cf50fc1f0 con 0x559cf50f9370 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6d648740 10 client.0 _ll_register_callbacks cb 0x559cf4d628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6880a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 0x559cf50f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6980c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf50f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6880a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 0x559cf50f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53232/0 (socket says 172.21.15.122:53232) 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6880a640 1 -- 172.21.15.122:0/3143581276 learned_addr learned my addr 172.21.15.122:0/3143581276 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.262+0000 7f6c6980c640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6880a640 1 -- 172.21.15.122:0/3143581276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 msgr2=0x559cf50f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6880a640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6880a640 1 -- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 msgr2=0x559cf50f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6880a640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf50f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6880a640 1 -- 172.21.15.122:0/3143581276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf4dc53c0 con 0x559cf50efb40 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6980c640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6880a640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 0x559cf50f5f80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c54005940 tx=0x7f6c54031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419fe8504c1b6109 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c4cff9640 1 -- 172.21.15.122:0/3143581276 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54035040 con 0x559cf50efb40 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c4cff9640 1 -- 172.21.15.122:0/3143581276 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6c54004240 con 0x559cf50efb40 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c5c003d10 con 0x559cf50efb40 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c4cff9640 1 -- 172.21.15.122:0/3143581276 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c54004440 con 0x559cf50efb40 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6d648740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559cf5107b00 con 0x559cf50efb40 2024-09-12T01:48:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.263+0000 7f6c4cff9640 1 -- 172.21.15.122:0/3143581276 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c003d10 con 0x559cf50efb40 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c4cff9640 1 -- 172.21.15.122:0/3143581276 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6c54004bc0 con 0x559cf50efb40 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6d648740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6d648740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6d648740 20 client.16026 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6d648740 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf5109060 0x559cf510b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6d648740 4 client.16026 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cf5061860 con 0x559cf5109060 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6980c640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf5109060 0x559cf510b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c6980c640 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf5109060 0x559cf510b470 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82af29ab6923be8 server_cookie=b85e1c7673c7a80f in_seq=0 out_seq=0 2024-09-12T01:48:53.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.264+0000 7f6c4cff9640 10 client.16026 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c4cff9640 1 -- 172.21.15.122:0/3143581276 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cf5061860 con 0x559cf5109060 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c4cff9640 10 client.16026 handle_command_reply: tid=0 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 client.16026 shutdown 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 msgr2=0x559cf50f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 0x559cf50f5f80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c54005940 tx=0x7f6c54031920 comp rx=0 tx=0).stop 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 shutdown_connections 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf5109060 0x559cf510b470 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50efb40 0x559cf50f5f80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c4cff9640 0 client.16026 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:53.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 --2- 172.21.15.122:0/3143581276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf50f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 >> 172.21.15.122:0/3143581276 conn(0x559cf50b7310 msgr2=0x559cf50f0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:53.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 shutdown_connections 2024-09-12T01:48:53.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 1 -- 172.21.15.122:0/3143581276 wait complete. 2024-09-12T01:48:53.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.265+0000 7f6c6d648740 20 client.16026 trim_cache size 0 max 16384 2024-09-12T01:48:53.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 Processor -- start 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 -- start start 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5105340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 0x559cf5110d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641b20a0 con 0x559cf5109060 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641af970 con 0x559cf50ee5d0 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6d648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ad7a0 con 0x559cf50f9370 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 0x559cf5110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 0x559cf5110d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53238/0 (socket says 172.21.15.122:53238) 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 -- 172.21.15.122:0/1880835733 learned_addr learned my addr 172.21.15.122:0/1880835733 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:53.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5105340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6880a640 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 -- 172.21.15.122:0/1880835733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 msgr2=0x559cf50f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 -- 172.21.15.122:0/1880835733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 msgr2=0x559cf5105340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5105340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 -- 172.21.15.122:0/1880835733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf50f1120 con 0x559cf5109060 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6880a640 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.279+0000 7f6c6980c640 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 0x559cf5110d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c01a040 tx=0x7f6c5c00a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5857c02472fc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c4effd640 1 -- 172.21.15.122:0/1880835733 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c004a00 con 0x559cf5109060 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c4effd640 1 -- 172.21.15.122:0/1880835733 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c004ba0 con 0x559cf5109060 2024-09-12T01:48:53.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c4effd640 1 -- 172.21.15.122:0/1880835733 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0055f0 con 0x559cf5109060 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 -- 172.21.15.122:0/1880835733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 msgr2=0x559cf5110d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 0x559cf5110d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c01a040 tx=0x7f6c5c00a4d0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 -- 172.21.15.122:0/1880835733 shutdown_connections 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf5109060 0x559cf5110d40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf50f9370 0x559cf50f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 --2- 172.21.15.122:0/1880835733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 -- 172.21.15.122:0/1880835733 >> 172.21.15.122:0/1880835733 conn(0x559cf50b7310 msgr2=0x559cf50f4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 -- 172.21.15.122:0/1880835733 shutdown_connections 2024-09-12T01:48:53.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.280+0000 7f6c6d648740 1 -- 172.21.15.122:0/1880835733 wait complete. 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 -- start start 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 Processor -- start 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5101a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 0x559cf50fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf5109060 0x559cf50fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ade30 con 0x559cf50f9370 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641af2a0 con 0x559cf50ee5d0 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ae500 con 0x559cf5109060 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6d648740 10 client.0 _ll_register_callbacks cb 0x7f6c54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6880a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 0x559cf50fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6880a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 0x559cf50fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53246/0 (socket says 172.21.15.122:53246) 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6880a640 1 -- 172.21.15.122:0/362559548 learned_addr learned my addr 172.21.15.122:0/362559548 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6980c640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.281+0000 7f6c6980c640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf5109060 0x559cf50fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6880a640 1 -- 172.21.15.122:0/362559548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf5109060 msgr2=0x559cf50fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6880a640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf5109060 0x559cf50fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6880a640 1 -- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 msgr2=0x559cf5101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6880a640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6880a640 1 -- 172.21.15.122:0/362559548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf50fd9a0 con 0x559cf50f9370 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6980c640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf5109060 0x559cf50fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6880a640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 0x559cf50fcdc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c54004f60 tx=0x7f6c54005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290dd28a70bc68f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c4cff9640 1 -- 172.21.15.122:0/362559548 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54035070 con 0x559cf50f9370 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c4cff9640 1 -- 172.21.15.122:0/362559548 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6c540046c0 con 0x559cf50f9370 2024-09-12T01:48:53.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c5c004ba0 con 0x559cf50f9370 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c4cff9640 1 -- 172.21.15.122:0/362559548 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c54031bd0 con 0x559cf50f9370 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6d648740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559cf5173ba0 con 0x559cf50f9370 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c4cff9640 1 -- 172.21.15.122:0/362559548 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c004ba0 con 0x559cf50f9370 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.282+0000 7f6c4cff9640 1 -- 172.21.15.122:0/362559548 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6c54003d70 con 0x559cf50f9370 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T01:48:53.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 20 client.16032 populate_metadata read hostname 'smithi122' 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf51742f0 0x559cf5176710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 4 client.16032 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cf505bd70 con 0x559cf51742f0 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6980c640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf51742f0 0x559cf5176710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c4cff9640 10 client.16032 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6980c640 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf51742f0 0x559cf5176710 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ee4dad9ea2e5d5 server_cookie=d9c916b05ac05067 in_seq=0 out_seq=0 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c4cff9640 1 -- 172.21.15.122:0/362559548 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cf505bd70 con 0x559cf51742f0 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c4cff9640 10 client.16032 handle_command_reply: tid=0 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 client.16032 shutdown 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 msgr2=0x559cf50fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 0x559cf50fcdc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c54004f60 tx=0x7f6c54005210 comp rx=0 tx=0).stop 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 shutdown_connections 2024-09-12T01:48:53.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559cf51742f0 0x559cf5176710 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf5109060 0x559cf50fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf50f9370 0x559cf50fcdc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c4cff9640 0 client.16032 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.283+0000 7f6c6d648740 1 --2- 172.21.15.122:0/362559548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cf50ee5d0 0x559cf5101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 >> 172.21.15.122:0/362559548 conn(0x559cf50b7310 msgr2=0x559cf50f5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 shutdown_connections 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6d648740 1 -- 172.21.15.122:0/362559548 wait complete. 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6d648740 20 client.16032 trim_cache size 0 max 16384 2024-09-12T01:48:53.290 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6c500a2b80 msgr2=0x7f6c500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6c500a2b80 0x7f6c500a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6c600a4380 tx=0x7f6c600d2e60 comp rx=0 tx=0).stop 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 msgr2=0x7f6c64100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 0x7f6c64100990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e9f0 tx=0x7f6c6000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 shutdown_connections 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6c500a2b80 0x7f6c500a5040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c641082a0 0x7f6c64101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c641036e0 0x7f6c64100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 --2- 172.21.15.122:0/400909717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c64102d30 0x7f6c64100990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 >> 172.21.15.122:0/400909717 conn(0x7f6c640fc7d0 msgr2=0x7f6c64105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 shutdown_connections 2024-09-12T01:48:53.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:53.284+0000 7f6c6b26f640 1 -- 172.21.15.122:0/400909717 wait complete. 2024-09-12T01:48:53.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 112201 8.73 MB/sec warmup 71 sec latency 20.604 ms 2024-09-12T01:48:53.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:53 smithi151 ceph-mon[35229]: pgmap v407: 97 pgs: 97 active+clean; 61 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-09-12T01:48:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:53 smithi183 ceph-mon[35127]: pgmap v407: 97 pgs: 97 active+clean; 61 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-09-12T01:48:53.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:53 smithi122 ceph-mon[30925]: pgmap v407: 97 pgs: 97 active+clean; 61 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-09-12T01:48:54.036 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 49878873, 'ino': 1099511628345, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 110090539, 'ino': 1099511628337, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG0'}, {'damage_type': 'backtrace', 'id': 328977311, 'ino': 1099511628309, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL'}, {'damage_type': 'backtrace', 'id': 400223667, 'ino': 1099511628332, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.X04'}, {'damage_type': 'backtrace', 'id': 431945318, 'ino': 1099511628333, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.Y04'}, {'damage_type': 'backtrace', 'id': 458145652, 'ino': 1099511628295, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.004'}, {'damage_type': 'backtrace', 'id': 475304002, 'ino': 1099511628311, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 573335918, 'ino': 1099511628335, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.PX'}, {'damage_type': 'backtrace', 'id': 605755074, 'ino': 1099511628321, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORDPRO'}, {'damage_type': 'backtrace', 'id': 606885819, 'ino': 1099511628297, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 692289559, 'ino': 1099511628348, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 801928416, 'ino': 1099511628316, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 1024362100, 'ino': 1099511628339, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 1033846540, 'ino': 1099511628299, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 1105167294, 'ino': 1099511628304, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 1131112544, 'ino': 1099511628346, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 1229489712, 'ino': 1099511628318, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 1259429202, 'ino': 1099511628341, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 1291118371, 'ino': 1099511628355, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 1303562831, 'ino': 1099511628329, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX'}, {'damage_type': 'backtrace', 'id': 1346304330, 'ino': 1099511628325, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM'}, {'damage_type': 'backtrace', 'id': 1392191144, 'ino': 1099511628347, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 1394214234, 'ino': 1099511628308, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 1459630752, 'ino': 1099511628317, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 1654266283, 'ino': 1099511628327, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 1690769850, 'ino': 1099511627785, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp'}, {'damage_type': 'backtrace', 'id': 1744402955, 'ino': 1099511628306, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 1857508100, 'ino': 1099511628314, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 1859171131, 'ino': 1099511628303, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT'}, {'damage_type': 'backtrace', 'id': 2061731507, 'ino': 1099511627784, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0'}, {'damage_type': 'backtrace', 'id': 2114237310, 'ino': 1099511628338, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 2165299386, 'ino': 1099511628294, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.003'}, {'damage_type': 'backtrace', 'id': 2280732680, 'ino': 1099511628298, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 2283962765, 'ino': 1099511628320, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 2420400403, 'ino': 1099511628343, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.PX'}, {'damage_type': 'backtrace', 'id': 2437247993, 'ino': 1099511628319, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2448760793, 'ino': 1099511628340, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.VAL'}, {'damage_type': 'backtrace', 'id': 2502401533, 'ino': 1099511627786, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.000'}, {'damage_type': 'backtrace', 'id': 2813821176, 'ino': 1099511628342, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 2836932630, 'ino': 1099511628334, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB'}, {'damage_type': 'backtrace', 'id': 2932813215, 'ino': 1099511628354, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 2991007203, 'ino': 1099511628312, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 2991690014, 'ino': 1099511628328, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 3018225338, 'ino': 1099511628353, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL'}, {'damage_type': 'backtrace', 'id': 3055812580, 'ino': 1099511628310, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 3085745597, 'ino': 1099511628315, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 3118380721, 'ino': 1099511628296, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/WORD'}, {'damage_type': 'backtrace', 'id': 3134662022, 'ino': 1099511628292, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.001'}, {'damage_type': 'backtrace', 'id': 3140997507, 'ino': 1099511628351, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 3184517587, 'ino': 1099511628350, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/ACCESS'}, {'damage_type': 'backtrace', 'id': 3247996233, 'ino': 1099511628305, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 3276761257, 'ino': 1099511628302, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 3452356296, 'ino': 1099511628336, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG0'}, {'damage_type': 'backtrace', 'id': 3536023104, 'ino': 1099511628331, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.PX'}, {'damage_type': 'backtrace', 'id': 3585474328, 'ino': 1099511628307, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT'}, {'damage_type': 'backtrace', 'id': 3598582488, 'ino': 1099511628330, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.DB'}, {'damage_type': 'backtrace', 'id': 3769615942, 'ino': 1099511628313, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 3781786739, 'ino': 1099511628349, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL'}, {'damage_type': 'backtrace', 'id': 3794561998, 'ino': 1099511628300, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL'}, {'damage_type': 'backtrace', 'id': 3957869387, 'ino': 1099511628293, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/filler.002'}, {'damage_type': 'backtrace', 'id': 3994436319, 'ino': 1099511628344, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 4010718180, 'ino': 1099511628301, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 4062411078, 'ino': 1099511628352, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/SEED'}, {'damage_type': 'backtrace', 'id': 4133343826, 'ino': 1099511628326, 'path': '/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}] 2024-09-12T01:48:54.037 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-09-12T01:48:54.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114698 8.82 MB/sec warmup 72 sec latency 12.509 ms 2024-09-12T01:48:55.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 117661 8.93 MB/sec warmup 73 sec latency 64.901 ms 2024-09-12T01:48:55.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:55 smithi151 ceph-mon[35229]: pgmap v408: 97 pgs: 97 active+clean; 64 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2024-09-12T01:48:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:55 smithi183 ceph-mon[35127]: pgmap v408: 97 pgs: 97 active+clean; 64 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2024-09-12T01:48:55.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:55 smithi122 ceph-mon[30925]: pgmap v408: 97 pgs: 97 active+clean; 64 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2024-09-12T01:48:56.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 120929 9.07 MB/sec warmup 74 sec latency 9.954 ms 2024-09-12T01:48:57.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 123995 9.14 MB/sec warmup 75 sec latency 10.875 ms 2024-09-12T01:48:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:57 smithi151 ceph-mon[35229]: pgmap v409: 97 pgs: 97 active+clean; 61 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-09-12T01:48:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:57 smithi183 ceph-mon[35127]: pgmap v409: 97 pgs: 97 active+clean; 61 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-09-12T01:48:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:57 smithi122 ceph-mon[30925]: pgmap v409: 97 pgs: 97 active+clean; 61 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-09-12T01:48:58.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 126959 9.23 MB/sec warmup 76 sec latency 10.965 ms 2024-09-12T01:48:59.184 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T01:48:59.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:48:59.453 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:48:59.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 129419 9.28 MB/sec warmup 77 sec latency 19.256 ms 2024-09-12T01:48:59.858 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:48:59 smithi122 ceph-mon[30925]: pgmap v410: 97 pgs: 97 active+clean; 67 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2024-09-12T01:48:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:48:59 smithi151 ceph-mon[35229]: pgmap v410: 97 pgs: 97 active+clean; 67 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2024-09-12T01:48:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:48:59 smithi183 ceph-mon[35127]: pgmap v410: 97 pgs: 97 active+clean; 67 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2024-09-12T01:48:59.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 -- 172.21.15.122:0/2541407846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 msgr2=0x7f1ef8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:59.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 --2- 172.21.15.122:0/2541407846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8101200 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009900 tx=0x7f1ef402a480 comp rx=0 tx=0).stop 2024-09-12T01:48:59.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 -- 172.21.15.122:0/2541407846 shutdown_connections 2024-09-12T01:48:59.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 --2- 172.21.15.122:0/2541407846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 0x7f1ef810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:59.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 --2- 172.21.15.122:0/2541407846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ef8102570 0x7f1ef81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:59.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 --2- 172.21.15.122:0/2541407846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8101200 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 -- 172.21.15.122:0/2541407846 >> 172.21.15.122:0/2541407846 conn(0x7f1ef80fc7d0 msgr2=0x7f1ef80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 -- 172.21.15.122:0/2541407846 shutdown_connections 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.939+0000 7f1eff798640 1 -- 172.21.15.122:0/2541407846 wait complete. 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 Processor -- start 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 -- start start 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ef8102570 0x7f1ef8103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 0x7f1ef81041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8110f80 con 0x7f1ef8100e20 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8110e00 con 0x7f1ef8102ef0 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1eff798640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8111100 con 0x7f1ef8102570 2024-09-12T01:48:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1efd534640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1efd534640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53260/0 (socket says 172.21.15.122:53260) 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1efd534640 1 -- 172.21.15.122:0/1610101996 learned_addr learned my addr 172.21.15.122:0/1610101996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1efdd35640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 0x7f1ef81041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.940+0000 7f1efcd33640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ef8102570 0x7f1ef8103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efd534640 1 -- 172.21.15.122:0/1610101996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ef8102570 msgr2=0x7f1ef8103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efd534640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ef8102570 0x7f1ef8103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efd534640 1 -- 172.21.15.122:0/1610101996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 msgr2=0x7f1ef81041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efd534640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 0x7f1ef81041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efd534640 1 -- 172.21.15.122:0/1610101996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef81a69e0 con 0x7f1ef8100e20 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efdd35640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 0x7f1ef81041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1efd534640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8103750 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4006010 tx=0x7f1ef4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef403f070 con 0x7f1ef8100e20 2024-09-12T01:48:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef4031c30 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef81a6c70 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4035030 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.941+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef81a7150 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.942+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef8101200 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.943+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1ef4042070 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.944+0000 7f1eee7fc640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ee40a2b80 0x7f1ee40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.944+0000 7f1efcd33640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ee40a2b80 0x7f1ee40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.944+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1ef40ef6a0 con 0x7f1ef8100e20 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.945+0000 7f1efcd33640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ee40a2b80 0x7f1ee40a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ef81051e0 tx=0x7f1ee8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:48:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:48:59.949+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ef4031de0 con 0x7f1ef8100e20 2024-09-12T01:49:00.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.172+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ef81a7320 con 0x7f1ef8100e20 2024-09-12T01:49:00.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.172+0000 7f1eee7fc640 1 -- 172.21.15.122:0/1610101996 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11934 (secure 0 0 0) 0x7f1ef40bbfb0 con 0x7f1ef8100e20 2024-09-12T01:49:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:49:00.173 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"btime":"2024-09-12T01:48:18:666269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:48:17.839016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:49:00.173 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ee40a2b80 msgr2=0x7f1ee40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ee40a2b80 0x7f1ee40a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ef81051e0 tx=0x7f1ee8005f90 comp rx=0 tx=0).stop 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 msgr2=0x7f1ef8103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8103750 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4006010 tx=0x7f1ef4003c10 comp rx=0 tx=0).stop 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 shutdown_connections 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ee40a2b80 0x7f1ee40a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ef8102ef0 0x7f1ef81041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ef8102570 0x7f1ef8103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:00.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 --2- 172.21.15.122:0/1610101996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ef8100e20 0x7f1ef8103750 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:00.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 >> 172.21.15.122:0/1610101996 conn(0x7f1ef80fc7d0 msgr2=0x7f1ef806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:00.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 shutdown_connections 2024-09-12T01:49:00.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:00.175+0000 7f1eff798640 1 -- 172.21.15.122:0/1610101996 wait complete. 2024-09-12T01:49:00.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:00 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1610101996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:49:00.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 132223 9.35 MB/sec warmup 78 sec latency 14.280 ms 2024-09-12T01:49:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:00 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1610101996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:49:00.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T01:49:00.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T01:49:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:00 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1610101996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:49:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:49:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 -- 172.21.15.122:0/2484253040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 msgr2=0x7f5cdc10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2484253040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc10c9f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009900 tx=0x7f5cd002a410 comp rx=0 tx=0).stop 2024-09-12T01:49:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 -- 172.21.15.122:0/2484253040 shutdown_connections 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2484253040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cdc10cf30 0x7f5cdc119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2484253040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc10c9f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2484253040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cdc112640 0x7f5cdc112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 -- 172.21.15.122:0/2484253040 >> 172.21.15.122:0/2484253040 conn(0x7f5cdc06cdc0 msgr2=0x7f5cdc06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 -- 172.21.15.122:0/2484253040 shutdown_connections 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.657+0000 7f5ce4062640 1 -- 172.21.15.122:0/2484253040 wait complete. 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 Processor -- start 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 -- start start 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cdc10cf30 0x7f5cdc1a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cdc112640 0x7f5cdc1a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc11c9a0 con 0x7f5cdc10c590 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc11c820 con 0x7f5cdc112640 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce4062640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc11cb20 con 0x7f5cdc10cf30 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc1a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39044/0 (socket says 172.21.15.122:39044) 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/2559757825 learned_addr learned my addr 172.21.15.122:0/2559757825 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cdc10cf30 0x7f5cdc1a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.658+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cdc112640 0x7f5cdc1a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/2559757825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cdc10cf30 msgr2=0x7f5cdc1a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cdc10cf30 0x7f5cdc1a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/2559757825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cdc112640 msgr2=0x7f5cdc1a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cdc112640 0x7f5cdc1a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/2559757825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc1a7e60 con 0x7f5cdc10c590 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc1a2eb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9f0 tx=0x7f5cd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:01.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5cceffd640 1 -- 172.21.15.122:0/2559757825 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800cd80 con 0x7f5cdc10c590 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5cceffd640 1 -- 172.21.15.122:0/2559757825 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd8005500 con 0x7f5cdc10c590 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc1a8150 con 0x7f5cdc10c590 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5cceffd640 1 -- 172.21.15.122:0/2559757825 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8010690 con 0x7f5cdc10c590 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.659+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc1afa30 con 0x7f5cdc10c590 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.661+0000 7f5cceffd640 1 -- 172.21.15.122:0/2559757825 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5cd80046c0 con 0x7f5cdc10c590 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 Processor -- start 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 -- start start 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5cceffd640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5cc80a2c00 0x7f5cc80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084e760 0x561e2084eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 0x561e20859400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e20859940 0x561e2085bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561e207b42a0 con 0x561e2084e760 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561e2075bb10 con 0x561e2084f080 2024-09-12T01:49:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce643b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e207b0260 con 0x561e20859940 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5cceffd640 1 -- 172.21.15.122:0/2559757825 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5cd80cf3d0 con 0x7f5cdc10c590 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.662+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5cc80a2c00 0x7f5cc80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 0x561e20859400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 0x561e20859400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:44260/0 (socket says 172.21.15.122:44260) 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 -- 172.21.15.122:0/224825124 learned_addr learned my addr 172.21.15.122:0/224825124 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084e760 0x561e2084eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 -- 172.21.15.122:0/224825124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e20859940 msgr2=0x561e2085bd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e20859940 0x561e2085bd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 -- 172.21.15.122:0/224825124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084e760 msgr2=0x561e2084eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084e760 0x561e2084eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 -- 172.21.15.122:0/224825124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e20525b90 con 0x561e2084f080 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5cc80a2c00 0x7f5cc80a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4005fd0 tx=0x7f5cd4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 0x561e20859400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5cd002a930 tx=0x7f5cd0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f9f3c5da029394 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5cc7fff640 1 -- 172.21.15.122:0/224825124 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0035030 con 0x561e2084f080 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5cc7fff640 1 -- 172.21.15.122:0/224825124 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0040070 con 0x561e2084f080 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.663+0000 7f5cc7fff640 1 -- 172.21.15.122:0/224825124 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0031810 con 0x561e2084f080 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 -- 172.21.15.122:0/224825124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 msgr2=0x561e20859400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 0x561e20859400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5cd002a930 tx=0x7f5cd0005dd0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 -- 172.21.15.122:0/224825124 shutdown_connections 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e20859940 0x561e2085bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084f080 0x561e20859400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 --2- 172.21.15.122:0/224825124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084e760 0x561e2084eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 -- 172.21.15.122:0/224825124 >> 172.21.15.122:0/224825124 conn(0x561e2084acf0 msgr2=0x561e2084b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 -- 172.21.15.122:0/224825124 shutdown_connections 2024-09-12T01:49:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 -- 172.21.15.122:0/224825124 wait complete. 2024-09-12T01:49:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.664+0000 7f5ce643b740 1 -- start start 2024-09-12T01:49:01.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 Processor -- start 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20856d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20857270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561e2085c510 con 0x561e20859940 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561e2073c6e0 con 0x561e2084e760 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e2085c690 con 0x561e2084f080 2024-09-12T01:49:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce643b740 10 client.0 _ll_register_callbacks cb 0x561e20760a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce15fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20856d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce25ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20857270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce15fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20856d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:60996/0 (socket says 172.21.15.122:60996) 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:44270/0 (socket says 172.21.15.122:44270) 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce15fd640 1 -- 172.21.15.122:0/133947681 learned_addr learned my addr 172.21.15.122:0/133947681 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce15fd640 1 -- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 msgr2=0x561e208567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.665+0000 7f5ce15fd640 1 -- 172.21.15.122:0/133947681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 msgr2=0x561e20857270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20857270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce15fd640 1 -- 172.21.15.122:0/133947681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e208659d0 con 0x561e2084f080 2024-09-12T01:49:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20857270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20856d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006010 tx=0x7f5cd00044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a2649f7c6f095f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5cc57fa640 1 -- 172.21.15.122:0/133947681 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0035070 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd0040070 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5cc57fa640 1 -- 172.21.15.122:0/133947681 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5cd0005500 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5cc57fa640 1 -- 172.21.15.122:0/133947681 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cd00046c0 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce643b740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e2085c510 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5cc57fa640 1 -- 172.21.15.122:0/133947681 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0040070 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.666+0000 7f5cc57fa640 1 -- 172.21.15.122:0/133947681 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5cd0031c30 con 0x561e2084f080 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce643b740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce643b740 10 client.25399 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce643b740 20 client.25399 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce643b740 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208694e0 0x561e2086b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce643b740 4 client.25399 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e207c19d0 con 0x561e208694e0 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208694e0 0x561e2086b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5cc57fa640 10 client.25399 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.667+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208694e0 0x561e2086b8d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224087a4ec11097b server_cookie=27040677d679b05c in_seq=0 out_seq=0 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5cc57fa640 1 -- 172.21.15.122:0/133947681 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e207c19d0 con 0x561e208694e0 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5cc57fa640 10 client.25399 handle_command_reply: tid=0 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 client.25399 shutdown 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 msgr2=0x561e20856d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20856d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006010 tx=0x7f5cd00044b0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 shutdown_connections 2024-09-12T01:49:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208694e0 0x561e2086b8d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20857270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20856d30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 --2- 172.21.15.122:0/133947681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 >> 172.21.15.122:0/133947681 conn(0x561e2084acf0 msgr2=0x561e20850a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5cc57fa640 0 client.25399 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 shutdown_connections 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 1 -- 172.21.15.122:0/133947681 wait complete. 2024-09-12T01:49:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.668+0000 7f5ce643b740 20 client.25399 trim_cache size 0 max 16384 2024-09-12T01:49:01.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.685+0000 7f5ce643b740 1 Processor -- start 2024-09-12T01:49:01.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.685+0000 7f5ce643b740 1 -- start start 2024-09-12T01:49:01.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20849d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20871090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce643b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1ba8a0 con 0x561e20859940 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce643b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b8170 con 0x561e2084e760 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce643b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b5fa0 con 0x561e2084f080 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20871090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce15fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20849d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20871090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39086/0 (socket says 172.21.15.122:39086) 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 -- 172.21.15.122:0/173533902 learned_addr learned my addr 172.21.15.122:0/173533902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 -- 172.21.15.122:0/173533902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 msgr2=0x561e20849d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20849d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 -- 172.21.15.122:0/173533902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 msgr2=0x561e208570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 -- 172.21.15.122:0/173533902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2084eeb0 con 0x561e20859940 2024-09-12T01:49:01.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20871090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009a80 tx=0x7f5cd4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c54b9e00b9bb976 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:01.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5cc77fe640 1 -- 172.21.15.122:0/173533902 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4004b20 con 0x561e20859940 2024-09-12T01:49:01.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5cc77fe640 1 -- 172.21.15.122:0/173533902 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd4004cc0 con 0x561e20859940 2024-09-12T01:49:01.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.686+0000 7f5cc77fe640 1 -- 172.21.15.122:0/173533902 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4005240 con 0x561e20859940 2024-09-12T01:49:01.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 -- 172.21.15.122:0/173533902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 msgr2=0x561e20871090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20871090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009a80 tx=0x7f5cd4014000 comp rx=0 tx=0).stop 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 -- 172.21.15.122:0/173533902 shutdown_connections 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e20859940 0x561e20871090 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084f080 0x561e20849d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 --2- 172.21.15.122:0/173533902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e2084e760 0x561e208570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 -- 172.21.15.122:0/173533902 >> 172.21.15.122:0/173533902 conn(0x561e2084acf0 msgr2=0x561e20849650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 -- 172.21.15.122:0/173533902 shutdown_connections 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 -- 172.21.15.122:0/173533902 wait complete. 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.687+0000 7f5ce643b740 1 -- start start 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 Processor -- start 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 0x561e20861c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084f080 0x561e208621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e20859940 0x561e2085d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b6630 con 0x561e2084f080 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b7aa0 con 0x561e20859940 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b6d00 con 0x561e2084e760 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce643b740 10 client.0 _ll_register_callbacks cb 0x7f5cd0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce15fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084f080 0x561e208621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 0x561e20861c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 0x561e20861c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:32786/0 (socket says 172.21.15.122:32786) 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/379816849 learned_addr learned my addr 172.21.15.122:0/379816849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce25ff640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e20859940 0x561e2085d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e20859940 msgr2=0x561e2085d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e20859940 0x561e2085d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/379816849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084f080 msgr2=0x561e208621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084f080 0x561e208621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce1dfe640 1 -- 172.21.15.122:0/379816849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2085da40 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.688+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084f080 0x561e208621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce1dfe640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 0x561e20861c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9c0 tx=0x7f5cd809bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=227d11a7f834506 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5cc57fa640 1 -- 172.21.15.122:0/379816849 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80a8070 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5cc57fa640 1 -- 172.21.15.122:0/379816849 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5cd8099ef0 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd4004cc0 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5cc57fa640 1 -- 172.21.15.122:0/379816849 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cd808b050 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 10 client.25405 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e208d3d10 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5cc57fa640 1 -- 172.21.15.122:0/379816849 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd4004cc0 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5cc57fa640 1 -- 172.21.15.122:0/379816849 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5cd809b040 con 0x561e2084e760 2024-09-12T01:49:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 10 client.25405 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 10 client.25405 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 20 client.25405 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208d4450 0x561e208d6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 4 client.25405 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x561e207bbd20 con 0x561e208d4450 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.689+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208d4450 0x561e208d6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.690+0000 7f5ce15fd640 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208d4450 0x561e208d6840 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91033a4e91ca7a5 server_cookie=b460f36a4098a280 in_seq=0 out_seq=0 2024-09-12T01:49:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.690+0000 7f5cc57fa640 10 client.25405 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:01.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5cc57fa640 1 -- 172.21.15.122:0/379816849 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e207bbd20 con 0x561e208d4450 2024-09-12T01:49:01.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5cc57fa640 10 client.25405 handle_command_reply: tid=0 2024-09-12T01:49:01.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 client.25405 shutdown 2024-09-12T01:49:01.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 msgr2=0x561e20861c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 0x561e20861c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9c0 tx=0x7f5cd809bcb0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 shutdown_connections 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561e208d4450 0x561e208d6840 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e20859940 0x561e2085d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e2084f080 0x561e208621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 --2- 172.21.15.122:0/379816849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e2084e760 0x561e20861c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5cc57fa640 0 client.25405 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 >> 172.21.15.122:0/379816849 conn(0x561e2084acf0 msgr2=0x561e20849e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 shutdown_connections 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 1 -- 172.21.15.122:0/379816849 wait complete. 2024-09-12T01:49:01.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.798+0000 7f5ce643b740 20 client.25405 trim_cache size 0 max 16384 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 28, 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 10, 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "863f85e1": { 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 10, 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:49:01.800 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.7, 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:01.801 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:01.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:01.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:01.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:01.802 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5cc80a2c00 msgr2=0x7f5cc80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5cc80a2c00 0x7f5cc80a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4005fd0 tx=0x7f5cd4009500 comp rx=0 tx=0).stop 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 msgr2=0x7f5cdc1a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc1a2eb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9f0 tx=0x7f5cd800edf0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 shutdown_connections 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5cc80a2c00 0x7f5cc80a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cdc112640 0x7f5cdc1a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cdc10cf30 0x7f5cdc1a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 --2- 172.21.15.122:0/2559757825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cdc10c590 0x7f5cdc1a2eb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 >> 172.21.15.122:0/2559757825 conn(0x7f5cdc06cdc0 msgr2=0x7f5cdc10af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 shutdown_connections 2024-09-12T01:49:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:01.799+0000 7f5ce4062640 1 -- 172.21.15.122:0/2559757825 wait complete. 2024-09-12T01:49:01.817 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 9.46 MB/sec warmup 79 sec latency 97.973 ms 2024-09-12T01:49:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:01 smithi151 ceph-mon[35229]: pgmap v411: 97 pgs: 97 active+clean; 68 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 326 op/s 2024-09-12T01:49:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:01 smithi183 ceph-mon[35127]: pgmap v411: 97 pgs: 97 active+clean; 68 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 326 op/s 2024-09-12T01:49:01.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:01 smithi122 ceph-mon[30925]: pgmap v411: 97 pgs: 97 active+clean; 68 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 326 op/s 2024-09-12T01:49:02.595 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '863f85e1', quiesce took 3.4 seconds. Will release after: 21.3 2024-09-12T01:49:02.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 9.35 MB/sec warmup 80 sec latency 1018.305 ms 2024-09-12T01:49:03.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 9.23 MB/sec warmup 81 sec latency 2018.362 ms 2024-09-12T01:49:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:03 smithi183 ceph-mon[35127]: pgmap v412: 97 pgs: 97 active+clean; 69 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 266 op/s 2024-09-12T01:49:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:03 smithi122 ceph-mon[30925]: pgmap v412: 97 pgs: 97 active+clean; 69 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 266 op/s 2024-09-12T01:49:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:03 smithi151 ceph-mon[35229]: pgmap v412: 97 pgs: 97 active+clean; 69 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 266 op/s 2024-09-12T01:49:04.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 9.12 MB/sec warmup 82 sec latency 3018.419 ms 2024-09-12T01:49:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/591167627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18102600 msgr2=0x7f6e18108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/591167627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18102600 0x7f6e18108e90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009ca0 tx=0x7f6e1002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/591167627 shutdown_connections 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/591167627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18102600 0x7f6e18108e90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/591167627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18101c40 0x7f6e181020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/591167627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e18100a50 0x7f6e18100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/591167627 >> 172.21.15.122:0/591167627 conn(0x7f6e180fc1b0 msgr2=0x7f6e180fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/591167627 shutdown_connections 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.923+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/591167627 wait complete. 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 Processor -- start 2024-09-12T01:49:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 -- start start 2024-09-12T01:49:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18100a50 0x7f6e1806ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 0x7f6e1819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e18102600 0x7f6e1819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1810c6b0 con 0x7f6e18101c40 2024-09-12T01:49:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1810c530 con 0x7f6e18100a50 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1f9e4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1810c830 con 0x7f6e18102600 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 0x7f6e1819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1d780640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18100a50 0x7f6e1806ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 0x7f6e1819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39110/0 (socket says 172.21.15.122:39110) 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1d780640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18100a50 0x7f6e1806ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:44302/0 (socket says 172.21.15.122:44302) 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/3205294609 learned_addr learned my addr 172.21.15.122:0/3205294609 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1df81640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e18102600 0x7f6e1819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/3205294609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e18102600 msgr2=0x7f6e1819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e18102600 0x7f6e1819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/3205294609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18100a50 msgr2=0x7f6e1806ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.924+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18100a50 0x7f6e1806ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/3205294609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1819f9a0 con 0x7f6e18101c40 2024-09-12T01:49:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 0x7f6e1819ca70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800d950 tx=0x7f6e0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e0e7fc640 1 -- 172.21.15.122:0/3205294609 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08017030 con 0x7f6e18101c40 2024-09-12T01:49:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1819fc30 con 0x7f6e18101c40 2024-09-12T01:49:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e0e7fc640 1 -- 172.21.15.122:0/3205294609 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e08004020 con 0x7f6e18101c40 2024-09-12T01:49:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e0e7fc640 1 -- 172.21.15.122:0/3205294609 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080047a0 con 0x7f6e18101c40 2024-09-12T01:49:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.925+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e181a0180 con 0x7f6e18101c40 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e0e7fc640 1 -- 172.21.15.122:0/3205294609 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6e080041d0 con 0x7f6e18101c40 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 Processor -- start 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 -- start start 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288c9a0 0x5558c288cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288d2e0 0x5558c2891da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c2894830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5558c27f02f0 con 0x5558c2892370 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558c2798270 con 0x5558c288c9a0 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.928+0000 7f6e22858740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558c27ec2b0 con 0x5558c288d2e0 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c2894830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c2894830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39116/0 (socket says 172.21.15.122:39116) 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/1337434069 learned_addr learned my addr 172.21.15.122:0/1337434069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1d780640 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288d2e0 0x5558c2891da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1df81640 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288c9a0 0x5558c288cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e0e7fc640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e040a2c30 0x7f6e040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/1337434069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288d2e0 msgr2=0x5558c2891da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288d2e0 0x5558c2891da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/1337434069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288c9a0 msgr2=0x5558c288cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288c9a0 0x5558c288cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/1337434069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c2797340 con 0x5558c2892370 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1df81640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e040a2c30 0x7f6e040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e0e7fc640 1 -- 172.21.15.122:0/3205294609 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6e080ce970 con 0x7f6e18101c40 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c2894830 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6e080b29c0 tx=0x7f6e080af330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59be93e477035a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e037fe640 1 -- 172.21.15.122:0/1337434069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0809afa0 con 0x5558c2892370 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e037fe640 1 -- 172.21.15.122:0/1337434069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0809b8d0 con 0x5558c2892370 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.929+0000 7f6e037fe640 1 -- 172.21.15.122:0/1337434069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080a0710 con 0x5558c2892370 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 -- 172.21.15.122:0/1337434069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 msgr2=0x5558c2894830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e1df81640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e040a2c30 0x7f6e040a50f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e10005e30 tx=0x7f6e100095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c2894830 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6e080b29c0 tx=0x7f6e080af330 comp rx=0 tx=0).stop 2024-09-12T01:49:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 -- 172.21.15.122:0/1337434069 shutdown_connections 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c2894830 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288d2e0 0x5558c2891da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 --2- 172.21.15.122:0/1337434069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288c9a0 0x5558c288cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 -- 172.21.15.122:0/1337434069 >> 172.21.15.122:0/1337434069 conn(0x5558c2888010 msgr2=0x5558c2888410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 -- 172.21.15.122:0/1337434069 shutdown_connections 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.930+0000 7f6e22858740 1 -- 172.21.15.122:0/1337434069 wait complete. 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 -- start start 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 Processor -- start 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c2896c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 0x5558c2897160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c2892370 0x5558c289e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5558c2895340 con 0x5558c288d2e0 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558c2779290 con 0x5558c2892370 2024-09-12T01:49:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558c28954c0 con 0x5558c288c9a0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e22858740 10 client.0 _ll_register_callbacks cb 0x5558c27aa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e1cf7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 0x5558c2897160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e1cf7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 0x5558c2897160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39132/0 (socket says 172.21.15.122:39132) 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/2994009518 learned_addr learned my addr 172.21.15.122:0/2994009518 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e1df81640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c2892370 0x5558c289e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.931+0000 7f6e1d780640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c2896c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/2994009518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 msgr2=0x5558c2896c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c2896c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c2892370 msgr2=0x5558c289e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c2892370 0x5558c289e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e1cf7f640 1 -- 172.21.15.122:0/2994009518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c2503210 con 0x5558c288d2e0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 0x5558c2897160 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6e08097630 tx=0x7f6e08004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d1673f425f67f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e00ff9640 1 -- 172.21.15.122:0/2994009518 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080a0a90 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e00ff9640 1 -- 172.21.15.122:0/2994009518 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6e08097840 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0809b8d0 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e00ff9640 1 -- 172.21.15.122:0/2994009518 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e0809ab30 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558c25cf620 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e00ff9640 1 -- 172.21.15.122:0/2994009518 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0809b8d0 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e00ff9640 1 -- 172.21.15.122:0/2994009518 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6e080bb0c0 con 0x5558c288d2e0 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 20 client.16068 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.932+0000 7f6e22858740 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c28a24f0 0x5558c28a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.933+0000 7f6e22858740 4 client.16068 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.933+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558c27fda20 con 0x5558c28a24f0 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.933+0000 7f6e1d780640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c28a24f0 0x5558c28a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.933+0000 7f6e00ff9640 10 client.16068 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.933+0000 7f6e1d780640 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c28a24f0 0x5558c28a48e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3707af56d71f0b7 server_cookie=605a36e2042cfa0 in_seq=0 out_seq=0 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e00ff9640 1 -- 172.21.15.122:0/2994009518 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558c27fda20 con 0x5558c28a24f0 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e00ff9640 10 client.16068 handle_command_reply: tid=0 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 client.16068 shutdown 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 msgr2=0x5558c2897160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 0x5558c2897160 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6e08097630 tx=0x7f6e08004540 comp rx=0 tx=0).stop 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 shutdown_connections 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c28a24f0 0x5558c28a48e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c2892370 0x5558c289e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c288d2e0 0x5558c2897160 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e00ff9640 0 client.16068 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 --2- 172.21.15.122:0/2994009518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c2896c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 >> 172.21.15.122:0/2994009518 conn(0x5558c2888010 msgr2=0x5558c2889820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 shutdown_connections 2024-09-12T01:49:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 1 -- 172.21.15.122:0/2994009518 wait complete. 2024-09-12T01:49:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.934+0000 7f6e22858740 20 client.16068 trim_cache size 0 max 16384 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 Processor -- start 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 -- start start 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c28a9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c28a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181ab8c0 con 0x5558c2892370 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181a9190 con 0x5558c288d2e0 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e22858740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181a6fc0 con 0x5558c288c9a0 2024-09-12T01:49:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c28a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c28a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39138/0 (socket says 172.21.15.122:39138) 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 -- 172.21.15.122:0/459881219 learned_addr learned my addr 172.21.15.122:0/459881219 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1d780640 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c28a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 -- 172.21.15.122:0/459881219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 msgr2=0x5558c28a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c28a9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 -- 172.21.15.122:0/459881219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 msgr2=0x5558c289e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 -- 172.21.15.122:0/459881219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c288d7c0 con 0x5558c2892370 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e1df81640 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c28a9670 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6e10035000 tx=0x7f6e10044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6c73cdac9ca3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.949+0000 7f6e02ffd640 1 -- 172.21.15.122:0/459881219 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1002fce0 con 0x5558c2892370 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e02ffd640 1 -- 172.21.15.122:0/459881219 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e10036040 con 0x5558c2892370 2024-09-12T01:49:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e02ffd640 1 -- 172.21.15.122:0/459881219 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10046820 con 0x5558c2892370 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 -- 172.21.15.122:0/459881219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 msgr2=0x5558c28a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c28a9670 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6e10035000 tx=0x7f6e10044000 comp rx=0 tx=0).stop 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 -- 172.21.15.122:0/459881219 shutdown_connections 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c28a9670 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 --2- 172.21.15.122:0/459881219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c28a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 -- 172.21.15.122:0/459881219 >> 172.21.15.122:0/459881219 conn(0x5558c2888010 msgr2=0x5558c28871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 -- 172.21.15.122:0/459881219 shutdown_connections 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 -- 172.21.15.122:0/459881219 wait complete. 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 -- start start 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.950+0000 7f6e22858740 1 Processor -- start 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c289db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c289c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181a7650 con 0x5558c2892370 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181a8ac0 con 0x5558c288d2e0 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e181a7d20 con 0x5558c288c9a0 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e22858740 10 client.0 _ll_register_callbacks cb 0x7f6e10005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c289c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c289c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39142/0 (socket says 172.21.15.122:39142) 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 -- 172.21.15.122:0/831341756 learned_addr learned my addr 172.21.15.122:0/831341756 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1d780640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c289db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 -- 172.21.15.122:0/831341756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 msgr2=0x5558c289db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c289db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 -- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 msgr2=0x5558c289e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 -- 172.21.15.122:0/831341756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c289c9d0 con 0x5558c2892370 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1cf7f640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e1df81640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c289c1a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6e10031ae0 tx=0x7f6e100307f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02f60d56c42a440 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.951+0000 7f6e00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e00ff9640 1 -- 172.21.15.122:0/831341756 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10030cf0 con 0x5558c2892370 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e00ff9640 1 -- 172.21.15.122:0/831341756 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6e10046580 con 0x5558c2892370 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e10036040 con 0x5558c2892370 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e00ff9640 1 -- 172.21.15.122:0/831341756 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e10046780 con 0x5558c2892370 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558c290dbd0 con 0x5558c2892370 2024-09-12T01:49:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e00ff9640 1 -- 172.21.15.122:0/831341756 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e10036040 con 0x5558c2892370 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e00ff9640 1 -- 172.21.15.122:0/831341756 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6e10051070 con 0x5558c2892370 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 20 client.16074 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c290e550 0x5558c2910970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 4 client.16074 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558c28920c0 con 0x5558c290e550 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.952+0000 7f6e1d780640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c290e550 0x5558c2910970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.953+0000 7f6e1d780640 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c290e550 0x5558c2910970 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace838d350376630 server_cookie=7c0cd136eac56663 in_seq=0 out_seq=0 2024-09-12T01:49:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.953+0000 7f6e00ff9640 10 client.16074 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.961+0000 7f6e00ff9640 1 -- 172.21.15.122:0/831341756 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558c28920c0 con 0x5558c290e550 2024-09-12T01:49:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.961+0000 7f6e00ff9640 10 client.16074 handle_command_reply: tid=0 2024-09-12T01:49:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.961+0000 7f6e22858740 1 client.16074 shutdown 2024-09-12T01:49:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 msgr2=0x5558c289c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c289c1a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6e10031ae0 tx=0x7f6e100307f0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 shutdown_connections 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5558c290e550 0x5558c2910970 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558c2892370 0x5558c289c1a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e00ff9640 0 client.16074 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558c288d2e0 0x5558c289e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 --2- 172.21.15.122:0/831341756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558c288c9a0 0x5558c289db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 >> 172.21.15.122:0/831341756 conn(0x5558c2888010 msgr2=0x5558c2888ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 shutdown_connections 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 1 -- 172.21.15.122:0/831341756 wait complete. 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e22858740 20 client.16074 trim_cache size 0 max 16384 2024-09-12T01:49:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:49:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:49:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:49:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e040a2c30 msgr2=0x7f6e040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e040a2c30 0x7f6e040a50f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e10005e30 tx=0x7f6e100095c0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 msgr2=0x7f6e1819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 0x7f6e1819ca70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800d950 tx=0x7f6e0800de10 comp rx=0 tx=0).stop 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 shutdown_connections 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6e040a2c30 0x7f6e040a50f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e18102600 0x7f6e1819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e18101c40 0x7f6e1819ca70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 --2- 172.21.15.122:0/3205294609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e18100a50 0x7f6e1806ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 >> 172.21.15.122:0/3205294609 conn(0x7f6e180fc1b0 msgr2=0x7f6e18104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 shutdown_connections 2024-09-12T01:49:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:04.962+0000 7f6e1f9e4640 1 -- 172.21.15.122:0/3205294609 wait complete. 2024-09-12T01:49:05.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 9.01 MB/sec warmup 83 sec latency 4018.523 ms 2024-09-12T01:49:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:05 smithi183 ceph-mon[35127]: pgmap v413: 97 pgs: 97 active+clean; 71 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-09-12T01:49:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:05 smithi122 ceph-mon[30925]: pgmap v413: 97 pgs: 97 active+clean; 71 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-09-12T01:49:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:05 smithi151 ceph-mon[35229]: pgmap v413: 97 pgs: 97 active+clean; 71 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-09-12T01:49:06.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.90 MB/sec warmup 84 sec latency 5018.621 ms 2024-09-12T01:49:07.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.80 MB/sec warmup 85 sec latency 6018.670 ms 2024-09-12T01:49:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:07 smithi183 ceph-mon[35127]: pgmap v414: 97 pgs: 97 active+clean; 72 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2024-09-12T01:49:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:07 smithi122 ceph-mon[30925]: pgmap v414: 97 pgs: 97 active+clean; 72 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2024-09-12T01:49:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:07 smithi151 ceph-mon[35229]: pgmap v414: 97 pgs: 97 active+clean; 72 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2024-09-12T01:49:08.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.69 MB/sec warmup 86 sec latency 7018.742 ms 2024-09-12T01:49:09.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.59 MB/sec warmup 87 sec latency 8018.816 ms 2024-09-12T01:49:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:09 smithi183 ceph-mon[35127]: pgmap v415: 97 pgs: 97 active+clean; 73 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-09-12T01:49:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:09 smithi122 ceph-mon[30925]: pgmap v415: 97 pgs: 97 active+clean; 73 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-09-12T01:49:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:09 smithi151 ceph-mon[35229]: pgmap v415: 97 pgs: 97 active+clean; 73 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-09-12T01:49:10.818 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.50 MB/sec warmup 88 sec latency 9018.877 ms 2024-09-12T01:49:11.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.40 MB/sec warmup 89 sec latency 10018.983 ms 2024-09-12T01:49:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:11 smithi183 ceph-mon[35127]: pgmap v416: 97 pgs: 97 active+clean; 73 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1019 KiB/s wr, 114 op/s 2024-09-12T01:49:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:11 smithi122 ceph-mon[30925]: pgmap v416: 97 pgs: 97 active+clean; 73 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1019 KiB/s wr, 114 op/s 2024-09-12T01:49:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:11 smithi151 ceph-mon[35229]: pgmap v416: 97 pgs: 97 active+clean; 73 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1019 KiB/s wr, 114 op/s 2024-09-12T01:49:12.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.31 MB/sec warmup 90 sec latency 11019.080 ms 2024-09-12T01:49:13.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.22 MB/sec warmup 91 sec latency 12019.180 ms 2024-09-12T01:49:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:13 smithi183 ceph-mon[35127]: pgmap v417: 97 pgs: 97 active+clean; 74 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-12T01:49:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:13 smithi122 ceph-mon[30925]: pgmap v417: 97 pgs: 97 active+clean; 74 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-12T01:49:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:13 smithi151 ceph-mon[35229]: pgmap v417: 97 pgs: 97 active+clean; 74 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-12T01:49:14.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.13 MB/sec warmup 92 sec latency 13019.269 ms 2024-09-12T01:49:15.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 8.04 MB/sec warmup 93 sec latency 14019.356 ms 2024-09-12T01:49:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:15 smithi183 ceph-mon[35127]: pgmap v418: 97 pgs: 97 active+clean; 75 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-09-12T01:49:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:15 smithi122 ceph-mon[30925]: pgmap v418: 97 pgs: 97 active+clean; 75 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-09-12T01:49:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:15 smithi151 ceph-mon[35229]: pgmap v418: 97 pgs: 97 active+clean; 75 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-09-12T01:49:16.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.95 MB/sec warmup 94 sec latency 15019.473 ms 2024-09-12T01:49:17.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.87 MB/sec warmup 95 sec latency 16019.593 ms 2024-09-12T01:49:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:17 smithi183 ceph-mon[35127]: pgmap v419: 97 pgs: 97 active+clean; 76 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-12T01:49:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:17 smithi122 ceph-mon[30925]: pgmap v419: 97 pgs: 97 active+clean; 76 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- 172.21.15.122:0/1067770651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7138102df0 msgr2=0x7f7138103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- 172.21.15.122:0/1067770651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7138102df0 0x7f7138103270 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f712c009900 tx=0x7f712c02a410 comp rx=0 tx=0).stop 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- 172.21.15.122:0/1067770651 shutdown_connections 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- 172.21.15.122:0/1067770651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71381037b0 0x7f713810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- 172.21.15.122:0/1067770651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7138102df0 0x7f7138103270 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- 172.21.15.122:0/1067770651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71380ffe20 0x7f7138100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- 172.21.15.122:0/1067770651 >> 172.21.15.122:0/1067770651 conn(0x7f71380fb980 msgr2=0x7f71380fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- 172.21.15.122:0/1067770651 shutdown_connections 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- 172.21.15.122:0/1067770651 wait complete. 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 Processor -- start 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- start start 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 0x7f7138102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102df0 0x7f7138100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71381037b0 0x7f71381012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f713810d840 con 0x7f71380ffe20 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f713810d6c0 con 0x7f71381037b0 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.057+0000 7f713ef6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f713810d9c0 con 0x7f7138102df0 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 0x7f7138102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 0x7f7138102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60538/0 (socket says 172.21.15.122:60538) 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 -- 172.21.15.122:0/2626806630 learned_addr learned my addr 172.21.15.122:0/2626806630 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f7137fff640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102df0 0x7f7138100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713d50a640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71381037b0 0x7f71381012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 -- 172.21.15.122:0/2626806630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102df0 msgr2=0x7f7138100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102df0 0x7f7138100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 -- 172.21.15.122:0/2626806630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71381037b0 msgr2=0x7f71381012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71381037b0 0x7f71381012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 -- 172.21.15.122:0/2626806630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7138101b70 con 0x7f71380ffe20 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713cd09640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 0x7f7138102760 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f712800c970 tx=0x7f712800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f7135ffb640 1 -- 172.21.15.122:0/2626806630 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128004220 con 0x7f71380ffe20 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f7135ffb640 1 -- 172.21.15.122:0/2626806630 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71280043c0 con 0x7f71380ffe20 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71381a1ef0 con 0x7f71380ffe20 2024-09-12T01:49:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.058+0000 7f7135ffb640 1 -- 172.21.15.122:0/2626806630 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712800f450 con 0x7f71380ffe20 2024-09-12T01:49:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.059+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381a22f0 con 0x7f71380ffe20 2024-09-12T01:49:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.060+0000 7f7135ffb640 1 -- 172.21.15.122:0/2626806630 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f71280079b0 con 0x7f71380ffe20 2024-09-12T01:49:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 Processor -- start 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7135ffb640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71200a2c30 0x7f71200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 -- start start 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc69a0 0x55ce18cc6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 0x55ce18ccbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ce18c2a2f0 con 0x55ce18ccc370 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f713d50a640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71200a2c30 0x7f71200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ce18bd2270 con 0x55ce18cc69a0 2024-09-12T01:49:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7135ffb640 1 -- 172.21.15.122:0/2626806630 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f71280ced30 con 0x7f71380ffe20 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.061+0000 7f7141de3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce18c262b0 con 0x55ce18cc72e0 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 0x55ce18ccbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f713d50a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 0x55ce18ccbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34548/0 (socket says 172.21.15.122:34548) 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 -- 172.21.15.122:0/90702734 learned_addr learned my addr 172.21.15.122:0/90702734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f713cd09640 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc69a0 0x55ce18cc6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 -- 172.21.15.122:0/90702734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc69a0 msgr2=0x55ce18cc6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc69a0 0x55ce18cc6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 -- 172.21.15.122:0/90702734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 msgr2=0x55ce18cce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 -- 172.21.15.122:0/90702734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce18bd1340 con 0x55ce18cc72e0 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f713d50a640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71200a2c30 0x7f71200a50f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f71300136d0 tx=0x7f7130013ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f7137fff640 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 0x55ce18ccbda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712c009900 tx=0x7f712c0043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289740cf21e355c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f711effd640 1 -- 172.21.15.122:0/90702734 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c040070 con 0x55ce18cc72e0 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f711effd640 1 -- 172.21.15.122:0/90702734 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712c0047a0 con 0x55ce18cc72e0 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.062+0000 7f711effd640 1 -- 172.21.15.122:0/90702734 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c035040 con 0x55ce18cc72e0 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 -- 172.21.15.122:0/90702734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 msgr2=0x55ce18ccbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 0x55ce18ccbda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712c009900 tx=0x7f712c0043d0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 -- 172.21.15.122:0/90702734 shutdown_connections 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc72e0 0x55ce18ccbda0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 --2- 172.21.15.122:0/90702734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc69a0 0x55ce18cc6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 -- 172.21.15.122:0/90702734 >> 172.21.15.122:0/90702734 conn(0x55ce18cc2010 msgr2=0x55ce18cc2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 -- 172.21.15.122:0/90702734 shutdown_connections 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 -- 172.21.15.122:0/90702734 wait complete. 2024-09-12T01:49:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.063+0000 7f7141de3740 1 -- start start 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 Processor -- start 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc69a0 0x55ce18cd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cd80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ce18ccf2e0 con 0x55ce18ccc370 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ce18bb3290 con 0x55ce18cc72e0 2024-09-12T01:49:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce18ccf460 con 0x55ce18cc69a0 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7141de3740 10 client.0 _ll_register_callbacks cb 0x55ce18be4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cd80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cd80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60552/0 (socket says 172.21.15.122:60552) 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 -- 172.21.15.122:0/3704776962 learned_addr learned my addr 172.21.15.122:0/3704776962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713cd09640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc69a0 0x55ce18cd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f7137fff640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 -- 172.21.15.122:0/3704776962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc69a0 msgr2=0x55ce18cd0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc69a0 0x55ce18cd0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 -- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 msgr2=0x55ce18cd10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.064+0000 7f713d50a640 1 -- 172.21.15.122:0/3704776962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1893d210 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f713d50a640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cd80e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7130011f70 tx=0x7f7130019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94113be81ea56c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7117fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7117fff640 1 -- 172.21.15.122:0/3704776962 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130004d20 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7117fff640 1 -- 172.21.15.122:0/3704776962 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7130003970 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f712c0047a0 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7117fff640 1 -- 172.21.15.122:0/3704776962 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7130003b70 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce18a09620 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7117fff640 1 -- 172.21.15.122:0/3704776962 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712c0047a0 con 0x55ce18ccc370 2024-09-12T01:49:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7117fff640 1 -- 172.21.15.122:0/3704776962 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7130030020 con 0x55ce18ccc370 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 20 client.16086 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18cdc7f0 0x55ce18cdec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 4 client.16086 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.065+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce18c37a20 con 0x55ce18cdc7f0 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.066+0000 7f713cd09640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18cdc7f0 0x55ce18cdec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.066+0000 7f7117fff640 10 client.16086 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.066+0000 7f713cd09640 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18cdc7f0 0x55ce18cdec40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a34dc267ff067c server_cookie=bc2ad794d0b65e46 in_seq=0 out_seq=0 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7117fff640 1 -- 172.21.15.122:0/3704776962 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce18c37a20 con 0x55ce18cdc7f0 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7117fff640 10 client.16086 handle_command_reply: tid=0 2024-09-12T01:49:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 client.16086 shutdown 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 msgr2=0x55ce18cd80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cd80e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7130011f70 tx=0x7f7130019000 comp rx=0 tx=0).stop 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 shutdown_connections 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18cdc7f0 0x55ce18cdec40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18ccc370 0x55ce18cd80e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7117fff640 0 client.16086 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 --2- 172.21.15.122:0/3704776962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18cc69a0 0x55ce18cd0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 >> 172.21.15.122:0/3704776962 conn(0x55ce18cc2010 msgr2=0x55ce18cc3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 shutdown_connections 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 1 -- 172.21.15.122:0/3704776962 wait complete. 2024-09-12T01:49:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.067+0000 7f7141de3740 20 client.16086 trim_cache size 0 max 16384 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 Processor -- start 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 -- start start 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cc19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 0x55ce18cc35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71381adaf0 con 0x55ce18cc69a0 2024-09-12T01:49:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71381ab3c0 con 0x55ce18cc72e0 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.081+0000 7f7141de3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71381a91f0 con 0x55ce18ccc370 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cc19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cc19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60556/0 (socket says 172.21.15.122:60556) 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 -- 172.21.15.122:0/4268990903 learned_addr learned my addr 172.21.15.122:0/4268990903 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 -- 172.21.15.122:0/4268990903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 msgr2=0x55ce18cc35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 0x55ce18cc35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 -- 172.21.15.122:0/4268990903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 msgr2=0x55ce18cd80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 -- 172.21.15.122:0/4268990903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce18cc2ca0 con 0x55ce18cc69a0 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f713cd09640 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cc19b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f712809b5d0 tx=0x7f712801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895f598700624e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f711e7fc640 1 -- 172.21.15.122:0/4268990903 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712801d040 con 0x55ce18cc69a0 2024-09-12T01:49:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f711e7fc640 1 -- 172.21.15.122:0/4268990903 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71280b3030 con 0x55ce18cc69a0 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.082+0000 7f711e7fc640 1 -- 172.21.15.122:0/4268990903 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128097cb0 con 0x55ce18cc69a0 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- 172.21.15.122:0/4268990903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 msgr2=0x55ce18cc19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cc19b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f712809b5d0 tx=0x7f712801a040 comp rx=0 tx=0).stop 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- 172.21.15.122:0/4268990903 shutdown_connections 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 0x55ce18cc35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- 172.21.15.122:0/4268990903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cc19b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- 172.21.15.122:0/4268990903 >> 172.21.15.122:0/4268990903 conn(0x55ce18cc2010 msgr2=0x55ce18cc2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- 172.21.15.122:0/4268990903 shutdown_connections 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- 172.21.15.122:0/4268990903 wait complete. 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- start start 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 Processor -- start 2024-09-12T01:49:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cd55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 0x55ce18cd6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71381a9880 con 0x55ce18cc69a0 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71381aacf0 con 0x55ce18cc72e0 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.083+0000 7f7141de3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71381a9f50 con 0x55ce18ccc370 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7141de3740 10 client.0 _ll_register_callbacks cb 0x7f713000af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cd55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cd55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60560/0 (socket says 172.21.15.122:60560) 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 -- 172.21.15.122:0/2952577433 learned_addr learned my addr 172.21.15.122:0/2952577433 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7137fff640 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 -- 172.21.15.122:0/2952577433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 msgr2=0x55ce18cd6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 0x55ce18cd6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 -- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 msgr2=0x55ce18cd5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 -- 172.21.15.122:0/2952577433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce18cd7120 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f713cd09640 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cd55c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f71280a3cd0 tx=0x7f712809fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2111c0a9a4e2df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7117fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7117fff640 1 -- 172.21.15.122:0/2952577433 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71280b8020 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7117fff640 1 -- 172.21.15.122:0/2952577433 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7128003610 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71280b3030 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7117fff640 1 -- 172.21.15.122:0/2952577433 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f712801d070 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7141de3740 10 client.16092 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce18d47520 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7117fff640 1 -- 172.21.15.122:0/2952577433 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71280b3030 con 0x55ce18cc69a0 2024-09-12T01:49:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.084+0000 7f7117fff640 1 -- 172.21.15.122:0/2952577433 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f71280b4280 con 0x55ce18cc69a0 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7141de3740 10 client.16092 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7141de3740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7141de3740 20 client.16092 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7141de3740 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18d47e40 0x55ce18d4a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7141de3740 4 client.16092 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce18c31d70 con 0x55ce18d47e40 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7137fff640 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18d47e40 0x55ce18d4a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7117fff640 10 client.16092 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.085+0000 7f7137fff640 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18d47e40 0x55ce18d4a260 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1d5ff31e9b2a9b server_cookie=7a68b36376118b11 in_seq=0 out_seq=0 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7117fff640 1 -- 172.21.15.122:0/2952577433 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce18c31d70 con 0x55ce18d47e40 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7117fff640 10 client.16092 handle_command_reply: tid=0 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 client.16092 shutdown 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 msgr2=0x55ce18cd55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cd55c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f71280a3cd0 tx=0x7f712809fac0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 shutdown_connections 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ce18d47e40 0x55ce18d4a260 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce18ccc370 0x55ce18cd6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7117fff640 0 client.16092 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ce18cc72e0 0x55ce18cd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 --2- 172.21.15.122:0/2952577433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ce18cc69a0 0x55ce18cd55c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 >> 172.21.15.122:0/2952577433 conn(0x55ce18cc2010 msgr2=0x55ce18cc36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 shutdown_connections 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.094+0000 7f7141de3740 1 -- 172.21.15.122:0/2952577433 wait complete. 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f7141de3740 20 client.16092 trim_cache size 0 max 16384 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:49:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71200a2c30 msgr2=0x7f71200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71200a2c30 0x7f71200a50f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f71300136d0 tx=0x7f7130013ad0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 msgr2=0x7f7138102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 0x7f7138102760 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f712800c970 tx=0x7f712800ce30 comp rx=0 tx=0).stop 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 shutdown_connections 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71200a2c30 0x7f71200a50f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71381037b0 0x7f71381012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102df0 0x7f7138100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 --2- 172.21.15.122:0/2626806630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71380ffe20 0x7f7138102760 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 >> 172.21.15.122:0/2626806630 conn(0x7f71380fb980 msgr2=0x7f71381060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 shutdown_connections 2024-09-12T01:49:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:18.095+0000 7f713ef6d640 1 -- 172.21.15.122:0/2626806630 wait complete. 2024-09-12T01:49:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:17 smithi151 ceph-mon[35229]: pgmap v419: 97 pgs: 97 active+clean; 76 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-12T01:49:18.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.79 MB/sec warmup 96 sec latency 17019.706 ms 2024-09-12T01:49:19.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.71 MB/sec warmup 97 sec latency 18019.800 ms 2024-09-12T01:49:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:19 smithi183 ceph-mon[35127]: pgmap v420: 97 pgs: 97 active+clean; 74 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-09-12T01:49:19.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:19 smithi122 ceph-mon[30925]: pgmap v420: 97 pgs: 97 active+clean; 74 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-09-12T01:49:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:19 smithi151 ceph-mon[35229]: pgmap v420: 97 pgs: 97 active+clean; 74 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-09-12T01:49:20.819 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.63 MB/sec warmup 98 sec latency 19019.899 ms 2024-09-12T01:49:21.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.55 MB/sec warmup 99 sec latency 20020.044 ms 2024-09-12T01:49:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:21 smithi183 ceph-mon[35127]: pgmap v421: 97 pgs: 97 active+clean; 74 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 73 op/s 2024-09-12T01:49:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:21 smithi122 ceph-mon[30925]: pgmap v421: 97 pgs: 97 active+clean; 74 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 73 op/s 2024-09-12T01:49:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:21 smithi151 ceph-mon[35229]: pgmap v421: 97 pgs: 97 active+clean; 74 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 73 op/s 2024-09-12T01:49:22.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.48 MB/sec warmup 100 sec latency 21020.139 ms 2024-09-12T01:49:23.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.40 MB/sec warmup 101 sec latency 22020.240 ms 2024-09-12T01:49:23.896 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:49:23.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:23 smithi151 ceph-mon[35229]: pgmap v422: 97 pgs: 97 active+clean; 74 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.3 MiB/s wr, 83 op/s 2024-09-12T01:49:23.924 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:23 smithi122 ceph-mon[30925]: pgmap v422: 97 pgs: 97 active+clean; 74 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.3 MiB/s wr, 83 op/s 2024-09-12T01:49:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:23 smithi183 ceph-mon[35127]: pgmap v422: 97 pgs: 97 active+clean; 74 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.3 MiB/s wr, 83 op/s 2024-09-12T01:49:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:49:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 -- 172.21.15.122:0/1350106736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540106a80 msgr2=0x7fa540106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:24.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 --2- 172.21.15.122:0/1350106736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540106a80 0x7fa540106e60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009930 tx=0x7fa53c02a420 comp rx=0 tx=0).stop 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 -- 172.21.15.122:0/1350106736 shutdown_connections 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 --2- 172.21.15.122:0/1350106736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa540101420 0x7fa54010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 --2- 172.21.15.122:0/1350106736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa540100a80 0x7fa540100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 --2- 172.21.15.122:0/1350106736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540106a80 0x7fa540106e60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 -- 172.21.15.122:0/1350106736 >> 172.21.15.122:0/1350106736 conn(0x7fa5400fc7a0 msgr2=0x7fa5400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 -- 172.21.15.122:0/1350106736 shutdown_connections 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.625+0000 7fa5480b5640 1 -- 172.21.15.122:0/1350106736 wait complete. 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 Processor -- start 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 -- start start 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa540100a80 0x7fa540199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:24.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 0x7fa540199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa540106a80 0x7fa5401a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa540110f20 con 0x7fa540101420 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa540110da0 con 0x7fa540100a80 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa5480b5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401110a0 con 0x7fa540106a80 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 0x7fa540199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 0x7fa540199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47392/0 (socket says 172.21.15.122:47392) 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 -- 172.21.15.122:0/4047462084 learned_addr learned my addr 172.21.15.122:0/4047462084 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545e51640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa540100a80 0x7fa540199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 -- 172.21.15.122:0/4047462084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa540106a80 msgr2=0x7fa5401a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa540106a80 0x7fa5401a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 -- 172.21.15.122:0/4047462084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa540100a80 msgr2=0x7fa540199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa540100a80 0x7fa540199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.626+0000 7fa545650640 1 -- 172.21.15.122:0/4047462084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5401a49d0 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.627+0000 7fa545650640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 0x7fa540199950 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa53000b4a0 tx=0x7fa53000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.627+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530014070 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.627+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401a4ba0 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.627+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530010af0 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.627+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530017d10 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.627+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401a5090 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.628+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa540100f60 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.630+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa530017070 con 0x7fa540101420 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.630+0000 7fa536ffd640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa52c0a2b80 0x7fa52c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:24.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.630+0000 7fa545e51640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa52c0a2b80 0x7fa52c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.630+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa5300ce850 con 0x7fa540101420 2024-09-12T01:49:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.631+0000 7fa545e51640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa52c0a2b80 0x7fa52c0a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa53c000c00 tx=0x7fa53c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:24.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.633+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa53009b7b0 con 0x7fa540101420 2024-09-12T01:49:24.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.33 MB/sec warmup 102 sec latency 23020.321 ms 2024-09-12T01:49:24.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.837+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa54006ad80 con 0x7fa540101420 2024-09-12T01:49:24.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.837+0000 7fa536ffd640 1 -- 172.21.15.122:0/4047462084 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11934 (secure 0 0 0) 0x7fa53009b0b0 con 0x7fa540101420 2024-09-12T01:49:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:49:24.838 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"btime":"2024-09-12T01:48:18:666269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:48:17.839016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:49:24.838 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-09-12T01:49:24.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa52c0a2b80 msgr2=0x7fa52c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:24.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa52c0a2b80 0x7fa52c0a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa53c000c00 tx=0x7fa53c031490 comp rx=0 tx=0).stop 2024-09-12T01:49:24.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 msgr2=0x7fa540199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:24.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 0x7fa540199950 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa53000b4a0 tx=0x7fa53000b960 comp rx=0 tx=0).stop 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 shutdown_connections 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa52c0a2b80 0x7fa52c0a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa540106a80 0x7fa5401a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa540101420 0x7fa540199950 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 --2- 172.21.15.122:0/4047462084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa540100a80 0x7fa540199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 >> 172.21.15.122:0/4047462084 conn(0x7fa5400fc7a0 msgr2=0x7fa5400fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 shutdown_connections 2024-09-12T01:49:24.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:24.839+0000 7fa5480b5640 1 -- 172.21.15.122:0/4047462084 wait complete. 2024-09-12T01:49:25.557 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 863f85e1 --release --await' 2024-09-12T01:49:25.557 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 863f85e1 --release --await 2024-09-12T01:49:25.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135177 7.26 MB/sec warmup 103 sec latency 24020.359 ms 2024-09-12T01:49:25.841 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:49:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:25 smithi183 ceph-mon[35127]: pgmap v423: 97 pgs: 97 active+clean; 72 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.6 MiB/s wr, 95 op/s 2024-09-12T01:49:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:25 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4047462084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:49:25.954 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:25 smithi122 ceph-mon[30925]: pgmap v423: 97 pgs: 97 active+clean; 72 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.6 MiB/s wr, 95 op/s 2024-09-12T01:49:25.954 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:25 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4047462084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:49:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:25 smithi151 ceph-mon[35229]: pgmap v423: 97 pgs: 97 active+clean; 72 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.6 MiB/s wr, 95 op/s 2024-09-12T01:49:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4047462084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:49:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 -- 172.21.15.122:0/3782182870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 msgr2=0x7f2a0c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 --2- 172.21.15.122:0/3782182870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c100ee0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009cd0 tx=0x7f2a0002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 -- 172.21.15.122:0/3782182870 shutdown_connections 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 --2- 172.21.15.122:0/3782182870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0c101420 0x7f2a0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 --2- 172.21.15.122:0/3782182870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c100ee0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 --2- 172.21.15.122:0/3782182870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c106a80 0x7f2a0c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 -- 172.21.15.122:0/3782182870 >> 172.21.15.122:0/3782182870 conn(0x7f2a0c0fc7a0 msgr2=0x7f2a0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 -- 172.21.15.122:0/3782182870 shutdown_connections 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.311+0000 7f2a13588640 1 -- 172.21.15.122:0/3782182870 wait complete. 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 Processor -- start 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 -- start start 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0c101420 0x7f2a0c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c106a80 0x7f2a0c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c110e80 con 0x7f2a0c100a80 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c110d00 con 0x7f2a0c106a80 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a13588640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c111000 con 0x7f2a0c101420 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47420/0 (socket says 172.21.15.122:47420) 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 -- 172.21.15.122:0/138435782 learned_addr learned my addr 172.21.15.122:0/138435782 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11b25640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c106a80 0x7f2a0c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a10b23640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0c101420 0x7f2a0c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 -- 172.21.15.122:0/138435782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0c101420 msgr2=0x7f2a0c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0c101420 0x7f2a0c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 -- 172.21.15.122:0/138435782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c106a80 msgr2=0x7f2a0c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c106a80 0x7f2a0c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 -- 172.21.15.122:0/138435782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c1a69b0 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a11324640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c197140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9f0 tx=0x7f2a0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.312+0000 7f2a0a7fc640 1 -- 172.21.15.122:0/138435782 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400cd80 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.313+0000 7f2a0a7fc640 1 -- 172.21.15.122:0/138435782 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04010910 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.313+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c1a6ca0 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.313+0000 7f2a0a7fc640 1 -- 172.21.15.122:0/138435782 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04018990 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.313+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a71d0 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.315+0000 7f2a0a7fc640 1 -- 172.21.15.122:0/138435782 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2a04010ac0 con 0x7f2a0c100a80 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.315+0000 7f2a15961740 1 Processor -- start 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.315+0000 7f2a15961740 1 -- start start 2024-09-12T01:49:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.315+0000 7f2a15961740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.315+0000 7f2a15961740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.315+0000 7f2a15961740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bbe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a15961740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563b40b16620 con 0x563b40bb1060 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a15961740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563b40abde50 con 0x563b40bbc240 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a15961740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b40b125e0 con 0x563b40bb1980 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a11b25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a11b25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47426/0 (socket says 172.21.15.122:47426) 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a11b25640 1 -- 172.21.15.122:0/1708463071 learned_addr learned my addr 172.21.15.122:0/1708463071 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a0a7fc640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29f40a2c00 0x7f29f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a11324640 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a10b23640 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bbe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a11b25640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29f40a2c00 0x7f29f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.316+0000 7f2a0a7fc640 1 -- 172.21.15.122:0/138435782 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2a040cf370 con 0x7f2a0c100a80 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11b25640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29f40a2c00 0x7f29f40a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f29fc013950 tx=0x7f29fc00d810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11324640 1 -- 172.21.15.122:0/1708463071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 msgr2=0x563b40bbe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11324640 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bbe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11324640 1 -- 172.21.15.122:0/1708463071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 msgr2=0x563b40bb1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11324640 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11324640 1 -- 172.21.15.122:0/1708463071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b40aaed40 con 0x563b40bb1980 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a10b23640 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bbe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f2a11324640 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bbbd00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a040985a0 tx=0x7f2a04097870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b203db25ee448ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f29f37fe640 1 -- 172.21.15.122:0/1708463071 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0409bbd0 con 0x563b40bb1980 2024-09-12T01:49:26.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f29f37fe640 1 -- 172.21.15.122:0/1708463071 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a040a4840 con 0x563b40bb1980 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.317+0000 7f29f37fe640 1 -- 172.21.15.122:0/1708463071 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0409b280 con 0x563b40bb1980 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 -- 172.21.15.122:0/1708463071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 msgr2=0x563b40bbbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bbbd00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a040985a0 tx=0x7f2a04097870 comp rx=0 tx=0).stop 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 -- 172.21.15.122:0/1708463071 shutdown_connections 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bbe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bbbd00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 --2- 172.21.15.122:0/1708463071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 -- 172.21.15.122:0/1708463071 >> 172.21.15.122:0/1708463071 conn(0x563b40bad5f0 msgr2=0x563b40bad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 -- 172.21.15.122:0/1708463071 shutdown_connections 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.318+0000 7f2a15961740 1 -- 172.21.15.122:0/1708463071 wait complete. 2024-09-12T01:49:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 -- start start 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 Processor -- start 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bb9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bb9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563b40bbee10 con 0x563b40bb1060 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563b40a9ef10 con 0x563b40bbc240 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b40bbef90 con 0x563b40bb1980 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a15961740 10 client.0 _ll_register_callbacks cb 0x563b40ac2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a11324640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a11324640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47434/0 (socket says 172.21.15.122:47434) 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a11324640 1 -- 172.21.15.122:0/1428544160 learned_addr learned my addr 172.21.15.122:0/1428544160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.319+0000 7f2a10b23640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bb9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11324640 1 -- 172.21.15.122:0/1428544160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 msgr2=0x563b40bb9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11b25640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bb9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11324640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bb9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11324640 1 -- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 msgr2=0x563b40bb9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11324640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bb9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11324640 1 -- 172.21.15.122:0/1428544160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b40825580 con 0x563b40bb1060 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11b25640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bb9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a11324640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb9150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a04018460 tx=0x7f2a0409b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9b5d2ecf9acc44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f29f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f29f0ff9640 1 -- 172.21.15.122:0/1428544160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04018d30 con 0x563b40bb1060 2024-09-12T01:49:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f29f0ff9640 1 -- 172.21.15.122:0/1428544160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2a04010430 con 0x563b40bb1060 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a040a4840 con 0x563b40bb1060 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f29f0ff9640 1 -- 172.21.15.122:0/1428544160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a0409bbd0 con 0x563b40bb1060 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a15961740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b408ef960 con 0x563b40bb1060 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f29f0ff9640 1 -- 172.21.15.122:0/1428544160 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a040a4840 con 0x563b40bb1060 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f29f0ff9640 1 -- 172.21.15.122:0/1428544160 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a040a1230 con 0x563b40bb1060 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a15961740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a15961740 10 client.16110 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.320+0000 7f2a15961740 20 client.16110 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.321+0000 7f2a15961740 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40bcbc70 0x563b40bce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.321+0000 7f2a15961740 4 client.16110 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.321+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b40b23b90 con 0x563b40bcbc70 2024-09-12T01:49:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.321+0000 7f2a10b23640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40bcbc70 0x563b40bce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.321+0000 7f29f0ff9640 10 client.16110 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.321+0000 7f2a10b23640 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40bcbc70 0x563b40bce060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc57c84ddd0ceef server_cookie=b5b0c6fa7aee7b78 in_seq=0 out_seq=0 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f29f0ff9640 1 -- 172.21.15.122:0/1428544160 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b40b23b90 con 0x563b40bcbc70 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f29f0ff9640 10 client.16110 handle_command_reply: tid=0 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 client.16110 shutdown 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 msgr2=0x563b40bb9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb9150 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a04018460 tx=0x7f2a0409b470 comp rx=0 tx=0).stop 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 shutdown_connections 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40bcbc70 0x563b40bce060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bbc240 0x563b40bb9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1980 0x563b40bb9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 --2- 172.21.15.122:0/1428544160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1060 0x563b40bb9150 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 >> 172.21.15.122:0/1428544160 conn(0x563b40bad5f0 msgr2=0x563b40bb3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:26.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f29f0ff9640 0 client.16110 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:26.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 shutdown_connections 2024-09-12T01:49:26.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 1 -- 172.21.15.122:0/1428544160 wait complete. 2024-09-12T01:49:26.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.322+0000 7f2a15961740 20 client.16110 trim_cache size 0 max 16384 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 Processor -- start 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 -- start start 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1060 0x563b40bbb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 0x563b40bd0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bbc240 0x563b40bd11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1b21a0 con 0x563b40bb1980 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1afa70 con 0x563b40bb1060 2024-09-12T01:49:26.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a15961740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1ad8a0 con 0x563b40bbc240 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a10b23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 0x563b40bd0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a10b23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 0x563b40bd0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47444/0 (socket says 172.21.15.122:47444) 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a10b23640 1 -- 172.21.15.122:0/3668242635 learned_addr learned my addr 172.21.15.122:0/3668242635 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.339+0000 7f2a11324640 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1060 0x563b40bbb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a11b25640 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bbc240 0x563b40bd11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a10b23640 1 -- 172.21.15.122:0/3668242635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bbc240 msgr2=0x563b40bd11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a10b23640 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bbc240 0x563b40bd11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a10b23640 1 -- 172.21.15.122:0/3668242635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1060 msgr2=0x563b40bbb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a10b23640 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1060 0x563b40bbb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a10b23640 1 -- 172.21.15.122:0/3668242635 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b40bb8100 con 0x563b40bb1980 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f2a10b23640 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 0x563b40bd0ca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2a0002ae00 tx=0x7f2a00005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656ae3db03809e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:26.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f29f2ffd640 1 -- 172.21.15.122:0/3668242635 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000318c0 con 0x563b40bb1980 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f29f2ffd640 1 -- 172.21.15.122:0/3668242635 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a00036040 con 0x563b40bb1980 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.340+0000 7f29f2ffd640 1 -- 172.21.15.122:0/3668242635 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000044a0 con 0x563b40bb1980 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 -- 172.21.15.122:0/3668242635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 msgr2=0x563b40bd0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 0x563b40bd0ca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2a0002ae00 tx=0x7f2a00005c30 comp rx=0 tx=0).stop 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 -- 172.21.15.122:0/3668242635 shutdown_connections 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bbc240 0x563b40bd11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bb1980 0x563b40bd0ca0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 --2- 172.21.15.122:0/3668242635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1060 0x563b40bbb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 -- 172.21.15.122:0/3668242635 >> 172.21.15.122:0/3668242635 conn(0x563b40bad5f0 msgr2=0x563b40bba9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 -- 172.21.15.122:0/3668242635 shutdown_connections 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 -- 172.21.15.122:0/3668242635 wait complete. 2024-09-12T01:49:26.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 -- start start 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.341+0000 7f2a15961740 1 Processor -- start 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1060 0x563b40bcf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1980 0x563b40bcf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 0x563b40c2f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1adf30 con 0x563b40bbc240 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1af3a0 con 0x563b40bb1980 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1ae600 con 0x563b40bb1060 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 10 client.0 _ll_register_callbacks cb 0x7f2a0400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 0x563b40c2f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 0x563b40c2f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47456/0 (socket says 172.21.15.122:47456) 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 -- 172.21.15.122:0/3955486006 learned_addr learned my addr 172.21.15.122:0/3955486006 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11324640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1060 0x563b40bcf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a10b23640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1980 0x563b40bcf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 -- 172.21.15.122:0/3955486006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1060 msgr2=0x563b40bcf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1060 0x563b40bcf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 -- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1980 msgr2=0x563b40bcf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1980 0x563b40bcf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 -- 172.21.15.122:0/3955486006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b40c2fc30 con 0x563b40bbc240 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a11b25640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 0x563b40c2f5f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f29fc011c30 tx=0x7f29fc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e913c5747dceadc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f29f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f29f0ff9640 1 -- 172.21.15.122:0/3955486006 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc02d060 con 0x563b40bbc240 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f29f0ff9640 1 -- 172.21.15.122:0/3955486006 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f29fc004730 con 0x563b40bbc240 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.342+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a00036040 con 0x563b40bbc240 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f29f0ff9640 1 -- 172.21.15.122:0/3955486006 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29fc004930 con 0x563b40bbc240 2024-09-12T01:49:26.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 10 client.16116 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b40c37020 con 0x563b40bbc240 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f29f0ff9640 1 -- 172.21.15.122:0/3955486006 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a00036040 con 0x563b40bbc240 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f29f0ff9640 1 -- 172.21.15.122:0/3955486006 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f29fc032070 con 0x563b40bbc240 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 10 client.16116 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 20 client.16116 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40c37880 0x563b40c39c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 4 client.16116 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "863f85e1", "release": true, "await": true}] 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "863f85e1", "release": true, "await": true}) -- 0x563b40b1e0a0 con 0x563b40c37880 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a11324640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40c37880 0x563b40c39c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f2a11324640 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40c37880 0x563b40c39c70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffb5ee9a03cf92a server_cookie=c1c2790a062a7221 in_seq=0 out_seq=0 2024-09-12T01:49:26.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.343+0000 7f29f0ff9640 10 client.16116 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:26.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f29f0ff9640 1 -- 172.21.15.122:0/3955486006 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563b40b1e0a0 con 0x563b40c37880 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f29f0ff9640 10 client.16116 handle_command_reply: tid=0 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 client.16116 shutdown 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 msgr2=0x563b40c2f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 0x563b40c2f5f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f29fc011c30 tx=0x7f29fc01b000 comp rx=0 tx=0).stop 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 shutdown_connections 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563b40c37880 0x563b40c39c70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b40bbc240 0x563b40c2f5f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f29f0ff9640 0 client.16116 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b40bb1980 0x563b40bcf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 --2- 172.21.15.122:0/3955486006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b40bb1060 0x563b40bcf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 >> 172.21.15.122:0/3955486006 conn(0x563b40bad5f0 msgr2=0x563b40bb32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 shutdown_connections 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 1 -- 172.21.15.122:0/3955486006 wait complete. 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.353+0000 7f2a15961740 20 client.16116 trim_cache size 0 max 16384 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 28, 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:49:26.355 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 12, 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "863f85e1": { 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 12, 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.7, 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:49:26.356 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:49:26.357 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29f40a2c00 msgr2=0x7f29f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29f40a2c00 0x7f29f40a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f29fc013950 tx=0x7f29fc00d810 comp rx=0 tx=0).stop 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 msgr2=0x7f2a0c197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c197140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9f0 tx=0x7f2a0400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 shutdown_connections 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f29f40a2c00 0x7f29f40a50c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c106a80 0x7f2a0c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0c101420 0x7f2a0c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 --2- 172.21.15.122:0/138435782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a0c100a80 0x7f2a0c197140 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:26.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 >> 172.21.15.122:0/138435782 conn(0x7f2a0c0fc7a0 msgr2=0x7f2a0c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:26.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 shutdown_connections 2024-09-12T01:49:26.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:49:26.354+0000 7f2a13588640 1 -- 172.21.15.122:0/138435782 wait complete. 2024-09-12T01:49:26.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135999 7.21 MB/sec warmup 104 sec latency 24554.111 ms 2024-09-12T01:49:27.051 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '863f85e1', total seconds elapsed: 27.9 2024-09-12T01:49:27.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 138939 7.30 MB/sec warmup 105 sec latency 14.678 ms 2024-09-12T01:49:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:27 smithi183 ceph-mon[35127]: pgmap v424: 97 pgs: 97 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-09-12T01:49:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:27 smithi122 ceph-mon[30925]: pgmap v424: 97 pgs: 97 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-09-12T01:49:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:27 smithi151 ceph-mon[35229]: pgmap v424: 97 pgs: 97 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-09-12T01:49:28.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142221 7.41 MB/sec warmup 106 sec latency 15.160 ms 2024-09-12T01:49:29.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 145400 7.51 MB/sec warmup 107 sec latency 12.795 ms 2024-09-12T01:49:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:29 smithi183 ceph-mon[35127]: pgmap v425: 97 pgs: 97 active+clean; 67 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.5 MiB/s wr, 110 op/s 2024-09-12T01:49:29.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:29 smithi122 ceph-mon[30925]: pgmap v425: 97 pgs: 97 active+clean; 67 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.5 MiB/s wr, 110 op/s 2024-09-12T01:49:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:29 smithi151 ceph-mon[35229]: pgmap v425: 97 pgs: 97 active+clean; 67 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.5 MiB/s wr, 110 op/s 2024-09-12T01:49:30.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 148353 7.56 MB/sec warmup 108 sec latency 10.014 ms 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 -- 172.21.15.122:0/961368843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102000 msgr2=0x7ff9e8102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 --2- 172.21.15.122:0/961368843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102000 0x7ff9e8102400 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009900 tx=0x7ff9e402a410 comp rx=0 tx=0).stop 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 -- 172.21.15.122:0/961368843 shutdown_connections 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 --2- 172.21.15.122:0/961368843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e81016f0 0x7ff9e8108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 --2- 172.21.15.122:0/961368843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 --2- 172.21.15.122:0/961368843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102000 0x7ff9e8102400 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 -- 172.21.15.122:0/961368843 >> 172.21.15.122:0/961368843 conn(0x7ff9e80fb970 msgr2=0x7ff9e80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 -- 172.21.15.122:0/961368843 shutdown_connections 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.185+0000 7ff9f0186640 1 -- 172.21.15.122:0/961368843 wait complete. 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 Processor -- start 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 -- start start 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e81016f0 0x7ff9e819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e8102000 0x7ff9e819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e810c6e0 con 0x7ff9e81016f0 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e810c560 con 0x7ff9e8102000 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9f0186640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e810c860 con 0x7ff9e8100d30 2024-09-12T01:49:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9ed721640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e81016f0 0x7ff9e819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:35182/0 (socket says 172.21.15.122:35182) 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 -- 172.21.15.122:0/3308009865 learned_addr learned my addr 172.21.15.122:0/3308009865 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e8102000 0x7ff9e819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 -- 172.21.15.122:0/3308009865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e8102000 msgr2=0x7ff9e819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e8102000 0x7ff9e819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 -- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e81016f0 msgr2=0x7ff9e819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e81016f0 0x7ff9e819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9edf22640 1 -- 172.21.15.122:0/3308009865 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e819fa80 con 0x7ff9e8100d30 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e81016f0 0x7ff9e819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.186+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e8102000 0x7ff9e819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.187+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81007d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4006010 tx=0x7ff9e40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.187+0000 7ff9deffd640 1 -- 172.21.15.122:0/3308009865 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e403f030 con 0x7ff9e8100d30 2024-09-12T01:49:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.187+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e819fd10 con 0x7ff9e8100d30 2024-09-12T01:49:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.187+0000 7ff9deffd640 1 -- 172.21.15.122:0/3308009865 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e4003ce0 con 0x7ff9e8100d30 2024-09-12T01:49:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.187+0000 7ff9deffd640 1 -- 172.21.15.122:0/3308009865 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4035070 con 0x7ff9e8100d30 2024-09-12T01:49:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.187+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e81a01a0 con 0x7ff9e8100d30 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.189+0000 7ff9deffd640 1 -- 172.21.15.122:0/3308009865 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff9e4042030 con 0x7ff9e8100d30 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.189+0000 7ff9f2ffe740 1 Processor -- start 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.189+0000 7ff9f2ffe740 1 -- start start 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad91da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbad96da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563bbacf52f0 con 0x563bbad97370 2024-09-12T01:49:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563bbac9d270 con 0x563bbad919a0 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bbacf12b0 con 0x563bbad922e0 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ee723640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad91da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad99830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48376/0 (socket says 172.21.15.122:48376) 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 -- 172.21.15.122:0/383612878 learned_addr learned my addr 172.21.15.122:0/383612878 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ee723640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad91da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:41878/0 (socket says 172.21.15.122:41878) 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9edf22640 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbad96da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9deffd640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9d40a2c30 0x7ff9d40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 -- 172.21.15.122:0/383612878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 msgr2=0x563bbad96da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbad96da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 -- 172.21.15.122:0/383612878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 msgr2=0x563bbad91da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad91da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 -- 172.21.15.122:0/383612878 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bbac9c340 con 0x563bbad97370 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9d40a2c30 0x7ff9d40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9ed721640 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad99830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8010480 tx=0x7ff9d8010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a38b317180e8a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.190+0000 7ff9deffd640 1 -- 172.21.15.122:0/3308009865 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff9e40ef4d0 con 0x7ff9e8100d30 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9d3fff640 1 -- 172.21.15.122:0/383612878 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8011760 con 0x563bbad97370 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9d3fff640 1 -- 172.21.15.122:0/383612878 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8011de0 con 0x563bbad97370 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9d3fff640 1 -- 172.21.15.122:0/383612878 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d801a380 con 0x563bbad97370 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/383612878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 msgr2=0x563bbad99830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9d40a2c30 0x7ff9d40a50f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009fd0 tx=0x7ff9e0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad99830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8010480 tx=0x7ff9d8010940 comp rx=0 tx=0).stop 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/383612878 shutdown_connections 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad99830 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbad96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/383612878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad91da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.191+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/383612878 >> 172.21.15.122:0/383612878 conn(0x563bbad8d010 msgr2=0x563bbad8d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/383612878 shutdown_connections 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/383612878 wait complete. 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 -- start start 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 Processor -- start 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad919a0 0x563bbad9bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 0x563bbad9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563bbad9a340 con 0x563bbad922e0 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563bbac7e290 con 0x563bbad97370 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bbad9a4c0 con 0x563bbad919a0 2024-09-12T01:49:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9f2ffe740 10 client.0 _ll_register_callbacks cb 0x563bbacaf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9ed721640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 0x563bbad9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9edf22640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad919a0 0x563bbad9bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9ed721640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 0x563bbad9c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48378/0 (socket says 172.21.15.122:48378) 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.192+0000 7ff9ed721640 1 -- 172.21.15.122:0/3224075972 learned_addr learned my addr 172.21.15.122:0/3224075972 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9edf22640 1 -- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 msgr2=0x563bbada3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9edf22640 1 -- 172.21.15.122:0/3224075972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 msgr2=0x563bbad9c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 0x563bbad9c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9edf22640 1 -- 172.21.15.122:0/3224075972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bbaa08210 con 0x563bbad919a0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 0x563bbad9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad919a0 0x563bbad9bb90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x563bbad92d10 tx=0x7ff9e4004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d1b3af1b923159 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:49:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3224075972 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4049070 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d8011de0 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3224075972 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff9e40b8410 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3224075972 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9e40bbb70 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9f2ffe740 10 client.25459 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563bbaad4620 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.193+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3224075972 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8011de0 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3224075972 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff9e40bc000 con 0x563bbad919a0 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9f2ffe740 10 client.25459 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9f2ffe740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9f2ffe740 20 client.25459 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbada7870 0x563bbada9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9f2ffe740 4 client.25459 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bbad02a20 con 0x563bbada7870 2024-09-12T01:49:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbada7870 0x563bbada9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9d17fa640 10 client.25459 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.194+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbada7870 0x563bbada9cf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3642248b8a89e3 server_cookie=9ba4fe236bc00bf5 in_seq=0 out_seq=0 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3224075972 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bbad02a20 con 0x563bbada7870 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9d17fa640 10 client.25459 handle_command_reply: tid=0 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 client.25459 shutdown 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad919a0 msgr2=0x563bbad9bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad919a0 0x563bbad9bb90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x563bbad92d10 tx=0x7ff9e4004510 comp rx=0 tx=0).stop 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 shutdown_connections 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbada7870 0x563bbada9cf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad922e0 0x563bbad9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3224075972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad919a0 0x563bbad9bb90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9d17fa640 0 client.25459 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 >> 172.21.15.122:0/3224075972 conn(0x563bbad8d010 msgr2=0x563bbad8e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 shutdown_connections 2024-09-12T01:49:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3224075972 wait complete. 2024-09-12T01:49:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.195+0000 7ff9f2ffe740 20 client.25459 trim_cache size 0 max 16384 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 Processor -- start 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 -- start start 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad8e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81a96d0 con 0x563bbad97370 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81a6fa0 con 0x563bbad919a0 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81a4dd0 con 0x563bbad922e0 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad8e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad8e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48384/0 (socket says 172.21.15.122:48384) 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 -- 172.21.15.122:0/3300813503 learned_addr learned my addr 172.21.15.122:0/3300813503 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 -- 172.21.15.122:0/3300813503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 msgr2=0x563bbada30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 -- 172.21.15.122:0/3300813503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 msgr2=0x563bbad8c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad8c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.210+0000 7ff9ee723640 1 -- 172.21.15.122:0/3300813503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bbad92f70 con 0x563bbad97370 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad8e400 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0004590 tx=0x7ff9e001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de73c51a8b573f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9d37fe640 1 -- 172.21.15.122:0/3300813503 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0004680 con 0x563bbad97370 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9d37fe640 1 -- 172.21.15.122:0/3300813503 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e0032070 con 0x563bbad97370 2024-09-12T01:49:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9d37fe640 1 -- 172.21.15.122:0/3300813503 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e001e3b0 con 0x563bbad97370 2024-09-12T01:49:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3300813503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 msgr2=0x563bbad8e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad8e400 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0004590 tx=0x7ff9e001c000 comp rx=0 tx=0).stop 2024-09-12T01:49:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3300813503 shutdown_connections 2024-09-12T01:49:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad97370 0x563bbad8e400 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.211+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3300813503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad919a0 0x563bbad8c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3300813503 >> 172.21.15.122:0/3300813503 conn(0x563bbad8d010 msgr2=0x563bbad8b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3300813503 shutdown_connections 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3300813503 wait complete. 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- start start 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 Processor -- start 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 0x563bbada06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81a5460 con 0x563bbad919a0 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81a68d0 con 0x563bbad97370 2024-09-12T01:49:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81a5b30 con 0x563bbad922e0 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9f2ffe740 10 client.0 _ll_register_callbacks cb 0x7ff9e40f2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9edf22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 0x563bbada06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9edf22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 0x563bbada06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48394/0 (socket says 172.21.15.122:48394) 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9edf22640 1 -- 172.21.15.122:0/3752413324 learned_addr learned my addr 172.21.15.122:0/3752413324 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.212+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9edf22640 1 -- 172.21.15.122:0/3752413324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 msgr2=0x563bbada0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9edf22640 1 -- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 msgr2=0x563bbada1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9edf22640 1 -- 172.21.15.122:0/3752413324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bbada2210 con 0x563bbad919a0 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9ee723640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9edf22640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 0x563bbada06b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff9e403bed0 tx=0x7ff9e40ed030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2890277dfde19154 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3752413324 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4049040 con 0x563bbad919a0 2024-09-12T01:49:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3752413324 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff9e40c44d0 con 0x563bbad919a0 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e0032070 con 0x563bbad919a0 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3752413324 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9e40a1050 con 0x563bbad919a0 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9f2ffe740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563bbae12590 con 0x563bbad919a0 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3752413324 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e0032070 con 0x563bbad919a0 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.213+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3752413324 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff9e40bb1f0 con 0x563bbad919a0 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9f2ffe740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9f2ffe740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9f2ffe740 20 client.16134 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbae12cd0 0x563bbae150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9f2ffe740 4 client.16134 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:49:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bbacfcd70 con 0x563bbae12cd0 2024-09-12T01:49:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbae12cd0 0x563bbae150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9ed721640 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbae12cd0 0x563bbae150f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c5e324646fbd6c server_cookie=8d19b9b9737bf866 in_seq=0 out_seq=0 2024-09-12T01:49:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.214+0000 7ff9d17fa640 10 client.16134 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.222+0000 7ff9d17fa640 1 -- 172.21.15.122:0/3752413324 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bbacfcd70 con 0x563bbae12cd0 2024-09-12T01:49:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.222+0000 7ff9d17fa640 10 client.16134 handle_command_reply: tid=0 2024-09-12T01:49:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.222+0000 7ff9f2ffe740 1 client.16134 shutdown 2024-09-12T01:49:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.237+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 msgr2=0x563bbada06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.237+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 0x563bbada06b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff9e403bed0 tx=0x7ff9e40ed030 comp rx=0 tx=0).stop 2024-09-12T01:49:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.238+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 shutdown_connections 2024-09-12T01:49:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.238+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563bbae12cd0 0x563bbae150f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.238+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bbad97370 0x563bbada1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.238+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bbad922e0 0x563bbada0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.238+0000 7ff9f2ffe740 1 --2- 172.21.15.122:0/3752413324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bbad919a0 0x563bbada06b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.238+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 >> 172.21.15.122:0/3752413324 conn(0x563bbad8d010 msgr2=0x563bbad8b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.241+0000 7ff9d17fa640 0 client.16134 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.244+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 shutdown_connections 2024-09-12T01:49:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.244+0000 7ff9f2ffe740 1 -- 172.21.15.122:0/3752413324 wait complete. 2024-09-12T01:49:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.245+0000 7ff9f2ffe740 20 client.16134 trim_cache size 0 max 16384 2024-09-12T01:49:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:49:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:49:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:49:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:49:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.249+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9d40a2c30 msgr2=0x7ff9d40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.249+0000 7ff9f0186640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9d40a2c30 0x7ff9d40a50f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009fd0 tx=0x7ff9e0004a40 comp rx=0 tx=0).stop 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.249+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 msgr2=0x7ff9e81007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.249+0000 7ff9f0186640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81007d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4006010 tx=0x7ff9e40042f0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 shutdown_connections 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9d40a2c30 0x7ff9d40a50f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9e8102000 0x7ff9e819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e81016f0 0x7ff9e819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 --2- 172.21.15.122:0/3308009865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9e8100d30 0x7ff9e81007d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 >> 172.21.15.122:0/3308009865 conn(0x7ff9e80fb970 msgr2=0x7ff9e8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 shutdown_connections 2024-09-12T01:49:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:31.250+0000 7ff9f0186640 1 -- 172.21.15.122:0/3308009865 wait complete. 2024-09-12T01:49:31.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 150400 7.61 MB/sec warmup 109 sec latency 76.074 ms 2024-09-12T01:49:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:31 smithi183 ceph-mon[35127]: pgmap v426: 97 pgs: 97 active+clean; 67 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 141 op/s 2024-09-12T01:49:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:31 smithi122 ceph-mon[30925]: pgmap v426: 97 pgs: 97 active+clean; 67 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 141 op/s 2024-09-12T01:49:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:31 smithi151 ceph-mon[35229]: pgmap v426: 97 pgs: 97 active+clean; 67 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 141 op/s 2024-09-12T01:49:32.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 153227 7.67 MB/sec warmup 110 sec latency 16.766 ms 2024-09-12T01:49:33.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156485 7.77 MB/sec warmup 111 sec latency 10.506 ms 2024-09-12T01:49:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:33 smithi183 ceph-mon[35127]: pgmap v427: 97 pgs: 97 active+clean; 68 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-09-12T01:49:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:33 smithi122 ceph-mon[30925]: pgmap v427: 97 pgs: 97 active+clean; 68 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-09-12T01:49:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:33 smithi151 ceph-mon[35229]: pgmap v427: 97 pgs: 97 active+clean; 68 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.7 MiB/s wr, 157 op/s 2024-09-12T01:49:34.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 159716 7.86 MB/sec warmup 112 sec latency 10.838 ms 2024-09-12T01:49:35.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 162535 7.91 MB/sec warmup 113 sec latency 16.612 ms 2024-09-12T01:49:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:35 smithi183 ceph-mon[35127]: pgmap v428: 97 pgs: 97 active+clean; 72 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-12T01:49:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:35 smithi122 ceph-mon[30925]: pgmap v428: 97 pgs: 97 active+clean; 72 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-12T01:49:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:35 smithi151 ceph-mon[35229]: pgmap v428: 97 pgs: 97 active+clean; 72 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-12T01:49:36.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 164994 7.95 MB/sec warmup 114 sec latency 10.260 ms 2024-09-12T01:49:37.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 167712 8.01 MB/sec warmup 115 sec latency 14.056 ms 2024-09-12T01:49:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:37 smithi183 ceph-mon[35127]: pgmap v429: 97 pgs: 97 active+clean; 72 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 257 op/s 2024-09-12T01:49:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:37 smithi122 ceph-mon[30925]: pgmap v429: 97 pgs: 97 active+clean; 72 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 257 op/s 2024-09-12T01:49:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:37 smithi151 ceph-mon[35229]: pgmap v429: 97 pgs: 97 active+clean; 72 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 257 op/s 2024-09-12T01:49:38.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170677 8.09 MB/sec warmup 116 sec latency 12.170 ms 2024-09-12T01:49:39.820 INFO:tasks.workunit.client.0.smithi122.stdout: 1 173646 8.15 MB/sec warmup 117 sec latency 10.742 ms 2024-09-12T01:49:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:39 smithi183 ceph-mon[35127]: pgmap v430: 97 pgs: 97 active+clean; 75 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-09-12T01:49:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:39 smithi122 ceph-mon[30925]: pgmap v430: 97 pgs: 97 active+clean; 75 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-09-12T01:49:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:39 smithi151 ceph-mon[35229]: pgmap v430: 97 pgs: 97 active+clean; 75 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-09-12T01:49:40.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 176277 8.18 MB/sec warmup 118 sec latency 47.709 ms 2024-09-12T01:49:41.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178548 8.23 MB/sec warmup 119 sec latency 16.329 ms 2024-09-12T01:49:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:41 smithi183 ceph-mon[35127]: pgmap v431: 97 pgs: 97 active+clean; 78 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-09-12T01:49:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:41 smithi122 ceph-mon[30925]: pgmap v431: 97 pgs: 97 active+clean; 78 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-09-12T01:49:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:41 smithi151 ceph-mon[35229]: pgmap v431: 97 pgs: 97 active+clean; 78 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-09-12T01:49:43.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 184596 17.80 MB/sec execute 1 sec latency 11.607 ms 2024-09-12T01:49:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:43 smithi183 ceph-mon[35127]: pgmap v432: 97 pgs: 97 active+clean; 79 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-09-12T01:49:43.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:43 smithi122 ceph-mon[30925]: pgmap v432: 97 pgs: 97 active+clean; 79 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-09-12T01:49:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:43 smithi151 ceph-mon[35229]: pgmap v432: 97 pgs: 97 active+clean; 79 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-09-12T01:49:44.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 -- 172.21.15.122:0/207598983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89581025c0 msgr2=0x7f89581049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 --2- 172.21.15.122:0/207598983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89581025c0 0x7f89581049b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8954009900 tx=0x7f895402a410 comp rx=0 tx=0).stop 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 -- 172.21.15.122:0/207598983 shutdown_connections 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 --2- 172.21.15.122:0/207598983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8958107820 0x7f8958109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 --2- 172.21.15.122:0/207598983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8958104ef0 0x7f89581072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 --2- 172.21.15.122:0/207598983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89581025c0 0x7f89581049b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 -- 172.21.15.122:0/207598983 >> 172.21.15.122:0/207598983 conn(0x7f89580fc1e0 msgr2=0x7f89580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 -- 172.21.15.122:0/207598983 shutdown_connections 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.347+0000 7f895fcaf640 1 -- 172.21.15.122:0/207598983 wait complete. 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 Processor -- start 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 -- start start 2024-09-12T01:49:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89581025c0 0x7f8958197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 0x7f89581983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8958107820 0x7f895819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f895810d820 con 0x7f8958104ef0 2024-09-12T01:49:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f895810d6a0 con 0x7f8958107820 2024-09-12T01:49:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895fcaf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f895810d9a0 con 0x7f89581025c0 2024-09-12T01:49:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 0x7f89581983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 0x7f89581983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49560/0 (socket says 172.21.15.122:49560) 2024-09-12T01:49:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 -- 172.21.15.122:0/3610931275 learned_addr learned my addr 172.21.15.122:0/3610931275 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895e24c640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8958107820 0x7f895819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895da4b640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89581025c0 0x7f8958197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 -- 172.21.15.122:0/3610931275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89581025c0 msgr2=0x7f8958197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89581025c0 0x7f8958197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 -- 172.21.15.122:0/3610931275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8958107820 msgr2=0x7f895819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8958107820 0x7f895819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.348+0000 7f895d24a640 1 -- 172.21.15.122:0/3610931275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895819faa0 con 0x7f8958104ef0 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.349+0000 7f895d24a640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 0x7f89581983c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9f0 tx=0x7f894800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.349+0000 7f894effd640 1 -- 172.21.15.122:0/3610931275 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894800ccc0 con 0x7f8958104ef0 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.349+0000 7f894effd640 1 -- 172.21.15.122:0/3610931275 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8948005500 con 0x7f8958104ef0 2024-09-12T01:49:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.349+0000 7f894effd640 1 -- 172.21.15.122:0/3610931275 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948010690 con 0x7f8958104ef0 2024-09-12T01:49:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.349+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895819fd90 con 0x7f8958104ef0 2024-09-12T01:49:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.349+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581a0230 con 0x7f8958104ef0 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.351+0000 7f894effd640 1 -- 172.21.15.122:0/3610931275 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f89480046c0 con 0x7f8958104ef0 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.351+0000 7f8962a3d740 1 Processor -- start 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 -- start start 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d8780 0x558b4f9dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b4f9322f0 con 0x558b4f9d2db0 2024-09-12T01:49:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f894effd640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89440a2c30 0x7f89440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895da4b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895e24c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895d24a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d8780 0x558b4f9dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895da4b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:58752/0 (socket says 172.21.15.122:58752) 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895da4b640 1 -- 172.21.15.122:0/3602095553 learned_addr learned my addr 172.21.15.122:0/3602095553 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558b4f8da270 con 0x558b4f9d36f0 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b4f92e2b0 con 0x558b4f9d8780 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895e24c640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89440a2c30 0x7f89440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f894effd640 1 -- 172.21.15.122:0/3610931275 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89480cf460 con 0x7f8958104ef0 2024-09-12T01:49:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.352+0000 7f895da4b640 1 -- 172.21.15.122:0/3602095553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d8780 msgr2=0x558b4f9dac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895da4b640 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d8780 0x558b4f9dac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895da4b640 1 -- 172.21.15.122:0/3602095553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 msgr2=0x558b4f9d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895da4b640 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895da4b640 1 -- 172.21.15.122:0/3602095553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4f8d9340 con 0x558b4f9d36f0 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895d24a640 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d8780 0x558b4f9dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895e24c640 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895da4b640 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d81b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f895402a930 tx=0x7f8954032cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21391219cec2ed3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f895e24c640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89440a2c30 0x7f89440a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8950005c40 tx=0x7f8950004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f8943fff640 1 -- 172.21.15.122:0/3602095553 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895403f070 con 0x558b4f9d36f0 2024-09-12T01:49:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f8943fff640 1 -- 172.21.15.122:0/3602095553 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89540318b0 con 0x558b4f9d36f0 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.353+0000 7f8943fff640 1 -- 172.21.15.122:0/3602095553 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954035070 con 0x558b4f9d36f0 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 msgr2=0x558b4f9d81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d81b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f895402a930 tx=0x7f8954032cb0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 shutdown_connections 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d8780 0x558b4f9dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d81b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3602095553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 >> 172.21.15.122:0/3602095553 conn(0x558b4f9ca010 msgr2=0x558b4f9ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 shutdown_connections 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 -- 172.21.15.122:0/3602095553 wait complete. 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 -- start start 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.354+0000 7f8962a3d740 1 Processor -- start 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9dd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b4f9db6f0 con 0x558b4f9d2db0 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558b4f8bb290 con 0x558b4f9d8780 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b4f9db870 con 0x558b4f9d36f0 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 10 client.0 _ll_register_callbacks cb 0x558b4f8ec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9dcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49592/0 (socket says 172.21.15.122:49592) 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 -- 172.21.15.122:0/3463083954 learned_addr learned my addr 172.21.15.122:0/3463083954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895e24c640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895d24a640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 -- 172.21.15.122:0/3463083954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 msgr2=0x558b4f9dd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9dd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 -- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 msgr2=0x558b4f9e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 -- 172.21.15.122:0/3463083954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4f645210 con 0x558b4f9d2db0 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895d24a640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:49:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f895da4b640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9dcf20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f89540314e0 tx=0x7f895402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7852fa013e263de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f89417fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f89417fa640 1 -- 172.21.15.122:0/3463083954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895404a020 con 0x558b4f9d2db0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f89417fa640 1 -- 172.21.15.122:0/3463083954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8954003800 con 0x558b4f9d2db0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.355+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89540318b0 con 0x558b4f9d2db0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f89417fa640 1 -- 172.21.15.122:0/3463083954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f895403f070 con 0x558b4f9d2db0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b4f711620 con 0x558b4f9d2db0 2024-09-12T01:49:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f89417fa640 1 -- 172.21.15.122:0/3463083954 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89540318b0 con 0x558b4f9d2db0 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f89417fa640 1 -- 172.21.15.122:0/3463083954 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8954051c10 con 0x558b4f9d2db0 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 20 client.16146 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4f9e8760 0x558b4f9eab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 4 client.16146 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b4f93fa20 con 0x558b4f9e8760 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f895d24a640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4f9e8760 0x558b4f9eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f89417fa640 10 client.16146 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.356+0000 7f895d24a640 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4f9e8760 0x558b4f9eab70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba79a0b0249fc73 server_cookie=745231d12f425a9f in_seq=0 out_seq=0 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.357+0000 7f89417fa640 1 -- 172.21.15.122:0/3463083954 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b4f93fa20 con 0x558b4f9e8760 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.357+0000 7f89417fa640 10 client.16146 handle_command_reply: tid=0 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.357+0000 7f8962a3d740 1 client.16146 shutdown 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 msgr2=0x558b4f9dcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9dcf20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f89540314e0 tx=0x7f895402ab10 comp rx=0 tx=0).stop 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 shutdown_connections 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4f9e8760 0x558b4f9eab70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 --2- 172.21.15.122:0/3463083954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9dcf20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f89417fa640 0 client.16146 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 >> 172.21.15.122:0/3463083954 conn(0x558b4f9ca010 msgr2=0x558b4f9cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 shutdown_connections 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 1 -- 172.21.15.122:0/3463083954 wait complete. 2024-09-12T01:49:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.358+0000 7f8962a3d740 20 client.16146 trim_cache size 0 max 16384 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.373+0000 7f8962a3d740 1 Processor -- start 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 -- start start 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89581a9680 con 0x558b4f9d2db0 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89581a6f50 con 0x558b4f9d8780 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f8962a3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89581a4d80 con 0x558b4f9d36f0 2024-09-12T01:49:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49604/0 (socket says 172.21.15.122:49604) 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 -- 172.21.15.122:0/4226081955 learned_addr learned my addr 172.21.15.122:0/4226081955 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 -- 172.21.15.122:0/4226081955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 msgr2=0x558b4f9d2660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9d2660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 -- 172.21.15.122:0/4226081955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 msgr2=0x558b4f9dd3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9dd3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 -- 172.21.15.122:0/4226081955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4f9d4330 con 0x558b4f9d2db0 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f895da4b640 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d3190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8954006010 tx=0x7f8954031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7532b0abd110d62e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f89437fe640 1 -- 172.21.15.122:0/4226081955 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895404a070 con 0x558b4f9d2db0 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.374+0000 7f89437fe640 1 -- 172.21.15.122:0/4226081955 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895403b760 con 0x558b4f9d2db0 2024-09-12T01:49:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f89437fe640 1 -- 172.21.15.122:0/4226081955 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954035040 con 0x558b4f9d2db0 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 -- 172.21.15.122:0/4226081955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 msgr2=0x558b4f9d3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d3190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8954006010 tx=0x7f8954031400 comp rx=0 tx=0).stop 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 -- 172.21.15.122:0/4226081955 shutdown_connections 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d8780 0x558b4f9dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d36f0 0x558b4f9d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 --2- 172.21.15.122:0/4226081955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d2db0 0x558b4f9d3190 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 -- 172.21.15.122:0/4226081955 >> 172.21.15.122:0/4226081955 conn(0x558b4f9ca010 msgr2=0x558b4f9dccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 -- 172.21.15.122:0/4226081955 shutdown_connections 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.375+0000 7f8962a3d740 1 -- 172.21.15.122:0/4226081955 wait complete. 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 -- start start 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 Processor -- start 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d2db0 0x558b4f9d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 0x558b4fa55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89581a5410 con 0x558b4f9d8780 2024-09-12T01:49:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89581a6880 con 0x558b4f9d36f0 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89581a5ae0 con 0x558b4f9d2db0 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f8962a3d740 10 client.0 _ll_register_callbacks cb 0x7f8950009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 0x558b4fa55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895d24a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 0x558b4fa55c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49618/0 (socket says 172.21.15.122:49618) 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 -- 172.21.15.122:0/2387358210 learned_addr learned my addr 172.21.15.122:0/2387358210 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895da4b640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d2db0 0x558b4f9d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 -- 172.21.15.122:0/2387358210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d2db0 msgr2=0x558b4f9d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d2db0 0x558b4f9d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 -- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 msgr2=0x558b4f9d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.376+0000 7f895e24c640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f895e24c640 1 -- 172.21.15.122:0/2387358210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4f9e47a0 con 0x558b4f9d8780 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f895d24a640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f895e24c640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 0x558b4fa55c30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8950004600 tx=0x7f8950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191313ea82b51ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f89417fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f89417fa640 1 -- 172.21.15.122:0/2387358210 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500108b0 con 0x558b4f9d8780 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f89417fa640 1 -- 172.21.15.122:0/2387358210 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8950010a50 con 0x558b4f9d8780 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b4f939d70 con 0x558b4f9d8780 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f89417fa640 1 -- 172.21.15.122:0/2387358210 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8950010c50 con 0x558b4f9d8780 2024-09-12T01:49:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f8962a3d740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b4fa57240 con 0x558b4f9d8780 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f89417fa640 1 -- 172.21.15.122:0/2387358210 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558b4f939d70 con 0x558b4f9d8780 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f89417fa640 1 -- 172.21.15.122:0/2387358210 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f895001e420 con 0x558b4f9d8780 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f8962a3d740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.377+0000 7f8962a3d740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f8962a3d740 20 client.16152 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f8962a3d740 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4fa57cb0 0x558b4fa5a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f8962a3d740 4 client.16152 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b4f8b6900 con 0x558b4fa57cb0 2024-09-12T01:49:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f895da4b640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4fa57cb0 0x558b4fa5a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f89417fa640 10 client.16152 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.378+0000 7f895da4b640 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4fa57cb0 0x558b4fa5a0d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e16965f41857be server_cookie=49973ca829c2d2a5 in_seq=0 out_seq=0 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f89417fa640 1 -- 172.21.15.122:0/2387358210 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b4f8b6900 con 0x558b4fa57cb0 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f89417fa640 10 client.16152 handle_command_reply: tid=0 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 client.16152 shutdown 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 msgr2=0x558b4fa55c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 0x558b4fa55c30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8950004600 tx=0x7f8950004a40 comp rx=0 tx=0).stop 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 shutdown_connections 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558b4fa57cb0 0x558b4fa5a0d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b4f9d8780 0x558b4fa55c30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4f9d36f0 0x558b4f9d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 --2- 172.21.15.122:0/2387358210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b4f9d2db0 0x558b4f9d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f89417fa640 0 client.16152 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 >> 172.21.15.122:0/2387358210 conn(0x558b4f9ca010 msgr2=0x558b4f9c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 shutdown_connections 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 1 -- 172.21.15.122:0/2387358210 wait complete. 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.386+0000 7f8962a3d740 20 client.16152 trim_cache size 0 max 16384 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:49:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:49:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:49:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89440a2c30 msgr2=0x7f89440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89440a2c30 0x7f89440a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8950005c40 tx=0x7f8950004ab0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 msgr2=0x7f89581983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 0x7f89581983c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9f0 tx=0x7f894800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 shutdown_connections 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89440a2c30 0x7f89440a50f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8958107820 0x7f895819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8958104ef0 0x7f89581983c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 --2- 172.21.15.122:0/3610931275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89581025c0 0x7f8958197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 >> 172.21.15.122:0/3610931275 conn(0x7f89580fc1e0 msgr2=0x7f8958102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 shutdown_connections 2024-09-12T01:49:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:44.387+0000 7f895fcaf640 1 -- 172.21.15.122:0/3610931275 wait complete. 2024-09-12T01:49:44.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 187334 15.19 MB/sec execute 2 sec latency 10.769 ms 2024-09-12T01:49:45.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 189694 14.51 MB/sec execute 3 sec latency 11.395 ms 2024-09-12T01:49:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:45 smithi183 ceph-mon[35127]: pgmap v433: 97 pgs: 97 active+clean; 80 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-09-12T01:49:45.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:45 smithi122 ceph-mon[30925]: pgmap v433: 97 pgs: 97 active+clean; 80 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-09-12T01:49:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:45 smithi151 ceph-mon[35229]: pgmap v433: 97 pgs: 97 active+clean; 80 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-09-12T01:49:46.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 192166 14.35 MB/sec execute 4 sec latency 18.324 ms 2024-09-12T01:49:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:49:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:49:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:49:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:49:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:49:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:49:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:49:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:49:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:49:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:49:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:49:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:49:47.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 195338 15.24 MB/sec execute 5 sec latency 11.944 ms 2024-09-12T01:49:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:47 smithi183 ceph-mon[35127]: pgmap v434: 97 pgs: 97 active+clean; 85 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-09-12T01:49:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:47 smithi122 ceph-mon[30925]: pgmap v434: 97 pgs: 97 active+clean; 85 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-09-12T01:49:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:47 smithi151 ceph-mon[35229]: pgmap v434: 97 pgs: 97 active+clean; 85 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-09-12T01:49:48.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 198268 14.94 MB/sec execute 6 sec latency 15.619 ms 2024-09-12T01:49:49.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 201130 14.77 MB/sec execute 7 sec latency 10.863 ms 2024-09-12T01:49:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:49 smithi183 ceph-mon[35127]: pgmap v435: 97 pgs: 97 active+clean; 86 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2024-09-12T01:49:49.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:49 smithi122 ceph-mon[30925]: pgmap v435: 97 pgs: 97 active+clean; 86 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2024-09-12T01:49:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:49 smithi151 ceph-mon[35229]: pgmap v435: 97 pgs: 97 active+clean; 86 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2024-09-12T01:49:50.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 203611 14.97 MB/sec execute 8 sec latency 13.930 ms 2024-09-12T01:49:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:50 smithi183 ceph-mon[35127]: pgmap v436: 97 pgs: 97 active+clean; 90 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 259 op/s 2024-09-12T01:49:50.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:50 smithi122 ceph-mon[30925]: pgmap v436: 97 pgs: 97 active+clean; 90 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 259 op/s 2024-09-12T01:49:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:50 smithi151 ceph-mon[35229]: pgmap v436: 97 pgs: 97 active+clean; 90 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 259 op/s 2024-09-12T01:49:51.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 206418 14.90 MB/sec execute 9 sec latency 12.437 ms 2024-09-12T01:49:52.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 209529 15.27 MB/sec execute 10 sec latency 10.198 ms 2024-09-12T01:49:53.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 212451 15.10 MB/sec execute 11 sec latency 18.239 ms 2024-09-12T01:49:53.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:53 smithi151 ceph-mon[35229]: pgmap v437: 97 pgs: 97 active+clean; 91 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-09-12T01:49:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:53 smithi183 ceph-mon[35127]: pgmap v437: 97 pgs: 97 active+clean; 91 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-09-12T01:49:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:53 smithi122 ceph-mon[30925]: pgmap v437: 97 pgs: 97 active+clean; 91 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-09-12T01:49:54.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 215329 14.94 MB/sec execute 12 sec latency 15.594 ms 2024-09-12T01:49:55.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 217874 15.10 MB/sec execute 13 sec latency 10.458 ms 2024-09-12T01:49:55.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:55 smithi151 ceph-mon[35229]: pgmap v438: 97 pgs: 97 active+clean; 95 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 261 op/s 2024-09-12T01:49:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:55 smithi183 ceph-mon[35127]: pgmap v438: 97 pgs: 97 active+clean; 95 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 261 op/s 2024-09-12T01:49:55.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:55 smithi122 ceph-mon[30925]: pgmap v438: 97 pgs: 97 active+clean; 95 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 261 op/s 2024-09-12T01:49:56.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 220585 15.03 MB/sec execute 14 sec latency 54.778 ms 2024-09-12T01:49:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 -- 172.21.15.122:0/2154187434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8100a40 msgr2=0x7f42a8100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 --2- 172.21.15.122:0/2154187434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8100a40 0x7f42a8100e40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009900 tx=0x7f42a402a410 comp rx=0 tx=0).stop 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 -- 172.21.15.122:0/2154187434 shutdown_connections 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 --2- 172.21.15.122:0/2154187434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a81025f0 0x7f42a8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 --2- 172.21.15.122:0/2154187434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8101c30 0x7f42a81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 --2- 172.21.15.122:0/2154187434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8100a40 0x7f42a8100e40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 -- 172.21.15.122:0/2154187434 >> 172.21.15.122:0/2154187434 conn(0x7f42a80fc1e0 msgr2=0x7f42a80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 -- 172.21.15.122:0/2154187434 shutdown_connections 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.477+0000 7f42b014b640 1 -- 172.21.15.122:0/2154187434 wait complete. 2024-09-12T01:49:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 Processor -- start 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 -- start start 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a8100a40 0x7f42a806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 0x7f42a819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a81025f0 0x7f42a819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c6a0 con 0x7f42a8101c30 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c520 con 0x7f42a8100a40 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42b014b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c820 con 0x7f42a81025f0 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 0x7f42a819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 0x7f42a819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60868/0 (socket says 172.21.15.122:60868) 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 -- 172.21.15.122:0/3489341462 learned_addr learned my addr 172.21.15.122:0/3489341462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42adee7640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a8100a40 0x7f42a806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ae6e8640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a81025f0 0x7f42a819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 -- 172.21.15.122:0/3489341462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a81025f0 msgr2=0x7f42a819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a81025f0 0x7f42a819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 -- 172.21.15.122:0/3489341462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a8100a40 msgr2=0x7f42a806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a8100a40 0x7f42a806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 -- 172.21.15.122:0/3489341462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a819fa40 con 0x7f42a8101c30 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.478+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 0x7f42a819cb10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f429800e9f0 tx=0x7f429800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.479+0000 7f429effd640 1 -- 172.21.15.122:0/3489341462 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429800cd80 con 0x7f42a8101c30 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.479+0000 7f429effd640 1 -- 172.21.15.122:0/3489341462 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4298010910 con 0x7f42a8101c30 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.479+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a819fcd0 con 0x7f42a8101c30 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.479+0000 7f429effd640 1 -- 172.21.15.122:0/3489341462 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298018990 con 0x7f42a8101c30 2024-09-12T01:49:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.479+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a81a01d0 con 0x7f42a8101c30 2024-09-12T01:49:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.481+0000 7f429effd640 1 -- 172.21.15.122:0/3489341462 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4298010ac0 con 0x7f42a8101c30 2024-09-12T01:49:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.481+0000 7f42b2fc2740 1 Processor -- start 2024-09-12T01:49:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 -- start start 2024-09-12T01:49:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17e9a0 0x557f4b17eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17f2e0 0x557f4b183da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b186830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f4b0e22f0 con 0x557f4b184370 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557f4b08a270 con 0x557f4b17f2e0 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f4b0de2b0 con 0x557f4b17e9a0 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f429effd640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f42940a2c30 0x7f42940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42ad6e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b186830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42ae6e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17e9a0 0x557f4b17eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42ae6e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17e9a0 0x557f4b17eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:54772/0 (socket says 172.21.15.122:54772) 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42adee7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17f2e0 0x557f4b183da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42ae6e8640 1 -- 172.21.15.122:0/3229958980 learned_addr learned my addr 172.21.15.122:0/3229958980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42ae6e8640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f42940a2c30 0x7f42940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f429effd640 1 -- 172.21.15.122:0/3489341462 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f42980cf330 con 0x7f42a8101c30 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42adee7640 1 -- 172.21.15.122:0/3229958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17e9a0 msgr2=0x557f4b17eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42adee7640 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17e9a0 0x557f4b17eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42adee7640 1 -- 172.21.15.122:0/3229958980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 msgr2=0x557f4b186830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42adee7640 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b186830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42adee7640 1 -- 172.21.15.122:0/3229958980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f4b089340 con 0x557f4b17f2e0 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.482+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b186830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f42adee7640 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17f2e0 0x557f4b183da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009230 tx=0x7f42a4005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6baf3d924d2b38e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f42ae6e8640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f42940a2c30 0x7f42940a50f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005c20 tx=0x7f42a0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f4293fff640 1 -- 172.21.15.122:0/3229958980 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4040070 con 0x557f4b17f2e0 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f4293fff640 1 -- 172.21.15.122:0/3229958980 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42a40318b0 con 0x557f4b17f2e0 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f4293fff640 1 -- 172.21.15.122:0/3229958980 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4035040 con 0x557f4b17f2e0 2024-09-12T01:49:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f42b2fc2740 1 -- 172.21.15.122:0/3229958980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17f2e0 msgr2=0x557f4b183da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.483+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17f2e0 0x557f4b183da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009230 tx=0x7f42a4005bf0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- 172.21.15.122:0/3229958980 shutdown_connections 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b186830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17f2e0 0x557f4b183da0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/3229958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17e9a0 0x557f4b17eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- 172.21.15.122:0/3229958980 >> 172.21.15.122:0/3229958980 conn(0x557f4b17a010 msgr2=0x557f4b17a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- 172.21.15.122:0/3229958980 shutdown_connections 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- 172.21.15.122:0/3229958980 wait complete. 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- start start 2024-09-12T01:49:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 Processor -- start 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b188b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 0x557f4b189080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b184370 0x557f4b1900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f4b1872e0 con 0x557f4b17f2e0 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557f4b06b290 con 0x557f4b17e9a0 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f4b187460 con 0x557f4b184370 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.484+0000 7f42b2fc2740 10 client.0 _ll_register_callbacks cb 0x557f4b09c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 0x557f4b189080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42adee7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b188b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ae6e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b184370 0x557f4b1900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 0x557f4b189080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60890/0 (socket says 172.21.15.122:60890) 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 -- 172.21.15.122:0/2734435513 learned_addr learned my addr 172.21.15.122:0/2734435513 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 -- 172.21.15.122:0/2734435513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b184370 msgr2=0x557f4b1900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b184370 0x557f4b1900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 -- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 msgr2=0x557f4b188b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b188b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 -- 172.21.15.122:0/2734435513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f4adf5210 con 0x557f4b17f2e0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ae6e8640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b184370 0x557f4b1900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42adee7640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b188b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 0x557f4b189080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f429800c6a0 tx=0x7f429809ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0412245e760ce48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42917fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42917fa640 1 -- 172.21.15.122:0/2734435513 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429809bc40 con 0x557f4b17f2e0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42917fa640 1 -- 172.21.15.122:0/2734435513 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4298099f40 con 0x557f4b17f2e0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42a40318b0 con 0x557f4b17f2e0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42917fa640 1 -- 172.21.15.122:0/2734435513 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f429809ff40 con 0x557f4b17f2e0 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42b2fc2740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-09-12T01:49:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.485+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f4aec1620 con 0x557f4b17f2e0 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42917fa640 1 -- 172.21.15.122:0/2734435513 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42a40318b0 con 0x557f4b17f2e0 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42917fa640 1 -- 172.21.15.122:0/2734435513 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f42980ce030 con 0x557f4b17f2e0 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42b2fc2740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42b2fc2740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42b2fc2740 20 client.16164 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b194860 0x557f4b196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42b2fc2740 4 client.16164 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:49:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f4b0efa20 con 0x557f4b194860 2024-09-12T01:49:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42adee7640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b194860 0x557f4b196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42adee7640 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b194860 0x557f4b196cb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e267aabafe58c56d server_cookie=d315ab2c1ade67ba in_seq=0 out_seq=0 2024-09-12T01:49:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.486+0000 7f42917fa640 10 client.16164 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42917fa640 1 -- 172.21.15.122:0/2734435513 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f4b0efa20 con 0x557f4b194860 2024-09-12T01:49:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42917fa640 10 client.16164 handle_command_reply: tid=0 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 client.16164 shutdown 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 msgr2=0x557f4b189080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 0x557f4b189080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f429800c6a0 tx=0x7f429809ba10 comp rx=0 tx=0).stop 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 shutdown_connections 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b194860 0x557f4b196cb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b184370 0x557f4b1900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17f2e0 0x557f4b189080 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42917fa640 0 client.16164 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2734435513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b188b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 >> 172.21.15.122:0/2734435513 conn(0x557f4b17a010 msgr2=0x557f4b17b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 shutdown_connections 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2734435513 wait complete. 2024-09-12T01:49:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.487+0000 7f42b2fc2740 20 client.16164 trim_cache size 0 max 16384 2024-09-12T01:49:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.502+0000 7f42b2fc2740 1 Processor -- start 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 -- start start 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b1790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b190080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42a81ab830 con 0x557f4b184370 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42a81a9100 con 0x557f4b17e9a0 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42a81a6f30 con 0x557f4b17f2e0 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42ae6e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b1790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b1790b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:58360/0 (socket says 172.21.15.122:58360) 2024-09-12T01:49:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42ae6e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b19b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60906/0 (socket says 172.21.15.122:60906) 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 -- 172.21.15.122:0/2670248730 learned_addr learned my addr 172.21.15.122:0/2670248730 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b190080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 -- 172.21.15.122:0/2670248730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 msgr2=0x557f4b190080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b190080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 -- 172.21.15.122:0/2670248730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 msgr2=0x557f4b19b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b19b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 -- 172.21.15.122:0/2670248730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f4b17f7c0 con 0x557f4b17e9a0 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42adee7640 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b1790b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42a402a930 tx=0x7f42a4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87b208fb54391fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42937fe640 1 -- 172.21.15.122:0/2670248730 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4035040 con 0x557f4b17e9a0 2024-09-12T01:49:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42937fe640 1 -- 172.21.15.122:0/2670248730 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42a4004030 con 0x557f4b17e9a0 2024-09-12T01:49:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.503+0000 7f42937fe640 1 -- 172.21.15.122:0/2670248730 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4040040 con 0x557f4b17e9a0 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2670248730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 msgr2=0x557f4b1790b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b1790b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42a402a930 tx=0x7f42a4005b50 comp rx=0 tx=0).stop 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2670248730 shutdown_connections 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b184370 0x557f4b19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b190080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/2670248730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b17e9a0 0x557f4b1790b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2670248730 >> 172.21.15.122:0/2670248730 conn(0x557f4b17a010 msgr2=0x557f4b1789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2670248730 shutdown_connections 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 -- 172.21.15.122:0/2670248730 wait complete. 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.504+0000 7f42b2fc2740 1 -- start start 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 Processor -- start 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 0x557f4b18d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b18d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b184370 0x557f4b18e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42a81a75c0 con 0x557f4b17e9a0 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42a81a8a30 con 0x557f4b184370 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42a81a7c90 con 0x557f4b17f2e0 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42b2fc2740 10 client.0 _ll_register_callbacks cb 0x7f429800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 0x557f4b18d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 0x557f4b18d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60912/0 (socket says 172.21.15.122:60912) 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 -- 172.21.15.122:0/1992189902 learned_addr learned my addr 172.21.15.122:0/1992189902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b18d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42ae6e8640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b184370 0x557f4b18e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 -- 172.21.15.122:0/1992189902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 msgr2=0x557f4b18d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b18d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 -- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b184370 msgr2=0x557f4b18e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b184370 0x557f4b18e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 -- 172.21.15.122:0/1992189902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f4b18ed70 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42adee7640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 0x557f4b18d210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f42a402a450 tx=0x7f42a4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444803189667bb38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.505+0000 7f42917fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42917fa640 1 -- 172.21.15.122:0/1992189902 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a405e060 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42917fa640 1 -- 172.21.15.122:0/1992189902 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f42a4005230 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42a4004030 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42917fa640 1 -- 172.21.15.122:0/1992189902 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42a4040040 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f4b190080 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42917fa640 1 -- 172.21.15.122:0/1992189902 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42a4004030 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42917fa640 1 -- 172.21.15.122:0/1992189902 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f42a4063070 con 0x557f4b17e9a0 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:49:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 20 client.16170 populate_metadata read hostname 'smithi122' 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b1ff630 0x557f4b201a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 4 client.16170 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f4b0e9d70 con 0x557f4b1ff630 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.506+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b1ff630 0x557f4b201a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.507+0000 7f42917fa640 10 client.16170 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.507+0000 7f42ad6e6640 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b1ff630 0x557f4b201a50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca82cad577a11b3 server_cookie=dbe92eede46eccdc in_seq=0 out_seq=0 2024-09-12T01:49:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42917fa640 1 -- 172.21.15.122:0/1992189902 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f4b0e9d70 con 0x557f4b1ff630 2024-09-12T01:49:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42917fa640 10 client.16170 handle_command_reply: tid=0 2024-09-12T01:49:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 client.16170 shutdown 2024-09-12T01:49:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 msgr2=0x557f4b18d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 0x557f4b18d210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f42a402a450 tx=0x7f42a4031a90 comp rx=0 tx=0).stop 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 shutdown_connections 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557f4b1ff630 0x557f4b201a50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f4b184370 0x557f4b18e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42917fa640 0 client.16170 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f4b17f2e0 0x557f4b18d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 --2- 172.21.15.122:0/1992189902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f4b17e9a0 0x557f4b18d210 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 >> 172.21.15.122:0/1992189902 conn(0x557f4b17a010 msgr2=0x557f4b1789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 shutdown_connections 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 1 -- 172.21.15.122:0/1992189902 wait complete. 2024-09-12T01:49:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.515+0000 7f42b2fc2740 20 client.16170 trim_cache size 0 max 16384 2024-09-12T01:49:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:49:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:49:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:49:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:49:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f42940a2c30 msgr2=0x7f42940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f42940a2c30 0x7f42940a50f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005c20 tx=0x7f42a0009600 comp rx=0 tx=0).stop 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 msgr2=0x7f42a819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 0x7f42a819cb10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f429800e9f0 tx=0x7f429800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 shutdown_connections 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f42940a2c30 0x7f42940a50f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a81025f0 0x7f42a819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42a8101c30 0x7f42a819cb10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 --2- 172.21.15.122:0/3489341462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a8100a40 0x7f42a806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 >> 172.21.15.122:0/3489341462 conn(0x7f42a80fc1e0 msgr2=0x7f42a8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 shutdown_connections 2024-09-12T01:49:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:49:57.516+0000 7f42b014b640 1 -- 172.21.15.122:0/3489341462 wait complete. 2024-09-12T01:49:57.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 223782 15.28 MB/sec execute 15 sec latency 20.498 ms 2024-09-12T01:49:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:57 smithi151 ceph-mon[35229]: pgmap v439: 97 pgs: 97 active+clean; 96 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-12T01:49:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:57 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:49:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:57 smithi183 ceph-mon[35127]: pgmap v439: 97 pgs: 97 active+clean; 96 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-12T01:49:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:57 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:49:57.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:57 smithi122 ceph-mon[30925]: pgmap v439: 97 pgs: 97 active+clean; 96 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-12T01:49:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:57 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:49:58.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 226862 15.25 MB/sec execute 16 sec latency 11.912 ms 2024-09-12T01:49:59.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 229808 15.27 MB/sec execute 17 sec latency 10.215 ms 2024-09-12T01:49:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:49:59 smithi151 ceph-mon[35229]: pgmap v440: 97 pgs: 97 active+clean; 101 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-12T01:49:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:49:59 smithi183 ceph-mon[35127]: pgmap v440: 97 pgs: 97 active+clean; 101 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-12T01:49:59.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:49:59 smithi122 ceph-mon[30925]: pgmap v440: 97 pgs: 97 active+clean; 101 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-12T01:50:00.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T01:49:59.999+0000 7f42afbd7640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T01:50:00.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T01:49:59.999+0000 7f42afbd7640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T01:50:00.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T01:49:59.999+0000 7f42afbd7640 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2024-09-12T01:50:00.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 232412 15.17 MB/sec execute 18 sec latency 10.978 ms 2024-09-12T01:50:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:00 smithi151 ceph-mon[35229]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T01:50:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:00 smithi151 ceph-mon[35229]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T01:50:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:00 smithi151 ceph-mon[35229]: mds.h(mds.2): Metadata damage detected 2024-09-12T01:50:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:00 smithi183 ceph-mon[35127]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T01:50:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:00 smithi183 ceph-mon[35127]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T01:50:00.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:00 smithi183 ceph-mon[35127]: mds.h(mds.2): Metadata damage detected 2024-09-12T01:50:00.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:00 smithi122 ceph-mon[30925]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T01:50:00.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:00 smithi122 ceph-mon[30925]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T01:50:00.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:00 smithi122 ceph-mon[30925]: mds.h(mds.2): Metadata damage detected 2024-09-12T01:50:01.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 235137 15.15 MB/sec execute 19 sec latency 10.849 ms 2024-09-12T01:50:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:01 smithi151 ceph-mon[35229]: pgmap v441: 97 pgs: 97 active+clean; 102 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 268 op/s 2024-09-12T01:50:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:01 smithi183 ceph-mon[35127]: pgmap v441: 97 pgs: 97 active+clean; 102 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 268 op/s 2024-09-12T01:50:01.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:01 smithi122 ceph-mon[30925]: pgmap v441: 97 pgs: 97 active+clean; 102 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 268 op/s 2024-09-12T01:50:02.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 237973 15.28 MB/sec execute 20 sec latency 74.469 ms 2024-09-12T01:50:03.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 241043 15.26 MB/sec execute 21 sec latency 11.547 ms 2024-09-12T01:50:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:03 smithi151 ceph-mon[35229]: pgmap v442: 97 pgs: 97 active+clean; 103 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-12T01:50:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:03 smithi183 ceph-mon[35127]: pgmap v442: 97 pgs: 97 active+clean; 103 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-12T01:50:03.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:03 smithi122 ceph-mon[30925]: pgmap v442: 97 pgs: 97 active+clean; 103 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-12T01:50:04.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 243810 15.14 MB/sec execute 22 sec latency 11.121 ms 2024-09-12T01:50:05.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 246038 15.06 MB/sec execute 23 sec latency 15.969 ms 2024-09-12T01:50:05.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:05 smithi151 ceph-mon[35229]: pgmap v443: 97 pgs: 97 active+clean; 110 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-09-12T01:50:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:05 smithi183 ceph-mon[35127]: pgmap v443: 97 pgs: 97 active+clean; 110 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-09-12T01:50:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:05 smithi122 ceph-mon[30925]: pgmap v443: 97 pgs: 97 active+clean; 110 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-09-12T01:50:06.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 248926 15.14 MB/sec execute 24 sec latency 8.979 ms 2024-09-12T01:50:07.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 251885 15.23 MB/sec execute 25 sec latency 60.822 ms 2024-09-12T01:50:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:07 smithi151 ceph-mon[35229]: pgmap v444: 97 pgs: 97 active+clean; 110 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-09-12T01:50:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:07 smithi183 ceph-mon[35127]: pgmap v444: 97 pgs: 97 active+clean; 110 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-09-12T01:50:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:07 smithi122 ceph-mon[30925]: pgmap v444: 97 pgs: 97 active+clean; 110 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-09-12T01:50:08.821 INFO:tasks.workunit.client.0.smithi122.stdout: 1 254777 15.17 MB/sec execute 26 sec latency 16.727 ms 2024-09-12T01:50:09.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 257565 15.09 MB/sec execute 27 sec latency 17.682 ms 2024-09-12T01:50:09.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:09 smithi151 ceph-mon[35229]: pgmap v445: 97 pgs: 97 active+clean; 114 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-09-12T01:50:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:09 smithi183 ceph-mon[35127]: pgmap v445: 97 pgs: 97 active+clean; 114 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-09-12T01:50:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:09 smithi122 ceph-mon[30925]: pgmap v445: 97 pgs: 97 active+clean; 114 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 -- 172.21.15.122:0/1750447915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20040ffe30 msgr2=0x7f2004100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 --2- 172.21.15.122:0/1750447915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20040ffe30 0x7f2004100230 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f200002a410 comp rx=0 tx=0).stop 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 -- 172.21.15.122:0/1750447915 shutdown_connections 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 --2- 172.21.15.122:0/1750447915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103860 0x7f200410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 --2- 172.21.15.122:0/1750447915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2004102ea0 0x7f2004103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 --2- 172.21.15.122:0/1750447915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20040ffe30 0x7f2004100230 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.609+0000 7f200c1f8640 1 -- 172.21.15.122:0/1750447915 >> 172.21.15.122:0/1750447915 conn(0x7f20040fb930 msgr2=0x7f20040fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 -- 172.21.15.122:0/1750447915 shutdown_connections 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 -- 172.21.15.122:0/1750447915 wait complete. 2024-09-12T01:50:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 Processor -- start 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 -- start start 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20040ffe30 0x7f2004197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 0x7f2004198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2004103860 0x7f200419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f200410d7e0 con 0x7f2004102ea0 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f200410d660 con 0x7f2004103860 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200c1f8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f200410d960 con 0x7f20040ffe30 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f2009793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 0x7f2004198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f2009793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 0x7f2004198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39088/0 (socket says 172.21.15.122:39088) 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f2009793640 1 -- 172.21.15.122:0/474002529 learned_addr learned my addr 172.21.15.122:0/474002529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f200a795640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2004103860 0x7f200419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.610+0000 7f2009f94640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20040ffe30 0x7f2004197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f2009793640 1 -- 172.21.15.122:0/474002529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20040ffe30 msgr2=0x7f2004197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f2009793640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20040ffe30 0x7f2004197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f2009793640 1 -- 172.21.15.122:0/474002529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2004103860 msgr2=0x7f200419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f2009793640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2004103860 0x7f200419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f2009793640 1 -- 172.21.15.122:0/474002529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200419f9f0 con 0x7f2004102ea0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f2009793640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 0x7f2004198310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400e9f0 tx=0x7f1ff400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f1ffaffd640 1 -- 172.21.15.122:0/474002529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400cd80 con 0x7f2004102ea0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f1ffaffd640 1 -- 172.21.15.122:0/474002529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff4010910 con 0x7f2004102ea0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200419fce0 con 0x7f2004102ea0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f1ffaffd640 1 -- 172.21.15.122:0/474002529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4018990 con 0x7f2004102ea0 2024-09-12T01:50:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.611+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a0220 con 0x7f2004102ea0 2024-09-12T01:50:10.621 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.613+0000 7f1ffaffd640 1 -- 172.21.15.122:0/474002529 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1ff4010ac0 con 0x7f2004102ea0 2024-09-12T01:50:10.621 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.613+0000 7f200f066740 1 Processor -- start 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 -- start start 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 0x563f482a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f482082f0 con 0x563f482a49a0 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563f481b0270 con 0x563f482a52e0 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200f066740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563f482042b0 con 0x563f482aa370 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200a795640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009793640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 0x563f482a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 0x563f482a9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:43592/0 (socket says 172.21.15.122:43592) 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 -- 172.21.15.122:0/1253049178 learned_addr learned my addr 172.21.15.122:0/1253049178 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 -- 172.21.15.122:0/1253049178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 msgr2=0x563f482ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f1ffaffd640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ff00a2cb0 0x7f1ff00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 -- 172.21.15.122:0/1253049178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 msgr2=0x563f482a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f2009f94640 1 -- 172.21.15.122:0/1253049178 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f481af340 con 0x563f482a52e0 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f1ffaffd640 1 -- 172.21.15.122:0/474002529 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1ff40cf2a0 con 0x7f2004102ea0 2024-09-12T01:50:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.614+0000 7f200a795640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ff00a2cb0 0x7f1ff00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.615+0000 7f2009f94640 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 0x563f482a9da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2000031610 tx=0x7f2000005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96ffe78299aa7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.615+0000 7f1feffff640 1 -- 172.21.15.122:0/1253049178 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000040070 con 0x563f482a52e0 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.615+0000 7f1feffff640 1 -- 172.21.15.122:0/1253049178 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000004030 con 0x563f482a52e0 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.615+0000 7f1feffff640 1 -- 172.21.15.122:0/1253049178 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000035040 con 0x563f482a52e0 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 -- 172.21.15.122:0/1253049178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 msgr2=0x563f482a9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 0x563f482a9da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2000031610 tx=0x7f2000005de0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200a795640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ff00a2cb0 0x7f1ff00a5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009fd0 tx=0x7f1ffc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 -- 172.21.15.122:0/1253049178 shutdown_connections 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a52e0 0x563f482a9da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- 172.21.15.122:0/1253049178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 -- 172.21.15.122:0/1253049178 >> 172.21.15.122:0/1253049178 conn(0x563f482a0010 msgr2=0x563f482a0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 -- 172.21.15.122:0/1253049178 shutdown_connections 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 -- 172.21.15.122:0/1253049178 wait complete. 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 -- start start 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 Processor -- start 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a49a0 0x563f482aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.616+0000 7f200f066740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f482ad2e0 con 0x563f482a52e0 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563f48191290 con 0x563f482aa370 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563f482ad460 con 0x563f482a49a0 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 10 client.0 _ll_register_callbacks cb 0x563f481c2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009f94640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a49a0 0x563f482aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009f94640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a49a0 0x563f482aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:45510/0 (socket says 172.21.15.122:45510) 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39120/0 (socket says 172.21.15.122:39120) 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200a795640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009f94640 1 -- 172.21.15.122:0/3440474062 learned_addr learned my addr 172.21.15.122:0/3440474062 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 -- 172.21.15.122:0/3440474062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a49a0 msgr2=0x563f482aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a49a0 0x563f482aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 -- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 msgr2=0x563f482b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 -- 172.21.15.122:0/3440474062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f47f1b210 con 0x563f482a52e0 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f2009793640 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482af160 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40ccda0 tx=0x7f1ff40d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3981124eabae761f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f1fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f1fed7fa640 1 -- 172.21.15.122:0/3440474062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff409b510 con 0x563f482a52e0 2024-09-12T01:50:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f1fed7fa640 1 -- 172.21.15.122:0/3440474062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1ff4018b30 con 0x563f482a52e0 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2000004030 con 0x563f482a52e0 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f1fed7fa640 1 -- 172.21.15.122:0/3440474062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ff40b4050 con 0x563f482a52e0 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 10 client.16182 fetch_fsmap learned FSMap version 29 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.617+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f47fe7620 con 0x563f482a52e0 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f1fed7fa640 1 -- 172.21.15.122:0/3440474062 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000004030 con 0x563f482a52e0 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f1fed7fa640 1 -- 172.21.15.122:0/3440474062 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f1ff409b000 con 0x563f482a52e0 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f200f066740 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f200f066740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f200f066740 20 client.16182 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f200f066740 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f482ba900 0x563f482bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f200f066740 4 client.16182 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f48215a20 con 0x563f482ba900 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f2009f94640 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f482ba900 0x563f482bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f1fed7fa640 10 client.16182 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.618+0000 7f2009f94640 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f482ba900 0x563f482bcd50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0bc584bf80d16 server_cookie=ea138382a5f38d76 in_seq=0 out_seq=0 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f1fed7fa640 1 -- 172.21.15.122:0/3440474062 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f48215a20 con 0x563f482ba900 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f1fed7fa640 10 client.16182 handle_command_reply: tid=0 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 client.16182 shutdown 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 msgr2=0x563f482af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482af160 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40ccda0 tx=0x7f1ff40d2f90 comp rx=0 tx=0).stop 2024-09-12T01:50:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 shutdown_connections 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f482ba900 0x563f482bcd50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482af160 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f1fed7fa640 0 client.16182 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 --2- 172.21.15.122:0/3440474062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a49a0 0x563f482aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 >> 172.21.15.122:0/3440474062 conn(0x563f482a0010 msgr2=0x563f482a1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 shutdown_connections 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 1 -- 172.21.15.122:0/3440474062 wait complete. 2024-09-12T01:50:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.619+0000 7f200f066740 20 client.16182 trim_cache size 0 max 16384 2024-09-12T01:50:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 Processor -- start 2024-09-12T01:50:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 -- start start 2024-09-12T01:50:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a49a0 0x563f4829f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20041adba0 con 0x563f482a52e0 2024-09-12T01:50:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20041ab470 con 0x563f482a49a0 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f200f066740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20041a92a0 con 0x563f482aa370 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39134/0 (socket says 172.21.15.122:39134) 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 -- 172.21.15.122:0/1690251941 learned_addr learned my addr 172.21.15.122:0/1690251941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009f94640 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a49a0 0x563f4829f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 -- 172.21.15.122:0/1690251941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 msgr2=0x563f482c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 -- 172.21.15.122:0/1690251941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a49a0 msgr2=0x563f4829f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a49a0 0x563f4829f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.634+0000 7f2009793640 1 -- 172.21.15.122:0/1690251941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f482a57c0 con 0x563f482a52e0 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f2009793640 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482b6160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1ff409bb40 tx=0x7f1ff409a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804e9911cf0e82b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f1fef7fe640 1 -- 172.21.15.122:0/1690251941 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40b8020 con 0x563f482a52e0 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f1fef7fe640 1 -- 172.21.15.122:0/1690251941 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff4097c70 con 0x563f482a52e0 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f1fef7fe640 1 -- 172.21.15.122:0/1690251941 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4018b30 con 0x563f482a52e0 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 -- 172.21.15.122:0/1690251941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 msgr2=0x563f482b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482b6160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1ff409bb40 tx=0x7f1ff409a0c0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 -- 172.21.15.122:0/1690251941 shutdown_connections 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482aa370 0x563f482c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a52e0 0x563f482b6160 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 --2- 172.21.15.122:0/1690251941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482a49a0 0x563f4829f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 -- 172.21.15.122:0/1690251941 >> 172.21.15.122:0/1690251941 conn(0x563f482a0010 msgr2=0x563f4829e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.635+0000 7f200f066740 1 -- 172.21.15.122:0/1690251941 shutdown_connections 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 -- 172.21.15.122:0/1690251941 wait complete. 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 -- start start 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 Processor -- start 2024-09-12T01:50:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a52e0 0x563f482b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20041a9930 con 0x563f482a49a0 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20041aada0 con 0x563f482aa370 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20041aa000 con 0x563f482a52e0 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200f066740 10 client.0 _ll_register_callbacks cb 0x7f1ff40d2a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f2009f94640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f2009f94640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39136/0 (socket says 172.21.15.122:39136) 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f2009f94640 1 -- 172.21.15.122:0/1738290243 learned_addr learned my addr 172.21.15.122:0/1738290243 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f200a795640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.636+0000 7f2009793640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a52e0 0x563f482b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f2009f94640 1 -- 172.21.15.122:0/1738290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a52e0 msgr2=0x563f482b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f2009f94640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a52e0 0x563f482b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f2009f94640 1 -- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 msgr2=0x563f482b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f2009f94640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f2009f94640 1 -- 172.21.15.122:0/1738290243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f482b58c0 con 0x563f482a49a0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f2009f94640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482b3dd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f2000005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a587a54e42cf6987 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f1fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f1fed7fa640 1 -- 172.21.15.122:0/1738290243 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000040040 con 0x563f482a49a0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f1fed7fa640 1 -- 172.21.15.122:0/1738290243 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2000004040 con 0x563f482a49a0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff4097c70 con 0x563f482a49a0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f1fed7fa640 1 -- 172.21.15.122:0/1738290243 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2000035040 con 0x563f482a49a0 2024-09-12T01:50:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f48325c40 con 0x563f482a49a0 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f1fed7fa640 1 -- 172.21.15.122:0/1738290243 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff4097c70 con 0x563f482a49a0 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f1fed7fa640 1 -- 172.21.15.122:0/1738290243 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2000004780 con 0x563f482a49a0 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 20 client.16188 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f48326140 0x563f48328560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 4 client.16188 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.637+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f4820fd70 con 0x563f48326140 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.638+0000 7f2009793640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f48326140 0x563f48328560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.638+0000 7f2009793640 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f48326140 0x563f48328560 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a641054dd484e8a0 server_cookie=be0064e69fc22fdc in_seq=0 out_seq=0 2024-09-12T01:50:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.638+0000 7f1fed7fa640 10 client.16188 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f1fed7fa640 1 -- 172.21.15.122:0/1738290243 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f4820fd70 con 0x563f48326140 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f1fed7fa640 10 client.16188 handle_command_reply: tid=0 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 client.16188 shutdown 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 msgr2=0x563f482b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482b3dd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f2000005850 comp rx=0 tx=0).stop 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 shutdown_connections 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563f48326140 0x563f48328560 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563f482aa370 0x563f482b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f1fed7fa640 0 client.16188 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f482a52e0 0x563f482b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 --2- 172.21.15.122:0/1738290243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f482a49a0 0x563f482b3dd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 >> 172.21.15.122:0/1738290243 conn(0x563f482a0010 msgr2=0x563f4829f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 shutdown_connections 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 1 -- 172.21.15.122:0/1738290243 wait complete. 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200f066740 20 client.16188 trim_cache size 0 max 16384 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:50:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:50:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:50:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:50:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ff00a2cb0 msgr2=0x7f1ff00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.647+0000 7f200c1f8640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ff00a2cb0 0x7f1ff00a5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009fd0 tx=0x7f1ffc004a40 comp rx=0 tx=0).stop 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 msgr2=0x7f2004198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 0x7f2004198310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400e9f0 tx=0x7f1ff400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 shutdown_connections 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1ff00a2cb0 0x7f1ff00a5170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2004103860 0x7f200419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2004102ea0 0x7f2004198310 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 --2- 172.21.15.122:0/474002529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20040ffe30 0x7f2004197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 >> 172.21.15.122:0/474002529 conn(0x7f20040fb930 msgr2=0x7f2004106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 shutdown_connections 2024-09-12T01:50:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:10.648+0000 7f200c1f8640 1 -- 172.21.15.122:0/474002529 wait complete. 2024-09-12T01:50:10.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 260041 15.09 MB/sec execute 28 sec latency 12.076 ms 2024-09-12T01:50:11.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 262887 15.17 MB/sec execute 29 sec latency 135.075 ms 2024-09-12T01:50:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:11 smithi151 ceph-mon[35229]: pgmap v446: 97 pgs: 97 active+clean; 119 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-09-12T01:50:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:11 smithi183 ceph-mon[35127]: pgmap v446: 97 pgs: 97 active+clean; 119 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-09-12T01:50:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:11 smithi122 ceph-mon[30925]: pgmap v446: 97 pgs: 97 active+clean; 119 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-09-12T01:50:12.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 265509 15.08 MB/sec execute 30 sec latency 145.620 ms 2024-09-12T01:50:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:12 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:50:12.923 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:50:12.923 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T01:50:12.923 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:50:12.948 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:12 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:50:12.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:12 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T01:50:13.185 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.661+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/3282532390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 msgr2=0x7f0f481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.661+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/3282532390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481029b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02a480 comp rx=0 tx=0).stop 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/3282532390 shutdown_connections 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/3282532390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 0x7f0f4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/3282532390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481029b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/3282532390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f48100e20 0x7f0f48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/3282532390 >> 172.21.15.122:0/3282532390 conn(0x7f0f480fc7d0 msgr2=0x7f0f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/3282532390 shutdown_connections 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/3282532390 wait complete. 2024-09-12T01:50:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 Processor -- start 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.662+0000 7f0f4d2b2640 1 -- start start 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f48100e20 0x7f0f48199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 0x7f0f481a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110f50 con 0x7f0f48102570 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110dd0 con 0x7f0f48102ef0 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481110d0 con 0x7f0f48100e20 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39142/0 (socket says 172.21.15.122:39142) 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 -- 172.21.15.122:0/154398304 learned_addr learned my addr 172.21.15.122:0/154398304 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f48100e20 0x7f0f48199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f477fe640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 0x7f0f481a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:13.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 -- 172.21.15.122:0/154398304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f48100e20 msgr2=0x7f0f48199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f48100e20 0x7f0f48199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 -- 172.21.15.122:0/154398304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 msgr2=0x7f0f481a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 0x7f0f481a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 -- 172.21.15.122:0/154398304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f481a4920 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f477fe640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 0x7f0f481a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f467fc640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481998a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c006010 tx=0x7f0f3c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c03f070 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f481a4af0 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3c031c30 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.663+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c035030 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.664+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f481a4fd0 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.665+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f48101200 con 0x7f0f48102570 2024-09-12T01:50:13.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.666+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0f3c042070 con 0x7f0f48102570 2024-09-12T01:50:13.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.667+0000 7f0f2ffff640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f300a2b80 0x7f0f300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:13.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.667+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f300a2b80 0x7f0f300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:13.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.667+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0f3c0ef690 con 0x7f0f48102570 2024-09-12T01:50:13.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.667+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f300a2b80 0x7f0f300a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005d00 tx=0x7f0f3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:13.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.671+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f3c0bbfa0 con 0x7f0f48102570 2024-09-12T01:50:13.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 268039 14.98 MB/sec execute 31 sec latency 22.087 ms 2024-09-12T01:50:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:13 smithi151 ceph-mon[35229]: pgmap v447: 97 pgs: 97 active+clean; 118 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-09-12T01:50:13.876 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:13 smithi122 ceph-mon[30925]: pgmap v447: 97 pgs: 97 active+clean; 118 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-09-12T01:50:13.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.905+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f48109090 con 0x7f0f48102570 2024-09-12T01:50:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.906+0000 7f0f2ffff640 1 -- 172.21.15.122:0/154398304 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f0f3c0c02a0 con 0x7f0f48102570 2024-09-12T01:50:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:50:13.906 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:50:13.907 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.908+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f300a2b80 msgr2=0x7f0f300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.908+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f300a2b80 0x7f0f300a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005d00 tx=0x7f0f3800b410 comp rx=0 tx=0).stop 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.908+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 msgr2=0x7f0f481998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.908+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481998a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c006010 tx=0x7f0f3c003c10 comp rx=0 tx=0).stop 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 shutdown_connections 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0f300a2b80 0x7f0f300a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48102ef0 0x7f0f481a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48102570 0x7f0f481998a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 --2- 172.21.15.122:0/154398304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f48100e20 0x7f0f48199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 >> 172.21.15.122:0/154398304 conn(0x7f0f480fc7d0 msgr2=0x7f0f480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 shutdown_connections 2024-09-12T01:50:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:13.909+0000 7f0f4d2b2640 1 -- 172.21.15.122:0/154398304 wait complete. 2024-09-12T01:50:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:13 smithi183 ceph-mon[35127]: pgmap v447: 97 pgs: 97 active+clean; 118 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-09-12T01:50:14.705 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T01:50:14.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T01:50:14.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 270726 15.00 MB/sec execute 32 sec latency 14.850 ms 2024-09-12T01:50:14.904 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:14 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/154398304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:14.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:14 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/154398304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:14 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/154398304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:15.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 -- 172.21.15.122:0/1955368787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102ef0 msgr2=0x7f235010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 --2- 172.21.15.122:0/1955368787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102ef0 0x7f235010da40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2348009ca0 tx=0x7f234802a950 comp rx=0 tx=0).stop 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 -- 172.21.15.122:0/1955368787 shutdown_connections 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 --2- 172.21.15.122:0/1955368787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102ef0 0x7f235010da40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 --2- 172.21.15.122:0/1955368787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102570 0x7f23501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 --2- 172.21.15.122:0/1955368787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f2350101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 -- 172.21.15.122:0/1955368787 >> 172.21.15.122:0/1955368787 conn(0x7f23500fc7d0 msgr2=0x7f23500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 -- 172.21.15.122:0/1955368787 shutdown_connections 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.524+0000 7f235704c640 1 -- 172.21.15.122:0/1955368787 wait complete. 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 Processor -- start 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 -- start start 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f235010e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 0x7f235010e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102ef0 0x7f235010ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2350112090 con 0x7f2350102570 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2350111f10 con 0x7f2350102ef0 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f235704c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2350112210 con 0x7f2350100e20 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f234ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 0x7f235010e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f2354de8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f235010e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f2354de8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f235010e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:45568/0 (socket says 172.21.15.122:45568) 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f23555e9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102ef0 0x7f235010ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f234ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 0x7f235010e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39172/0 (socket says 172.21.15.122:39172) 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.525+0000 7f2354de8640 1 -- 172.21.15.122:0/1621764794 learned_addr learned my addr 172.21.15.122:0/1621764794 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:15.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f2354de8640 1 -- 172.21.15.122:0/1621764794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102ef0 msgr2=0x7f235010ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f2354de8640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102ef0 0x7f235010ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f2354de8640 1 -- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 msgr2=0x7f235010e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f2354de8640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 0x7f235010e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f2354de8640 1 -- 172.21.15.122:0/1621764794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23501a69e0 con 0x7f2350100e20 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f23555e9640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102ef0 0x7f235010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f234ffff640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 0x7f235010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f2354de8640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f235010e1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f234400d770 tx=0x7f234400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f234dffb640 1 -- 172.21.15.122:0/1621764794 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344017070 con 0x7f2350100e20 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23501a6cd0 con 0x7f2350100e20 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f234dffb640 1 -- 172.21.15.122:0/1621764794 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2344005500 con 0x7f2350100e20 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f234dffb640 1 -- 172.21.15.122:0/1621764794 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344004e00 con 0x7f2350100e20 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.526+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501a71c0 con 0x7f2350100e20 2024-09-12T01:50:15.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.528+0000 7f234dffb640 1 -- 172.21.15.122:0/1621764794 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2344004fa0 con 0x7f2350100e20 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 Processor -- start 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 -- start start 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234dffb640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23400a2b80 0x7f23400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931a9850 0x5571931a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 0x5571931b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55719310e160 con 0x5571931a9850 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571930b5720 con 0x5571931aa170 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2359425740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55719310a120 con 0x5571931b4a30 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f23555e9640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23400a2b80 0x7f23400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234dffb640 1 -- 172.21.15.122:0/1621764794 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f23440cea10 con 0x7f2350100e20 2024-09-12T01:50:15.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 0x5571931b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f23555e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f2354de8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931a9850 0x5571931a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 0x5571931b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:43672/0 (socket says 172.21.15.122:43672) 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 -- 172.21.15.122:0/578050813 learned_addr learned my addr 172.21.15.122:0/578050813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 -- 172.21.15.122:0/578050813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 msgr2=0x5571931b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 -- 172.21.15.122:0/578050813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931a9850 msgr2=0x5571931a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931a9850 0x5571931a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.529+0000 7f234ffff640 1 -- 172.21.15.122:0/578050813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557192e7fba0 con 0x5571931aa170 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.530+0000 7f23555e9640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23400a2b80 0x7f23400a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f234802fd80 tx=0x7f234802ff50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.530+0000 7f234ffff640 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 0x5571931b44f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f233c011470 tx=0x7f233c011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79ed2bff9c612b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:15.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.530+0000 7f233affd640 1 -- 172.21.15.122:0/578050813 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c004020 con 0x5571931aa170 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.530+0000 7f233affd640 1 -- 172.21.15.122:0/578050813 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f233c0041c0 con 0x5571931aa170 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.530+0000 7f233affd640 1 -- 172.21.15.122:0/578050813 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c00f8e0 con 0x5571931aa170 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.530+0000 7f2359425740 1 -- 172.21.15.122:0/578050813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 msgr2=0x5571931b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 0x5571931b44f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f233c011470 tx=0x7f233c011930 comp rx=0 tx=0).stop 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- 172.21.15.122:0/578050813 shutdown_connections 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931aa170 0x5571931b44f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- 172.21.15.122:0/578050813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931a9850 0x5571931a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- 172.21.15.122:0/578050813 >> 172.21.15.122:0/578050813 conn(0x5571931a5de0 msgr2=0x5571931a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- 172.21.15.122:0/578050813 shutdown_connections 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- 172.21.15.122:0/578050813 wait complete. 2024-09-12T01:50:15.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- start start 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 Processor -- start 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571931b7600 con 0x5571931b4a30 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557193096900 con 0x5571931a9850 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.531+0000 7f2359425740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5571931b7780 con 0x5571931aa170 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f2359425740 10 client.0 _ll_register_callbacks cb 0x557193094780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39196/0 (socket says 172.21.15.122:39196) 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 -- 172.21.15.122:0/99558600 learned_addr learned my addr 172.21.15.122:0/99558600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:15.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f2354de8640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f234ffff640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 -- 172.21.15.122:0/99558600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 msgr2=0x5571931b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 -- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 msgr2=0x5571931b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 -- 172.21.15.122:0/99558600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571930d6c50 con 0x5571931b4a30 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f23555e9640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931b2350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2348030040 tx=0x7f234802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ace5fe9dabee1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f232ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f232ffff640 1 -- 172.21.15.122:0/99558600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348034040 con 0x5571931b4a30 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f232ffff640 1 -- 172.21.15.122:0/99558600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2348054640 con 0x5571931b4a30 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.532+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f233c0041c0 con 0x5571931b4a30 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f232ffff640 1 -- 172.21.15.122:0/99558600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2348054840 con 0x5571931b4a30 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 10 client.16206 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:15.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571931c3260 con 0x5571931b4a30 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f232ffff640 1 -- 172.21.15.122:0/99558600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f233c0041c0 con 0x5571931b4a30 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f232ffff640 1 -- 172.21.15.122:0/99558600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2348052020 con 0x5571931b4a30 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 10 client.16206 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 20 client.16206 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x5571931c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 4 client.16206 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55719311b890 con 0x5571931c4710 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.533+0000 7f2354de8640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x5571931c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f2354de8640 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x5571931c6b80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d48689284739eb9 server_cookie=a5b1600229697dba in_seq=0 out_seq=0 2024-09-12T01:50:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f232ffff640 10 client.16206 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:15.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f232ffff640 1 -- 172.21.15.122:0/99558600 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55719311b890 con 0x5571931c4710 2024-09-12T01:50:15.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f232ffff640 10 client.16206 handle_command_reply: tid=0 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f2359425740 1 client.16206 shutdown 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 msgr2=0x5571931b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.534+0000 7f2359425740 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931b2350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2348030040 tx=0x7f234802a950 comp rx=0 tx=0).stop 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 shutdown_connections 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x5571931c6b80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931b2350 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f232ffff640 0 client.16206 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 --2- 172.21.15.122:0/99558600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 >> 172.21.15.122:0/99558600 conn(0x5571931a5de0 msgr2=0x5571931abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 shutdown_connections 2024-09-12T01:50:15.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 1 -- 172.21.15.122:0/99558600 wait complete. 2024-09-12T01:50:15.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.535+0000 7f2359425740 20 client.16206 trim_cache size 0 max 16384 2024-09-12T01:50:15.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 Processor -- start 2024-09-12T01:50:15.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 -- start start 2024-09-12T01:50:15.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b2020 con 0x5571931b4a30 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23501af8f0 con 0x5571931a9850 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f2359425740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23501ad720 con 0x5571931aa170 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.552+0000 7f234ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f2354de8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:45594/0 (socket says 172.21.15.122:45594) 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f23555e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 -- 172.21.15.122:0/3117742374 learned_addr learned my addr 172.21.15.122:0/3117742374 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 -- 172.21.15.122:0/3117742374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 msgr2=0x5571931cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 -- 172.21.15.122:0/3117742374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 msgr2=0x5571931aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 -- 172.21.15.122:0/3117742374 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571931a9fa0 con 0x5571931aa170 2024-09-12T01:50:15.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f234ffff640 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931cc990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f233c014040 tx=0x7f233c003e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78531f07d0e9d3fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:15.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f233a7fc640 1 -- 172.21.15.122:0/3117742374 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c027020 con 0x5571931aa170 2024-09-12T01:50:15.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f233a7fc640 1 -- 172.21.15.122:0/3117742374 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f233c003ad0 con 0x5571931aa170 2024-09-12T01:50:15.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.553+0000 7f233a7fc640 1 -- 172.21.15.122:0/3117742374 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c00f600 con 0x5571931aa170 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 -- 172.21.15.122:0/3117742374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 msgr2=0x5571931cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931cc990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f233c014040 tx=0x7f233c003e40 comp rx=0 tx=0).stop 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 -- 172.21.15.122:0/3117742374 shutdown_connections 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931b4a30 0x5571931aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931aa170 0x5571931cc990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 --2- 172.21.15.122:0/3117742374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 -- 172.21.15.122:0/3117742374 >> 172.21.15.122:0/3117742374 conn(0x5571931a5de0 msgr2=0x5571931b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 -- 172.21.15.122:0/3117742374 shutdown_connections 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 -- 172.21.15.122:0/3117742374 wait complete. 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 -- start start 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.554+0000 7f2359425740 1 Processor -- start 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 0x5571931b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23501addb0 con 0x5571931aa170 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23501af220 con 0x5571931a9850 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23501ae480 con 0x5571931b4a30 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2359425740 10 client.0 _ll_register_callbacks cb 0x7f2348009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 0x5571931b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2354de8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 0x5571931b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39226/0 (socket says 172.21.15.122:39226) 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f2354de8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:43688/0 (socket says 172.21.15.122:43688) 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 -- 172.21.15.122:0/4257461941 learned_addr learned my addr 172.21.15.122:0/4257461941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f23555e9640 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 -- 172.21.15.122:0/4257461941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 msgr2=0x5571931b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 -- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 msgr2=0x5571931b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 -- 172.21.15.122:0/4257461941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719322e420 con 0x5571931aa170 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f234ffff640 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 0x5571931b8c70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f233c011e30 tx=0x7f233c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6d18cf293ff7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.555+0000 7f232ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f232ffff640 1 -- 172.21.15.122:0/4257461941 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c027070 con 0x5571931aa170 2024-09-12T01:50:15.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f232ffff640 1 -- 172.21.15.122:0/4257461941 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f233c005280 con 0x5571931aa170 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f233c003ad0 con 0x5571931aa170 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f232ffff640 1 -- 172.21.15.122:0/4257461941 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f233c0035f0 con 0x5571931aa170 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 10 client.16212 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55719322f6e0 con 0x5571931aa170 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f232ffff640 1 -- 172.21.15.122:0/4257461941 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f233c003ad0 con 0x5571931aa170 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f232ffff640 1 -- 172.21.15.122:0/4257461941 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f233c023460 con 0x5571931aa170 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 10 client.16212 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 10 client.16212 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 20 client.16212 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x557193231be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 4 client.16212 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.556+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x557193115be0 con 0x5571931c4710 2024-09-12T01:50:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.557+0000 7f2354de8640 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x557193231be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.557+0000 7f232ffff640 10 client.16212 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.557+0000 7f2354de8640 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x557193231be0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78241207afa911cb server_cookie=c15b386b9240a866 in_seq=0 out_seq=0 2024-09-12T01:50:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f232ffff640 1 -- 172.21.15.122:0/4257461941 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557193115be0 con 0x5571931c4710 2024-09-12T01:50:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f232ffff640 10 client.16212 handle_command_reply: tid=0 2024-09-12T01:50:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 client.16212 shutdown 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 msgr2=0x5571931b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 0x5571931b8c70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f233c011e30 tx=0x7f233c005720 comp rx=0 tx=0).stop 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 shutdown_connections 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571931c4710 0x557193231be0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571931b4a30 0x5571931b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571931aa170 0x5571931b8c70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f232ffff640 0 client.16212 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 --2- 172.21.15.122:0/4257461941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571931a9850 0x5571931b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 >> 172.21.15.122:0/4257461941 conn(0x5571931a5de0 msgr2=0x5571931ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 shutdown_connections 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 1 -- 172.21.15.122:0/4257461941 wait complete. 2024-09-12T01:50:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.667+0000 7f2359425740 20 client.16212 trim_cache size 0 max 16384 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 14, 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "e8a7a64d": { 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 14, 2024-09-12T01:50:15.669 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 140.2, 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:15.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23400a2b80 msgr2=0x7f23400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23400a2b80 0x7f23400a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f234802fd80 tx=0x7f234802ff50 comp rx=0 tx=0).stop 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 msgr2=0x7f235010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f235010e1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f234400d770 tx=0x7f234400dc30 comp rx=0 tx=0).stop 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 shutdown_connections 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23400a2b80 0x7f23400a5040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2350102ef0 0x7f235010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2350102570 0x7f235010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 --2- 172.21.15.122:0/1621764794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350100e20 0x7f235010e1d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 >> 172.21.15.122:0/1621764794 conn(0x7f23500fc7d0 msgr2=0x7f23501068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 shutdown_connections 2024-09-12T01:50:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:15.668+0000 7f235704c640 1 -- 172.21.15.122:0/1621764794 wait complete. 2024-09-12T01:50:15.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:15 smithi122 ceph-mon[30925]: pgmap v448: 97 pgs: 97 active+clean; 121 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-09-12T01:50:15.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 15.03 MB/sec execute 33 sec latency 154.514 ms 2024-09-12T01:50:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:15 smithi183 ceph-mon[35127]: pgmap v448: 97 pgs: 97 active+clean; 121 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-09-12T01:50:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:15 smithi151 ceph-mon[35229]: pgmap v448: 97 pgs: 97 active+clean; 121 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 370 op/s 2024-09-12T01:50:16.555 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e8a7a64d', quiesce took 3.6 seconds. Will release after: 16.549999999999997 2024-09-12T01:50:16.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 14.58 MB/sec execute 34 sec latency 1154.587 ms 2024-09-12T01:50:17.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 14.17 MB/sec execute 35 sec latency 2154.643 ms 2024-09-12T01:50:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:17 smithi183 ceph-mon[35127]: pgmap v449: 97 pgs: 97 active+clean; 120 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-12T01:50:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:17 smithi122 ceph-mon[30925]: pgmap v449: 97 pgs: 97 active+clean; 120 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-12T01:50:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:17 smithi151 ceph-mon[35229]: pgmap v449: 97 pgs: 97 active+clean; 120 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-12T01:50:18.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 13.77 MB/sec execute 36 sec latency 3154.702 ms 2024-09-12T01:50:19.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 13.40 MB/sec execute 37 sec latency 4154.809 ms 2024-09-12T01:50:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:19 smithi183 ceph-mon[35127]: pgmap v450: 97 pgs: 97 active+clean; 124 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:50:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:19 smithi122 ceph-mon[30925]: pgmap v450: 97 pgs: 97 active+clean; 124 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:50:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:19 smithi151 ceph-mon[35229]: pgmap v450: 97 pgs: 97 active+clean; 124 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:50:20.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 13.05 MB/sec execute 38 sec latency 5154.883 ms 2024-09-12T01:50:21.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 12.71 MB/sec execute 39 sec latency 6154.957 ms 2024-09-12T01:50:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:21 smithi183 ceph-mon[35127]: pgmap v451: 97 pgs: 97 active+clean; 127 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2024-09-12T01:50:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:21 smithi122 ceph-mon[30925]: pgmap v451: 97 pgs: 97 active+clean; 127 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2024-09-12T01:50:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:21 smithi151 ceph-mon[35229]: pgmap v451: 97 pgs: 97 active+clean; 127 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2024-09-12T01:50:22.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 12.40 MB/sec execute 40 sec latency 7155.028 ms 2024-09-12T01:50:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 -- 172.21.15.122:0/4049303278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab40102e00 msgr2=0x7fab40103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 --2- 172.21.15.122:0/4049303278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab40102e00 0x7fab40103280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fab34009900 tx=0x7fab3402a410 comp rx=0 tx=0).stop 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 -- 172.21.15.122:0/4049303278 shutdown_connections 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 --2- 172.21.15.122:0/4049303278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 0x7fab4010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 --2- 172.21.15.122:0/4049303278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab40102e00 0x7fab40103280 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 --2- 172.21.15.122:0/4049303278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab400ffe30 0x7fab40100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 -- 172.21.15.122:0/4049303278 >> 172.21.15.122:0/4049303278 conn(0x7fab400fb970 msgr2=0x7fab400fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 -- 172.21.15.122:0/4049303278 shutdown_connections 2024-09-12T01:50:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.745+0000 7fab45a8d640 1 -- 172.21.15.122:0/4049303278 wait complete. 2024-09-12T01:50:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 Processor -- start 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 -- start start 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 0x7fab40197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab40102e00 0x7fab40198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 0x7fab4019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab4010d800 con 0x7fab400ffe30 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab4010d680 con 0x7fab401037c0 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab45a8d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab4010d980 con 0x7fab40102e00 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 0x7fab40197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 0x7fab40197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40792/0 (socket says 172.21.15.122:40792) 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.746+0000 7fab3f7fe640 1 -- 172.21.15.122:0/1657381147 learned_addr learned my addr 172.21.15.122:0/1657381147 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3effd640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab40102e00 0x7fab40198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3ffff640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 0x7fab4019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3f7fe640 1 -- 172.21.15.122:0/1657381147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab40102e00 msgr2=0x7fab40198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab40102e00 0x7fab40198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3f7fe640 1 -- 172.21.15.122:0/1657381147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 msgr2=0x7fab4019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 0x7fab4019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3f7fe640 1 -- 172.21.15.122:0/1657381147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4019fa70 con 0x7fab400ffe30 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3ffff640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 0x7fab4019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 0x7fab40197e50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fab3000ccb0 tx=0x7fab3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3cff9640 1 -- 172.21.15.122:0/1657381147 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30004b20 con 0x7fab400ffe30 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3cff9640 1 -- 172.21.15.122:0/1657381147 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab30004cc0 con 0x7fab400ffe30 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4019fd60 con 0x7fab400ffe30 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab3cff9640 1 -- 172.21.15.122:0/1657381147 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30005020 con 0x7fab400ffe30 2024-09-12T01:50:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.747+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab401a01d0 con 0x7fab400ffe30 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.749+0000 7fab3cff9640 1 -- 172.21.15.122:0/1657381147 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fab30004000 con 0x7fab400ffe30 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 Processor -- start 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 -- start start 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3cff9640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab280a2c30 0x7fab280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935559da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b93555eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b935561830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b9354bd2f0 con 0x55b9355599a0 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b935465270 con 0x55b93555a2e0 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab48828740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b9354b92b0 con 0x55b93555f370 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3cff9640 1 -- 172.21.15.122:0/1657381147 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fab300cf150 con 0x7fab400ffe30 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3ffff640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab280a2c30 0x7fab280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935559da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935559da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40800/0 (socket says 172.21.15.122:40800) 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3f7fe640 1 -- 172.21.15.122:0/2177652043 learned_addr learned my addr 172.21.15.122:0/2177652043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.750+0000 7fab3effd640 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b93555eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3ffff640 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b935561830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3f7fe640 1 -- 172.21.15.122:0/2177652043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 msgr2=0x55b935561830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b935561830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3f7fe640 1 -- 172.21.15.122:0/2177652043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 msgr2=0x55b93555eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b93555eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3f7fe640 1 -- 172.21.15.122:0/2177652043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b935464340 con 0x55b9355599a0 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3ffff640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab280a2c30 0x7fab280a50f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab380149e0 tx=0x7fab3800b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935559da0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fab30097dd0 tx=0x7fab30004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f521ebc3377ca2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab25ffb640 1 -- 172.21.15.122:0/2177652043 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3001d040 con 0x55b9355599a0 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab25ffb640 1 -- 172.21.15.122:0/2177652043 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3009c070 con 0x55b9355599a0 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.751+0000 7fab25ffb640 1 -- 172.21.15.122:0/2177652043 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab300a45e0 con 0x55b9355599a0 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 -- 172.21.15.122:0/2177652043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 msgr2=0x55b935559da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935559da0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fab30097dd0 tx=0x7fab30004f60 comp rx=0 tx=0).stop 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 -- 172.21.15.122:0/2177652043 shutdown_connections 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b935561830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b93555eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 --2- 172.21.15.122:0/2177652043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935559da0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 -- 172.21.15.122:0/2177652043 >> 172.21.15.122:0/2177652043 conn(0x55b935555010 msgr2=0x55b935555410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 -- 172.21.15.122:0/2177652043 shutdown_connections 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 -- 172.21.15.122:0/2177652043 wait complete. 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.752+0000 7fab48828740 1 -- start start 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 Processor -- start 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935563c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935564140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b93556b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b935562340 con 0x55b9355599a0 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b935446290 con 0x55b93555a2e0 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b9355624c0 con 0x55b93555f370 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935563c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab48828740 10 client.0 _ll_register_callbacks cb 0x55b935477140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935563c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40804/0 (socket says 172.21.15.122:40804) 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 -- 172.21.15.122:0/575587494 learned_addr learned my addr 172.21.15.122:0/575587494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3effd640 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935564140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 -- 172.21.15.122:0/575587494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 msgr2=0x55b93556b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b93556b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 -- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 msgr2=0x55b935564140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935564140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 -- 172.21.15.122:0/575587494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9351d0210 con 0x55b9355599a0 2024-09-12T01:50:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935563c00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fab30004af0 tx=0x7fab3009fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51f7515b4058576 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.753+0000 7fab1effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab1effd640 1 -- 172.21.15.122:0/575587494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab300a8070 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab1effd640 1 -- 172.21.15.122:0/575587494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fab300afe90 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab3009c070 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab1effd640 1 -- 172.21.15.122:0/575587494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab3001d070 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 10 client.16224 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b93529c620 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab1effd640 1 -- 172.21.15.122:0/575587494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3009c070 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab1effd640 1 -- 172.21.15.122:0/575587494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fab30086110 con 0x55b9355599a0 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 10 client.16224 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 20 client.16224 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b93556f480 0x55b935571870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 4 client.16224 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9354caa20 con 0x55b93556f480 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.754+0000 7fab3effd640 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b93556f480 0x55b935571870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.755+0000 7fab3effd640 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b93556f480 0x55b935571870 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1be724ad58d34c9 server_cookie=ac3cd36ce9c81f2 in_seq=0 out_seq=0 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.755+0000 7fab1effd640 10 client.16224 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab1effd640 1 -- 172.21.15.122:0/575587494 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9354caa20 con 0x55b93556f480 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab1effd640 10 client.16224 handle_command_reply: tid=0 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 client.16224 shutdown 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 msgr2=0x55b935563c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935563c00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fab30004af0 tx=0x7fab3009fd90 comp rx=0 tx=0).stop 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 shutdown_connections 2024-09-12T01:50:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b93556f480 0x55b935571870 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93555f370 0x55b93556b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935564140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 --2- 172.21.15.122:0/575587494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b9355599a0 0x55b935563c00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab1effd640 0 client.16224 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 >> 172.21.15.122:0/575587494 conn(0x55b935555010 msgr2=0x55b935556820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 shutdown_connections 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 1 -- 172.21.15.122:0/575587494 wait complete. 2024-09-12T01:50:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.756+0000 7fab48828740 20 client.16224 trim_cache size 0 max 16384 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 Processor -- start 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 -- start start 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b935559d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935556400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 0x55b935556940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab401ab890 con 0x55b93555f370 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab401a9160 con 0x55b93555a2e0 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab48828740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab401a6f90 con 0x55b9355599a0 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 0x55b935556940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 0x55b935556940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40814/0 (socket says 172.21.15.122:40814) 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.770+0000 7fab3ffff640 1 -- 172.21.15.122:0/2217132059 learned_addr learned my addr 172.21.15.122:0/2217132059 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b935559d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3effd640 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935556400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3ffff640 1 -- 172.21.15.122:0/2217132059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 msgr2=0x55b935559d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3ffff640 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b935559d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3ffff640 1 -- 172.21.15.122:0/2217132059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 msgr2=0x55b935556400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3ffff640 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935556400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3ffff640 1 -- 172.21.15.122:0/2217132059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b935564090 con 0x55b93555f370 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab3ffff640 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 0x55b935556940 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fab38014380 tx=0x7fab380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835cd8717b9af799 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab257fa640 1 -- 172.21.15.122:0/2217132059 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38020640 con 0x55b93555f370 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab257fa640 1 -- 172.21.15.122:0/2217132059 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab38026070 con 0x55b93555f370 2024-09-12T01:50:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.771+0000 7fab257fa640 1 -- 172.21.15.122:0/2217132059 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38011930 con 0x55b93555f370 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- 172.21.15.122:0/2217132059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 msgr2=0x55b935556940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 0x55b935556940 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fab38014380 tx=0x7fab380046c0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- 172.21.15.122:0/2217132059 shutdown_connections 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555f370 0x55b935556940 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555a2e0 0x55b935556400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- 172.21.15.122:0/2217132059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b935559d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- 172.21.15.122:0/2217132059 >> 172.21.15.122:0/2217132059 conn(0x55b935555010 msgr2=0x55b9355541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- 172.21.15.122:0/2217132059 shutdown_connections 2024-09-12T01:50:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- 172.21.15.122:0/2217132059 wait complete. 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- start start 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 Processor -- start 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b9355d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 0x55b9355d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555f370 0x55b9355dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab401a7620 con 0x55b93555a2e0 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab401a8a90 con 0x55b93555f370 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.772+0000 7fab48828740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab401a7cf0 con 0x55b9355599a0 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab48828740 10 client.0 _ll_register_callbacks cb 0x7fab34005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 0x55b9355d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 0x55b9355d5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40816/0 (socket says 172.21.15.122:40816) 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 -- 172.21.15.122:0/1074590436 learned_addr learned my addr 172.21.15.122:0/1074590436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b9355d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3ffff640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555f370 0x55b9355dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 -- 172.21.15.122:0/1074590436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 msgr2=0x55b9355d5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b9355d5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 -- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555f370 msgr2=0x55b9355dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555f370 0x55b9355dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 -- 172.21.15.122:0/1074590436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9355dd2c0 con 0x55b93555a2e0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3ffff640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555f370 0x55b9355dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab3effd640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 0x55b9355d5c70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fab3402a930 tx=0x7fab3402aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf376fda04832f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab1effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab1effd640 1 -- 172.21.15.122:0/1074590436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34035040 con 0x55b93555a2e0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab1effd640 1 -- 172.21.15.122:0/1074590436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fab34004040 con 0x55b93555a2e0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab38026070 con 0x55b93555a2e0 2024-09-12T01:50:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab1effd640 1 -- 172.21.15.122:0/1074590436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab34004240 con 0x55b93555a2e0 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab48828740 10 client.16230 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.773+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b9355de640 con 0x55b93555a2e0 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab1effd640 1 -- 172.21.15.122:0/1074590436 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab38026070 con 0x55b93555a2e0 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab1effd640 1 -- 172.21.15.122:0/1074590436 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fab3404f020 con 0x55b93555a2e0 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab48828740 10 client.16230 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab48828740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab48828740 20 client.16230 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab48828740 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b9355df070 0x55b9355e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab48828740 4 client.16230 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9354c4d70 con 0x55b9355df070 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b9355df070 0x55b9355e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab3f7fe640 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b9355df070 0x55b9355e1490 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cafb73435a86db server_cookie=1863ba56152e3987 in_seq=0 out_seq=0 2024-09-12T01:50:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.774+0000 7fab1effd640 10 client.16230 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab1effd640 1 -- 172.21.15.122:0/1074590436 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9354c4d70 con 0x55b9355df070 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab1effd640 10 client.16230 handle_command_reply: tid=0 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 client.16230 shutdown 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 msgr2=0x55b9355d5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 0x55b9355d5c70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fab3402a930 tx=0x7fab3402aba0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 shutdown_connections 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b9355df070 0x55b9355e1490 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b93555f370 0x55b9355dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab1effd640 0 client.16230 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b93555a2e0 0x55b9355d5c70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 --2- 172.21.15.122:0/1074590436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9355599a0 0x55b9355d5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 >> 172.21.15.122:0/1074590436 conn(0x55b935555010 msgr2=0x55b9355553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 shutdown_connections 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 1 -- 172.21.15.122:0/1074590436 wait complete. 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.784+0000 7fab48828740 20 client.16230 trim_cache size 0 max 16384 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:50:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:50:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:50:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:50:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab280a2c30 msgr2=0x7fab280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab280a2c30 0x7fab280a50f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab380149e0 tx=0x7fab3800b580 comp rx=0 tx=0).stop 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 msgr2=0x7fab40197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 0x7fab40197e50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fab3000ccb0 tx=0x7fab3000a620 comp rx=0 tx=0).stop 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 shutdown_connections 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab280a2c30 0x7fab280a50f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab401037c0 0x7fab4019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab40102e00 0x7fab40198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 --2- 172.21.15.122:0/1657381147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab400ffe30 0x7fab40197e50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 >> 172.21.15.122:0/1657381147 conn(0x7fab400fb970 msgr2=0x7fab40106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 shutdown_connections 2024-09-12T01:50:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:23.785+0000 7fab45a8d640 1 -- 172.21.15.122:0/1657381147 wait complete. 2024-09-12T01:50:23.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 12.09 MB/sec execute 41 sec latency 8155.121 ms 2024-09-12T01:50:23.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:23 smithi151 ceph-mon[35229]: pgmap v452: 97 pgs: 97 active+clean; 126 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 222 op/s 2024-09-12T01:50:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:23 smithi183 ceph-mon[35127]: pgmap v452: 97 pgs: 97 active+clean; 126 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 222 op/s 2024-09-12T01:50:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:23 smithi122 ceph-mon[30925]: pgmap v452: 97 pgs: 97 active+clean; 126 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 222 op/s 2024-09-12T01:50:24.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 11.81 MB/sec execute 42 sec latency 9155.153 ms 2024-09-12T01:50:25.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 11.53 MB/sec execute 43 sec latency 10155.197 ms 2024-09-12T01:50:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:25 smithi183 ceph-mon[35127]: pgmap v453: 97 pgs: 97 active+clean; 123 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T01:50:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:25 smithi122 ceph-mon[30925]: pgmap v453: 97 pgs: 97 active+clean; 123 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T01:50:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:25 smithi151 ceph-mon[35229]: pgmap v453: 97 pgs: 97 active+clean; 123 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T01:50:26.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 11.27 MB/sec execute 44 sec latency 11155.252 ms 2024-09-12T01:50:27.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 11.02 MB/sec execute 45 sec latency 12155.285 ms 2024-09-12T01:50:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:27 smithi183 ceph-mon[35127]: pgmap v454: 97 pgs: 97 active+clean; 122 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:50:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:27 smithi122 ceph-mon[30925]: pgmap v454: 97 pgs: 97 active+clean; 122 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:50:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:27 smithi151 ceph-mon[35229]: pgmap v454: 97 pgs: 97 active+clean; 122 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:50:28.822 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 10.78 MB/sec execute 46 sec latency 13155.314 ms 2024-09-12T01:50:29.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 10.55 MB/sec execute 47 sec latency 14155.364 ms 2024-09-12T01:50:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:29 smithi183 ceph-mon[35127]: pgmap v455: 97 pgs: 97 active+clean; 123 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 2.7 MiB/s wr, 148 op/s 2024-09-12T01:50:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:29 smithi122 ceph-mon[30925]: pgmap v455: 97 pgs: 97 active+clean; 123 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 2.7 MiB/s wr, 148 op/s 2024-09-12T01:50:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:29 smithi151 ceph-mon[35229]: pgmap v455: 97 pgs: 97 active+clean; 123 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 2.7 MiB/s wr, 148 op/s 2024-09-12T01:50:30.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 10.33 MB/sec execute 48 sec latency 15155.440 ms 2024-09-12T01:50:31.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 10.12 MB/sec execute 49 sec latency 16155.529 ms 2024-09-12T01:50:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:31 smithi183 ceph-mon[35127]: pgmap v456: 97 pgs: 97 active+clean; 122 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-12T01:50:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:31 smithi122 ceph-mon[30925]: pgmap v456: 97 pgs: 97 active+clean; 122 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-12T01:50:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:31 smithi151 ceph-mon[35229]: pgmap v456: 97 pgs: 97 active+clean; 122 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-12T01:50:32.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 9.92 MB/sec execute 50 sec latency 17155.612 ms 2024-09-12T01:50:33.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:50:33.365 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:33.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:33 smithi122 ceph-mon[30925]: pgmap v457: 97 pgs: 97 active+clean; 122 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.3 MiB/s wr, 95 op/s 2024-09-12T01:50:33.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 9.72 MB/sec execute 51 sec latency 18155.656 ms 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 -- 172.21.15.122:0/497641375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102ef0 msgr2=0x7f7e2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/497641375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102ef0 0x7f7e2810da40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009ca0 tx=0x7f7e2002a950 comp rx=0 tx=0).stop 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 -- 172.21.15.122:0/497641375 shutdown_connections 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/497641375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102ef0 0x7f7e2810da40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/497641375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e28102570 0x7f7e281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/497641375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 0x7f7e28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:33.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 -- 172.21.15.122:0/497641375 >> 172.21.15.122:0/497641375 conn(0x7f7e280fc7d0 msgr2=0x7f7e280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 -- 172.21.15.122:0/497641375 shutdown_connections 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.822+0000 7f7e2ff41640 1 -- 172.21.15.122:0/497641375 wait complete. 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 Processor -- start 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 -- start start 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 0x7f7e28197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 0x7f7e28197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e28102ef0 0x7f7e28197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28111020 con 0x7f7e28102570 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28110ea0 con 0x7f7e28102ef0 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2ff41640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281111a0 con 0x7f7e28100e20 2024-09-12T01:50:33.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 0x7f7e28197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 0x7f7e28197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44512/0 (socket says 172.21.15.122:44512) 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 -- 172.21.15.122:0/4258983444 learned_addr learned my addr 172.21.15.122:0/4258983444 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2e4de640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e28102ef0 0x7f7e28197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2dcdd640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 0x7f7e28197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 -- 172.21.15.122:0/4258983444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 msgr2=0x7f7e28197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 0x7f7e28197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 -- 172.21.15.122:0/4258983444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e28102ef0 msgr2=0x7f7e28197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e28102ef0 0x7f7e28197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 -- 172.21.15.122:0/4258983444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e281a6ab0 con 0x7f7e28102570 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2dcdd640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 0x7f7e28197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e2d4dc640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 0x7f7e28197780 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400b4a0 tx=0x7f7e2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:33.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.823+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24014070 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.824+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e281a6da0 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.824+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24010af0 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.824+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24017d10 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.824+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281a72d0 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.825+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e28101280 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.826+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7e24017070 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.826+0000 7f7e1affd640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e140a2b80 0x7f7e140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.827+0000 7f7e2dcdd640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e140a2b80 0x7f7e140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.827+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7e240ce3a0 con 0x7f7e28102570 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.827+0000 7f7e2dcdd640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e140a2b80 0x7f7e140a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c006fd0 tx=0x7f7e1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:33.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:33.831+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e2409b300 con 0x7f7e28102570 2024-09-12T01:50:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:33 smithi183 ceph-mon[35127]: pgmap v457: 97 pgs: 97 active+clean; 122 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.3 MiB/s wr, 95 op/s 2024-09-12T01:50:34.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.030+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e281090c0 con 0x7f7e28102570 2024-09-12T01:50:34.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.031+0000 7f7e1affd640 1 -- 172.21.15.122:0/4258983444 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f7e2409ac00 con 0x7f7e28102570 2024-09-12T01:50:34.032 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:50:34.032 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:50:34.032 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:50:34.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e140a2b80 msgr2=0x7f7e140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e140a2b80 0x7f7e140a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c006fd0 tx=0x7f7e1c008040 comp rx=0 tx=0).stop 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 msgr2=0x7f7e28197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 0x7f7e28197780 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400b4a0 tx=0x7f7e2400b960 comp rx=0 tx=0).stop 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 shutdown_connections 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e140a2b80 0x7f7e140a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e28102ef0 0x7f7e28197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:34.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28102570 0x7f7e28197780 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:34.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 --2- 172.21.15.122:0/4258983444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28100e20 0x7f7e28197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:34.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 >> 172.21.15.122:0/4258983444 conn(0x7f7e280fc7d0 msgr2=0x7f7e280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:34.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 shutdown_connections 2024-09-12T01:50:34.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:34.033+0000 7f7e2ff41640 1 -- 172.21.15.122:0/4258983444 wait complete. 2024-09-12T01:50:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:33 smithi151 ceph-mon[35229]: pgmap v457: 97 pgs: 97 active+clean; 122 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.3 MiB/s wr, 95 op/s 2024-09-12T01:50:34.815 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:34 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4258983444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:34.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 9.54 MB/sec execute 52 sec latency 19155.949 ms 2024-09-12T01:50:34.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id e8a7a64d --release --await' 2024-09-12T01:50:34.844 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id e8a7a64d --release --await 2024-09-12T01:50:34.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:34 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4258983444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:34 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4258983444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 -- 172.21.15.122:0/1240519522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4106f40 msgr2=0x7f68a410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 --2- 172.21.15.122:0/1240519522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4106f40 0x7f68a410eac0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f689c009ca0 tx=0x7f689c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 -- 172.21.15.122:0/1240519522 shutdown_connections 2024-09-12T01:50:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 --2- 172.21.15.122:0/1240519522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4106f40 0x7f68a410eac0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 --2- 172.21.15.122:0/1240519522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a41027c0 0x7f68a4106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 --2- 172.21.15.122:0/1240519522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68a4101e10 0x7f68a41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 -- 172.21.15.122:0/1240519522 >> 172.21.15.122:0/1240519522 conn(0x7f68a40fd8d0 msgr2=0x7f68a40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 -- 172.21.15.122:0/1240519522 shutdown_connections 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.781+0000 7f68aae7e640 1 -- 172.21.15.122:0/1240519522 wait complete. 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 Processor -- start 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 -- start start 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68a4101e10 0x7f68a4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 0x7f68a4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a4106f40 0x7f68a419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4111fa0 con 0x7f68a41027c0 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4111e20 con 0x7f68a4101e10 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68aae7e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4112120 con 0x7f68a4106f40 2024-09-12T01:50:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 0x7f68a4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 0x7f68a4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44528/0 (socket says 172.21.15.122:44528) 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68a3fff640 1 -- 172.21.15.122:0/2908623343 learned_addr learned my addr 172.21.15.122:0/2908623343 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.782+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68a4101e10 0x7f68a4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a941b640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a4106f40 0x7f68a419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a3fff640 1 -- 172.21.15.122:0/2908623343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a4106f40 msgr2=0x7f68a419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a3fff640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a4106f40 0x7f68a419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a3fff640 1 -- 172.21.15.122:0/2908623343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68a4101e10 msgr2=0x7f68a4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a3fff640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68a4101e10 0x7f68a4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a3fff640 1 -- 172.21.15.122:0/2908623343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a419d3b0 con 0x7f68a41027c0 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a941b640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a4106f40 0x7f68a419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a3fff640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 0x7f68a4198940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f689000d770 tx=0x7f689000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a1ffb640 1 -- 172.21.15.122:0/2908623343 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f68a41027c0 2024-09-12T01:50:35.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a419d6a0 con 0x7f68a41027c0 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a1ffb640 1 -- 172.21.15.122:0/2908623343 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6890005500 con 0x7f68a41027c0 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68a1ffb640 1 -- 172.21.15.122:0/2908623343 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890004e00 con 0x7f68a41027c0 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.783+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a41a4f80 con 0x7f68a41027c0 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.785+0000 7f68a1ffb640 1 -- 172.21.15.122:0/2908623343 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6890004fa0 con 0x7f68a41027c0 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 Processor -- start 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 -- start start 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a1ffb640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f68940a2b80 0x7f68940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 0x55cccc78f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f980 0x55cccc799d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc79c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cccc6f4620 con 0x55cccc78f060 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cccc69be50 con 0x55cccc79a240 2024-09-12T01:50:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68ad257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cccc6f05e0 con 0x55cccc78f980 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a941b640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f68940a2b80 0x7f68940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 0x55cccc78f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a1ffb640 1 -- 172.21.15.122:0/2908623343 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f68900ceaa0 con 0x7f68a41027c0 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 0x55cccc78f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44538/0 (socket says 172.21.15.122:44538) 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 -- 172.21.15.122:0/2920595965 learned_addr learned my addr 172.21.15.122:0/2920595965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a941b640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc79c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a3fff640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f980 0x55cccc799d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 -- 172.21.15.122:0/2920595965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f980 msgr2=0x55cccc799d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f980 0x55cccc799d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 -- 172.21.15.122:0/2920595965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 msgr2=0x55cccc79c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc79c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a8c1a640 1 -- 172.21.15.122:0/2920595965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccc68cd40 con 0x55cccc78f060 2024-09-12T01:50:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a3fff640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f980 0x55cccc799d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.786+0000 7f68a941b640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc79c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 0x55cccc78f440 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6898013ca0 tx=0x7f6898009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ceada0809757da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68a941b640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f68940a2b80 0x7f68940a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f689c006010 tx=0x7f689c009960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f688effd640 1 -- 172.21.15.122:0/2920595965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898009ca0 con 0x55cccc78f060 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f688effd640 1 -- 172.21.15.122:0/2920595965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6898009e40 con 0x55cccc78f060 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f688effd640 1 -- 172.21.15.122:0/2920595965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898003a50 con 0x55cccc78f060 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 -- 172.21.15.122:0/2920595965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 msgr2=0x55cccc78f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 0x55cccc78f440 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6898013ca0 tx=0x7f6898009b70 comp rx=0 tx=0).stop 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 -- 172.21.15.122:0/2920595965 shutdown_connections 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc79c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f980 0x55cccc799d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 --2- 172.21.15.122:0/2920595965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f060 0x55cccc78f440 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.787+0000 7f68ad257740 1 -- 172.21.15.122:0/2920595965 >> 172.21.15.122:0/2920595965 conn(0x55cccc78b5f0 msgr2=0x55cccc78b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 -- 172.21.15.122:0/2920595965 shutdown_connections 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 -- 172.21.15.122:0/2920595965 wait complete. 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 -- start start 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 Processor -- start 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f060 0x55cccc796fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc797520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc79a240 0x55cccc797a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cccc79ce10 con 0x55cccc78f980 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cccc67cf10 con 0x55cccc78f060 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cccc79cf90 con 0x55cccc79a240 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68ad257740 10 client.0 _ll_register_callbacks cb 0x55cccc6a0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:35.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.788+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc797520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc797520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44552/0 (socket says 172.21.15.122:44552) 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 -- 172.21.15.122:0/1865138989 learned_addr learned my addr 172.21.15.122:0/1865138989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f060 0x55cccc796fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a941b640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc79a240 0x55cccc797a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 -- 172.21.15.122:0/1865138989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc79a240 msgr2=0x55cccc797a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc79a240 0x55cccc797a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 -- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f060 msgr2=0x55cccc796fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f060 0x55cccc796fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 -- 172.21.15.122:0/1865138989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cccc403580 con 0x55cccc78f980 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68a3fff640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc797520 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6890021000 tx=0x7f689009af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4e0337f0344c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f6883fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f6883fff640 1 -- 172.21.15.122:0/1865138989 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890020050 con 0x55cccc78f980 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f6883fff640 1 -- 172.21.15.122:0/1865138989 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6890009cc0 con 0x55cccc78f980 2024-09-12T01:50:35.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cccc701b90 con 0x55cccc78f980 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f6883fff640 1 -- 172.21.15.122:0/1865138989 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68900046c0 con 0x55cccc78f980 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68ad257740 10 client.16248 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cccc4cd960 con 0x55cccc78f980 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.789+0000 7f6883fff640 1 -- 172.21.15.122:0/1865138989 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cccc701b90 con 0x55cccc78f980 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f6883fff640 1 -- 172.21.15.122:0/1865138989 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6890097930 con 0x55cccc78f980 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68ad257740 10 client.16248 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68ad257740 10 client.16248 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68ad257740 20 client.16248 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68ad257740 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc7a9b30 0x55cccc7abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68ad257740 4 client.16248 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cccc6fc0a0 con 0x55cccc7a9b30 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc7a9b30 0x55cccc7abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc7a9b30 0x55cccc7abfa0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaeaa9d37cbf0e25 server_cookie=5e65698cae55112f in_seq=0 out_seq=0 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.790+0000 7f6883fff640 10 client.16248 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f6883fff640 1 -- 172.21.15.122:0/1865138989 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cccc6fc0a0 con 0x55cccc7a9b30 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f6883fff640 10 client.16248 handle_command_reply: tid=0 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 client.16248 shutdown 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 msgr2=0x55cccc797520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc797520 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6890021000 tx=0x7f689009af10 comp rx=0 tx=0).stop 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 shutdown_connections 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc7a9b30 0x55cccc7abfa0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc79a240 0x55cccc797a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc797520 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 --2- 172.21.15.122:0/1865138989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f060 0x55cccc796fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f6883fff640 0 client.16248 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.791+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 >> 172.21.15.122:0/1865138989 conn(0x55cccc78b5f0 msgr2=0x55cccc7912d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.792+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 shutdown_connections 2024-09-12T01:50:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.792+0000 7f68ad257740 1 -- 172.21.15.122:0/1865138989 wait complete. 2024-09-12T01:50:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.792+0000 7f68ad257740 20 client.16248 trim_cache size 0 max 16384 2024-09-12T01:50:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.808+0000 7f68ad257740 1 Processor -- start 2024-09-12T01:50:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.808+0000 7f68ad257740 1 -- start start 2024-09-12T01:50:35.809 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:35 smithi122 ceph-mon[30925]: pgmap v458: 97 pgs: 97 active+clean; 119 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:50:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68ad257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc78a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68ad257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc7b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68ad257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc7b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68ad257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41adc90 con 0x55cccc78f980 2024-09-12T01:50:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68ad257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ab560 con 0x55cccc79a240 2024-09-12T01:50:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68ad257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41a9390 con 0x55cccc78f060 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc7b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc7b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44566/0 (socket says 172.21.15.122:44566) 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 -- 172.21.15.122:0/3076541521 learned_addr learned my addr 172.21.15.122:0/3076541521 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc78a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a941b640 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc7b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 -- 172.21.15.122:0/3076541521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 msgr2=0x55cccc78a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc78a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 -- 172.21.15.122:0/3076541521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 msgr2=0x55cccc7b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc7b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 -- 172.21.15.122:0/3076541521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccc78b420 con 0x55cccc78f980 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.809+0000 7f68a3fff640 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc7b1840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f68900b4250 tx=0x7f68900997d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94902674b55b60c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f688e7fc640 1 -- 172.21.15.122:0/3076541521 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890020070 con 0x55cccc78f980 2024-09-12T01:50:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f688e7fc640 1 -- 172.21.15.122:0/3076541521 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689009f670 con 0x55cccc78f980 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f688e7fc640 1 -- 172.21.15.122:0/3076541521 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890097930 con 0x55cccc78f980 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 -- 172.21.15.122:0/3076541521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 msgr2=0x55cccc7b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc7b1840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f68900b4250 tx=0x7f68900997d0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 -- 172.21.15.122:0/3076541521 shutdown_connections 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc79a240 0x55cccc7b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc78f980 0x55cccc7b1840 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 --2- 172.21.15.122:0/3076541521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc78a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 -- 172.21.15.122:0/3076541521 >> 172.21.15.122:0/3076541521 conn(0x55cccc78b5f0 msgr2=0x55cccc789f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 -- 172.21.15.122:0/3076541521 shutdown_connections 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.810+0000 7f68ad257740 1 -- 172.21.15.122:0/3076541521 wait complete. 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 -- start start 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 Processor -- start 2024-09-12T01:50:35.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc7a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f980 0x55cccc7a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 0x55cccc79d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41a9a20 con 0x55cccc79a240 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41aae90 con 0x55cccc78f980 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41aa0f0 con 0x55cccc78f060 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68ad257740 10 client.0 _ll_register_callbacks cb 0x7f689009b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68a941b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 0x55cccc79d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68a941b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 0x55cccc79d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44574/0 (socket says 172.21.15.122:44574) 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.811+0000 7f68a941b640 1 -- 172.21.15.122:0/3044502957 learned_addr learned my addr 172.21.15.122:0/3044502957 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc7a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a3fff640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f980 0x55cccc7a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a941b640 1 -- 172.21.15.122:0/3044502957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 msgr2=0x55cccc7a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a941b640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc7a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a941b640 1 -- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f980 msgr2=0x55cccc7a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a941b640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f980 0x55cccc7a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a941b640 1 -- 172.21.15.122:0/3044502957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cccc79dba0 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68a941b640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 0x55cccc79d3a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f689c02ade0 tx=0x7f689c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b842d2ef754451ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f6883fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f6883fff640 1 -- 172.21.15.122:0/3044502957 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c04e070 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f6883fff640 1 -- 172.21.15.122:0/3044502957 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f689c031d30 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cccc6785a0 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f6883fff640 1 -- 172.21.15.122:0/3044502957 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f689c03c110 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68ad257740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.812+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cccc815540 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f6883fff640 1 -- 172.21.15.122:0/3044502957 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cccc6785a0 con 0x55cccc79a240 2024-09-12T01:50:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f6883fff640 1 -- 172.21.15.122:0/3044502957 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f689c047040 con 0x55cccc79a240 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68ad257740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68ad257740 10 client.16254 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68ad257740 20 client.16254 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68ad257740 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc815cc0 0x55cccc8180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68ad257740 4 client.16254 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e8a7a64d", "release": true, "await": true}] 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e8a7a64d", "release": true, "await": true}) -- 0x55cccc6c5ce0 con 0x55cccc815cc0 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.813+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc815cc0 0x55cccc8180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:35.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.814+0000 7f6883fff640 10 client.16254 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:35.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.814+0000 7f68a8c1a640 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc815cc0 0x55cccc8180e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a141e699560fa5d3 server_cookie=436f11d5d43cb3fb in_seq=0 out_seq=0 2024-09-12T01:50:35.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f6883fff640 1 -- 172.21.15.122:0/3044502957 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cccc6c5ce0 con 0x55cccc815cc0 2024-09-12T01:50:35.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f6883fff640 10 client.16254 handle_command_reply: tid=0 2024-09-12T01:50:35.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 client.16254 shutdown 2024-09-12T01:50:35.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273269 9.36 MB/sec execute 53 sec latency 20155.988 ms 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 msgr2=0x55cccc79d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 0x55cccc79d3a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f689c02ade0 tx=0x7f689c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 shutdown_connections 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55cccc815cc0 0x55cccc8180e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f6883fff640 0 client.16254 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cccc79a240 0x55cccc79d3a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cccc78f980 0x55cccc7a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 --2- 172.21.15.122:0/3044502957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cccc78f060 0x55cccc7a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.823+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 >> 172.21.15.122:0/3044502957 conn(0x55cccc78b5f0 msgr2=0x55cccc7977c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 shutdown_connections 2024-09-12T01:50:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68ad257740 1 -- 172.21.15.122:0/3044502957 wait complete. 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68ad257740 20 client.16254 trim_cache size 0 max 16384 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 16, 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:50:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: "e8a7a64d": { 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 16, 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 140.2, 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:50:35.827 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:35.828 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:50:35.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f68940a2b80 msgr2=0x7f68940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f68940a2b80 0x7f68940a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f689c006010 tx=0x7f689c009960 comp rx=0 tx=0).stop 2024-09-12T01:50:35.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 msgr2=0x7f68a4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:35.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 0x7f68a4198940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f689000d770 tx=0x7f689000dc30 comp rx=0 tx=0).stop 2024-09-12T01:50:35.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 shutdown_connections 2024-09-12T01:50:35.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f68940a2b80 0x7f68940a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68a4106f40 0x7f68a419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a41027c0 0x7f68a4198940 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 --2- 172.21.15.122:0/2908623343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68a4101e10 0x7f68a4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:35.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 >> 172.21.15.122:0/2908623343 conn(0x7f68a40fd8d0 msgr2=0x7f68a40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:35.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.824+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 shutdown_connections 2024-09-12T01:50:35.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:35.825+0000 7f68aae7e640 1 -- 172.21.15.122:0/2908623343 wait complete. 2024-09-12T01:50:35.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:35 smithi183 ceph-mon[35127]: pgmap v458: 97 pgs: 97 active+clean; 119 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:50:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:35 smithi151 ceph-mon[35229]: pgmap v458: 97 pgs: 97 active+clean; 119 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:50:36.596 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e8a7a64d', total seconds elapsed: 23.7 2024-09-12T01:50:36.596 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T01:50:36.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:50:36.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 275722 9.36 MB/sec execute 54 sec latency 20157.322 ms 2024-09-12T01:50:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 -- 172.21.15.122:0/1786358505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101025c0 msgr2=0x7fa1101049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 --2- 172.21.15.122:0/1786358505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101025c0 0x7fa1101049b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa100009860 tx=0x7fa10002a340 comp rx=0 tx=0).stop 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 -- 172.21.15.122:0/1786358505 shutdown_connections 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 --2- 172.21.15.122:0/1786358505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa110107820 0x7fa110109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 --2- 172.21.15.122:0/1786358505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110104ef0 0x7fa1101072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 --2- 172.21.15.122:0/1786358505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1101025c0 0x7fa1101049b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 -- 172.21.15.122:0/1786358505 >> 172.21.15.122:0/1786358505 conn(0x7fa1100fc1e0 msgr2=0x7fa1100fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 -- 172.21.15.122:0/1786358505 shutdown_connections 2024-09-12T01:50:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.880+0000 7fa11540e640 1 -- 172.21.15.122:0/1786358505 wait complete. 2024-09-12T01:50:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.881+0000 7fa11540e640 1 Processor -- start 2024-09-12T01:50:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.881+0000 7fa11540e640 1 -- start start 2024-09-12T01:50:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa11540e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1101025c0 0x7fa110197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa11540e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 0x7fa110198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa11540e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110107820 0x7fa11019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa11540e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010d830 con 0x7fa110107820 2024-09-12T01:50:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa11540e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010d6b0 con 0x7fa110104ef0 2024-09-12T01:50:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa11540e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010d9b0 con 0x7fa1101025c0 2024-09-12T01:50:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1101025c0 0x7fa110197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110107820 0x7fa11019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 0x7fa110198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 0x7fa110198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45808/0 (socket says 172.21.15.122:45808) 2024-09-12T01:50:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1101025c0 0x7fa110197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41628/0 (socket says 172.21.15.122:41628) 2024-09-12T01:50:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 -- 172.21.15.122:0/2989905077 learned_addr learned my addr 172.21.15.122:0/2989905077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 -- 172.21.15.122:0/2989905077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1101025c0 msgr2=0x7fa110197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1101025c0 0x7fa110197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 -- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110107820 msgr2=0x7fa11019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110107820 0x7fa11019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 -- 172.21.15.122:0/2989905077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11019fa60 con 0x7fa110104ef0 2024-09-12T01:50:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110107820 0x7fa11019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 0x7fa110198380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e9f0 tx=0x7fa10400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa0f7fff640 1 -- 172.21.15.122:0/2989905077 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400ccc0 con 0x7fa110104ef0 2024-09-12T01:50:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa0f7fff640 1 -- 172.21.15.122:0/2989905077 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104010910 con 0x7fa110104ef0 2024-09-12T01:50:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.882+0000 7fa0f7fff640 1 -- 172.21.15.122:0/2989905077 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104018960 con 0x7fa110104ef0 2024-09-12T01:50:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.883+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11019fd50 con 0x7fa110104ef0 2024-09-12T01:50:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.883+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101a01f0 con 0x7fa110104ef0 2024-09-12T01:50:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa0f7fff640 1 -- 172.21.15.122:0/2989905077 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa104010ac0 con 0x7fa110104ef0 2024-09-12T01:50:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 Processor -- start 2024-09-12T01:50:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 -- start start 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac52da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac55830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55877abb12f0 con 0x55877ac4e2e0 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55877ab59270 con 0x55877ac4d9a0 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa118285740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55877abad2b0 con 0x55877ac53370 2024-09-12T01:50:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa10effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac52da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa0f7fff640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa0f80a2cb0 0x7fa0f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45820/0 (socket says 172.21.15.122:45820) 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa10effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac52da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44608/0 (socket says 172.21.15.122:44608) 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3041139476 learned_addr learned my addr 172.21.15.122:0/3041139476 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.885+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa0f80a2cb0 0x7fa0f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa0f7fff640 1 -- 172.21.15.122:0/2989905077 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa1040cf290 con 0x7fa110104ef0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3041139476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 msgr2=0x55877ac55830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac55830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3041139476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 msgr2=0x55877ac52da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac52da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3041139476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877ab58340 con 0x55877ac4d9a0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10effd640 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dda0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa108013e80 tx=0x7fa10800f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea082240cfe4aa56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa0f80a2cb0 0x7fa0f80a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa108008000 tx=0x7fa10801f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa0f57fa640 1 -- 172.21.15.122:0/3041139476 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108003e00 con 0x55877ac4d9a0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa0f57fa640 1 -- 172.21.15.122:0/3041139476 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa108022020 con 0x55877ac4d9a0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.886+0000 7fa0f57fa640 1 -- 172.21.15.122:0/3041139476 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10800e910 con 0x55877ac4d9a0 2024-09-12T01:50:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 -- 172.21.15.122:0/3041139476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 msgr2=0x55877ac4dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dda0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa108013e80 tx=0x7fa10800f7d0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 -- 172.21.15.122:0/3041139476 shutdown_connections 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac55830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 --2- 172.21.15.122:0/3041139476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dda0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 -- 172.21.15.122:0/3041139476 >> 172.21.15.122:0/3041139476 conn(0x55877ac49010 msgr2=0x55877ac49410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 -- 172.21.15.122:0/3041139476 shutdown_connections 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 -- 172.21.15.122:0/3041139476 wait complete. 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 -- start start 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.887+0000 7fa118285740 1 Processor -- start 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55877ac562e0 con 0x55877ac4e2e0 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55877ab3a290 con 0x55877ac4d9a0 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55877ac56460 con 0x55877ac53370 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 10 client.0 _ll_register_callbacks cb 0x55877ab6b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44616/0 (socket says 172.21.15.122:44616) 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 -- 172.21.15.122:0/3346148772 learned_addr learned my addr 172.21.15.122:0/3346148772 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10effd640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 -- 172.21.15.122:0/3346148772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 msgr2=0x55877ac5f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac5f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 -- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 msgr2=0x55877ac57b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac57b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 -- 172.21.15.122:0/3346148772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877a8c4210 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10effd640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac580b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e9c0 tx=0x7fa10409ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a300bba87e402c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa0f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3346148772 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104026040 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3346148772 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa10409b1f0 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa108022020 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3346148772 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa10409fea0 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 10 client.16266 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.888+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55877a990620 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3346148772 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa108022020 con 0x55877ac4e2e0 2024-09-12T01:50:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3346148772 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa1040978f0 con 0x55877ac4e2e0 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa118285740 10 client.16266 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa118285740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa118285740 20 client.16266 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa118285740 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877ac63740 0x55877ac65b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa118285740 4 client.16266 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55877abbea20 con 0x55877ac63740 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa10effd640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877ac63740 0x55877ac65b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa10effd640 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877ac63740 0x55877ac65b30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41347f37b6477606 server_cookie=4d237ae5f4268e35 in_seq=0 out_seq=0 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.889+0000 7fa0f2ff5640 10 client.16266 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3346148772 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55877abbea20 con 0x55877ac63740 2024-09-12T01:50:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa0f2ff5640 10 client.16266 handle_command_reply: tid=0 2024-09-12T01:50:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 client.16266 shutdown 2024-09-12T01:50:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 msgr2=0x55877ac580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac580b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e9c0 tx=0x7fa10409ba80 comp rx=0 tx=0).stop 2024-09-12T01:50:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 shutdown_connections 2024-09-12T01:50:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877ac63740 0x55877ac65b30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac53370 0x55877ac5f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac4e2e0 0x55877ac580b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa0f2ff5640 0 client.16266 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 --2- 172.21.15.122:0/3346148772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 >> 172.21.15.122:0/3346148772 conn(0x55877ac49010 msgr2=0x55877ac4a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 shutdown_connections 2024-09-12T01:50:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 1 -- 172.21.15.122:0/3346148772 wait complete. 2024-09-12T01:50:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.890+0000 7fa118285740 20 client.16266 trim_cache size 0 max 16384 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 Processor -- start 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 -- start start 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4e2e0 0x55877ac5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877ac480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101ab880 con 0x55877ac53370 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101a9150 con 0x55877ac4d9a0 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa118285740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101a6f80 con 0x55877ac4e2e0 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877ac480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877ac480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44620/0 (socket says 172.21.15.122:44620) 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 -- 172.21.15.122:0/2802182442 learned_addr learned my addr 172.21.15.122:0/2802182442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4e2e0 0x55877ac5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 -- 172.21.15.122:0/2802182442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4e2e0 msgr2=0x55877ac5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4e2e0 0x55877ac5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 -- 172.21.15.122:0/2802182442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 msgr2=0x55877ac4dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 -- 172.21.15.122:0/2802182442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877ac48e30 con 0x55877ac53370 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10e7fc640 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4e2e0 0x55877ac5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877ac480b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa108013e80 tx=0x7fa108005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d7eb3395c6059b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.905+0000 7fa0f4ff9640 1 -- 172.21.15.122:0/2802182442 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108003e00 con 0x55877ac53370 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa0f4ff9640 1 -- 172.21.15.122:0/2802182442 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa108022050 con 0x55877ac53370 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa0f4ff9640 1 -- 172.21.15.122:0/2802182442 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10800e4d0 con 0x55877ac53370 2024-09-12T01:50:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 -- 172.21.15.122:0/2802182442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 msgr2=0x55877ac480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877ac480b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa108013e80 tx=0x7fa108005c40 comp rx=0 tx=0).stop 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 -- 172.21.15.122:0/2802182442 shutdown_connections 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877ac480b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4e2e0 0x55877ac5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 --2- 172.21.15.122:0/2802182442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4d9a0 0x55877ac4dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 -- 172.21.15.122:0/2802182442 >> 172.21.15.122:0/2802182442 conn(0x55877ac49010 msgr2=0x55877ac479a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 -- 172.21.15.122:0/2802182442 shutdown_connections 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.906+0000 7fa118285740 1 -- 172.21.15.122:0/2802182442 wait complete. 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 -- start start 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 Processor -- start 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4d9a0 0x55877ac5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4e2e0 0x55877ac5c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877accc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101a7610 con 0x55877ac53370 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101a8a80 con 0x55877ac4e2e0 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1101a7ce0 con 0x55877ac4d9a0 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa118285740 10 client.0 _ll_register_callbacks cb 0x7fa1040977d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877accc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4d9a0 0x55877ac5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4d9a0 0x55877ac5c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41658/0 (socket says 172.21.15.122:41658) 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10e7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4e2e0 0x55877ac5c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10effd640 1 -- 172.21.15.122:0/3441546936 learned_addr learned my addr 172.21.15.122:0/3441546936 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877accc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44636/0 (socket says 172.21.15.122:44636) 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3441546936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4d9a0 msgr2=0x55877ac5c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4d9a0 0x55877ac5c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4e2e0 msgr2=0x55877ac5c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4e2e0 0x55877ac5c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 -- 172.21.15.122:0/3441546936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877accc930 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa10f7fe640 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877accc3f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa10800a010 tx=0x7fa108013ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0dea827306be699 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.907+0000 7fa0f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3441546936 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080120d0 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3441546936 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa10800e4d0 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa108022050 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3441546936 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa108012270 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 10 client.16272 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55877accdbf0 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3441546936 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa108022050 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3441546936 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa108042050 con 0x55877ac53370 2024-09-12T01:50:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 20 client.16272 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877acce340 0x55877acd0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 4 client.16272 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55877abb8d70 con 0x55877acce340 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.908+0000 7fa10effd640 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877acce340 0x55877acd0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.909+0000 7fa0f2ff5640 10 client.16272 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.909+0000 7fa10effd640 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877acce340 0x55877acd0760 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f881c0fa3d0504 server_cookie=cfc85bb416db03a7 in_seq=0 out_seq=0 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa0f2ff5640 1 -- 172.21.15.122:0/3441546936 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55877abb8d70 con 0x55877acce340 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa0f2ff5640 10 client.16272 handle_command_reply: tid=0 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 client.16272 shutdown 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 msgr2=0x55877accc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877accc3f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa10800a010 tx=0x7fa108013ae0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 shutdown_connections 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55877acce340 0x55877acd0760 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa0f2ff5640 0 client.16272 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55877ac53370 0x55877accc3f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55877ac4e2e0 0x55877ac5c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 --2- 172.21.15.122:0/3441546936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877ac4d9a0 0x55877ac5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 >> 172.21.15.122:0/3441546936 conn(0x55877ac49010 msgr2=0x55877ac481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 shutdown_connections 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 1 -- 172.21.15.122:0/3441546936 wait complete. 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa118285740 20 client.16272 trim_cache size 0 max 16384 2024-09-12T01:50:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:50:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:50:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:50:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa0f80a2cb0 msgr2=0x7fa0f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.918+0000 7fa11540e640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa0f80a2cb0 0x7fa0f80a5170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa108008000 tx=0x7fa10801f000 comp rx=0 tx=0).stop 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 msgr2=0x7fa110198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 0x7fa110198380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e9f0 tx=0x7fa10400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 shutdown_connections 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa0f80a2cb0 0x7fa0f80a5170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa110107820 0x7fa11019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa110104ef0 0x7fa110198380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 --2- 172.21.15.122:0/2989905077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1101025c0 0x7fa110197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 >> 172.21.15.122:0/2989905077 conn(0x7fa1100fc1e0 msgr2=0x7fa110102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 shutdown_connections 2024-09-12T01:50:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:36.919+0000 7fa11540e640 1 -- 172.21.15.122:0/2989905077 wait complete. 2024-09-12T01:50:37.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 -- 172.21.15.122:0/4239495537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 msgr2=0x7fdfc8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:37.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4239495537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc8101200 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009900 tx=0x7fdfbc02a480 comp rx=0 tx=0).stop 2024-09-12T01:50:37.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 -- 172.21.15.122:0/4239495537 shutdown_connections 2024-09-12T01:50:37.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4239495537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8102ef0 0x7fdfc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4239495537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfc8102570 0x7fdfc81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4239495537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc8101200 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 -- 172.21.15.122:0/4239495537 >> 172.21.15.122:0/4239495537 conn(0x7fdfc80fc7d0 msgr2=0x7fdfc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 -- 172.21.15.122:0/4239495537 shutdown_connections 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 -- 172.21.15.122:0/4239495537 wait complete. 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.348+0000 7fdfceecd640 1 Processor -- start 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 -- start start 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8102570 0x7fdfc8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfc8102ef0 0x7fdfc81a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8110f80 con 0x7fdfc8100e20 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8110e00 con 0x7fdfc8102ef0 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfceecd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8111100 con 0x7fdfc8102570 2024-09-12T01:50:37.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44648/0 (socket says 172.21.15.122:44648) 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 -- 172.21.15.122:0/4010708906 learned_addr learned my addr 172.21.15.122:0/4010708906 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfc7fff640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8102570 0x7fdfc8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfcd46a640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfc8102ef0 0x7fdfc81a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 -- 172.21.15.122:0/4010708906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8102570 msgr2=0x7fdfc8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8102570 0x7fdfc8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 -- 172.21.15.122:0/4010708906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfc8102ef0 msgr2=0x7fdfc81a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfc8102ef0 0x7fdfc81a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 -- 172.21.15.122:0/4010708906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc81a48b0 con 0x7fdfc8100e20 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfccc69640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc81992f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc006010 tx=0x7fdfbc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:37.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.349+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc03f070 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.350+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc81a4a80 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.350+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc031c30 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.350+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc035030 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.350+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc81a4f10 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.351+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8101200 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.352+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdfbc042070 con 0x7fdfc8100e20 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.352+0000 7fdfc5ffb640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdfb80a2b80 0x7fdfb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.353+0000 7fdfc7fff640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdfb80a2b80 0x7fdfb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:37.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.353+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdfbc0efba0 con 0x7fdfc8100e20 2024-09-12T01:50:37.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.353+0000 7fdfc7fff640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdfb80a2b80 0x7fdfb80a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806ab00 tx=0x7fdfb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:37.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.356+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfbc031de0 con 0x7fdfc8100e20 2024-09-12T01:50:37.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.564+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfc8109090 con 0x7fdfc8100e20 2024-09-12T01:50:37.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.564+0000 7fdfc5ffb640 1 -- 172.21.15.122:0/4010708906 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fdfbc0bc4b0 con 0x7fdfc8100e20 2024-09-12T01:50:37.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:50:37.565 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:50:37.565 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.566+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdfb80a2b80 msgr2=0x7fdfb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.566+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdfb80a2b80 0x7fdfb80a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806ab00 tx=0x7fdfb4005f90 comp rx=0 tx=0).stop 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.566+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 msgr2=0x7fdfc81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.566+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc81992f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc006010 tx=0x7fdfbc003c10 comp rx=0 tx=0).stop 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 shutdown_connections 2024-09-12T01:50:37.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdfb80a2b80 0x7fdfb80a5040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfc8102ef0 0x7fdfc81a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8102570 0x7fdfc8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 --2- 172.21.15.122:0/4010708906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc8100e20 0x7fdfc81992f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:37.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 >> 172.21.15.122:0/4010708906 conn(0x7fdfc80fc7d0 msgr2=0x7fdfc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:37.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 shutdown_connections 2024-09-12T01:50:37.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:37.567+0000 7fdfceecd640 1 -- 172.21.15.122:0/4010708906 wait complete. 2024-09-12T01:50:37.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277951 9.43 MB/sec execute 55 sec latency 31.961 ms 2024-09-12T01:50:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:37 smithi183 ceph-mon[35127]: pgmap v459: 97 pgs: 97 active+clean; 118 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.3 MiB/s wr, 88 op/s 2024-09-12T01:50:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:37 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4010708906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:37 smithi122 ceph-mon[30925]: pgmap v459: 97 pgs: 97 active+clean; 118 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.3 MiB/s wr, 88 op/s 2024-09-12T01:50:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:37 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4010708906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:37 smithi151 ceph-mon[35229]: pgmap v459: 97 pgs: 97 active+clean; 118 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.3 MiB/s wr, 88 op/s 2024-09-12T01:50:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:37 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4010708906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:38.330 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T01:50:38.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T01:50:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:38.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 280681 9.56 MB/sec execute 56 sec latency 74.891 ms 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/1367477032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 msgr2=0x7f0adc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/1367477032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc101200 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009900 tx=0x7f0ad802a480 comp rx=0 tx=0).stop 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/1367477032 shutdown_connections 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/1367477032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102ef0 0x7f0adc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/1367477032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102570 0x7f0adc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/1367477032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc101200 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/1367477032 >> 172.21.15.122:0/1367477032 conn(0x7f0adc0fc7d0 msgr2=0x7f0adc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/1367477032 shutdown_connections 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.082+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/1367477032 wait complete. 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 Processor -- start 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 -- start start 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102570 0x7f0adc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102ef0 0x7f0adc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc110f80 con 0x7f0adc100e20 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc110e00 con 0x7f0adc102ef0 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae3f0c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc111100 con 0x7f0adc102570 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44658/0 (socket says 172.21.15.122:44658) 2024-09-12T01:50:39.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2200285838 learned_addr learned my addr 172.21.15.122:0/2200285838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102570 0x7f0adc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.083+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102ef0 0x7f0adc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2200285838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102570 msgr2=0x7f0adc198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102570 0x7f0adc198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2200285838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102ef0 msgr2=0x7f0adc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102ef0 0x7f0adc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2200285838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc19d4c0 con 0x7f0adc100e20 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102570 0x7f0adc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102ef0 0x7f0adc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:39.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc198510 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8006010 tx=0x7f0ad8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:39.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ad2ffd640 1 -- 172.21.15.122:0/2200285838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad803f070 con 0x7f0adc100e20 2024-09-12T01:50:39.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ad2ffd640 1 -- 172.21.15.122:0/2200285838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad8031c30 con 0x7f0adc100e20 2024-09-12T01:50:39.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc19d750 con 0x7f0adc100e20 2024-09-12T01:50:39.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ad2ffd640 1 -- 172.21.15.122:0/2200285838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8035030 con 0x7f0adc100e20 2024-09-12T01:50:39.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.084+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc103680 con 0x7f0adc100e20 2024-09-12T01:50:39.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.087+0000 7f0ad2ffd640 1 -- 172.21.15.122:0/2200285838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0ad8042070 con 0x7f0adc100e20 2024-09-12T01:50:39.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.087+0000 7f0ae62e5740 1 Processor -- start 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 -- start start 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ad2ffd640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ac80a2c00 0x7f0ac80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a48621dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a4862284f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a486228a30 0x55a48622ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a486182160 con 0x55a486228a30 2024-09-12T01:50:39.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a486129720 con 0x55a48621d850 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a48617e120 con 0x55a48621e170 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ad2ffd640 1 -- 172.21.15.122:0/2200285838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0ad80ef620 con 0x7f0adc100e20 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae14a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a4862284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ac80a2c00 0x7f0ac80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a48621dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae14a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a4862284f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41694/0 (socket says 172.21.15.122:41694) 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.088+0000 7f0ae14a7640 1 -- 172.21.15.122:0/392912355 learned_addr learned my addr 172.21.15.122:0/392912355 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a486228a30 0x55a48622ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/392912355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 msgr2=0x55a4862284f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a4862284f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/392912355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a486228a30 msgr2=0x55a48622ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a486228a30 0x55a48622ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/392912355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a485ef3ba0 con 0x55a48621d850 2024-09-12T01:50:39.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ac80a2c00 0x7f0ac80a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009fd0 tx=0x7f0ad4011f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:39.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a48621dc30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80ed120 tx=0x7f0ad80058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90575aae6cacf56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:39.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ac7fff640 1 -- 172.21.15.122:0/392912355 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8049070 con 0x55a48621d850 2024-09-12T01:50:39.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ac7fff640 1 -- 172.21.15.122:0/392912355 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad8031de0 con 0x55a48621d850 2024-09-12T01:50:39.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.089+0000 7f0ac7fff640 1 -- 172.21.15.122:0/392912355 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80c4af0 con 0x55a48621d850 2024-09-12T01:50:39.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 -- 172.21.15.122:0/392912355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 msgr2=0x55a48621dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a48621dc30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80ed120 tx=0x7f0ad80058b0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 -- 172.21.15.122:0/392912355 shutdown_connections 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a486228a30 0x55a48622ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a4862284f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/392912355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a48621dc30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 -- 172.21.15.122:0/392912355 >> 172.21.15.122:0/392912355 conn(0x55a486219de0 msgr2=0x55a48621a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 -- 172.21.15.122:0/392912355 shutdown_connections 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 -- 172.21.15.122:0/392912355 wait complete. 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 -- start start 2024-09-12T01:50:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.090+0000 7f0ae62e5740 1 Processor -- start 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a4862259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a486225ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a486228a30 0x55a486226420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a48622b3c0 con 0x55a48621e170 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a48610a900 con 0x55a48621d850 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a48622b540 con 0x55a486228a30 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae62e5740 10 client.0 _ll_register_callbacks cb 0x55a486108780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a4862259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a4862259a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45908/0 (socket says 172.21.15.122:45908) 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/3900168895 learned_addr learned my addr 172.21.15.122:0/3900168895 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae24a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a486228a30 0x55a486226420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.091+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a486225ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/3900168895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a486228a30 msgr2=0x55a486226420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a486228a30 0x55a486226420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/3900168895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 msgr2=0x55a486225ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a486225ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/3900168895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48614ac50 con 0x55a48621d850 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a486228a30 0x55a486226420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a4862259a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80bbc30 tx=0x7f0ad8005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7876a6630972856d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ac57fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ac57fa640 1 -- 172.21.15.122:0/3900168895 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8049070 con 0x55a48621d850 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ac57fa640 1 -- 172.21.15.122:0/3900168895 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0ad803b460 con 0x55a48621d850 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad8031de0 con 0x55a48621d850 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ac57fa640 1 -- 172.21.15.122:0/3900168895 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ad80ce980 con 0x55a48621d850 2024-09-12T01:50:39.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae62e5740 10 client.25619 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a48622b3c0 con 0x55a48621d850 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.092+0000 7f0ac57fa640 1 -- 172.21.15.122:0/3900168895 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad8031de0 con 0x55a48621d850 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ac57fa640 1 -- 172.21.15.122:0/3900168895 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0ad80b85a0 con 0x55a48621d850 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae62e5740 10 client.25619 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae62e5740 10 client.25619 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae62e5740 20 client.25619 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862382f0 0x55a48623a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae62e5740 4 client.25619 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a48618f890 con 0x55a4862382f0 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862382f0 0x55a48623a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ac57fa640 10 client.25619 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.093+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862382f0 0x55a48623a710 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe8d938407a1996 server_cookie=4cb579a309b4e766 in_seq=0 out_seq=0 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ac57fa640 1 -- 172.21.15.122:0/3900168895 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a48618f890 con 0x55a4862382f0 2024-09-12T01:50:39.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ac57fa640 10 client.25619 handle_command_reply: tid=0 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 client.25619 shutdown 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 msgr2=0x55a4862259a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a4862259a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80bbc30 tx=0x7f0ad8005b70 comp rx=0 tx=0).stop 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 shutdown_connections 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862382f0 0x55a48623a710 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a486228a30 0x55a486226420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a486225ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ac57fa640 0 client.25619 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/3900168895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a48621d850 0x55a4862259a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 >> 172.21.15.122:0/3900168895 conn(0x55a486219de0 msgr2=0x55a48621fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 shutdown_connections 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.094+0000 7f0ae62e5740 1 -- 172.21.15.122:0/3900168895 wait complete. 2024-09-12T01:50:39.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.095+0000 7f0ae62e5740 20 client.25619 trim_cache size 0 max 16384 2024-09-12T01:50:39.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 Processor -- start 2024-09-12T01:50:39.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 -- start start 2024-09-12T01:50:39.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 0x55a486226270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a486218db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48623ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1b2090 con 0x55a48621d850 2024-09-12T01:50:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1af960 con 0x55a486228a30 2024-09-12T01:50:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.112+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1ad790 con 0x55a48621e170 2024-09-12T01:50:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 0x55a486226270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 0x55a486226270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44690/0 (socket says 172.21.15.122:44690) 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/975511311 learned_addr learned my addr 172.21.15.122:0/975511311 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48623ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a486218db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/975511311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 msgr2=0x55a486218db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a486218db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/975511311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 msgr2=0x55a48623ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48623ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/975511311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48621dfa0 con 0x55a48621d850 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48623ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 0x55a486226270 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c45c0 tx=0x7f0ad80ba440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4125d6263186a268 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ac77fe640 1 -- 172.21.15.122:0/975511311 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8031de0 con 0x55a48621d850 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ac77fe640 1 -- 172.21.15.122:0/975511311 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad80c4af0 con 0x55a48621d850 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.113+0000 7f0ac77fe640 1 -- 172.21.15.122:0/975511311 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80b85a0 con 0x55a48621d850 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 -- 172.21.15.122:0/975511311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 msgr2=0x55a486226270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 0x55a486226270 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c45c0 tx=0x7f0ad80ba440 comp rx=0 tx=0).stop 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 -- 172.21.15.122:0/975511311 shutdown_connections 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48623ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621e170 0x55a486218db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/975511311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621d850 0x55a486226270 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 -- 172.21.15.122:0/975511311 >> 172.21.15.122:0/975511311 conn(0x55a486219de0 msgr2=0x55a4862186a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 -- 172.21.15.122:0/975511311 shutdown_connections 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.114+0000 7f0ae62e5740 1 -- 172.21.15.122:0/975511311 wait complete. 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 -- start start 2024-09-12T01:50:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 Processor -- start 2024-09-12T01:50:39.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621d850 0x55a48622c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a48622cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48622d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1ade20 con 0x55a48621e170 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1af290 con 0x55a486228a30 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc1ae4f0 con 0x55a48621d850 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae62e5740 10 client.0 _ll_register_callbacks cb 0x7f0acc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae14a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a48622cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae24a9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48622d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae24a9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48622d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45916/0 (socket says 172.21.15.122:45916) 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae14a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a48622cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44694/0 (socket says 172.21.15.122:44694) 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae24a9640 1 -- 172.21.15.122:0/2499277998 learned_addr learned my addr 172.21.15.122:0/2499277998 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.115+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621d850 0x55a48622c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 msgr2=0x55a48622d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48622d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2499277998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 msgr2=0x55a48622cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a48622cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae1ca8640 1 -- 172.21.15.122:0/2499277998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4862a2280 con 0x55a48621d850 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae24a9640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48622d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a48622cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae1ca8640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621d850 0x55a48622c740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c45c0 tx=0x7f0ad80d1f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6613e2572e93a319 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ac57fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ac57fa640 1 -- 172.21.15.122:0/2499277998 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8031de0 con 0x55a48621d850 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ac57fa640 1 -- 172.21.15.122:0/2499277998 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0ad80d03a0 con 0x55a48621d850 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad80c4af0 con 0x55a48621d850 2024-09-12T01:50:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ac57fa640 1 -- 172.21.15.122:0/2499277998 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ad80ce750 con 0x55a48621d850 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae62e5740 10 client.25603 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.116+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4862a2280 con 0x55a48621d850 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ac57fa640 1 -- 172.21.15.122:0/2499277998 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad80c4af0 con 0x55a48621d850 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ac57fa640 1 -- 172.21.15.122:0/2499277998 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0ad80ce0b0 con 0x55a48621d850 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae62e5740 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae62e5740 10 client.25603 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae62e5740 20 client.25603 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862a3820 0x55a4862a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae62e5740 4 client.25603 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55a486189be0 con 0x55a4862a3820 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862a3820 0x55a4862a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ae14a7640 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862a3820 0x55a4862a5c10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156f185f0b960019 server_cookie=8278271eb2c23bb0 in_seq=0 out_seq=0 2024-09-12T01:50:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.117+0000 7f0ac57fa640 10 client.25603 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ac57fa640 1 -- 172.21.15.122:0/2499277998 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a486189be0 con 0x55a4862a3820 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ac57fa640 10 client.25603 handle_command_reply: tid=0 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 client.25603 shutdown 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621d850 msgr2=0x55a48622c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621d850 0x55a48622c740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c45c0 tx=0x7f0ad80d1f60 comp rx=0 tx=0).stop 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 shutdown_connections 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a4862a3820 0x55a4862a5c10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a486228a30 0x55a48622d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ac57fa640 0 client.25603 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a48621e170 0x55a48622cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 --2- 172.21.15.122:0/2499277998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a48621d850 0x55a48622c740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 >> 172.21.15.122:0/2499277998 conn(0x55a486219de0 msgr2=0x55a486218ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 shutdown_connections 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 1 -- 172.21.15.122:0/2499277998 wait complete. 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.182+0000 7f0ae62e5740 20 client.25603 trim_cache size 0 max 16384 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 18, 2024-09-12T01:50:39.184 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "94ceae38": { 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 18, 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 140.2, 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:39.185 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:39.186 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ac80a2c00 msgr2=0x7f0ac80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ac80a2c00 0x7f0ac80a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009fd0 tx=0x7f0ad4011f50 comp rx=0 tx=0).stop 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 msgr2=0x7f0adc198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc198510 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8006010 tx=0x7f0ad8003c10 comp rx=0 tx=0).stop 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 shutdown_connections 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ac80a2c00 0x7f0ac80a50c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0adc102ef0 0x7f0adc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc102570 0x7f0adc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 --2- 172.21.15.122:0/2200285838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0adc100e20 0x7f0adc198510 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 >> 172.21.15.122:0/2200285838 conn(0x7f0adc0fc7d0 msgr2=0x7f0adc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 shutdown_connections 2024-09-12T01:50:39.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:39.183+0000 7f0ae3f0c640 1 -- 172.21.15.122:0/2200285838 wait complete. 2024-09-12T01:50:39.823 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 9.43 MB/sec execute 57 sec latency 697.926 ms 2024-09-12T01:50:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:39 smithi183 ceph-mon[35127]: pgmap v460: 97 pgs: 97 active+clean; 116 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.5 MiB/s wr, 100 op/s 2024-09-12T01:50:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:39 smithi122 ceph-mon[30925]: pgmap v460: 97 pgs: 97 active+clean; 116 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.5 MiB/s wr, 100 op/s 2024-09-12T01:50:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:39 smithi151 ceph-mon[35229]: pgmap v460: 97 pgs: 97 active+clean; 116 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 2.5 MiB/s wr, 100 op/s 2024-09-12T01:50:40.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '94ceae38', quiesce took 3.6 seconds. Will release after: 16.549999999999997 2024-09-12T01:50:40.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 9.26 MB/sec execute 58 sec latency 1698.046 ms 2024-09-12T01:50:41.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 9.11 MB/sec execute 59 sec latency 2698.143 ms 2024-09-12T01:50:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:41 smithi183 ceph-mon[35127]: pgmap v461: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-09-12T01:50:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:41 smithi122 ceph-mon[30925]: pgmap v461: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-09-12T01:50:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:41 smithi151 ceph-mon[35229]: pgmap v461: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.3 MiB/s wr, 143 op/s 2024-09-12T01:50:42.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.96 MB/sec execute 60 sec latency 3698.241 ms 2024-09-12T01:50:43.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.81 MB/sec execute 61 sec latency 4698.331 ms 2024-09-12T01:50:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:43 smithi183 ceph-mon[35127]: pgmap v462: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:50:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:43 smithi122 ceph-mon[30925]: pgmap v462: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:50:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:43 smithi151 ceph-mon[35229]: pgmap v462: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:50:44.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.67 MB/sec execute 62 sec latency 5698.447 ms 2024-09-12T01:50:45.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.53 MB/sec execute 63 sec latency 6698.545 ms 2024-09-12T01:50:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:45 smithi183 ceph-mon[35127]: pgmap v463: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-09-12T01:50:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:45 smithi122 ceph-mon[30925]: pgmap v463: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-09-12T01:50:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:45 smithi151 ceph-mon[35229]: pgmap v463: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-09-12T01:50:46.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.40 MB/sec execute 64 sec latency 7698.581 ms 2024-09-12T01:50:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:46 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:50:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:46 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:50:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:46 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:50:47.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.27 MB/sec execute 65 sec latency 8698.665 ms 2024-09-12T01:50:47.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:47 smithi183 ceph-mon[35127]: pgmap v464: 97 pgs: 97 active+clean; 120 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-09-12T01:50:47.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:50:47.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:50:47.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:50:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:47 smithi122 ceph-mon[30925]: pgmap v464: 97 pgs: 97 active+clean; 120 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-09-12T01:50:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:50:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:50:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:50:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:47 smithi151 ceph-mon[35229]: pgmap v464: 97 pgs: 97 active+clean; 120 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-09-12T01:50:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:50:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:50:48.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:50:48.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.14 MB/sec execute 66 sec latency 9698.734 ms 2024-09-12T01:50:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:48 smithi183 ceph-mon[35127]: pgmap v465: 97 pgs: 97 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 168 op/s 2024-09-12T01:50:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:48 smithi122 ceph-mon[30925]: pgmap v465: 97 pgs: 97 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 168 op/s 2024-09-12T01:50:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:48 smithi151 ceph-mon[35229]: pgmap v465: 97 pgs: 97 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 168 op/s 2024-09-12T01:50:49.824 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 8.02 MB/sec execute 67 sec latency 10698.817 ms 2024-09-12T01:50:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 -- 172.21.15.122:0/919280148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c1016d0 msgr2=0x7f3e2c108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 --2- 172.21.15.122:0/919280148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c1016d0 0x7f3e2c108f70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009900 tx=0x7f3e2402a410 comp rx=0 tx=0).stop 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 -- 172.21.15.122:0/919280148 shutdown_connections 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 --2- 172.21.15.122:0/919280148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c1016d0 0x7f3e2c108f70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 --2- 172.21.15.122:0/919280148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c100d10 0x7f3e2c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 --2- 172.21.15.122:0/919280148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 0x7f3e2c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 -- 172.21.15.122:0/919280148 >> 172.21.15.122:0/919280148 conn(0x7f3e2c0fb970 msgr2=0x7f3e2c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 -- 172.21.15.122:0/919280148 shutdown_connections 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.018+0000 7f3e30e77640 1 -- 172.21.15.122:0/919280148 wait complete. 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 Processor -- start 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 -- start start 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 0x7f3e2c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c1016d0 0x7f3e2c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 0x7f3e2c19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c10c6e0 con 0x7f3e2c100d10 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c10c560 con 0x7f3e2c101fe0 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e30e77640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c10c860 con 0x7f3e2c1016d0 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e2a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 0x7f3e2c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e2a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 0x7f3e2c197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45130/0 (socket says 172.21.15.122:45130) 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.019+0000 7f3e2a59c640 1 -- 172.21.15.122:0/2421933294 learned_addr learned my addr 172.21.15.122:0/2421933294 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e29d9b640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c1016d0 0x7f3e2c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 0x7f3e2c19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2a59c640 1 -- 172.21.15.122:0/2421933294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c1016d0 msgr2=0x7f3e2c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c1016d0 0x7f3e2c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2a59c640 1 -- 172.21.15.122:0/2421933294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 msgr2=0x7f3e2c19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 0x7f3e2c19f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2a59c640 1 -- 172.21.15.122:0/2421933294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c19f990 con 0x7f3e2c100d10 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 0x7f3e2c19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 0x7f3e2c197d70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009fd0 tx=0x7f3e1c00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e1b7fe640 1 -- 172.21.15.122:0/2421933294 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c011470 con 0x7f3e2c100d10 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c19fc80 con 0x7f3e2c100d10 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e1b7fe640 1 -- 172.21.15.122:0/2421933294 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1c011af0 con 0x7f3e2c100d10 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e1b7fe640 1 -- 172.21.15.122:0/2421933294 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01ab90 con 0x7f3e2c100d10 2024-09-12T01:50:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.020+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c1a01d0 con 0x7f3e2c100d10 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 Processor -- start 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e1b7fe640 1 -- 172.21.15.122:0/2421933294 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3e1c011610 con 0x7f3e2c100d10 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 -- start start 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e1bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e1c370 0x55e233e1e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e233d7a2f0 con 0x55e233e169a0 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e233d22270 con 0x55e233e1c370 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.023+0000 7f3e33c09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e233d762b0 con 0x55e233e172e0 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e16da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e16da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45132/0 (socket says 172.21.15.122:45132) 2024-09-12T01:50:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/3571236823 learned_addr learned my addr 172.21.15.122:0/3571236823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e29d9b640 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e1c370 0x55e233e1e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e1bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e1b7fe640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3e140a2c30 0x7f3e140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/3571236823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 msgr2=0x55e233e1bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e1bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/3571236823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e1c370 msgr2=0x55e233e1e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e1c370 0x55e233e1e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/3571236823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e233d21340 con 0x55e233e169a0 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e1b7fe640 1 -- 172.21.15.122:0/2421933294 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3e1c0cf610 con 0x7f3e2c100d10 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e16da0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009230 tx=0x7f3e24032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cbb611610dfcf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3e140a2c30 0x7f3e140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.024+0000 7f3e18ff9640 1 -- 172.21.15.122:0/3571236823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2403f070 con 0x55e233e169a0 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e18ff9640 1 -- 172.21.15.122:0/3571236823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e24005040 con 0x55e233e169a0 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e18ff9640 1 -- 172.21.15.122:0/3571236823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24035070 con 0x55e233e169a0 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 -- 172.21.15.122:0/3571236823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 msgr2=0x55e233e16da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3e140a2c30 0x7f3e140a50f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3e24005ab0 tx=0x7f3e240035f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e16da0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009230 tx=0x7f3e24032d20 comp rx=0 tx=0).stop 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 -- 172.21.15.122:0/3571236823 shutdown_connections 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e1c370 0x55e233e1e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e1bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 --2- 172.21.15.122:0/3571236823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e16da0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 -- 172.21.15.122:0/3571236823 >> 172.21.15.122:0/3571236823 conn(0x55e233e12010 msgr2=0x55e233e12410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 -- 172.21.15.122:0/3571236823 shutdown_connections 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.025+0000 7f3e33c09740 1 -- 172.21.15.122:0/3571236823 wait complete. 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 -- start start 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 Processor -- start 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e169a0 0x55e233e20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e233e1f2e0 con 0x55e233e1c370 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e233d03290 con 0x55e233e172e0 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e233e1f460 con 0x55e233e169a0 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e33c09740 10 client.0 _ll_register_callbacks cb 0x55e233d34140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e2ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e2ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e280c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45140/0 (socket says 172.21.15.122:45140) 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2170817953 learned_addr learned my addr 172.21.15.122:0/2170817953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.026+0000 7f3e29d9b640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e169a0 0x55e233e20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2170817953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e169a0 msgr2=0x55e233e20b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e169a0 0x55e233e20b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 msgr2=0x55e233e21080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e21080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2170817953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e233a8d210 con 0x55e233e1c370 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e169a0 0x55e233e20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e280c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3e24004de0 tx=0x7f3e24044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cac1a531b39182c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2170817953 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2405e060 con 0x55e233e1c370 2024-09-12T01:50:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2170817953 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3e24005400 con 0x55e233e1c370 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e24005040 con 0x55e233e1c370 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2170817953 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3e2403f050 con 0x55e233e1c370 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e33c09740 10 client.16308 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e233b59620 con 0x55e233e1c370 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.027+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2170817953 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e24005040 con 0x55e233e1c370 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2170817953 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3e24063070 con 0x55e233e1c370 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e33c09740 10 client.16308 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e33c09740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e33c09740 20 client.16308 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e2c410 0x55e233e2e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e33c09740 4 client.16308 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e233d87a20 con 0x55e233e2c410 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e2c410 0x55e233e2e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e11ffb640 10 client.16308 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.028+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e2c410 0x55e233e2e860 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af3996ff1f114dc server_cookie=b001af14bd16db27 in_seq=0 out_seq=0 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2170817953 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e233d87a20 con 0x55e233e2c410 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e11ffb640 10 client.16308 handle_command_reply: tid=0 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 client.16308 shutdown 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 msgr2=0x55e233e280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e280c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3e24004de0 tx=0x7f3e24044000 comp rx=0 tx=0).stop 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 shutdown_connections 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e2c410 0x55e233e2e860 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e280c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e21080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2170817953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e169a0 0x55e233e20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e11ffb640 0 client.16308 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.029+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 >> 172.21.15.122:0/2170817953 conn(0x55e233e12010 msgr2=0x55e233e13820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.030+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 shutdown_connections 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.030+0000 7f3e33c09740 1 -- 172.21.15.122:0/2170817953 wait complete. 2024-09-12T01:50:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.030+0000 7f3e33c09740 20 client.16308 trim_cache size 0 max 16384 2024-09-12T01:50:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 Processor -- start 2024-09-12T01:50:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 -- start start 2024-09-12T01:50:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e12ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e13400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e1c370 0x55e233e13940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ab7d0 con 0x55e233e169a0 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1a90a0 con 0x55e233e172e0 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e33c09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1a6ed0 con 0x55e233e1c370 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e2a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e12ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e2a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e12ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45146/0 (socket says 172.21.15.122:45146) 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.044+0000 7f3e2a59c640 1 -- 172.21.15.122:0/627316175 learned_addr learned my addr 172.21.15.122:0/627316175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e2a59c640 1 -- 172.21.15.122:0/627316175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e1c370 msgr2=0x55e233e13940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e1c370 0x55e233e13940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e2a59c640 1 -- 172.21.15.122:0/627316175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 msgr2=0x55e233e13400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e13400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e2a59c640 1 -- 172.21.15.122:0/627316175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e233e12ca0 con 0x55e233e169a0 2024-09-12T01:50:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e12ec0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0cd110 tx=0x7f3e1c0cf890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9ed30ac1eb13f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e13fff640 1 -- 172.21.15.122:0/627316175 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c09c570 con 0x55e233e169a0 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e13fff640 1 -- 172.21.15.122:0/627316175 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1c09be70 con 0x55e233e169a0 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.045+0000 7f3e13fff640 1 -- 172.21.15.122:0/627316175 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01ad30 con 0x55e233e169a0 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- 172.21.15.122:0/627316175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 msgr2=0x55e233e12ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e12ec0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0cd110 tx=0x7f3e1c0cf890 comp rx=0 tx=0).stop 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- 172.21.15.122:0/627316175 shutdown_connections 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e1c370 0x55e233e13940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e172e0 0x55e233e13400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- 172.21.15.122:0/627316175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e169a0 0x55e233e12ec0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- 172.21.15.122:0/627316175 >> 172.21.15.122:0/627316175 conn(0x55e233e12010 msgr2=0x55e233e20b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- 172.21.15.122:0/627316175 shutdown_connections 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- 172.21.15.122:0/627316175 wait complete. 2024-09-12T01:50:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- start start 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 Processor -- start 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e169a0 0x55e233e923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e99920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1a7560 con 0x55e233e1c370 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1a89d0 con 0x55e233e169a0 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1a7c30 con 0x55e233e172e0 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e33c09740 10 client.0 _ll_register_callbacks cb 0x7f3e20004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e2ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e99920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e2ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e99920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45156/0 (socket says 172.21.15.122:45156) 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2614287129 learned_addr learned my addr 172.21.15.122:0/2614287129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.046+0000 7f3e29d9b640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e169a0 0x55e233e923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2614287129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 msgr2=0x55e233e928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e169a0 msgr2=0x55e233e923a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e169a0 0x55e233e923a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2ad9d640 1 -- 172.21.15.122:0/2614287129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e233e99f30 con 0x55e233e1c370 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e29d9b640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e2ad9d640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e99920 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3e2402a930 tx=0x7f3e24004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de246c84da12e1cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2614287129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2405e070 con 0x55e233e1c370 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2614287129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3e24050690 con 0x55e233e1c370 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1c09be70 con 0x55e233e1c370 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2614287129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3e24035040 con 0x55e233e1c370 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e33c09740 10 client.16314 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e233e9b240 con 0x55e233e1c370 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2614287129 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1c09be70 con 0x55e233e1c370 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2614287129 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3e24063070 con 0x55e233e1c370 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e33c09740 10 client.16314 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e33c09740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.047+0000 7f3e33c09740 20 client.16314 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.048+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e9ba90 0x55e233e9deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.048+0000 7f3e33c09740 4 client.16314 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.048+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e233d81d70 con 0x55e233e9ba90 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.048+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e9ba90 0x55e233e9deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:50.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.048+0000 7f3e11ffb640 10 client.16314 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.048+0000 7f3e2a59c640 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e9ba90 0x55e233e9deb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d1ec7fdab44572 server_cookie=7293352972c35844 in_seq=0 out_seq=0 2024-09-12T01:50:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e11ffb640 1 -- 172.21.15.122:0/2614287129 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e233d81d70 con 0x55e233e9ba90 2024-09-12T01:50:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e11ffb640 10 client.16314 handle_command_reply: tid=0 2024-09-12T01:50:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 client.16314 shutdown 2024-09-12T01:50:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 msgr2=0x55e233e99920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e99920 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3e2402a930 tx=0x7f3e24004970 comp rx=0 tx=0).stop 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 shutdown_connections 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e233e9ba90 0x55e233e9deb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e233e1c370 0x55e233e99920 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e233e172e0 0x55e233e928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 --2- 172.21.15.122:0/2614287129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e233e169a0 0x55e233e923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e11ffb640 0 client.16314 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 >> 172.21.15.122:0/2614287129 conn(0x55e233e12010 msgr2=0x55e233e109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 shutdown_connections 2024-09-12T01:50:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 1 -- 172.21.15.122:0/2614287129 wait complete. 2024-09-12T01:50:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.058+0000 7f3e33c09740 20 client.16314 trim_cache size 0 max 16384 2024-09-12T01:50:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:50:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:50:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:50:50.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3e140a2c30 msgr2=0x7f3e140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3e140a2c30 0x7f3e140a50f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3e24005ab0 tx=0x7f3e240035f0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 msgr2=0x7f3e2c197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 0x7f3e2c197d70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009fd0 tx=0x7f3e1c00c270 comp rx=0 tx=0).stop 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 shutdown_connections 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3e140a2c30 0x7f3e140a50f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e2c101fe0 0x7f3e2c19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e2c1016d0 0x7f3e2c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 --2- 172.21.15.122:0/2421933294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100d10 0x7f3e2c197d70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 >> 172.21.15.122:0/2421933294 conn(0x7f3e2c0fb970 msgr2=0x7f3e2c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 shutdown_connections 2024-09-12T01:50:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:50:50.059+0000 7f3e30e77640 1 -- 172.21.15.122:0/2421933294 wait complete. 2024-09-12T01:50:50.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.90 MB/sec execute 68 sec latency 11698.934 ms 2024-09-12T01:50:51.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.79 MB/sec execute 69 sec latency 12699.064 ms 2024-09-12T01:50:51.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:51 smithi151 ceph-mon[35229]: pgmap v466: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-09-12T01:50:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:51 smithi183 ceph-mon[35127]: pgmap v466: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-09-12T01:50:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:51 smithi122 ceph-mon[30925]: pgmap v466: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-09-12T01:50:52.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.68 MB/sec execute 70 sec latency 13699.128 ms 2024-09-12T01:50:53.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.57 MB/sec execute 71 sec latency 14699.188 ms 2024-09-12T01:50:53.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:53 smithi151 ceph-mon[35229]: pgmap v467: 97 pgs: 97 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-12T01:50:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:53 smithi183 ceph-mon[35127]: pgmap v467: 97 pgs: 97 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-12T01:50:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:53 smithi122 ceph-mon[30925]: pgmap v467: 97 pgs: 97 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 2.1 MiB/s wr, 92 op/s 2024-09-12T01:50:54.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.46 MB/sec execute 72 sec latency 15699.296 ms 2024-09-12T01:50:55.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.36 MB/sec execute 73 sec latency 16699.369 ms 2024-09-12T01:50:55.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:55 smithi151 ceph-mon[35229]: pgmap v468: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-12T01:50:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:55 smithi183 ceph-mon[35127]: pgmap v468: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-12T01:50:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:55 smithi122 ceph-mon[30925]: pgmap v468: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-12T01:50:56.733 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:50:56.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.26 MB/sec execute 74 sec latency 17699.425 ms 2024-09-12T01:50:57.000 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:57.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:57 smithi122 ceph-mon[30925]: pgmap v469: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-09-12T01:50:57.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 -- 172.21.15.122:0/1533629124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104100e20 msgr2=0x7fd104101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:57.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1533629124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104100e20 0x7fd104101200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009900 tx=0x7fd0f802a480 comp rx=0 tx=0).stop 2024-09-12T01:50:57.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 -- 172.21.15.122:0/1533629124 shutdown_connections 2024-09-12T01:50:57.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1533629124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 0x7fd10410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1533629124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104102570 0x7fd1041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1533629124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104100e20 0x7fd104101200 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 -- 172.21.15.122:0/1533629124 >> 172.21.15.122:0/1533629124 conn(0x7fd1040fc7d0 msgr2=0x7fd1040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 -- 172.21.15.122:0/1533629124 shutdown_connections 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.745+0000 7fd10ac40640 1 -- 172.21.15.122:0/1533629124 wait complete. 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 Processor -- start 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 -- start start 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104100e20 0x7fd104199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 0x7fd104199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 0x7fd1041a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd104110f50 con 0x7fd104102570 2024-09-12T01:50:57.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd104110dd0 con 0x7fd104102ef0 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd10ac40640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041110d0 con 0x7fd104100e20 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 0x7fd104199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 0x7fd104199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54788/0 (socket says 172.21.15.122:54788) 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd103fff640 1 -- 172.21.15.122:0/1743012285 learned_addr learned my addr 172.21.15.122:0/1743012285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.746+0000 7fd1089dc640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104100e20 0x7fd104199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd1091dd640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 0x7fd1041a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd103fff640 1 -- 172.21.15.122:0/1743012285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104100e20 msgr2=0x7fd104199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd103fff640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104100e20 0x7fd104199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd103fff640 1 -- 172.21.15.122:0/1743012285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 msgr2=0x7fd1041a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd103fff640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 0x7fd1041a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd103fff640 1 -- 172.21.15.122:0/1743012285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041a47e0 con 0x7fd104102570 2024-09-12T01:50:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd1091dd640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 0x7fd1041a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd103fff640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 0x7fd104199760 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c920 tx=0x7fd0f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0007c70 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041a49b0 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f0007e10 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0018890 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.747+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a4ef0 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.748+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd104101200 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.749+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd0f0018a70 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.750+0000 7fd101ffb640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0f40a2b80 0x7fd0f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.750+0000 7fd1089dc640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0f40a2b80 0x7fd0f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.750+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd0f00cf190 con 0x7fd104102570 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.751+0000 7fd1089dc640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0f40a2b80 0x7fd0f40a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.758+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0f0014030 con 0x7fd104102570 2024-09-12T01:50:57.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.16 MB/sec execute 75 sec latency 18699.456 ms 2024-09-12T01:50:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:57 smithi151 ceph-mon[35229]: pgmap v469: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-09-12T01:50:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:57 smithi183 ceph-mon[35127]: pgmap v469: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-09-12T01:50:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.968+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1041a51e0 con 0x7fd104102570 2024-09-12T01:50:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.969+0000 7fd101ffb640 1 -- 172.21.15.122:0/1743012285 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fd0f009c0f0 con 0x7fd104102570 2024-09-12T01:50:57.969 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:50:57.969 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:50:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:50:57.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0f40a2b80 msgr2=0x7fd0f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:57.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0f40a2b80 0x7fd0f40a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f80058d0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 msgr2=0x7fd104199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 0x7fd104199760 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c920 tx=0x7fd0f000cde0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 shutdown_connections 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0f40a2b80 0x7fd0f40a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd104102ef0 0x7fd1041a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd104102570 0x7fd104199760 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 --2- 172.21.15.122:0/1743012285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104100e20 0x7fd104199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 >> 172.21.15.122:0/1743012285 conn(0x7fd1040fc7d0 msgr2=0x7fd1041067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 shutdown_connections 2024-09-12T01:50:57.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:57.971+0000 7fd10ac40640 1 -- 172.21.15.122:0/1743012285 wait complete. 2024-09-12T01:50:58.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:58 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1743012285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:58.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281310 7.07 MB/sec execute 76 sec latency 19699.562 ms 2024-09-12T01:50:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:58 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1743012285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:58.895 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 94ceae38 --release --await' 2024-09-12T01:50:58.895 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 94ceae38 --release --await 2024-09-12T01:50:58.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:58 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1743012285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:50:59.141 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 -- 172.21.15.122:0/1352379339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98101bc0 msgr2=0x7fec9810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1352379339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98101bc0 0x7fec9810d910 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fec90009ca0 tx=0x7fec9002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 -- 172.21.15.122:0/1352379339 shutdown_connections 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1352379339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98101bc0 0x7fec9810d910 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1352379339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101240 0x7fec98101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1352379339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98105a90 0x7fec98105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 -- 172.21.15.122:0/1352379339 >> 172.21.15.122:0/1352379339 conn(0x7fec98077210 msgr2=0x7fec980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 -- 172.21.15.122:0/1352379339 shutdown_connections 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.660+0000 7fec9ef19640 1 -- 172.21.15.122:0/1352379339 wait complete. 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 Processor -- start 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 -- start start 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98101240 0x7fec98198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101bc0 0x7fec98198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 0x7fec9819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec98112130 con 0x7fec98105a90 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec98111fb0 con 0x7fec98101240 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ef19640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec981122b0 con 0x7fec98101bc0 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 0x7fec9819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 0x7fec9819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54794/0 (socket says 172.21.15.122:54794) 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 -- 172.21.15.122:0/1377329736 learned_addr learned my addr 172.21.15.122:0/1377329736 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98101240 0x7fec98198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101bc0 0x7fec98198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 -- 172.21.15.122:0/1377329736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101bc0 msgr2=0x7fec98198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101bc0 0x7fec98198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 -- 172.21.15.122:0/1377329736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98101240 msgr2=0x7fec98198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98101240 0x7fec98198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9d4b6640 1 -- 172.21.15.122:0/1377329736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9819d510 con 0x7fec98105a90 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101bc0 0x7fec98198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.661+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98101240 0x7fec98198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:50:59.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.662+0000 7fec9d4b6640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 0x7fec9819ce30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fec900097b0 tx=0x7fec90003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.662+0000 7fec8dffb640 1 -- 172.21.15.122:0/1377329736 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90031810 con 0x7fec98105a90 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.662+0000 7fec8dffb640 1 -- 172.21.15.122:0/1377329736 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec900319b0 con 0x7fec98105a90 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.662+0000 7fec8dffb640 1 -- 172.21.15.122:0/1377329736 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90030c90 con 0x7fec98105a90 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.662+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9819d7a0 con 0x7fec98105a90 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.662+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a50e0 con 0x7fec98105a90 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.665+0000 7fec8dffb640 1 -- 172.21.15.122:0/1377329736 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fec90004040 con 0x7fec98105a90 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.665+0000 7feca12f2740 1 Processor -- start 2024-09-12T01:50:59.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.665+0000 7fec8dffb640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec800a2c80 0x7fec800a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.665+0000 7feca12f2740 1 -- start start 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec800a2c80 0x7fec800a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8dffb640 1 -- 172.21.15.122:0/1377329736 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fec900f08d0 con 0x7fec98105a90 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7feca12f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff3f20 0x562491ff4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7feca12f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x562492003110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7feca12f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x562492005a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7feca12f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562491f59620 con 0x562491ff4840 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7feca12f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562491f00e50 con 0x562491ff3f20 2024-09-12T01:50:59.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7feca12f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562491f555e0 con 0x562492003650 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec9d4b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x562492005a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec9d4b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x562492005a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:40522/0 (socket says 172.21.15.122:40522) 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec9d4b6640 1 -- 172.21.15.122:0/725821976 learned_addr learned my addr 172.21.15.122:0/725821976 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x562492003110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff3f20 0x562491ff4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec800a2c80 0x7fec800a5140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fec88006fd0 tx=0x7fec8800ad10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 -- 172.21.15.122:0/725821976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 msgr2=0x562492005a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x562492005a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 -- 172.21.15.122:0/725821976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff3f20 msgr2=0x562491ff4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff3f20 0x562491ff4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 -- 172.21.15.122:0/725821976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562491ef1d40 con 0x562491ff4840 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.666+0000 7fec8ffff640 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x562492003110 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fec980ff710 tx=0x7fec88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57786077e254b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:59.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.667+0000 7fec7effd640 1 -- 172.21.15.122:0/725821976 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88023070 con 0x562491ff4840 2024-09-12T01:50:59.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.667+0000 7fec7effd640 1 -- 172.21.15.122:0/725821976 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8801a040 con 0x562491ff4840 2024-09-12T01:50:59.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.667+0000 7fec7effd640 1 -- 172.21.15.122:0/725821976 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88015680 con 0x562491ff4840 2024-09-12T01:50:59.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.667+0000 7feca12f2740 1 -- 172.21.15.122:0/725821976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 msgr2=0x562492003110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.667+0000 7feca12f2740 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x562492003110 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fec980ff710 tx=0x7fec88017000 comp rx=0 tx=0).stop 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- 172.21.15.122:0/725821976 shutdown_connections 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x562492005a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x562492003110 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 --2- 172.21.15.122:0/725821976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff3f20 0x562491ff4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- 172.21.15.122:0/725821976 >> 172.21.15.122:0/725821976 conn(0x562491ff05f0 msgr2=0x562491ff09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- 172.21.15.122:0/725821976 shutdown_connections 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- 172.21.15.122:0/725821976 wait complete. 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- start start 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 Processor -- start 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff4840 0x562492000b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492001080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562492006220 con 0x562491ff3f20 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562491ee1f10 con 0x562492003650 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624920063a0 con 0x562491ff4840 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.668+0000 7feca12f2740 10 client.0 _ll_register_callbacks cb 0x562491f05db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54826/0 (socket says 172.21.15.122:54826) 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 -- 172.21.15.122:0/1069626691 learned_addr learned my addr 172.21.15.122:0/1069626691 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff4840 0x562492000b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9d4b6640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492001080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 -- 172.21.15.122:0/1069626691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff4840 msgr2=0x562492000b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff4840 0x562492000b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 -- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 msgr2=0x562492001080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492001080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 -- 172.21.15.122:0/1069626691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562491c68580 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000600 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fec9400f530 tx=0x7fec9400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a752ae8fafe3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec77fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec77fff640 1 -- 172.21.15.122:0/1069626691 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94013850 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec77fff640 1 -- 172.21.15.122:0/1069626691 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fec940139f0 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec8801a040 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7fec77fff640 1 -- 172.21.15.122:0/1069626691 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec94013bf0 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7feca12f2740 10 client.16332 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.669+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562491d32960 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7fec77fff640 1 -- 172.21.15.122:0/1069626691 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8801a040 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7fec77fff640 1 -- 172.21.15.122:0/1069626691 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fec9401b400 con 0x562491ff3f20 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7feca12f2740 10 client.16332 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:59.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7feca12f2740 10 client.16332 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7feca12f2740 20 client.16332 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7feca12f2740 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492015750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7feca12f2740 4 client.16332 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562491f66b90 con 0x562492013320 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.670+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492015750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.671+0000 7fec8ffff640 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492015750 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5df8e81988ed164 server_cookie=2af325395655de7a in_seq=0 out_seq=0 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.671+0000 7fec77fff640 10 client.16332 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:59.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.672+0000 7fec77fff640 1 -- 172.21.15.122:0/1069626691 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562491f66b90 con 0x562492013320 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.672+0000 7fec77fff640 10 client.16332 handle_command_reply: tid=0 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.672+0000 7feca12f2740 1 client.16332 shutdown 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 msgr2=0x562492000600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000600 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fec9400f530 tx=0x7fec9400f9f0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 shutdown_connections 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492015750 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7fec77fff640 0 client.16332 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492001080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff4840 0x562492000b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 --2- 172.21.15.122:0/1069626691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000600 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 >> 172.21.15.122:0/1069626691 conn(0x562491ff05f0 msgr2=0x562491ff6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:59.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 shutdown_connections 2024-09-12T01:50:59.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 1 -- 172.21.15.122:0/1069626691 wait complete. 2024-09-12T01:50:59.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.673+0000 7feca12f2740 20 client.16332 trim_cache size 0 max 16384 2024-09-12T01:50:59.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 Processor -- start 2024-09-12T01:50:59.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 -- start start 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff4840 0x56249201b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x56249201b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec981add30 con 0x562491ff3f20 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec981ab600 con 0x562491ff4840 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7feca12f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a9450 con 0x562492003650 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54836/0 (socket says 172.21.15.122:54836) 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 -- 172.21.15.122:0/2083599454 learned_addr learned my addr 172.21.15.122:0/2083599454 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 -- 172.21.15.122:0/2083599454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 msgr2=0x56249201b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x56249201b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 -- 172.21.15.122:0/2083599454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff4840 msgr2=0x56249201b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff4840 0x56249201b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 -- 172.21.15.122:0/2083599454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562491fff5b0 con 0x562491ff3f20 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000c30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fec94000c00 tx=0x7fec94006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1880d0fa802cf2b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.690+0000 7fec7e7fc640 1 -- 172.21.15.122:0/2083599454 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94034070 con 0x562491ff3f20 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7fec7e7fc640 1 -- 172.21.15.122:0/2083599454 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9401b600 con 0x562491ff3f20 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7fec7e7fc640 1 -- 172.21.15.122:0/2083599454 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9401be60 con 0x562491ff3f20 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 -- 172.21.15.122:0/2083599454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 msgr2=0x562492000c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000c30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fec94000c00 tx=0x7fec94006be0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 -- 172.21.15.122:0/2083599454 shutdown_connections 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562492003650 0x56249201b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562491ff4840 0x56249201b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 --2- 172.21.15.122:0/2083599454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff3f20 0x562492000c30 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 -- 172.21.15.122:0/2083599454 >> 172.21.15.122:0/2083599454 conn(0x562491ff05f0 msgr2=0x562491fef750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 -- 172.21.15.122:0/2083599454 shutdown_connections 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 -- 172.21.15.122:0/2083599454 wait complete. 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.691+0000 7feca12f2740 1 -- start start 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 Processor -- start 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff3f20 0x56249200ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x56249200bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492008040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a9ac0 con 0x562491ff4840 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec981aaf30 con 0x562492003650 2024-09-12T01:50:59.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec981aa190 con 0x562491ff3f20 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7feca12f2740 10 client.0 _ll_register_callbacks cb 0x7fec9400aed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x56249200bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x56249200bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54842/0 (socket says 172.21.15.122:54842) 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 -- 172.21.15.122:0/3217465987 learned_addr learned my addr 172.21.15.122:0/3217465987 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff3f20 0x56249200ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec9d4b6640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492008040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 -- 172.21.15.122:0/3217465987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff3f20 msgr2=0x56249200ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff3f20 0x56249200ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 -- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 msgr2=0x562492008040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492008040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.692+0000 7fec8ffff640 1 -- 172.21.15.122:0/3217465987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562492008870 con 0x562491ff4840 2024-09-12T01:50:59.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec8ffff640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x56249200bf50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fec8800ca60 tx=0x7fec88004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e376e91699a99f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec77fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec77fff640 1 -- 172.21.15.122:0/3217465987 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88023040 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec77fff640 1 -- 172.21.15.122:0/3217465987 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fec8801e430 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec9401b600 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec77fff640 1 -- 172.21.15.122:0/3217465987 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec8801e630 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 10 client.16338 fetch_fsmap learned FSMap version 30 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56249207ed30 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec77fff640 1 -- 172.21.15.122:0/3217465987 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9401b600 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7fec77fff640 1 -- 172.21.15.122:0/3217465987 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fec8803b030 con 0x562491ff4840 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 10 client.16338 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 10 client.16338 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 20 client.16338 populate_metadata read hostname 'smithi122' 2024-09-12T01:50:59.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492081100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:50:59.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 4 client.16338 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "94ceae38", "release": true, "await": true}] 2024-09-12T01:50:59.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.693+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "94ceae38", "release": true, "await": true}) -- 0x562491f610a0 con 0x562492013320 2024-09-12T01:50:59.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.694+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492081100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:50:59.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.694+0000 7fec9ccb5640 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492081100 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5190cef8af7e23 server_cookie=72635a1de7d72440 in_seq=0 out_seq=0 2024-09-12T01:50:59.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.694+0000 7fec77fff640 10 client.16338 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:59.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7fec77fff640 1 -- 172.21.15.122:0/3217465987 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562491f610a0 con 0x562492013320 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7fec77fff640 10 client.16338 handle_command_reply: tid=0 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 client.16338 shutdown 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 msgr2=0x56249200bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x56249200bf50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fec8800ca60 tx=0x7fec88004660 comp rx=0 tx=0).stop 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 shutdown_connections 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562492013320 0x562492081100 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562492003650 0x562492008040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562491ff4840 0x56249200bf50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 --2- 172.21.15.122:0/3217465987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562491ff3f20 0x56249200ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 >> 172.21.15.122:0/3217465987 conn(0x562491ff05f0 msgr2=0x56249200e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:59.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7fec77fff640 0 client.16338 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 shutdown_connections 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.702+0000 7feca12f2740 1 -- 172.21.15.122:0/3217465987 wait complete. 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7feca12f2740 20 client.16338 trim_cache size 0 max 16384 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 20, 2024-09-12T01:50:59.704 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "94ceae38": { 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 20, 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 140.2, 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:50:59.705 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:50:59.706 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:50:59.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec800a2c80 msgr2=0x7fec800a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec800a2c80 0x7fec800a5140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fec88006fd0 tx=0x7fec8800ad10 comp rx=0 tx=0).stop 2024-09-12T01:50:59.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 msgr2=0x7fec9819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:50:59.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 0x7fec9819ce30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fec900097b0 tx=0x7fec90003f10 comp rx=0 tx=0).stop 2024-09-12T01:50:59.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 shutdown_connections 2024-09-12T01:50:59.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fec800a2c80 0x7fec800a5140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec98105a90 0x7fec9819ce30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec98101bc0 0x7fec98198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 --2- 172.21.15.122:0/1377329736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec98101240 0x7fec98198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:50:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 >> 172.21.15.122:0/1377329736 conn(0x7fec98077210 msgr2=0x7fec98103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:50:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 shutdown_connections 2024-09-12T01:50:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:50:59.703+0000 7fec9ef19640 1 -- 172.21.15.122:0/1377329736 wait complete. 2024-09-12T01:50:59.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281738 7.02 MB/sec execute 77 sec latency 20580.258 ms 2024-09-12T01:50:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:50:59 smithi151 ceph-mon[35229]: pgmap v470: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 779 KiB/s wr, 46 op/s 2024-09-12T01:50:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:50:59 smithi183 ceph-mon[35127]: pgmap v470: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 779 KiB/s wr, 46 op/s 2024-09-12T01:50:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:50:59 smithi122 ceph-mon[30925]: pgmap v470: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 779 KiB/s wr, 46 op/s 2024-09-12T01:51:00.552 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '94ceae38', total seconds elapsed: 24.0 2024-09-12T01:51:00.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 284451 7.11 MB/sec execute 78 sec latency 10.366 ms 2024-09-12T01:51:01.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 287613 7.26 MB/sec execute 79 sec latency 9.665 ms 2024-09-12T01:51:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:01 smithi151 ceph-mon[35229]: pgmap v471: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 501 KiB/s wr, 31 op/s 2024-09-12T01:51:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:01 smithi183 ceph-mon[35127]: pgmap v471: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 501 KiB/s wr, 31 op/s 2024-09-12T01:51:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:01 smithi122 ceph-mon[30925]: pgmap v471: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 501 KiB/s wr, 31 op/s 2024-09-12T01:51:02.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 290520 7.34 MB/sec execute 80 sec latency 10.169 ms 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 -- 172.21.15.122:0/4141790968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14081025f0 msgr2=0x7f1408108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 --2- 172.21.15.122:0/4141790968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14081025f0 0x7f1408108e80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1400009ca0 tx=0x7f140002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 -- 172.21.15.122:0/4141790968 shutdown_connections 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 --2- 172.21.15.122:0/4141790968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14081025f0 0x7f1408108e80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 --2- 172.21.15.122:0/4141790968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1408101c30 0x7f14081020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 --2- 172.21.15.122:0/4141790968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408100a40 0x7f1408100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 -- 172.21.15.122:0/4141790968 >> 172.21.15.122:0/4141790968 conn(0x7f14080fc1e0 msgr2=0x7f14080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 -- 172.21.15.122:0/4141790968 shutdown_connections 2024-09-12T01:51:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 -- 172.21.15.122:0/4141790968 wait complete. 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.153+0000 7f140dd86640 1 Processor -- start 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 -- start start 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 0x7f140806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101c30 0x7f140819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14081025f0 0x7f140819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f140810c6a0 con 0x7f1408100a40 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f140810c520 con 0x7f1408101c30 2024-09-12T01:51:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f140dd86640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f140810c820 con 0x7f14081025f0 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 0x7f140806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f1406ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101c30 0x7f140819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 0x7f140806a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59458/0 (socket says 172.21.15.122:59458) 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 -- 172.21.15.122:0/610025660 learned_addr learned my addr 172.21.15.122:0/610025660 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f1407fff640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14081025f0 0x7f140819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 -- 172.21.15.122:0/610025660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14081025f0 msgr2=0x7f140819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14081025f0 0x7f140819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 -- 172.21.15.122:0/610025660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101c30 msgr2=0x7f140819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101c30 0x7f140819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f14077fe640 1 -- 172.21.15.122:0/610025660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140819f9e0 con 0x7f1408100a40 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.154+0000 7f1406ffd640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101c30 0x7f140819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.155+0000 7f14077fe640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 0x7f140806a410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f13f800e9c0 tx=0x7f13f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.155+0000 7f1404ff9640 1 -- 172.21.15.122:0/610025660 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f800cd80 con 0x7f1408100a40 2024-09-12T01:51:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.155+0000 7f1404ff9640 1 -- 172.21.15.122:0/610025660 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f80035f0 con 0x7f1408100a40 2024-09-12T01:51:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.155+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140819fc70 con 0x7f1408100a40 2024-09-12T01:51:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.155+0000 7f1404ff9640 1 -- 172.21.15.122:0/610025660 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8010690 con 0x7f1408100a40 2024-09-12T01:51:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.155+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14081a01c0 con 0x7f1408100a40 2024-09-12T01:51:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.157+0000 7f1404ff9640 1 -- 172.21.15.122:0/610025660 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f13f8010830 con 0x7f1408100a40 2024-09-12T01:51:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.157+0000 7f1410bf8740 1 Processor -- start 2024-09-12T01:51:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.157+0000 7f1410bf8740 1 -- start start 2024-09-12T01:51:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x55785378d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378d6f0 0x5578537921b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853794c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1410bf8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5578536ec2f0 con 0x557853792780 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1410bf8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557853694270 con 0x55785378d6f0 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1410bf8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578536e82b0 con 0x55785378cdb0 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853794c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853794c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59472/0 (socket says 172.21.15.122:59472) 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 -- 172.21.15.122:0/3172694354 learned_addr learned my addr 172.21.15.122:0/3172694354 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f14077fe640 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378d6f0 0x5578537921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1407fff640 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x55785378d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1404ff9640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f00a2cb0 0x7f13f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 -- 172.21.15.122:0/3172694354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 msgr2=0x55785378d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x55785378d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 -- 172.21.15.122:0/3172694354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378d6f0 msgr2=0x5578537921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378d6f0 0x5578537921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 -- 172.21.15.122:0/3172694354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557853693340 con 0x557853792780 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1407fff640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f00a2cb0 0x7f13f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1404ff9640 1 -- 172.21.15.122:0/610025660 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f13f80cf3a0 con 0x7f1408100a40 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.158+0000 7f1406ffd640 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853794c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00bfd0 tx=0x7f13fc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c9c13eb63ab10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f13edffb640 1 -- 172.21.15.122:0/3172694354 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc005220 con 0x557853792780 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f13edffb640 1 -- 172.21.15.122:0/3172694354 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13fc004c80 con 0x557853792780 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f13edffb640 1 -- 172.21.15.122:0/3172694354 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc011c40 con 0x557853792780 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 -- 172.21.15.122:0/3172694354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 msgr2=0x557853794c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853794c40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00bfd0 tx=0x7f13fc0108e0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1407fff640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f00a2cb0 0x7f13f00a5170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1400006010 tx=0x7f140003f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 -- 172.21.15.122:0/3172694354 shutdown_connections 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853794c40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378d6f0 0x5578537921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 --2- 172.21.15.122:0/3172694354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x55785378d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.159+0000 7f1410bf8740 1 -- 172.21.15.122:0/3172694354 >> 172.21.15.122:0/3172694354 conn(0x557853784010 msgr2=0x557853784410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 -- 172.21.15.122:0/3172694354 shutdown_connections 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 -- 172.21.15.122:0/3172694354 wait complete. 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 -- start start 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 Processor -- start 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x557853796fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 0x5578537974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557853795750 con 0x55785378d6f0 2024-09-12T01:51:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557853675290 con 0x557853792780 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578537958d0 con 0x55785378cdb0 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1410bf8740 10 client.0 _ll_register_callbacks cb 0x5578536a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1406ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 0x5578537974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1406ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 0x5578537974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59482/0 (socket says 172.21.15.122:59482) 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1406ffd640 1 -- 172.21.15.122:0/2724886736 learned_addr learned my addr 172.21.15.122:0/2724886736 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f14077fe640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x557853796fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.160+0000 7f1407fff640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f14077fe640 1 -- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 msgr2=0x55785379e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f14077fe640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f14077fe640 1 -- 172.21.15.122:0/2724886736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 msgr2=0x5578537974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f14077fe640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 0x5578537974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f14077fe640 1 -- 172.21.15.122:0/2724886736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578533ff210 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f1406ffd640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 0x5578537974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f14077fe640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x557853796fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13f8098060 tx=0x7f13f8097fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2be701f3e6f9e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f13e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f13e6ffd640 1 -- 172.21.15.122:0/2724886736 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80b2060 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13fc004c80 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f13e6ffd640 1 -- 172.21.15.122:0/2724886736 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f13f80037a0 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f13e6ffd640 1 -- 172.21.15.122:0/2724886736 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f13f809b560 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f1410bf8740 10 client.25642 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.161+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578534cb620 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f13e6ffd640 1 -- 172.21.15.122:0/2724886736 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13fc004c80 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f13e6ffd640 1 -- 172.21.15.122:0/2724886736 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f13f809b050 con 0x55785378cdb0 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1410bf8740 10 client.25642 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1410bf8740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1410bf8740 20 client.25642 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1410bf8740 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5578537a2bc0 0x5578537a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1410bf8740 4 client.25642 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578536f9a20 con 0x5578537a2bc0 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1406ffd640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5578537a2bc0 0x5578537a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f13e6ffd640 10 client.25642 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.162+0000 7f1406ffd640 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5578537a2bc0 0x5578537a4fb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1478034b5841dc server_cookie=aa6f63a2c431b07c in_seq=0 out_seq=0 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f13e6ffd640 1 -- 172.21.15.122:0/2724886736 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578536f9a20 con 0x5578537a2bc0 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f13e6ffd640 10 client.25642 handle_command_reply: tid=0 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 client.25642 shutdown 2024-09-12T01:51:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 msgr2=0x557853796fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x557853796fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13f8098060 tx=0x7f13f8097fa0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 shutdown_connections 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5578537a2bc0 0x5578537a4fb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378d6f0 0x5578537974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 --2- 172.21.15.122:0/2724886736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378cdb0 0x557853796fa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f13e6ffd640 0 client.25642 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 >> 172.21.15.122:0/2724886736 conn(0x557853784010 msgr2=0x557853785820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 shutdown_connections 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 1 -- 172.21.15.122:0/2724886736 wait complete. 2024-09-12T01:51:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.163+0000 7f1410bf8740 20 client.25642 trim_cache size 0 max 16384 2024-09-12T01:51:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 Processor -- start 2024-09-12T01:51:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 -- start start 2024-09-12T01:51:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378cdb0 0x55785378d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853785400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14081a9660 con 0x557853792780 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14081a6f30 con 0x55785378cdb0 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1410bf8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14081a4d60 con 0x55785378d6f0 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853785400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f14077fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378cdb0 0x55785378d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1406ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853785400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59488/0 (socket says 172.21.15.122:59488) 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 -- 172.21.15.122:0/1741641874 learned_addr learned my addr 172.21.15.122:0/1741641874 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 -- 172.21.15.122:0/1741641874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 msgr2=0x55785379e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 -- 172.21.15.122:0/1741641874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378cdb0 msgr2=0x55785378d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378cdb0 0x55785378d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1407fff640 1 -- 172.21.15.122:0/1741641874 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557853797640 con 0x557853792780 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.179+0000 7f1406ffd640 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1407fff640 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853785400 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1400035000 tx=0x7f1400043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59d017ef729c6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f13ed7fa640 1 -- 172.21.15.122:0/1741641874 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140002fc90 con 0x557853792780 2024-09-12T01:51:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f13ed7fa640 1 -- 172.21.15.122:0/1741641874 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400036040 con 0x557853792780 2024-09-12T01:51:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f13ed7fa640 1 -- 172.21.15.122:0/1741641874 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000466a0 con 0x557853792780 2024-09-12T01:51:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1410bf8740 1 -- 172.21.15.122:0/1741641874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 msgr2=0x557853785400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1410bf8740 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853785400 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1400035000 tx=0x7f1400043000 comp rx=0 tx=0).stop 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1410bf8740 1 -- 172.21.15.122:0/1741641874 shutdown_connections 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1410bf8740 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557853792780 0x557853785400 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1410bf8740 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.180+0000 7f1410bf8740 1 --2- 172.21.15.122:0/1741641874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55785378cdb0 0x55785378d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- 172.21.15.122:0/1741641874 >> 172.21.15.122:0/1741641874 conn(0x557853784010 msgr2=0x5578537831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- 172.21.15.122:0/1741641874 shutdown_connections 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- 172.21.15.122:0/1741641874 wait complete. 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- start start 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 Processor -- start 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 0x55785379bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14081a53f0 con 0x55785378cdb0 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14081a6860 con 0x557853792780 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14081a5ac0 con 0x55785378d6f0 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f1410bf8740 10 client.0 _ll_register_callbacks cb 0x7f13f80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.181+0000 7f14077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 0x55785379bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f14077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 0x55785379bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59492/0 (socket says 172.21.15.122:59492) 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f14077fe640 1 -- 172.21.15.122:0/38765586 learned_addr learned my addr 172.21.15.122:0/38765586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1406ffd640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 -- 172.21.15.122:0/38765586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 msgr2=0x55785379c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 -- 172.21.15.122:0/38765586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 msgr2=0x55785379bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 0x55785379bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 -- 172.21.15.122:0/38765586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785379d670 con 0x557853792780 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1406ffd640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f14077fe640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 0x55785379bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1407fff640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379d060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1400031ae0 tx=0x7f1400009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b184b3e6b86db768 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f13e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f13e6ffd640 1 -- 172.21.15.122:0/38765586 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000098b0 con 0x557853792780 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f13e6ffd640 1 -- 172.21.15.122:0/38765586 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f14000307d0 con 0x557853792780 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.182+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1400036040 con 0x557853792780 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f13e6ffd640 1 -- 172.21.15.122:0/38765586 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14000309d0 con 0x557853792780 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 10 client.25670 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55785380d890 con 0x557853792780 2024-09-12T01:51:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f13e6ffd640 1 -- 172.21.15.122:0/38765586 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400036040 con 0x557853792780 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f13e6ffd640 1 -- 172.21.15.122:0/38765586 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1400063030 con 0x557853792780 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 20 client.25670 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55785380e230 0x557853810650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 4 client.25670 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578536f3d70 con 0x55785380e230 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f14077fe640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55785380e230 0x557853810650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f14077fe640 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55785380e230 0x557853810650 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=456f3d307e78cf57 server_cookie=d4787ff4145242f6 in_seq=0 out_seq=0 2024-09-12T01:51:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.183+0000 7f13e6ffd640 10 client.25670 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f13e6ffd640 1 -- 172.21.15.122:0/38765586 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578536f3d70 con 0x55785380e230 2024-09-12T01:51:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f13e6ffd640 10 client.25670 handle_command_reply: tid=0 2024-09-12T01:51:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 client.25670 shutdown 2024-09-12T01:51:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 msgr2=0x55785379d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379d060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1400031ae0 tx=0x7f1400009360 comp rx=0 tx=0).stop 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 shutdown_connections 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55785380e230 0x557853810650 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557853792780 0x55785379d060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55785378d6f0 0x55785379c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 --2- 172.21.15.122:0/38765586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55785378cdb0 0x55785379bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f13e6ffd640 0 client.25670 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 >> 172.21.15.122:0/38765586 conn(0x557853784010 msgr2=0x5578537843f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 shutdown_connections 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 1 -- 172.21.15.122:0/38765586 wait complete. 2024-09-12T01:51:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.192+0000 7f1410bf8740 20 client.25670 trim_cache size 0 max 16384 2024-09-12T01:51:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:51:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:51:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:51:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:51:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f00a2cb0 msgr2=0x7f13f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f00a2cb0 0x7f13f00a5170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1400006010 tx=0x7f140003f040 comp rx=0 tx=0).stop 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 msgr2=0x7f140806a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 0x7f140806a410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f13f800e9c0 tx=0x7f13f800ee80 comp rx=0 tx=0).stop 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 shutdown_connections 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13f00a2cb0 0x7f13f00a5170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14081025f0 0x7f140819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1408101c30 0x7f140819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 --2- 172.21.15.122:0/610025660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1408100a40 0x7f140806a410 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 >> 172.21.15.122:0/610025660 conn(0x7f14080fc1e0 msgr2=0x7f1408104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 shutdown_connections 2024-09-12T01:51:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:03.193+0000 7f140dd86640 1 -- 172.21.15.122:0/610025660 wait complete. 2024-09-12T01:51:03.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 293343 7.41 MB/sec execute 81 sec latency 11.961 ms 2024-09-12T01:51:03.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:03 smithi151 ceph-mon[35229]: pgmap v472: 97 pgs: 97 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 356 KiB/s wr, 25 op/s 2024-09-12T01:51:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:03 smithi183 ceph-mon[35127]: pgmap v472: 97 pgs: 97 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 356 KiB/s wr, 25 op/s 2024-09-12T01:51:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:03 smithi122 ceph-mon[30925]: pgmap v472: 97 pgs: 97 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 356 KiB/s wr, 25 op/s 2024-09-12T01:51:04.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295682 7.49 MB/sec execute 82 sec latency 14.954 ms 2024-09-12T01:51:05.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 298584 7.60 MB/sec execute 83 sec latency 10.852 ms 2024-09-12T01:51:05.975 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:05 smithi183 ceph-mon[35127]: pgmap v473: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 559 KiB/s wr, 75 op/s 2024-09-12T01:51:05.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:05 smithi122 ceph-mon[30925]: pgmap v473: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 559 KiB/s wr, 75 op/s 2024-09-12T01:51:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:05 smithi151 ceph-mon[35229]: pgmap v473: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 559 KiB/s wr, 75 op/s 2024-09-12T01:51:06.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 301755 7.74 MB/sec execute 84 sec latency 10.420 ms 2024-09-12T01:51:07.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 304695 7.80 MB/sec execute 85 sec latency 11.510 ms 2024-09-12T01:51:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:07 smithi183 ceph-mon[35127]: pgmap v474: 97 pgs: 97 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 454 KiB/s wr, 75 op/s 2024-09-12T01:51:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:07 smithi122 ceph-mon[30925]: pgmap v474: 97 pgs: 97 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 454 KiB/s wr, 75 op/s 2024-09-12T01:51:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:07 smithi151 ceph-mon[35229]: pgmap v474: 97 pgs: 97 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 454 KiB/s wr, 75 op/s 2024-09-12T01:51:08.825 INFO:tasks.workunit.client.0.smithi122.stdout: 1 307443 7.86 MB/sec execute 86 sec latency 43.182 ms 2024-09-12T01:51:09.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 309672 7.93 MB/sec execute 87 sec latency 19.792 ms 2024-09-12T01:51:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:09 smithi183 ceph-mon[35127]: pgmap v475: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 996 KiB/s wr, 151 op/s 2024-09-12T01:51:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:09 smithi122 ceph-mon[30925]: pgmap v475: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 996 KiB/s wr, 151 op/s 2024-09-12T01:51:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:09 smithi151 ceph-mon[35229]: pgmap v475: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 996 KiB/s wr, 151 op/s 2024-09-12T01:51:10.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 312775 8.04 MB/sec execute 88 sec latency 10.742 ms 2024-09-12T01:51:11.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 315980 8.16 MB/sec execute 89 sec latency 11.007 ms 2024-09-12T01:51:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:11 smithi183 ceph-mon[35127]: pgmap v476: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-09-12T01:51:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:11 smithi122 ceph-mon[30925]: pgmap v476: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-09-12T01:51:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:11 smithi151 ceph-mon[35229]: pgmap v476: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 173 op/s 2024-09-12T01:51:12.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 319074 8.23 MB/sec execute 90 sec latency 13.313 ms 2024-09-12T01:51:13.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 322013 8.32 MB/sec execute 91 sec latency 10.669 ms 2024-09-12T01:51:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:13 smithi183 ceph-mon[35127]: pgmap v477: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-12T01:51:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:13 smithi122 ceph-mon[30925]: pgmap v477: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-12T01:51:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:13 smithi151 ceph-mon[35229]: pgmap v477: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-09-12T01:51:14.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 324571 8.37 MB/sec execute 92 sec latency 11.916 ms 2024-09-12T01:51:15.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 327224 8.44 MB/sec execute 93 sec latency 10.807 ms 2024-09-12T01:51:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:15 smithi183 ceph-mon[35127]: pgmap v478: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 309 op/s 2024-09-12T01:51:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:15 smithi122 ceph-mon[30925]: pgmap v478: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 309 op/s 2024-09-12T01:51:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:15 smithi151 ceph-mon[35229]: pgmap v478: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 309 op/s 2024-09-12T01:51:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 -- 172.21.15.122:0/2790445475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4102620 msgr2=0x7fb9c4104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/2790445475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4102620 0x7fb9c4104a10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009900 tx=0x7fb9b402a410 comp rx=0 tx=0).stop 2024-09-12T01:51:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 -- 172.21.15.122:0/2790445475 shutdown_connections 2024-09-12T01:51:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/2790445475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4107880 0x7fb9c4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/2790445475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 0x7fb9c4107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/2790445475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4102620 0x7fb9c4104a10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 -- 172.21.15.122:0/2790445475 >> 172.21.15.122:0/2790445475 conn(0x7fb9c40fc1e0 msgr2=0x7fb9c40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 -- 172.21.15.122:0/2790445475 shutdown_connections 2024-09-12T01:51:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.285+0000 7fb9ca39f640 1 -- 172.21.15.122:0/2790445475 wait complete. 2024-09-12T01:51:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 Processor -- start 2024-09-12T01:51:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 -- start start 2024-09-12T01:51:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 0x7fb9c4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 0x7fb9c41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 0x7fb9c419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410d830 con 0x7fb9c4107880 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410d6b0 con 0x7fb9c4102620 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9ca39f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410d9b0 con 0x7fb9c4104f50 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 0x7fb9c4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9c893c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 0x7fb9c419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 0x7fb9c4197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:39702/0 (socket says 172.21.15.122:39702) 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9c3fff640 1 -- 172.21.15.122:0/1345555340 learned_addr learned my addr 172.21.15.122:0/1345555340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9c893c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 0x7fb9c419f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47608/0 (socket says 172.21.15.122:47608) 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.286+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 0x7fb9c41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c37fe640 1 -- 172.21.15.122:0/1345555340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 msgr2=0x7fb9c4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 0x7fb9c4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c37fe640 1 -- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 msgr2=0x7fb9c419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 0x7fb9c419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c37fe640 1 -- 172.21.15.122:0/1345555340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419fab0 con 0x7fb9c4104f50 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c893c640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 0x7fb9c419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 0x7fb9c4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 0x7fb9c41983d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9c0 tx=0x7fb9b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c17fa640 1 -- 172.21.15.122:0/1345555340 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ccc0 con 0x7fb9c4104f50 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419fda0 con 0x7fb9c4104f50 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c17fa640 1 -- 172.21.15.122:0/1345555340 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8005020 con 0x7fb9c4104f50 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9c17fa640 1 -- 172.21.15.122:0/1345555340 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8004a00 con 0x7fb9c4104f50 2024-09-12T01:51:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.287+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c41a0210 con 0x7fb9c4104f50 2024-09-12T01:51:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.289+0000 7fb9c17fa640 1 -- 172.21.15.122:0/1345555340 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb9b80051d0 con 0x7fb9c4104f50 2024-09-12T01:51:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 Processor -- start 2024-09-12T01:51:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 -- start start 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b24f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b24fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b24fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5644b245a2f0 con 0x5644b24fc370 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644b2402270 con 0x5644b24f72e0 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9cd219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644b24562b0 con 0x5644b24f69a0 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b24fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c893c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b24f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b24fe830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47622/0 (socket says 172.21.15.122:47622) 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 -- 172.21.15.122:0/713680872 learned_addr learned my addr 172.21.15.122:0/713680872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b24fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c17fa640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9ac0a2c30 0x7fb9ac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c893c640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9ac0a2c30 0x7fb9ac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 -- 172.21.15.122:0/713680872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 msgr2=0x5644b24f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b24f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 -- 172.21.15.122:0/713680872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 msgr2=0x5644b24fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b24fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c37fe640 1 -- 172.21.15.122:0/713680872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b2401340 con 0x5644b24fc370 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.290+0000 7fb9c17fa640 1 -- 172.21.15.122:0/1345555340 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb9b80ceff0 con 0x7fb9c4104f50 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b24fe830 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8013fd0 tx=0x7fb9b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cc4d2bee8d1fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9c893c640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9ac0a2c30 0x7fb9ac0a50f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0049d0 tx=0x7fb9bc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9aa7fc640 1 -- 172.21.15.122:0/713680872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b809b190 con 0x5644b24fc370 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9aa7fc640 1 -- 172.21.15.122:0/713680872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8013190 con 0x5644b24fc370 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9aa7fc640 1 -- 172.21.15.122:0/713680872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8013bd0 con 0x5644b24fc370 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 -- 172.21.15.122:0/713680872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 msgr2=0x5644b24fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b24fe830 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8013fd0 tx=0x7fb9b801f000 comp rx=0 tx=0).stop 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 -- 172.21.15.122:0/713680872 shutdown_connections 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b24fe830 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b24fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 --2- 172.21.15.122:0/713680872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b24f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 -- 172.21.15.122:0/713680872 >> 172.21.15.122:0/713680872 conn(0x5644b24f2010 msgr2=0x5644b24f2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 -- 172.21.15.122:0/713680872 shutdown_connections 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.291+0000 7fb9cd219740 1 -- 172.21.15.122:0/713680872 wait complete. 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 -- start start 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 Processor -- start 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b2500b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b2501040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b2508080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5644b24ff2e0 con 0x5644b24fc370 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644b23e3290 con 0x5644b24f72e0 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644b24ff460 con 0x5644b24f69a0 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9cd219740 10 client.0 _ll_register_callbacks cb 0x5644b2414140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9c893c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b2508080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b2501040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b2500b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9c893c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b2508080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47638/0 (socket says 172.21.15.122:47638) 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.292+0000 7fb9c893c640 1 -- 172.21.15.122:0/3647201049 learned_addr learned my addr 172.21.15.122:0/3647201049 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c893c640 1 -- 172.21.15.122:0/3647201049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 msgr2=0x5644b2500b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c893c640 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b2500b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c893c640 1 -- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 msgr2=0x5644b2501040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c893c640 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b2501040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c893c640 1 -- 172.21.15.122:0/3647201049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b216d210 con 0x5644b24fc370 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b2501040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9c893c640 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b2508080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc010b90 tx=0x7fb9bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7eb8e589424724 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9a37fe640 1 -- 172.21.15.122:0/3647201049 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0208f0 con 0x5644b24fc370 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9a37fe640 1 -- 172.21.15.122:0/3647201049 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb9bc020a90 con 0x5644b24fc370 2024-09-12T01:51:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644b2467a20 con 0x5644b24fc370 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9a37fe640 1 -- 172.21.15.122:0/3647201049 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc020c90 con 0x5644b24fc370 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9cd219740 10 client.16368 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644b2239620 con 0x5644b24fc370 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9a37fe640 1 -- 172.21.15.122:0/3647201049 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644b2467a20 con 0x5644b24fc370 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.293+0000 7fb9a37fe640 1 -- 172.21.15.122:0/3647201049 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb9bc01f040 con 0x5644b24fc370 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9cd219740 10 client.16368 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9cd219740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9cd219740 20 client.16368 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9cd219740 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b250c820 0x5644b250ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9cd219740 4 client.16368 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b2461d70 con 0x5644b250c820 2024-09-12T01:51:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b250c820 0x5644b250ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b250c820 0x5644b250ec10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c4dadd64e7fe2e server_cookie=1c87862b18d569fe in_seq=0 out_seq=0 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.294+0000 7fb9a37fe640 10 client.16368 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9a37fe640 1 -- 172.21.15.122:0/3647201049 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644b2461d70 con 0x5644b250c820 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9a37fe640 10 client.16368 handle_command_reply: tid=0 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 client.16368 shutdown 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 msgr2=0x5644b2508080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b2508080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc010b90 tx=0x7fb9bc004a40 comp rx=0 tx=0).stop 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 shutdown_connections 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b250c820 0x5644b250ec10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24fc370 0x5644b2508080 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b2501040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9a37fe640 0 client.16368 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 --2- 172.21.15.122:0/3647201049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f69a0 0x5644b2500b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 >> 172.21.15.122:0/3647201049 conn(0x5644b24f2010 msgr2=0x5644b24f3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 shutdown_connections 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 1 -- 172.21.15.122:0/3647201049 wait complete. 2024-09-12T01:51:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.295+0000 7fb9cd219740 20 client.16368 trim_cache size 0 max 16384 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.309+0000 7fb9cd219740 1 Processor -- start 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.309+0000 7fb9cd219740 1 -- start start 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b24f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f72e0 0x5644b24f3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24fc370 0x5644b24f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9cd219740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41ab8a0 con 0x5644b24f69a0 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9cd219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41a9170 con 0x5644b24fc370 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9cd219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41a6fa0 con 0x5644b24f72e0 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b24f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b24f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47652/0 (socket says 172.21.15.122:47652) 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 -- 172.21.15.122:0/1540361855 learned_addr learned my addr 172.21.15.122:0/1540361855 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f72e0 0x5644b24f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c893c640 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24fc370 0x5644b24f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 -- 172.21.15.122:0/1540361855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f72e0 msgr2=0x5644b24f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f72e0 0x5644b24f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 -- 172.21.15.122:0/1540361855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24fc370 msgr2=0x5644b24f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24fc370 0x5644b24f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 -- 172.21.15.122:0/1540361855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b250b140 con 0x5644b24f69a0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b24f10b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009230 tx=0x7fb9b40040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a339f687d80f76c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.310+0000 7fb9a9ffb640 1 -- 172.21.15.122:0/1540361855 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4035040 con 0x5644b24f69a0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9a9ffb640 1 -- 172.21.15.122:0/1540361855 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4031db0 con 0x5644b24f69a0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9a9ffb640 1 -- 172.21.15.122:0/1540361855 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4004a00 con 0x5644b24f69a0 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 -- 172.21.15.122:0/1540361855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 msgr2=0x5644b24f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b24f10b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009230 tx=0x7fb9b40040f0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 -- 172.21.15.122:0/1540361855 shutdown_connections 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24fc370 0x5644b24f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24f72e0 0x5644b24f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 --2- 172.21.15.122:0/1540361855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b24f10b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 -- 172.21.15.122:0/1540361855 >> 172.21.15.122:0/1540361855 conn(0x5644b24f2010 msgr2=0x5644b24f09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 -- 172.21.15.122:0/1540361855 shutdown_connections 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.311+0000 7fb9cd219740 1 -- 172.21.15.122:0/1540361855 wait complete. 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 -- start start 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 Processor -- start 2024-09-12T01:51:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b2572390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b25728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24fc370 0x5644b2579910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41a7630 con 0x5644b24f69a0 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41a8aa0 con 0x5644b24f72e0 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41a7d00 con 0x5644b24fc370 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9cd219740 10 client.0 _ll_register_callbacks cb 0x7fb9bc010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b2572390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b2572390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47654/0 (socket says 172.21.15.122:47654) 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9c3fff640 1 -- 172.21.15.122:0/2551555970 learned_addr learned my addr 172.21.15.122:0/2551555970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b25728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.312+0000 7fb9c893c640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24fc370 0x5644b2579910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c3fff640 1 -- 172.21.15.122:0/2551555970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24fc370 msgr2=0x5644b2579910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24fc370 0x5644b2579910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c3fff640 1 -- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 msgr2=0x5644b25728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b25728d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c3fff640 1 -- 172.21.15.122:0/2551555970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b2579f20 con 0x5644b24f69a0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b25728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c893c640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24fc370 0x5644b2579910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9c3fff640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b2572390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402ac00 tx=0x7fb9b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1d94a343a01409 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9a37fe640 1 -- 172.21.15.122:0/2551555970 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4035040 con 0x5644b24f69a0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9a37fe640 1 -- 172.21.15.122:0/2551555970 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb9b4031ad0 con 0x5644b24f69a0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b4031db0 con 0x5644b24f69a0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9a37fe640 1 -- 172.21.15.122:0/2551555970 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9b40035f0 con 0x5644b24f69a0 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9cd219740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644b257b2a0 con 0x5644b24f69a0 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.313+0000 7fb9a37fe640 1 -- 172.21.15.122:0/2551555970 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4031db0 con 0x5644b24f69a0 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9a37fe640 1 -- 172.21.15.122:0/2551555970 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb9b404aa00 con 0x5644b24f69a0 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9cd219740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9cd219740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9cd219740 20 client.16374 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9cd219740 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b257bc00 0x5644b257e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9cd219740 4 client.16374 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644b23de900 con 0x5644b257bc00 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b257bc00 0x5644b257e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9c37fe640 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b257bc00 0x5644b257e020 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af9633b89dfd7ac server_cookie=cb5aaa612d68441e in_seq=0 out_seq=0 2024-09-12T01:51:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.314+0000 7fb9a37fe640 10 client.16374 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.327+0000 7fb9a37fe640 1 -- 172.21.15.122:0/2551555970 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644b23de900 con 0x5644b257bc00 2024-09-12T01:51:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.327+0000 7fb9a37fe640 10 client.16374 handle_command_reply: tid=0 2024-09-12T01:51:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.327+0000 7fb9cd219740 1 client.16374 shutdown 2024-09-12T01:51:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 msgr2=0x5644b2572390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b2572390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402ac00 tx=0x7fb9b4009260 comp rx=0 tx=0).stop 2024-09-12T01:51:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 shutdown_connections 2024-09-12T01:51:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5644b257bc00 0x5644b257e020 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644b24fc370 0x5644b2579910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644b24f72e0 0x5644b25728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 --2- 172.21.15.122:0/2551555970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644b24f69a0 0x5644b2572390 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 >> 172.21.15.122:0/2551555970 conn(0x5644b24f2010 msgr2=0x5644b24f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.328+0000 7fb9a37fe640 0 client.16374 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:51:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.329+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 shutdown_connections 2024-09-12T01:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.329+0000 7fb9cd219740 1 -- 172.21.15.122:0/2551555970 wait complete. 2024-09-12T01:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.329+0000 7fb9cd219740 20 client.16374 trim_cache size 0 max 16384 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9ac0a2c30 msgr2=0x7fb9ac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9ac0a2c30 0x7fb9ac0a50f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0049d0 tx=0x7fb9bc004ab0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 msgr2=0x7fb9c41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 0x7fb9c41983d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9c0 tx=0x7fb9b800ee80 comp rx=0 tx=0).stop 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 shutdown_connections 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9ac0a2c30 0x7fb9ac0a50f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4107880 0x7fb9c419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9c4104f50 0x7fb9c41983d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 --2- 172.21.15.122:0/1345555340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c4102620 0x7fb9c4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 >> 172.21.15.122:0/1345555340 conn(0x7fb9c40fc1e0 msgr2=0x7fb9c4103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 shutdown_connections 2024-09-12T01:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:16.330+0000 7fb9ca39f640 1 -- 172.21.15.122:0/1345555340 wait complete. 2024-09-12T01:51:16.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 330278 8.54 MB/sec execute 94 sec latency 16.905 ms 2024-09-12T01:51:17.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 333438 8.64 MB/sec execute 95 sec latency 10.780 ms 2024-09-12T01:51:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:17 smithi183 ceph-mon[35127]: pgmap v479: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2024-09-12T01:51:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:17 smithi122 ceph-mon[30925]: pgmap v479: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2024-09-12T01:51:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:17 smithi151 ceph-mon[35229]: pgmap v479: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2024-09-12T01:51:18.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 336225 8.68 MB/sec execute 96 sec latency 13.213 ms 2024-09-12T01:51:19.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 338670 8.73 MB/sec execute 97 sec latency 11.311 ms 2024-09-12T01:51:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:19 smithi183 ceph-mon[35127]: pgmap v480: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-09-12T01:51:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:19 smithi122 ceph-mon[30925]: pgmap v480: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-09-12T01:51:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:19 smithi151 ceph-mon[35229]: pgmap v480: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-09-12T01:51:20.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 341431 8.79 MB/sec execute 98 sec latency 10.894 ms 2024-09-12T01:51:21.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 344407 8.88 MB/sec execute 99 sec latency 69.678 ms 2024-09-12T01:51:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:21 smithi183 ceph-mon[35127]: pgmap v481: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-09-12T01:51:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:21 smithi122 ceph-mon[30925]: pgmap v481: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-09-12T01:51:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:21 smithi151 ceph-mon[35229]: pgmap v481: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-09-12T01:51:22.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 347693 8.97 MB/sec execute 100 sec latency 11.493 ms 2024-09-12T01:51:23.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 350580 9.02 MB/sec execute 101 sec latency 14.271 ms 2024-09-12T01:51:23.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:23 smithi151 ceph-mon[35229]: pgmap v482: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-09-12T01:51:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:23 smithi183 ceph-mon[35127]: pgmap v482: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-09-12T01:51:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:23 smithi122 ceph-mon[30925]: pgmap v482: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-09-12T01:51:24.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 353505 9.07 MB/sec execute 102 sec latency 12.718 ms 2024-09-12T01:51:25.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 355773 9.12 MB/sec execute 103 sec latency 17.480 ms 2024-09-12T01:51:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:25 smithi183 ceph-mon[35127]: pgmap v483: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-09-12T01:51:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:25 smithi122 ceph-mon[30925]: pgmap v483: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-09-12T01:51:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:25 smithi151 ceph-mon[35229]: pgmap v483: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-09-12T01:51:26.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 358832 9.19 MB/sec execute 104 sec latency 10.062 ms 2024-09-12T01:51:27.826 INFO:tasks.workunit.client.0.smithi122.stdout: 1 362015 9.28 MB/sec execute 105 sec latency 13.998 ms 2024-09-12T01:51:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:27 smithi183 ceph-mon[35127]: pgmap v484: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-09-12T01:51:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:27 smithi122 ceph-mon[30925]: pgmap v484: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-09-12T01:51:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:27 smithi151 ceph-mon[35229]: pgmap v484: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-09-12T01:51:28.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 364998 9.33 MB/sec execute 106 sec latency 10.737 ms 2024-09-12T01:51:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 -- 172.21.15.122:0/2091064279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 msgr2=0x7f566c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 --2- 172.21.15.122:0/2091064279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1072e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f565c009900 tx=0x7f565c02a410 comp rx=0 tx=0).stop 2024-09-12T01:51:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 -- 172.21.15.122:0/2091064279 shutdown_connections 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 --2- 172.21.15.122:0/2091064279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566c107820 0x7f566c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 --2- 172.21.15.122:0/2091064279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1072e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 --2- 172.21.15.122:0/2091064279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f566c1025c0 0x7f566c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 -- 172.21.15.122:0/2091064279 >> 172.21.15.122:0/2091064279 conn(0x7f566c0fc1e0 msgr2=0x7f566c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 -- 172.21.15.122:0/2091064279 shutdown_connections 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 -- 172.21.15.122:0/2091064279 wait complete. 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 Processor -- start 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.432+0000 7f5674742640 1 -- start start 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566c1025c0 0x7f566c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f566c107820 0x7f566c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f566c10d7d0 con 0x7f566c104ef0 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f566c10d650 con 0x7f566c1025c0 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f566c10d950 con 0x7f566c107820 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f56724de640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566c1025c0 0x7f566c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51354/0 (socket says 172.21.15.122:51354) 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 -- 172.21.15.122:0/3965171976 learned_addr learned my addr 172.21.15.122:0/3965171976 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5672cdf640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f566c107820 0x7f566c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 -- 172.21.15.122:0/3965171976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f566c107820 msgr2=0x7f566c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f566c107820 0x7f566c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 -- 172.21.15.122:0/3965171976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566c1025c0 msgr2=0x7f566c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566c1025c0 0x7f566c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 -- 172.21.15.122:0/3965171976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c19fa80 con 0x7f566c104ef0 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5671cdd640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1983a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f565c031610 tx=0x7f565c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f56637fe640 1 -- 172.21.15.122:0/3965171976 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c03f030 con 0x7f566c104ef0 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c19fd10 con 0x7f566c104ef0 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f56637fe640 1 -- 172.21.15.122:0/3965171976 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c003ce0 con 0x7f566c104ef0 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f56637fe640 1 -- 172.21.15.122:0/3965171976 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c035070 con 0x7f566c104ef0 2024-09-12T01:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.433+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a01b0 con 0x7f566c104ef0 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56637fe640 1 -- 172.21.15.122:0/3965171976 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f565c042030 con 0x7f566c104ef0 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 Processor -- start 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 -- start start 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 0x559ad4b38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b3dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559ad4a9c2f0 con 0x559ad4b389a0 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ad4a44270 con 0x559ad4b3e370 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56774d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ad4a982b0 con 0x559ad4b392e0 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f5672cdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 0x559ad4b38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56724de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b3dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f5672cdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 0x559ad4b38da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51364/0 (socket says 172.21.15.122:51364) 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f5672cdf640 1 -- 172.21.15.122:0/4198640309 learned_addr learned my addr 172.21.15.122:0/4198640309 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56637fe640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56580a2c30 0x7f56580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.436+0000 7f56724de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b3dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:39046/0 (socket says 172.21.15.122:39046) 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f5672cdf640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56580a2c30 0x7f56580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56724de640 1 -- 172.21.15.122:0/4198640309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 msgr2=0x559ad4b40830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56724de640 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b40830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56724de640 1 -- 172.21.15.122:0/4198640309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 msgr2=0x559ad4b38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56724de640 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 0x559ad4b38da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56724de640 1 -- 172.21.15.122:0/4198640309 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad4a43340 con 0x559ad4b392e0 2024-09-12T01:51:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56637fe640 1 -- 172.21.15.122:0/3965171976 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f565c0ef530 con 0x7f566c104ef0 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f5672cdf640 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 0x559ad4b38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f56724de640 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b3dda0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f56680107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d16e51507d831a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f5672cdf640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56580a2c30 0x7f56580a50f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5664009990 tx=0x7f5664004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f5660ff9640 1 -- 172.21.15.122:0/4198640309 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566801c680 con 0x559ad4b392e0 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f5660ff9640 1 -- 172.21.15.122:0/4198640309 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566800be20 con 0x559ad4b392e0 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.437+0000 7f5660ff9640 1 -- 172.21.15.122:0/4198640309 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668011c70 con 0x559ad4b392e0 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 -- 172.21.15.122:0/4198640309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 msgr2=0x559ad4b3dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b3dda0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f56680107a0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 -- 172.21.15.122:0/4198640309 shutdown_connections 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b40830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b3dda0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 --2- 172.21.15.122:0/4198640309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b389a0 0x559ad4b38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 -- 172.21.15.122:0/4198640309 >> 172.21.15.122:0/4198640309 conn(0x559ad4b34010 msgr2=0x559ad4b34410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 -- 172.21.15.122:0/4198640309 shutdown_connections 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 -- 172.21.15.122:0/4198640309 wait complete. 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 -- start start 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.438+0000 7f56774d9740 1 Processor -- start 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b42c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b4a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559ad4b41340 con 0x559ad4b392e0 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ad4a25290 con 0x559ad4b3e370 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ad4b414c0 con 0x559ad4b389a0 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 10 client.0 _ll_register_callbacks cb 0x559ad4a56140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b43160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51378/0 (socket says 172.21.15.122:51378) 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 -- 172.21.15.122:0/2997318097 learned_addr learned my addr 172.21.15.122:0/2997318097 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56724de640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b42c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5672cdf640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b4a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 -- 172.21.15.122:0/2997318097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 msgr2=0x559ad4b42c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b42c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 -- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 msgr2=0x559ad4b4a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b4a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 -- 172.21.15.122:0/2997318097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad47af210 con 0x559ad4b392e0 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5671cdd640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b43160 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f565c03bfd0 tx=0x7f565c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef961ef4133f966c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5655ffb640 1 -- 172.21.15.122:0/2997318097 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c049050 con 0x559ad4b392e0 2024-09-12T01:51:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f5655ffb640 1 -- 172.21.15.122:0/2997318097 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f565c0b84b0 con 0x559ad4b392e0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.439+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566800be20 con 0x559ad4b392e0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f5655ffb640 1 -- 172.21.15.122:0/2997318097 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f565c03bd70 con 0x559ad4b392e0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 10 client.16386 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559ad487b620 con 0x559ad4b392e0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f5655ffb640 1 -- 172.21.15.122:0/2997318097 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566800be20 con 0x559ad4b392e0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f5655ffb640 1 -- 172.21.15.122:0/2997318097 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f565c0d3020 con 0x559ad4b392e0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 10 client.16386 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 20 client.16386 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4b4e900 0x559ad4b50d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 4 client.16386 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ad4aa9a20 con 0x559ad4b4e900 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.440+0000 7f56724de640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4b4e900 0x559ad4b50d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56724de640 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4b4e900 0x559ad4b50d70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4eb2b21234d71e server_cookie=f2e8c67654dbcd0e in_seq=0 out_seq=0 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f5655ffb640 10 client.16386 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f5655ffb640 1 -- 172.21.15.122:0/2997318097 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ad4aa9a20 con 0x559ad4b4e900 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f5655ffb640 10 client.16386 handle_command_reply: tid=0 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 client.16386 shutdown 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 msgr2=0x559ad4b43160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b43160 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f565c03bfd0 tx=0x7f565c009560 comp rx=0 tx=0).stop 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 shutdown_connections 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4b4e900 0x559ad4b50d70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b4a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b43160 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f5655ffb640 0 client.16386 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 --2- 172.21.15.122:0/2997318097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b42c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 >> 172.21.15.122:0/2997318097 conn(0x559ad4b34010 msgr2=0x559ad4b35820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 shutdown_connections 2024-09-12T01:51:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.441+0000 7f56774d9740 1 -- 172.21.15.122:0/2997318097 wait complete. 2024-09-12T01:51:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.442+0000 7f56774d9740 20 client.16386 trim_cache size 0 max 16384 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 Processor -- start 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 -- start start 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b389a0 0x559ad4b34ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b4a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 0x559ad4b35400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ab890 con 0x559ad4b3e370 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1a9160 con 0x559ad4b389a0 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56774d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1a6f90 con 0x559ad4b392e0 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 0x559ad4b35400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 0x559ad4b35400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51384/0 (socket says 172.21.15.122:51384) 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f56724de640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b389a0 0x559ad4b34ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 -- 172.21.15.122:0/2395414085 learned_addr learned my addr 172.21.15.122:0/2395414085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b4a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 -- 172.21.15.122:0/2395414085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 msgr2=0x559ad4b4a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b4a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 -- 172.21.15.122:0/2395414085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b389a0 msgr2=0x559ad4b34ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b389a0 0x559ad4b34ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 -- 172.21.15.122:0/2395414085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad4b33e30 con 0x559ad4b3e370 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.457+0000 7f5672cdf640 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 0x559ad4b35400 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f56640127d0 tx=0x7f5664009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dcf9056f4e87aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f5657fff640 1 -- 172.21.15.122:0/2395414085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664027020 con 0x559ad4b3e370 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f5657fff640 1 -- 172.21.15.122:0/2395414085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5664012290 con 0x559ad4b3e370 2024-09-12T01:51:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f5657fff640 1 -- 172.21.15.122:0/2395414085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640107b0 con 0x559ad4b3e370 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 -- 172.21.15.122:0/2395414085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 msgr2=0x559ad4b35400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 0x559ad4b35400 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f56640127d0 tx=0x7f5664009ad0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 -- 172.21.15.122:0/2395414085 shutdown_connections 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b3e370 0x559ad4b35400 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b392e0 0x559ad4b4a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 --2- 172.21.15.122:0/2395414085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b389a0 0x559ad4b34ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 -- 172.21.15.122:0/2395414085 >> 172.21.15.122:0/2395414085 conn(0x559ad4b34010 msgr2=0x559ad4b42c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.458+0000 7f56774d9740 1 -- 172.21.15.122:0/2395414085 shutdown_connections 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 -- 172.21.15.122:0/2395414085 wait complete. 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 -- start start 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 Processor -- start 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b48990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1a7620 con 0x559ad4b392e0 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1a8a90 con 0x559ad4b3e370 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1a7cf0 con 0x559ad4b389a0 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56774d9740 10 client.0 _ll_register_callbacks cb 0x7f565c0b7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:39074/0 (socket says 172.21.15.122:39074) 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 -- 172.21.15.122:0/2475760632 learned_addr learned my addr 172.21.15.122:0/2475760632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f5671cdd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f5672cdf640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b48990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 -- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 msgr2=0x559ad4b48990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b48990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 -- 172.21.15.122:0/2475760632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 msgr2=0x559ad4b479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.459+0000 7f56724de640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f56724de640 1 -- 172.21.15.122:0/2475760632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad4b48fa0 con 0x559ad4b389a0 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5671cdd640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f56724de640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b474b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f566800ba60 tx=0x7f566800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26358576eec523e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:51:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5655ffb640 1 -- 172.21.15.122:0/2475760632 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668005450 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5664012290 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5655ffb640 1 -- 172.21.15.122:0/2475760632 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f56680055f0 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5655ffb640 1 -- 172.21.15.122:0/2475760632 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5668004660 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f56774d9740 10 client.25684 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559ad4b4a320 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5655ffb640 1 -- 172.21.15.122:0/2475760632 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5664012290 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.460+0000 7f5655ffb640 1 -- 172.21.15.122:0/2475760632 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f566803a050 con 0x559ad4b389a0 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f56774d9740 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f56774d9740 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f56774d9740 20 client.25684 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f56774d9740 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4bb9990 0x559ad4bbbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f56774d9740 4 client.25684 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:51:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ad4aa3d70 con 0x559ad4bb9990 2024-09-12T01:51:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f5671cdd640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4bb9990 0x559ad4bbbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f5655ffb640 10 client.25684 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.461+0000 7f5671cdd640 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4bb9990 0x559ad4bbbd80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8e760bbb99b83f server_cookie=af3d71c118fb65bb in_seq=0 out_seq=0 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f5655ffb640 1 -- 172.21.15.122:0/2475760632 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ad4aa3d70 con 0x559ad4bb9990 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f5655ffb640 10 client.25684 handle_command_reply: tid=0 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 client.25684 shutdown 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 msgr2=0x559ad4b474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b474b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f566800ba60 tx=0x7f566800be20 comp rx=0 tx=0).stop 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 shutdown_connections 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ad4bb9990 0x559ad4bbbd80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ad4b3e370 0x559ad4b48990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ad4b392e0 0x559ad4b479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f5655ffb640 0 client.25684 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 --2- 172.21.15.122:0/2475760632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad4b389a0 0x559ad4b474b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 >> 172.21.15.122:0/2475760632 conn(0x559ad4b34010 msgr2=0x559ad4b329a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 shutdown_connections 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 1 -- 172.21.15.122:0/2475760632 wait complete. 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.470+0000 7f56774d9740 20 client.25684 trim_cache size 0 max 16384 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:51:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:51:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56580a2c30 msgr2=0x7f56580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56580a2c30 0x7f56580a50f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5664009990 tx=0x7f5664004ab0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 msgr2=0x7f566c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1983a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f565c031610 tx=0x7f565c0043b0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 shutdown_connections 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56580a2c30 0x7f56580a50f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f566c107820 0x7f566c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f566c104ef0 0x7f566c1983a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 --2- 172.21.15.122:0/3965171976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566c1025c0 0x7f566c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 >> 172.21.15.122:0/3965171976 conn(0x7f566c0fc1e0 msgr2=0x7f566c102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 shutdown_connections 2024-09-12T01:51:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:29.471+0000 7f5674742640 1 -- 172.21.15.122:0/3965171976 wait complete. 2024-09-12T01:51:29.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 367825 9.36 MB/sec execute 107 sec latency 10.755 ms 2024-09-12T01:51:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:29 smithi183 ceph-mon[35127]: pgmap v485: 97 pgs: 97 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 219 op/s 2024-09-12T01:51:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:29 smithi122 ceph-mon[30925]: pgmap v485: 97 pgs: 97 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 219 op/s 2024-09-12T01:51:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:29 smithi151 ceph-mon[35229]: pgmap v485: 97 pgs: 97 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 219 op/s 2024-09-12T01:51:30.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 370293 9.43 MB/sec execute 108 sec latency 10.477 ms 2024-09-12T01:51:31.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 372974 9.47 MB/sec execute 109 sec latency 88.982 ms 2024-09-12T01:51:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:31 smithi183 ceph-mon[35127]: pgmap v486: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-09-12T01:51:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:31 smithi122 ceph-mon[30925]: pgmap v486: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-09-12T01:51:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:31 smithi151 ceph-mon[35229]: pgmap v486: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-09-12T01:51:32.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 376186 9.56 MB/sec execute 110 sec latency 10.530 ms 2024-09-12T01:51:33.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 378986 9.59 MB/sec execute 111 sec latency 15.164 ms 2024-09-12T01:51:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:33 smithi183 ceph-mon[35127]: pgmap v487: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-09-12T01:51:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:33 smithi122 ceph-mon[30925]: pgmap v487: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-09-12T01:51:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:33 smithi151 ceph-mon[35229]: pgmap v487: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-09-12T01:51:34.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 380998 9.61 MB/sec execute 112 sec latency 81.011 ms 2024-09-12T01:51:35.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 383885 9.65 MB/sec execute 113 sec latency 9.923 ms 2024-09-12T01:51:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:35 smithi183 ceph-mon[35127]: pgmap v488: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 281 op/s 2024-09-12T01:51:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:35 smithi122 ceph-mon[30925]: pgmap v488: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 281 op/s 2024-09-12T01:51:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:35 smithi151 ceph-mon[35229]: pgmap v488: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 281 op/s 2024-09-12T01:51:36.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 387115 9.73 MB/sec execute 114 sec latency 11.512 ms 2024-09-12T01:51:37.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 390358 9.80 MB/sec execute 115 sec latency 14.458 ms 2024-09-12T01:51:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:37 smithi183 ceph-mon[35127]: pgmap v489: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-12T01:51:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:37 smithi122 ceph-mon[30925]: pgmap v489: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-12T01:51:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:37 smithi151 ceph-mon[35229]: pgmap v489: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-12T01:51:38.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 393212 9.83 MB/sec execute 116 sec latency 11.527 ms 2024-09-12T01:51:39.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 395255 9.85 MB/sec execute 117 sec latency 70.615 ms 2024-09-12T01:51:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:39 smithi183 ceph-mon[35127]: pgmap v490: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 272 op/s 2024-09-12T01:51:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:39 smithi122 ceph-mon[30925]: pgmap v490: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 272 op/s 2024-09-12T01:51:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:39 smithi151 ceph-mon[35229]: pgmap v490: 97 pgs: 97 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 272 op/s 2024-09-12T01:51:40.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 398193 9.89 MB/sec execute 118 sec latency 11.869 ms 2024-09-12T01:51:41.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 401401 9.96 MB/sec execute 119 sec latency 10.487 ms 2024-09-12T01:51:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:41 smithi183 ceph-mon[35127]: pgmap v491: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-09-12T01:51:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:41 smithi122 ceph-mon[30925]: pgmap v491: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-09-12T01:51:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:41 smithi151 ceph-mon[35229]: pgmap v491: 97 pgs: 97 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-09-12T01:51:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 -- 172.21.15.122:0/2494626345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac8104ef0 msgr2=0x7f7ac81072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 --2- 172.21.15.122:0/2494626345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac8104ef0 0x7f7ac81072e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009900 tx=0x7f7ab802a410 comp rx=0 tx=0).stop 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 -- 172.21.15.122:0/2494626345 shutdown_connections 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 --2- 172.21.15.122:0/2494626345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ac8107820 0x7f7ac8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 --2- 172.21.15.122:0/2494626345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac8104ef0 0x7f7ac81072e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 --2- 172.21.15.122:0/2494626345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac81025c0 0x7f7ac81049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 -- 172.21.15.122:0/2494626345 >> 172.21.15.122:0/2494626345 conn(0x7f7ac80fc1e0 msgr2=0x7f7ac80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 -- 172.21.15.122:0/2494626345 shutdown_connections 2024-09-12T01:51:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 -- 172.21.15.122:0/2494626345 wait complete. 2024-09-12T01:51:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 Processor -- start 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.562+0000 7f7acf67b640 1 -- start start 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 0x7f7ac8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ac8104ef0 0x7f7ac8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac8107820 0x7f7ac819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac810d840 con 0x7f7ac81025c0 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac810d6c0 con 0x7f7ac8104ef0 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac810d9c0 con 0x7f7ac8107820 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 0x7f7ac8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 0x7f7ac8197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44158/0 (socket says 172.21.15.122:44158) 2024-09-12T01:51:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 -- 172.21.15.122:0/3990635746 learned_addr learned my addr 172.21.15.122:0/3990635746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7accc16640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ac8104ef0 0x7f7ac8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac8107820 0x7f7ac819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 -- 172.21.15.122:0/3990635746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac8107820 msgr2=0x7f7ac819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac8107820 0x7f7ac819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 -- 172.21.15.122:0/3990635746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ac8104ef0 msgr2=0x7f7ac8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ac8104ef0 0x7f7ac8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 -- 172.21.15.122:0/3990635746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac819fa60 con 0x7f7ac81025c0 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac8107820 0x7f7ac819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acd417640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 0x7f7ac8197e40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9f0 tx=0x7f7ac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7abe7fc640 1 -- 172.21.15.122:0/3990635746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400cd80 con 0x7f7ac81025c0 2024-09-12T01:51:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac819fd50 con 0x7f7ac81025c0 2024-09-12T01:51:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7abe7fc640 1 -- 172.21.15.122:0/3990635746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac4005500 con 0x7f7ac81025c0 2024-09-12T01:51:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7abe7fc640 1 -- 172.21.15.122:0/3990635746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4010690 con 0x7f7ac81025c0 2024-09-12T01:51:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.563+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81a01c0 con 0x7f7ac81025c0 2024-09-12T01:51:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.565+0000 7f7abe7fc640 1 -- 172.21.15.122:0/3990635746 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7ac40046c0 con 0x7f7ac81025c0 2024-09-12T01:51:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 Processor -- start 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 -- start start 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7abe7fc640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7ab40a2c30 0x7f7ab40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2de6f0 0x55993b2e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55993b23d2f0 con 0x55993b2de6f0 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55993b1e5270 con 0x55993b2e3780 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.566+0000 7f7ad2418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55993b2392b0 con 0x55993b2dddb0 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7ab40a2c30 0x7f7ab40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acd417640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2de6f0 0x55993b2e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acd417640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2de1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:35014/0 (socket says 172.21.15.122:35014) 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acd417640 1 -- 172.21.15.122:0/564470327 learned_addr learned my addr 172.21.15.122:0/564470327 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acdc18640 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7abe7fc640 1 -- 172.21.15.122:0/3990635746 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7ac40cf4d0 con 0x7f7ac81025c0 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 -- 172.21.15.122:0/564470327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 msgr2=0x55993b2de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 -- 172.21.15.122:0/564470327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 msgr2=0x55993b2e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 -- 172.21.15.122:0/564470327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993b1e4340 con 0x55993b2de6f0 2024-09-12T01:51:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acdc18640 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7ab40a2c30 0x7f7ab40a50f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005c40 tx=0x7f7ac0009db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7accc16640 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2de6f0 0x55993b2e31b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009230 tx=0x7f7ab8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85664c0068784c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7ab37fe640 1 -- 172.21.15.122:0/564470327 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8035040 con 0x55993b2de6f0 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7ab37fe640 1 -- 172.21.15.122:0/564470327 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8004490 con 0x55993b2de6f0 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.567+0000 7f7ab37fe640 1 -- 172.21.15.122:0/564470327 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80316b0 con 0x55993b2de6f0 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 -- 172.21.15.122:0/564470327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2de6f0 msgr2=0x55993b2e31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2de6f0 0x55993b2e31b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009230 tx=0x7f7ab8005c00 comp rx=0 tx=0).stop 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 -- 172.21.15.122:0/564470327 shutdown_connections 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2de6f0 0x55993b2e31b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 --2- 172.21.15.122:0/564470327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 -- 172.21.15.122:0/564470327 >> 172.21.15.122:0/564470327 conn(0x55993b2d5010 msgr2=0x55993b2d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 -- 172.21.15.122:0/564470327 shutdown_connections 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 -- 172.21.15.122:0/564470327 wait complete. 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.568+0000 7f7ad2418740 1 -- start start 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 Processor -- start 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 0x55993b2ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55993b2e66f0 con 0x55993b2e3780 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55993b1c6290 con 0x55993b2de6f0 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55993b2e6870 con 0x55993b2dddb0 2024-09-12T01:51:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7ad2418740 10 client.0 _ll_register_callbacks cb 0x55993b1f7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 0x55993b2ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 0x55993b2ef4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44172/0 (socket says 172.21.15.122:44172) 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acd417640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7accc16640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 -- 172.21.15.122:0/3321653503 learned_addr learned my addr 172.21.15.122:0/3321653503 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 -- 172.21.15.122:0/3321653503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 msgr2=0x55993b2e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 -- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 msgr2=0x55993b2e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2e8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7acdc18640 1 -- 172.21.15.122:0/3321653503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993af50210 con 0x55993b2e3780 2024-09-12T01:51:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.569+0000 7f7accc16640 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7acdc18640 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 0x55993b2ef4c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0010d80 tx=0x7f7ac00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b720a89ca5bbdd6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/3321653503 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0026070 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/3321653503 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7ac0004b20 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab8004490 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/3321653503 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0031050 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55993b01c620 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/3321653503 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab8004490 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/3321653503 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7ac001f070 con 0x55993b2e3780 2024-09-12T01:51:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 20 client.16404 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b2f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 4 client.16404 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55993b24aa20 con 0x55993b2f3bd0 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.570+0000 7f7acd417640 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b2f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.571+0000 7f7ab0ff9640 10 client.16404 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.571+0000 7f7acd417640 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b2f5fc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ab460c865e7652 server_cookie=8bc257c9591bd7a6 in_seq=0 out_seq=0 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.571+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/3321653503 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55993b24aa20 con 0x55993b2f3bd0 2024-09-12T01:51:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.571+0000 7f7ab0ff9640 10 client.16404 handle_command_reply: tid=0 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 client.16404 shutdown 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 msgr2=0x55993b2ef4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 0x55993b2ef4c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0010d80 tx=0x7f7ac00094a0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 shutdown_connections 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b2f5fc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2e3780 0x55993b2ef4c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ab0ff9640 0 client.16404 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 --2- 172.21.15.122:0/3321653503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2dddb0 0x55993b2e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 >> 172.21.15.122:0/3321653503 conn(0x55993b2d5010 msgr2=0x55993b2d6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 shutdown_connections 2024-09-12T01:51:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 1 -- 172.21.15.122:0/3321653503 wait complete. 2024-09-12T01:51:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.572+0000 7f7ad2418740 20 client.16404 trim_cache size 0 max 16384 2024-09-12T01:51:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 Processor -- start 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 -- start start 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2e3780 0x55993b2fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81a9660 con 0x55993b2dddb0 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81a6f30 con 0x55993b2de6f0 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7ad2418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81a4d60 con 0x55993b2e3780 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2d40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44180/0 (socket says 172.21.15.122:44180) 2024-09-12T01:51:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 -- 172.21.15.122:0/616997620 learned_addr learned my addr 172.21.15.122:0/616997620 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7accc16640 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acdc18640 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2e3780 0x55993b2fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 -- 172.21.15.122:0/616997620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2e3780 msgr2=0x55993b2fa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2e3780 0x55993b2fa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 -- 172.21.15.122:0/616997620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 msgr2=0x55993b2ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 -- 172.21.15.122:0/616997620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993b2debd0 con 0x55993b2dddb0 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.587+0000 7f7acd417640 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2d40b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40097d0 tx=0x7f7ac409bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25030bf74238e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ab2ffd640 1 -- 172.21.15.122:0/616997620 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac409c430 con 0x55993b2dddb0 2024-09-12T01:51:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ab2ffd640 1 -- 172.21.15.122:0/616997620 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac409bd30 con 0x55993b2dddb0 2024-09-12T01:51:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ab2ffd640 1 -- 172.21.15.122:0/616997620 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4099dd0 con 0x55993b2dddb0 2024-09-12T01:51:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 -- 172.21.15.122:0/616997620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 msgr2=0x55993b2d40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2d40b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40097d0 tx=0x7f7ac409bc20 comp rx=0 tx=0).stop 2024-09-12T01:51:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 -- 172.21.15.122:0/616997620 shutdown_connections 2024-09-12T01:51:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2e3780 0x55993b2fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2de6f0 0x55993b2ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 --2- 172.21.15.122:0/616997620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2d40b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 -- 172.21.15.122:0/616997620 >> 172.21.15.122:0/616997620 conn(0x55993b2d5010 msgr2=0x55993b2d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 -- 172.21.15.122:0/616997620 shutdown_connections 2024-09-12T01:51:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.588+0000 7f7ad2418740 1 -- 172.21.15.122:0/616997620 wait complete. 2024-09-12T01:51:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 -- start start 2024-09-12T01:51:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 Processor -- start 2024-09-12T01:51:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2de6f0 0x55993b2ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2edff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81a53f0 con 0x55993b2dddb0 2024-09-12T01:51:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81a6860 con 0x55993b2e3780 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81a5ac0 con 0x55993b2de6f0 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7ad2418740 10 client.0 _ll_register_callbacks cb 0x7f7ac00125e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7accc16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2de6f0 0x55993b2ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2ecaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44192/0 (socket says 172.21.15.122:44192) 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 -- 172.21.15.122:0/1508345776 learned_addr learned my addr 172.21.15.122:0/1508345776 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acdc18640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 -- 172.21.15.122:0/1508345776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2de6f0 msgr2=0x55993b2ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2de6f0 0x55993b2ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 -- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 msgr2=0x55993b2edff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2edff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.589+0000 7f7acd417640 1 -- 172.21.15.122:0/1508345776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993b2ee600 con 0x55993b2dddb0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7acdc18640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2edff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7acd417640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2ecaf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4098190 tx=0x7f7ac4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b0a15d4fa1d988 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/1508345776 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40a00e0 con 0x55993b2dddb0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/1508345776 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7ac40cd0e0 con 0x55993b2dddb0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac409bd30 con 0x55993b2dddb0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/1508345776 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ac40a2680 con 0x55993b2dddb0 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 10 client.16410 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55993b35e8a0 con 0x55993b2dddb0 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/1508345776 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac409bd30 con 0x55993b2dddb0 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/1508345776 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7ac40ad020 con 0x55993b2dddb0 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 10 client.16410 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 20 client.16410 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b361150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 4 client.16410 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.590+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55993b244d70 con 0x55993b2f3bd0 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.591+0000 7f7accc16640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b361150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.591+0000 7f7ab0ff9640 10 client.16410 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.591+0000 7f7accc16640 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b361150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6cbf6ab3659150 server_cookie=97df772e4097a645 in_seq=0 out_seq=0 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.599+0000 7f7ab0ff9640 1 -- 172.21.15.122:0/1508345776 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55993b244d70 con 0x55993b2f3bd0 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ab0ff9640 10 client.16410 handle_command_reply: tid=0 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 client.16410 shutdown 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 msgr2=0x55993b2ecaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2ecaf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4098190 tx=0x7f7ac4005740 comp rx=0 tx=0).stop 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 shutdown_connections 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55993b2f3bd0 0x55993b361150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55993b2e3780 0x55993b2edff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ab0ff9640 0 client.16410 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55993b2de6f0 0x55993b2ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 --2- 172.21.15.122:0/1508345776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55993b2dddb0 0x55993b2ecaf0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 >> 172.21.15.122:0/1508345776 conn(0x55993b2d5010 msgr2=0x55993b2d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 shutdown_connections 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 1 -- 172.21.15.122:0/1508345776 wait complete. 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7ad2418740 20 client.16410 trim_cache size 0 max 16384 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:51:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:51:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:51:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:51:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7ab40a2c30 msgr2=0x7f7ab40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7acf67b640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7ab40a2c30 0x7f7ab40a50f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005c40 tx=0x7f7ac0009db0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 msgr2=0x7f7ac8197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.600+0000 7f7acf67b640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 0x7f7ac8197e40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9f0 tx=0x7f7ac400edf0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 shutdown_connections 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7ab40a2c30 0x7f7ab40a50f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ac8107820 0x7f7ac819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ac8104ef0 0x7f7ac8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 --2- 172.21.15.122:0/3990635746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac81025c0 0x7f7ac8197e40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 >> 172.21.15.122:0/3990635746 conn(0x7f7ac80fc1e0 msgr2=0x7f7ac8102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 shutdown_connections 2024-09-12T01:51:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:42.601+0000 7f7acf67b640 1 -- 172.21.15.122:0/3990635746 wait complete. 2024-09-12T01:51:42.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 404645 10.04 MB/sec execute 120 sec latency 10.268 ms 2024-09-12T01:51:43.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 407587 10.07 MB/sec execute 121 sec latency 10.796 ms 2024-09-12T01:51:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:43 smithi183 ceph-mon[35127]: pgmap v492: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-09-12T01:51:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:43 smithi122 ceph-mon[30925]: pgmap v492: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-09-12T01:51:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:43 smithi151 ceph-mon[35229]: pgmap v492: 97 pgs: 97 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-09-12T01:51:44.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 409864 10.08 MB/sec execute 122 sec latency 66.888 ms 2024-09-12T01:51:45.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 412483 10.11 MB/sec execute 123 sec latency 14.140 ms 2024-09-12T01:51:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:45 smithi183 ceph-mon[35127]: pgmap v493: 97 pgs: 97 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 338 op/s 2024-09-12T01:51:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:45 smithi122 ceph-mon[30925]: pgmap v493: 97 pgs: 97 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 338 op/s 2024-09-12T01:51:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:45 smithi151 ceph-mon[35229]: pgmap v493: 97 pgs: 97 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 338 op/s 2024-09-12T01:51:46.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 415614 10.18 MB/sec execute 124 sec latency 8.927 ms 2024-09-12T01:51:47.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 418730 10.24 MB/sec execute 125 sec latency 14.254 ms 2024-09-12T01:51:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:47 smithi183 ceph-mon[35127]: pgmap v494: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-09-12T01:51:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:47 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:51:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:47 smithi122 ceph-mon[30925]: pgmap v494: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-09-12T01:51:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:47 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:51:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:47 smithi151 ceph-mon[35229]: pgmap v494: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-09-12T01:51:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:47 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:51:48.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 421514 10.27 MB/sec execute 126 sec latency 12.026 ms 2024-09-12T01:51:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:51:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:51:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:51:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:51:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:51:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:51:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:51:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:51:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:51:49.827 INFO:tasks.workunit.client.0.smithi122.stdout: 1 423542 10.28 MB/sec execute 127 sec latency 61.529 ms 2024-09-12T01:51:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:49 smithi183 ceph-mon[35127]: pgmap v495: 97 pgs: 97 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 368 op/s 2024-09-12T01:51:49.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:49 smithi122 ceph-mon[30925]: pgmap v495: 97 pgs: 97 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 368 op/s 2024-09-12T01:51:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:49 smithi151 ceph-mon[35229]: pgmap v495: 97 pgs: 97 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 368 op/s 2024-09-12T01:51:50.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 426238 10.31 MB/sec execute 128 sec latency 16.266 ms 2024-09-12T01:51:51.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 429341 10.37 MB/sec execute 129 sec latency 11.667 ms 2024-09-12T01:51:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:51 smithi183 ceph-mon[35127]: pgmap v496: 97 pgs: 97 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-12T01:51:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:51 smithi122 ceph-mon[30925]: pgmap v496: 97 pgs: 97 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-12T01:51:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:51 smithi151 ceph-mon[35229]: pgmap v496: 97 pgs: 97 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-12T01:51:52.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 432153 10.39 MB/sec execute 130 sec latency 11.178 ms 2024-09-12T01:51:53.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 434660 10.41 MB/sec execute 131 sec latency 18.889 ms 2024-09-12T01:51:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:53 smithi183 ceph-mon[35127]: pgmap v497: 97 pgs: 97 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-12T01:51:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:53 smithi122 ceph-mon[30925]: pgmap v497: 97 pgs: 97 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-12T01:51:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:53 smithi151 ceph-mon[35229]: pgmap v497: 97 pgs: 97 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-12T01:51:54.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 436905 10.43 MB/sec execute 132 sec latency 65.301 ms 2024-09-12T01:51:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.690+0000 7eff90d31640 1 -- 172.21.15.122:0/2298262870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c100a40 msgr2=0x7eff8c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.690+0000 7eff90d31640 1 --2- 172.21.15.122:0/2298262870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c100a40 0x7eff8c100e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009900 tx=0x7eff7c02a410 comp rx=0 tx=0).stop 2024-09-12T01:51:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 -- 172.21.15.122:0/2298262870 shutdown_connections 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 --2- 172.21.15.122:0/2298262870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c1025f0 0x7eff8c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 --2- 172.21.15.122:0/2298262870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff8c101c30 0x7eff8c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 --2- 172.21.15.122:0/2298262870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c100a40 0x7eff8c100e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 -- 172.21.15.122:0/2298262870 >> 172.21.15.122:0/2298262870 conn(0x7eff8c0fc1e0 msgr2=0x7eff8c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 -- 172.21.15.122:0/2298262870 shutdown_connections 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 -- 172.21.15.122:0/2298262870 wait complete. 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 Processor -- start 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.691+0000 7eff90d31640 1 -- start start 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c100a40 0x7eff8c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff8c101c30 0x7eff8c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 0x7eff8c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10c6c0 con 0x7eff8c1025f0 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10c540 con 0x7eff8c101c30 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10c840 con 0x7eff8c100a40 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 0x7eff8c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c100a40 0x7eff8c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff8c101c30 0x7eff8c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c100a40 0x7eff8c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33066/0 (socket says 172.21.15.122:33066) 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 0x7eff8c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46782/0 (socket says 172.21.15.122:46782) 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8a59c640 1 -- 172.21.15.122:0/3858506827 learned_addr learned my addr 172.21.15.122:0/3858506827 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 -- 172.21.15.122:0/3858506827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c100a40 msgr2=0x7eff8c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c100a40 0x7eff8c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 -- 172.21.15.122:0/3858506827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff8c101c30 msgr2=0x7eff8c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff8c101c30 0x7eff8c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 -- 172.21.15.122:0/3858506827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c19fac0 con 0x7eff8c1025f0 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff8ad9d640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 0x7eff8c19f450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7eff84009fd0 tx=0x7eff8400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff7b7fe640 1 -- 172.21.15.122:0/3858506827 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8400ce20 con 0x7eff8c1025f0 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff7b7fe640 1 -- 172.21.15.122:0/3858506827 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff84005020 con 0x7eff8c1025f0 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff7b7fe640 1 -- 172.21.15.122:0/3858506827 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84010720 con 0x7eff8c1025f0 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c19fdb0 con 0x7eff8c1025f0 2024-09-12T01:51:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.692+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a0270 con 0x7eff8c1025f0 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff7b7fe640 1 -- 172.21.15.122:0/3858506827 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7eff84005540 con 0x7eff8c1025f0 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 Processor -- start 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 -- start start 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26061da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26066da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee26069830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ee25fc52f0 con 0x55ee26067370 2024-09-12T01:51:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee25f6d270 con 0x55ee260619a0 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff93acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee25fc12b0 con 0x55ee260622e0 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee26069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee26069830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46790/0 (socket says 172.21.15.122:46790) 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26066da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33076/0 (socket says 172.21.15.122:33076) 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff89d9b640 1 -- 172.21.15.122:0/3658334292 learned_addr learned my addr 172.21.15.122:0/3658334292 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.695+0000 7eff89d9b640 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26061da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff7b7fe640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff740a2c30 0x7eff740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8a59c640 1 -- 172.21.15.122:0/3658334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 msgr2=0x55ee26061da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8a59c640 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26061da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8a59c640 1 -- 172.21.15.122:0/3658334292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 msgr2=0x55ee26069830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8a59c640 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee26069830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8a59c640 1 -- 172.21.15.122:0/3658334292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee25f6c340 con 0x55ee260622e0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff89d9b640 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee26069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8ad9d640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff740a2c30 0x7eff740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff7b7fe640 1 -- 172.21.15.122:0/3858506827 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7eff840cf650 con 0x7eff8c1025f0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8a59c640 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26066da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff7c031440 tx=0x7eff7c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b6f472c1f981f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff8ad9d640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff740a2c30 0x7eff740a50f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff840982c0 tx=0x7eff84096870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff78ff9640 1 -- 172.21.15.122:0/3658334292 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c040070 con 0x55ee260622e0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff78ff9640 1 -- 172.21.15.122:0/3658334292 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c004030 con 0x55ee260622e0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.696+0000 7eff78ff9640 1 -- 172.21.15.122:0/3658334292 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c035040 con 0x55ee260622e0 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 -- 172.21.15.122:0/3658334292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 msgr2=0x55ee26066da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26066da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff7c031440 tx=0x7eff7c009680 comp rx=0 tx=0).stop 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 -- 172.21.15.122:0/3658334292 shutdown_connections 2024-09-12T01:51:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee26069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26066da0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 --2- 172.21.15.122:0/3658334292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26061da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 -- 172.21.15.122:0/3658334292 >> 172.21.15.122:0/3658334292 conn(0x55ee2605d010 msgr2=0x55ee2605d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 -- 172.21.15.122:0/3658334292 shutdown_connections 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 -- 172.21.15.122:0/3658334292 wait complete. 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 -- start start 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.697+0000 7eff93acd740 1 Processor -- start 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 0x55ee2606bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee2606c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee26067370 0x55ee26073140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ee2606a2e0 con 0x55ee260619a0 2024-09-12T01:51:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee25f4e290 con 0x55ee26067370 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee2606a460 con 0x55ee260622e0 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 10 client.0 _ll_register_callbacks cb 0x55ee25f7f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 0x55ee2606bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 0x55ee2606bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46800/0 (socket says 172.21.15.122:46800) 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 -- 172.21.15.122:0/2649872510 learned_addr learned my addr 172.21.15.122:0/2649872510 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff8a59c640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee2606c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee26067370 0x55ee26073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 -- 172.21.15.122:0/2649872510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 msgr2=0x55ee2606c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee2606c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 -- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee26067370 msgr2=0x55ee26073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee26067370 0x55ee26073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 -- 172.21.15.122:0/2649872510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee25cd8210 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff8a59c640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee2606c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 0x55ee2606bbc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7eff80012d30 tx=0x7eff80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621012ee1256efa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff71ffb640 1 -- 172.21.15.122:0/2649872510 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8001a040 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7c004030 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.698+0000 7eff71ffb640 1 -- 172.21.15.122:0/2649872510 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7eff80004230 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff71ffb640 1 -- 172.21.15.122:0/2649872510 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff8000ee50 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 10 client.16422 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee25da4620 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff71ffb640 1 -- 172.21.15.122:0/2649872510 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c004030 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff71ffb640 1 -- 172.21.15.122:0/2649872510 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7eff800047a0 con 0x55ee260619a0 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 10 client.16422 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 20 client.16422 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee26079cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 4 client.16422 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee25fd2a20 con 0x55ee26077890 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff8a59c640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee26079cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff71ffb640 10 client.16422 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.699+0000 7eff8a59c640 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee26079cc0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bae967957791043 server_cookie=c8012953e87f4bed in_seq=0 out_seq=0 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff71ffb640 1 -- 172.21.15.122:0/2649872510 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee25fd2a20 con 0x55ee26077890 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff71ffb640 10 client.16422 handle_command_reply: tid=0 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 client.16422 shutdown 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 msgr2=0x55ee2606bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 0x55ee2606bbc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7eff80012d30 tx=0x7eff80004730 comp rx=0 tx=0).stop 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 shutdown_connections 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee26079cc0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee26067370 0x55ee26073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff71ffb640 0 client.16422 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee2606c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 --2- 172.21.15.122:0/2649872510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260619a0 0x55ee2606bbc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 >> 172.21.15.122:0/2649872510 conn(0x55ee2605d010 msgr2=0x55ee2605e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 shutdown_connections 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 1 -- 172.21.15.122:0/2649872510 wait complete. 2024-09-12T01:51:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.700+0000 7eff93acd740 20 client.16422 trim_cache size 0 max 16384 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 Processor -- start 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 -- start start 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee2605dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26073100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee2605e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1ab940 con 0x55ee26067370 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1a9210 con 0x55ee260619a0 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff93acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1a7040 con 0x55ee260622e0 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee2605e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee2605e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46812/0 (socket says 172.21.15.122:46812) 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 -- 172.21.15.122:0/2252386183 learned_addr learned my addr 172.21.15.122:0/2252386183 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff8a59c640 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee2605dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 -- 172.21.15.122:0/2252386183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 msgr2=0x55ee26073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26073100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 -- 172.21.15.122:0/2252386183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 msgr2=0x55ee2605dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee2605dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.715+0000 7eff89d9b640 1 -- 172.21.15.122:0/2252386183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2606b700 con 0x55ee26067370 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff89d9b640 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee2605e400 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7eff80001eb0 tx=0x7eff80005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139081b69a2637f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff73fff640 1 -- 172.21.15.122:0/2252386183 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80031030 con 0x55ee26067370 2024-09-12T01:51:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff73fff640 1 -- 172.21.15.122:0/2252386183 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff800047a0 con 0x55ee26067370 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff73fff640 1 -- 172.21.15.122:0/2252386183 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8003a7d0 con 0x55ee26067370 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 -- 172.21.15.122:0/2252386183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 msgr2=0x55ee2605e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee2605e400 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7eff80001eb0 tx=0x7eff80005f90 comp rx=0 tx=0).stop 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 -- 172.21.15.122:0/2252386183 shutdown_connections 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee26067370 0x55ee2605e400 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee260622e0 0x55ee26073100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 --2- 172.21.15.122:0/2252386183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee2605dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 -- 172.21.15.122:0/2252386183 >> 172.21.15.122:0/2252386183 conn(0x55ee2605d010 msgr2=0x55ee2606b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 -- 172.21.15.122:0/2252386183 shutdown_connections 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.716+0000 7eff93acd740 1 -- 172.21.15.122:0/2252386183 wait complete. 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 -- start start 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 Processor -- start 2024-09-12T01:51:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 0x55ee26070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee26067370 0x55ee26071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1a76d0 con 0x55ee260622e0 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1a8b40 con 0x55ee260619a0 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1a7da0 con 0x55ee26067370 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff93acd740 10 client.0 _ll_register_callbacks cb 0x7eff80009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 0x55ee26070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 0x55ee26070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46828/0 (socket says 172.21.15.122:46828) 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 -- 172.21.15.122:0/433142157 learned_addr learned my addr 172.21.15.122:0/433142157 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff89d9b640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff89d9b640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee26067370 0x55ee26071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 -- 172.21.15.122:0/433142157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee26067370 msgr2=0x55ee26071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee26067370 0x55ee26071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 -- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 msgr2=0x55ee26070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26070580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff8a59c640 1 -- 172.21.15.122:0/433142157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee26072090 con 0x55ee260622e0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.717+0000 7eff89d9b640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee26067370 0x55ee26071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff8a59c640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 0x55ee26070ac0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009650 tx=0x7eff7c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded40b32eb09cacc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff71ffb640 1 -- 172.21.15.122:0/433142157 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c040050 con 0x55ee260622e0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff71ffb640 1 -- 172.21.15.122:0/433142157 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7eff7c004040 con 0x55ee260622e0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff800047a0 con 0x55ee260622e0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff71ffb640 1 -- 172.21.15.122:0/433142157 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff7c035040 con 0x55ee260622e0 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 10 client.16428 fetch_fsmap learned FSMap version 30 2024-09-12T01:51:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee26073470 con 0x55ee260622e0 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff71ffb640 1 -- 172.21.15.122:0/433142157 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff800047a0 con 0x55ee260622e0 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff71ffb640 1 -- 172.21.15.122:0/433142157 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7eff7c04f3c0 con 0x55ee260622e0 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 10 client.16428 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 20 client.16428 populate_metadata read hostname 'smithi122' 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee260e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 4 client.16428 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee25fccd70 con 0x55ee26077890 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.718+0000 7eff89d9b640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee260e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.719+0000 7eff71ffb640 10 client.16428 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.719+0000 7eff89d9b640 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee260e4a10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61034a6eba0bbb32 server_cookie=1682b731e587e332 in_seq=0 out_seq=0 2024-09-12T01:51:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff71ffb640 1 -- 172.21.15.122:0/433142157 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee25fccd70 con 0x55ee26077890 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff71ffb640 10 client.16428 handle_command_reply: tid=0 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 client.16428 shutdown 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 msgr2=0x55ee26070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 0x55ee26070ac0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009650 tx=0x7eff7c005970 comp rx=0 tx=0).stop 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 shutdown_connections 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ee26077890 0x55ee260e4a10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee26067370 0x55ee26071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ee260622e0 0x55ee26070ac0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 --2- 172.21.15.122:0/433142157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee260619a0 0x55ee26070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff71ffb640 0 client.16428 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 >> 172.21.15.122:0/433142157 conn(0x55ee2605d010 msgr2=0x55ee2605b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 shutdown_connections 2024-09-12T01:51:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 1 -- 172.21.15.122:0/433142157 wait complete. 2024-09-12T01:51:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.728+0000 7eff93acd740 20 client.16428 trim_cache size 0 max 16384 2024-09-12T01:51:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:51:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:51:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:51:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff740a2c30 msgr2=0x7eff740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff740a2c30 0x7eff740a50f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff840982c0 tx=0x7eff84096870 comp rx=0 tx=0).stop 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 msgr2=0x7eff8c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 0x7eff8c19f450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7eff84009fd0 tx=0x7eff8400ef10 comp rx=0 tx=0).stop 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 shutdown_connections 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff740a2c30 0x7eff740a50f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff8c1025f0 0x7eff8c19f450 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff8c101c30 0x7eff8c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 --2- 172.21.15.122:0/3858506827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff8c100a40 0x7eff8c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:51:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 >> 172.21.15.122:0/3858506827 conn(0x7eff8c0fc1e0 msgr2=0x7eff8c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:51:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 shutdown_connections 2024-09-12T01:51:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:51:55.729+0000 7eff90d31640 1 -- 172.21.15.122:0/3858506827 wait complete. 2024-09-12T01:51:55.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 440111 10.49 MB/sec execute 133 sec latency 28.378 ms 2024-09-12T01:51:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:55 smithi183 ceph-mon[35127]: pgmap v498: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 334 op/s 2024-09-12T01:51:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:55 smithi122 ceph-mon[30925]: pgmap v498: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 334 op/s 2024-09-12T01:51:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:55 smithi151 ceph-mon[35229]: pgmap v498: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 334 op/s 2024-09-12T01:51:56.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 443062 10.52 MB/sec execute 134 sec latency 11.135 ms 2024-09-12T01:51:57.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:57 smithi122 ceph-mon[30925]: pgmap v499: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2024-09-12T01:51:57.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 445957 10.54 MB/sec execute 135 sec latency 10.984 ms 2024-09-12T01:51:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:57 smithi183 ceph-mon[35127]: pgmap v499: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2024-09-12T01:51:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:57 smithi151 ceph-mon[35229]: pgmap v499: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2024-09-12T01:51:58.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 448505 10.58 MB/sec execute 136 sec latency 10.380 ms 2024-09-12T01:51:59.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 451020 10.61 MB/sec execute 137 sec latency 75.757 ms 2024-09-12T01:51:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:51:59 smithi183 ceph-mon[35127]: pgmap v500: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-09-12T01:51:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:51:59 smithi122 ceph-mon[30925]: pgmap v500: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-09-12T01:52:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:51:59 smithi151 ceph-mon[35229]: pgmap v500: 97 pgs: 97 active+clean; 173 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-09-12T01:52:00.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 454051 10.66 MB/sec execute 138 sec latency 16.531 ms 2024-09-12T01:52:01.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 456756 10.67 MB/sec execute 139 sec latency 16.488 ms 2024-09-12T01:52:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:01 smithi183 ceph-mon[35127]: pgmap v501: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-12T01:52:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:01 smithi122 ceph-mon[30925]: pgmap v501: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-12T01:52:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:01 smithi151 ceph-mon[35229]: pgmap v501: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-12T01:52:02.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 458415 10.68 MB/sec execute 140 sec latency 55.533 ms 2024-09-12T01:52:03.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 461553 10.82 MB/sec execute 141 sec latency 11.073 ms 2024-09-12T01:52:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:03 smithi183 ceph-mon[35127]: pgmap v502: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 307 op/s 2024-09-12T01:52:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:03 smithi122 ceph-mon[30925]: pgmap v502: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 307 op/s 2024-09-12T01:52:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:03 smithi151 ceph-mon[35229]: pgmap v502: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 307 op/s 2024-09-12T01:52:04.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 464351 10.84 MB/sec execute 142 sec latency 11.795 ms 2024-09-12T01:52:05.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 466180 10.85 MB/sec execute 143 sec latency 48.875 ms 2024-09-12T01:52:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:05 smithi183 ceph-mon[35127]: pgmap v503: 97 pgs: 97 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 376 op/s 2024-09-12T01:52:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:05 smithi122 ceph-mon[30925]: pgmap v503: 97 pgs: 97 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 376 op/s 2024-09-12T01:52:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:05 smithi151 ceph-mon[35229]: pgmap v503: 97 pgs: 97 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 376 op/s 2024-09-12T01:52:06.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 469333 10.90 MB/sec execute 144 sec latency 10.526 ms 2024-09-12T01:52:07.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 472334 10.93 MB/sec execute 145 sec latency 18.925 ms 2024-09-12T01:52:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:07 smithi183 ceph-mon[35127]: pgmap v504: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-09-12T01:52:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:07 smithi122 ceph-mon[30925]: pgmap v504: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-09-12T01:52:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:07 smithi151 ceph-mon[35229]: pgmap v504: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-09-12T01:52:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 -- 172.21.15.122:0/2630557067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c104ef0 msgr2=0x7f176c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 --2- 172.21.15.122:0/2630557067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c104ef0 0x7f176c1072e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f175c009900 tx=0x7f175c02a410 comp rx=0 tx=0).stop 2024-09-12T01:52:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 -- 172.21.15.122:0/2630557067 shutdown_connections 2024-09-12T01:52:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 --2- 172.21.15.122:0/2630557067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c107820 0x7f176c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 --2- 172.21.15.122:0/2630557067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c104ef0 0x7f176c1072e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 --2- 172.21.15.122:0/2630557067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176c1025c0 0x7f176c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 -- 172.21.15.122:0/2630557067 >> 172.21.15.122:0/2630557067 conn(0x7f176c0fc1e0 msgr2=0x7f176c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 -- 172.21.15.122:0/2630557067 shutdown_connections 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.819+0000 7f177479f640 1 -- 172.21.15.122:0/2630557067 wait complete. 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 Processor -- start 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 -- start start 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176c1025c0 0x7f176c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 0x7f176c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c107820 0x7f176c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f176c10d800 con 0x7f176c107820 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f176c10d680 con 0x7f176c1025c0 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177479f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f176c10d980 con 0x7f176c104ef0 2024-09-12T01:52:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1771d3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 0x7f176c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1772d3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c107820 0x7f176c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f177253b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176c1025c0 0x7f176c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1771d3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 0x7f176c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:50468/0 (socket says 172.21.15.122:50468) 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1771d3a640 1 -- 172.21.15.122:0/3547097453 learned_addr learned my addr 172.21.15.122:0/3547097453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1772d3c640 1 -- 172.21.15.122:0/3547097453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 msgr2=0x7f176c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 0x7f176c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1772d3c640 1 -- 172.21.15.122:0/3547097453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176c1025c0 msgr2=0x7f176c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176c1025c0 0x7f176c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1772d3c640 1 -- 172.21.15.122:0/3547097453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c19fa50 con 0x7f176c107820 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.820+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 0x7f176c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.821+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c107820 0x7f176c19f3e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1764005fd0 tx=0x7f176400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.821+0000 7f17637fe640 1 -- 172.21.15.122:0/3547097453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764004d20 con 0x7f176c107820 2024-09-12T01:52:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.821+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c19fd40 con 0x7f176c107820 2024-09-12T01:52:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.821+0000 7f17637fe640 1 -- 172.21.15.122:0/3547097453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1764005020 con 0x7f176c107820 2024-09-12T01:52:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.821+0000 7f17637fe640 1 -- 172.21.15.122:0/3547097453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640116d0 con 0x7f176c107820 2024-09-12T01:52:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.821+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1a0250 con 0x7f176c107820 2024-09-12T01:52:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f17637fe640 1 -- 172.21.15.122:0/3547097453 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1764005540 con 0x7f176c107820 2024-09-12T01:52:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 Processor -- start 2024-09-12T01:52:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 -- start start 2024-09-12T01:52:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af414f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55af4144e2f0 con 0x55af414eedb0 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af413f6270 con 0x55af414ef6f0 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.823+0000 7f1777618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af4144a2b0 con 0x55af414f4780 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414ef1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45696/0 (socket says 172.21.15.122:45696) 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 -- 172.21.15.122:0/244467147 learned_addr learned my addr 172.21.15.122:0/244467147 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f177253b640 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af414f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 -- 172.21.15.122:0/244467147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 msgr2=0x55af414f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af414f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 -- 172.21.15.122:0/244467147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 msgr2=0x55af414f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 -- 172.21.15.122:0/244467147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af413f5340 con 0x55af414eedb0 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f177253b640 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f17637fe640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17580a2c30 0x7f17580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1771d3a640 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414ef1b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f175c0096c0 tx=0x7f175c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7982c34b09f53f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f177253b640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17580a2c30 0x7f17580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f17637fe640 1 -- 172.21.15.122:0/3547097453 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f17640cf400 con 0x7f176c107820 2024-09-12T01:52:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1760ff9640 1 -- 172.21.15.122:0/244467147 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c044030 con 0x55af414eedb0 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1760ff9640 1 -- 172.21.15.122:0/244467147 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c005020 con 0x55af414eedb0 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.824+0000 7f1760ff9640 1 -- 172.21.15.122:0/244467147 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c035040 con 0x55af414eedb0 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 -- 172.21.15.122:0/244467147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 msgr2=0x55af414ef1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414ef1b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f175c0096c0 tx=0x7f175c005c00 comp rx=0 tx=0).stop 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f177253b640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17580a2c30 0x7f17580a50f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55af414f0120 tx=0x7f176800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 -- 172.21.15.122:0/244467147 shutdown_connections 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af414f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 --2- 172.21.15.122:0/244467147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414ef1b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 -- 172.21.15.122:0/244467147 >> 172.21.15.122:0/244467147 conn(0x55af414e6010 msgr2=0x55af414e6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 -- 172.21.15.122:0/244467147 shutdown_connections 2024-09-12T01:52:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.825+0000 7f1777618740 1 -- 172.21.15.122:0/244467147 wait complete. 2024-09-12T01:52:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 -- start start 2024-09-12T01:52:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 Processor -- start 2024-09-12T01:52:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af41500550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55af414f76f0 con 0x55af414eedb0 2024-09-12T01:52:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af413d7290 con 0x55af414ef6f0 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af414f7870 con 0x55af414f4780 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1777618740 10 client.0 _ll_register_callbacks cb 0x55af41408140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f177253b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f177253b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45710/0 (socket says 172.21.15.122:45710) 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f177253b640 1 -- 172.21.15.122:0/3333251895 learned_addr learned my addr 172.21.15.122:0/3333251895 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af41500550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1772d3c640 1 -- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 msgr2=0x55af414f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1772d3c640 1 -- 172.21.15.122:0/3333251895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 msgr2=0x55af414f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1772d3c640 1 -- 172.21.15.122:0/3333251895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af41161210 con 0x55af414f4780 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.826+0000 7f177253b640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af41500550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f176400c640 tx=0x7f176409b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f8f4482afadebb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:52:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1755ffb640 1 -- 172.21.15.122:0/3333251895 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640b4e60 con 0x55af414f4780 2024-09-12T01:52:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1755ffb640 1 -- 172.21.15.122:0/3333251895 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f17640a4640 con 0x55af414f4780 2024-09-12T01:52:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f175c005020 con 0x55af414f4780 2024-09-12T01:52:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1755ffb640 1 -- 172.21.15.122:0/3333251895 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f176401f070 con 0x55af414f4780 2024-09-12T01:52:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 10 client.25732 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af4122d620 con 0x55af414f4780 2024-09-12T01:52:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1755ffb640 1 -- 172.21.15.122:0/3333251895 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c005020 con 0x55af414f4780 2024-09-12T01:52:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1755ffb640 1 -- 172.21.15.122:0/3333251895 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f176400f040 con 0x55af414f4780 2024-09-12T01:52:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 10 client.25732 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 20 client.25732 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41504ca0 0x55af41507090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 4 client.25732 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:52:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.827+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af4145ba20 con 0x55af41504ca0 2024-09-12T01:52:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.828+0000 7f177253b640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41504ca0 0x55af41507090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.828+0000 7f1755ffb640 10 client.25732 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.828+0000 7f177253b640 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41504ca0 0x55af41507090 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64560b2a0897674e server_cookie=9dff0e773023c2a7 in_seq=0 out_seq=0 2024-09-12T01:52:08.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 475122 10.94 MB/sec execute 146 sec latency 20.642 ms 2024-09-12T01:52:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1755ffb640 1 -- 172.21.15.122:0/3333251895 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af4145ba20 con 0x55af41504ca0 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1755ffb640 10 client.25732 handle_command_reply: tid=0 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 client.25732 shutdown 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 msgr2=0x55af41500550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af41500550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f176400c640 tx=0x7f176409b670 comp rx=0 tx=0).stop 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 shutdown_connections 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41504ca0 0x55af41507090 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414f4780 0x55af41500550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1755ffb640 0 client.25732 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 --2- 172.21.15.122:0/3333251895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414eedb0 0x55af414f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 >> 172.21.15.122:0/3333251895 conn(0x55af414e6010 msgr2=0x55af414e7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 shutdown_connections 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 1 -- 172.21.15.122:0/3333251895 wait complete. 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.829+0000 7f1777618740 20 client.25732 trim_cache size 0 max 16384 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 Processor -- start 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 -- start start 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 0x55af41500510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414f4780 0x55af414e7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1a9670 con 0x55af414ef6f0 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1a6f40 con 0x55af414f4780 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1777618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1a4d70 con 0x55af414eedb0 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 0x55af41500510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 0x55af41500510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45718/0 (socket says 172.21.15.122:45718) 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 -- 172.21.15.122:0/3664722496 learned_addr learned my addr 172.21.15.122:0/3664722496 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f177253b640 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414f4780 0x55af414e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 -- 172.21.15.122:0/3664722496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 msgr2=0x55af414e7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414e7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 -- 172.21.15.122:0/3664722496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414f4780 msgr2=0x55af414e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414f4780 0x55af414e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 -- 172.21.15.122:0/3664722496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af414f7080 con 0x55af414ef6f0 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.844+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 0x55af41500510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f175c009230 tx=0x7f175c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a674115f82739c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1757fff640 1 -- 172.21.15.122:0/3664722496 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c044070 con 0x55af414ef6f0 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1757fff640 1 -- 172.21.15.122:0/3664722496 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c051070 con 0x55af414ef6f0 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1757fff640 1 -- 172.21.15.122:0/3664722496 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c035040 con 0x55af414ef6f0 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 -- 172.21.15.122:0/3664722496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 msgr2=0x55af41500510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 0x55af41500510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f175c009230 tx=0x7f175c031b60 comp rx=0 tx=0).stop 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 -- 172.21.15.122:0/3664722496 shutdown_connections 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414f4780 0x55af414e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414ef6f0 0x55af41500510 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 --2- 172.21.15.122:0/3664722496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 -- 172.21.15.122:0/3664722496 >> 172.21.15.122:0/3664722496 conn(0x55af414e6010 msgr2=0x55af414e51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 -- 172.21.15.122:0/3664722496 shutdown_connections 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.845+0000 7f1777618740 1 -- 172.21.15.122:0/3664722496 wait complete. 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 -- start start 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 Processor -- start 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 0x55af414ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1a5400 con 0x55af414f4780 2024-09-12T01:52:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1a6870 con 0x55af414ef6f0 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1a5ad0 con 0x55af414eedb0 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1777618740 10 client.0 _ll_register_callbacks cb 0x7f17640053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 0x55af414ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 0x55af414ff160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45732/0 (socket says 172.21.15.122:45732) 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 -- 172.21.15.122:0/3794586119 learned_addr learned my addr 172.21.15.122:0/3794586119 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f177253b640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1771d3a640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 -- 172.21.15.122:0/3794586119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 msgr2=0x55af414fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 -- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 msgr2=0x55af414fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414fe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.846+0000 7f1772d3c640 1 -- 172.21.15.122:0/3794586119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af414ff770 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1772d3c640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 0x55af414ff160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f176409bed0 tx=0x7f176409b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217102fe7f34006 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1755ffb640 1 -- 172.21.15.122:0/3794586119 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640b4e60 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1755ffb640 1 -- 172.21.15.122:0/3794586119 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1764097ae0 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f175c051070 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1755ffb640 1 -- 172.21.15.122:0/3794586119 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f176409bae0 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 10 client.16446 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af4156f9a0 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1755ffb640 1 -- 172.21.15.122:0/3794586119 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c051070 con 0x55af414f4780 2024-09-12T01:52:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1755ffb640 1 -- 172.21.15.122:0/3794586119 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f176400cb20 con 0x55af414f4780 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 10 client.16446 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 20 client.16446 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41570190 0x55af415725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 4 client.16446 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af41455d70 con 0x55af41570190 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.847+0000 7f177253b640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41570190 0x55af415725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.848+0000 7f177253b640 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41570190 0x55af415725b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f8080c73d11c95 server_cookie=bc3ced1b96f4ac5f in_seq=0 out_seq=0 2024-09-12T01:52:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.848+0000 7f1755ffb640 10 client.16446 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1755ffb640 1 -- 172.21.15.122:0/3794586119 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af41455d70 con 0x55af41570190 2024-09-12T01:52:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1755ffb640 10 client.16446 handle_command_reply: tid=0 2024-09-12T01:52:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 client.16446 shutdown 2024-09-12T01:52:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 msgr2=0x55af414ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 0x55af414ff160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f176409bed0 tx=0x7f176409b520 comp rx=0 tx=0).stop 2024-09-12T01:52:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 shutdown_connections 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55af41570190 0x55af415725b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af414f4780 0x55af414ff160 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1755ffb640 0 client.16446 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af414ef6f0 0x55af414fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 --2- 172.21.15.122:0/3794586119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af414eedb0 0x55af414fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 >> 172.21.15.122:0/3794586119 conn(0x55af414e6010 msgr2=0x55af414e51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 shutdown_connections 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.857+0000 7f1777618740 1 -- 172.21.15.122:0/3794586119 wait complete. 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f1777618740 20 client.16446 trim_cache size 0 max 16384 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:52:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:52:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:52:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:52:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17580a2c30 msgr2=0x7f17580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17580a2c30 0x7f17580a50f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55af414f0120 tx=0x7f176800b390 comp rx=0 tx=0).stop 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c107820 msgr2=0x7f176c19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c107820 0x7f176c19f3e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1764005fd0 tx=0x7f176400edd0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 shutdown_connections 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17580a2c30 0x7f17580a50f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f176c107820 0x7f176c19f3e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f176c104ef0 0x7f176c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 --2- 172.21.15.122:0/3547097453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176c1025c0 0x7f176c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 >> 172.21.15.122:0/3547097453 conn(0x7f176c0fc1e0 msgr2=0x7f176c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 shutdown_connections 2024-09-12T01:52:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:08.858+0000 7f177479f640 1 -- 172.21.15.122:0/3547097453 wait complete. 2024-09-12T01:52:09.828 INFO:tasks.workunit.client.0.smithi122.stdout: 1 477648 10.98 MB/sec execute 147 sec latency 11.212 ms 2024-09-12T01:52:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:09 smithi183 ceph-mon[35127]: pgmap v505: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 303 op/s 2024-09-12T01:52:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:09 smithi122 ceph-mon[30925]: pgmap v505: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 303 op/s 2024-09-12T01:52:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:09 smithi151 ceph-mon[35229]: pgmap v505: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 303 op/s 2024-09-12T01:52:10.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 480216 11.00 MB/sec execute 148 sec latency 87.699 ms 2024-09-12T01:52:11.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 483586 11.05 MB/sec execute 149 sec latency 10.001 ms 2024-09-12T01:52:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:11 smithi183 ceph-mon[35127]: pgmap v506: 97 pgs: 97 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-09-12T01:52:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:11 smithi122 ceph-mon[30925]: pgmap v506: 97 pgs: 97 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-09-12T01:52:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:11 smithi151 ceph-mon[35229]: pgmap v506: 97 pgs: 97 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-09-12T01:52:12.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 487000 11.10 MB/sec execute 150 sec latency 12.567 ms 2024-09-12T01:52:13.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 489968 11.12 MB/sec execute 151 sec latency 19.473 ms 2024-09-12T01:52:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:13 smithi183 ceph-mon[35127]: pgmap v507: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-09-12T01:52:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:13 smithi122 ceph-mon[30925]: pgmap v507: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-09-12T01:52:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:13 smithi151 ceph-mon[35229]: pgmap v507: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-09-12T01:52:14.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 493223 11.17 MB/sec execute 152 sec latency 9.870 ms 2024-09-12T01:52:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:14 smithi151 ceph-mon[35229]: pgmap v508: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 351 op/s 2024-09-12T01:52:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:14 smithi183 ceph-mon[35127]: pgmap v508: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 351 op/s 2024-09-12T01:52:15.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:14 smithi122 ceph-mon[30925]: pgmap v508: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 351 op/s 2024-09-12T01:52:15.395 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T01:52:15.396 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:52:15.683 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:52:15.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 495345 11.17 MB/sec execute 153 sec latency 40.098 ms 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 -- 172.21.15.122:0/1181243170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014102ef0 msgr2=0x7f201410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1181243170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014102ef0 0x7f201410da40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f200c009ca0 tx=0x7f200c02a950 comp rx=0 tx=0).stop 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 -- 172.21.15.122:0/1181243170 shutdown_connections 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1181243170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014102ef0 0x7f201410da40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1181243170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102570 0x7f20141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1181243170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014100e20 0x7f2014101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 -- 172.21.15.122:0/1181243170 >> 172.21.15.122:0/1181243170 conn(0x7f20140fc7d0 msgr2=0x7f20140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 -- 172.21.15.122:0/1181243170 shutdown_connections 2024-09-12T01:52:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.177+0000 7f201ba6d640 1 -- 172.21.15.122:0/1181243170 wait complete. 2024-09-12T01:52:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 Processor -- start 2024-09-12T01:52:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 -- start start 2024-09-12T01:52:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 0x7f2014198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 0x7f2014198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102ef0 0x7f201419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2014110fd0 con 0x7f2014100e20 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2014110e50 con 0x7f2014102ef0 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201ba6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2014111150 con 0x7f2014102570 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f2019809640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 0x7f2014198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f2019008640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 0x7f2014198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f201a00a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102ef0 0x7f201419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f2019809640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 0x7f2014198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41662/0 (socket says 172.21.15.122:41662) 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f2019809640 1 -- 172.21.15.122:0/1030916489 learned_addr learned my addr 172.21.15.122:0/1030916489 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.178+0000 7f2019008640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 0x7f2014198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:39530/0 (socket says 172.21.15.122:39530) 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201a00a640 1 -- 172.21.15.122:0/1030916489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 msgr2=0x7f2014198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201a00a640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 0x7f2014198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201a00a640 1 -- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 msgr2=0x7f2014198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201a00a640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 0x7f2014198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201a00a640 1 -- 172.21.15.122:0/1030916489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201419d2f0 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f2019008640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 0x7f2014198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f2019809640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 0x7f2014198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201a00a640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102ef0 0x7f201419cc10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f200c02af40 tx=0x7f200c009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c034040 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201419d580 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c0308d0 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c004040 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.179+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20140fcbb0 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.180+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20141021b0 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.182+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f200c037020 con 0x7f2014102ef0 2024-09-12T01:52:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.182+0000 7f200affd640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f20000a2b80 0x7f20000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.182+0000 7f2019809640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f20000a2b80 0x7f20000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.182+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f200c0f42a0 con 0x7f2014102ef0 2024-09-12T01:52:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.183+0000 7f2019809640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f20000a2b80 0x7f20000a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2010005fd0 tx=0x7f201000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.187+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f200c0851b0 con 0x7f2014102ef0 2024-09-12T01:52:16.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.467+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20140fceb0 con 0x7f2014102ef0 2024-09-12T01:52:16.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.468+0000 7f200affd640 1 -- 172.21.15.122:0/1030916489 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f200c0098b0 con 0x7f2014102ef0 2024-09-12T01:52:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:52:16.469 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:52:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.470+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f20000a2b80 msgr2=0x7f20000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.470+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f20000a2b80 0x7f20000a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2010005fd0 tx=0x7f201000b4a0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.470+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102ef0 msgr2=0x7f201419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.470+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102ef0 0x7f201419cc10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f200c02af40 tx=0x7f200c009540 comp rx=0 tx=0).stop 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 shutdown_connections 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f20000a2b80 0x7f20000a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2014102ef0 0x7f201419cc10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2014102570 0x7f2014198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 --2- 172.21.15.122:0/1030916489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2014100e20 0x7f2014198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 >> 172.21.15.122:0/1030916489 conn(0x7f20140fc7d0 msgr2=0x7f201406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:16.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 shutdown_connections 2024-09-12T01:52:16.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:16.471+0000 7f201ba6d640 1 -- 172.21.15.122:0/1030916489 wait complete. 2024-09-12T01:52:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:16 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1030916489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:52:16.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 498121 11.19 MB/sec execute 154 sec latency 39.295 ms 2024-09-12T01:52:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:16 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1030916489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:52:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1030916489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:52:17.377 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T01:52:17.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T01:52:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:52:17.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:17 smithi122 ceph-mon[30925]: pgmap v509: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 4.2 MiB/s wr, 285 op/s 2024-09-12T01:52:17.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 501353 11.24 MB/sec execute 155 sec latency 9.947 ms 2024-09-12T01:52:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:17 smithi151 ceph-mon[35229]: pgmap v509: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 4.2 MiB/s wr, 285 op/s 2024-09-12T01:52:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:17 smithi183 ceph-mon[35127]: pgmap v509: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 4.2 MiB/s wr, 285 op/s 2024-09-12T01:52:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 -- 172.21.15.122:0/4191274236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 msgr2=0x7f7e2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 --2- 172.21.15.122:0/4191274236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2010da40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009ca0 tx=0x7f7e1802a950 comp rx=0 tx=0).stop 2024-09-12T01:52:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 -- 172.21.15.122:0/4191274236 shutdown_connections 2024-09-12T01:52:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 --2- 172.21.15.122:0/4191274236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2010da40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 --2- 172.21.15.122:0/4191274236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20102570 0x7f7e201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 --2- 172.21.15.122:0/4191274236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e20100e20 0x7f7e20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 -- 172.21.15.122:0/4191274236 >> 172.21.15.122:0/4191274236 conn(0x7f7e200fc7d0 msgr2=0x7f7e200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 -- 172.21.15.122:0/4191274236 shutdown_connections 2024-09-12T01:52:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 -- 172.21.15.122:0/4191274236 wait complete. 2024-09-12T01:52:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.656+0000 7f7e253b7640 1 Processor -- start 2024-09-12T01:52:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 -- start start 2024-09-12T01:52:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20100e20 0x7f7e20198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e20102570 0x7f7e20198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20110f80 con 0x7f7e20102ef0 2024-09-12T01:52:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20110e00 con 0x7f7e20102570 2024-09-12T01:52:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e253b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20111100 con 0x7f7e20100e20 2024-09-12T01:52:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41686/0 (socket says 172.21.15.122:41686) 2024-09-12T01:52:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 -- 172.21.15.122:0/2059461756 learned_addr learned my addr 172.21.15.122:0/2059461756 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e20102570 0x7f7e20198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1effd640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20100e20 0x7f7e20198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 -- 172.21.15.122:0/2059461756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20100e20 msgr2=0x7f7e20198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20100e20 0x7f7e20198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 -- 172.21.15.122:0/2059461756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e20102570 msgr2=0x7f7e20198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e20102570 0x7f7e20198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 -- 172.21.15.122:0/2059461756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2019d400 con 0x7f7e20102ef0 2024-09-12T01:52:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.657+0000 7f7e1f7fe640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2019cd20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7e18004af0 tx=0x7f7e1802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.658+0000 7f7e07fff640 1 -- 172.21.15.122:0/2059461756 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18034040 con 0x7f7e20102ef0 2024-09-12T01:52:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.658+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2019d690 con 0x7f7e20102ef0 2024-09-12T01:52:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.658+0000 7f7e07fff640 1 -- 172.21.15.122:0/2059461756 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e18030920 con 0x7f7e20102ef0 2024-09-12T01:52:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.658+0000 7f7e07fff640 1 -- 172.21.15.122:0/2059461756 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18004820 con 0x7f7e20102ef0 2024-09-12T01:52:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.658+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e201a4fd0 con 0x7f7e20102ef0 2024-09-12T01:52:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.660+0000 7f7e07fff640 1 -- 172.21.15.122:0/2059461756 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7e18037020 con 0x7f7e20102ef0 2024-09-12T01:52:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.660+0000 7f7e27790740 1 Processor -- start 2024-09-12T01:52:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.660+0000 7f7e27790740 1 -- start start 2024-09-12T01:52:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e27790740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e27790740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e27790740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e27790740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c3d22d2a0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e27790740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556c3d1d4b10 con 0x556c3d2c7680 2024-09-12T01:52:18.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e27790740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c3d229260 con 0x556c3d2d6d50 2024-09-12T01:52:18.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e07fff640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e080a2b80 0x7f7e080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41694/0 (socket says 172.21.15.122:41694) 2024-09-12T01:52:18.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 -- 172.21.15.122:0/1715063281 learned_addr learned my addr 172.21.15.122:0/1715063281 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e07fff640 1 -- 172.21.15.122:0/2059461756 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7e180efe60 con 0x7f7e20102ef0 2024-09-12T01:52:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1f7fe640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e080a2b80 0x7f7e080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 -- 172.21.15.122:0/1715063281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 msgr2=0x556c3d2d9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2d9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 -- 172.21.15.122:0/1715063281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 msgr2=0x556c3d2c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1effd640 1 -- 172.21.15.122:0/1715063281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3cf9eb90 con 0x556c3d2c7fa0 2024-09-12T01:52:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.661+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2d9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.662+0000 7f7e1effd640 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d6810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000bd10 tx=0x7f7e100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828c844a24526cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.662+0000 7f7e1f7fe640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e080a2b80 0x7f7e080a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006010 tx=0x7f7e180095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.662+0000 7f7e057fa640 1 -- 172.21.15.122:0/1715063281 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004030 con 0x556c3d2c7fa0 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.662+0000 7f7e057fa640 1 -- 172.21.15.122:0/1715063281 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e100041d0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.662+0000 7f7e057fa640 1 -- 172.21.15.122:0/1715063281 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10011820 con 0x556c3d2c7fa0 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 -- 172.21.15.122:0/1715063281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 msgr2=0x556c3d2d6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d6810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000bd10 tx=0x7f7e100107a0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 -- 172.21.15.122:0/1715063281 shutdown_connections 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2d9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d6810 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 --2- 172.21.15.122:0/1715063281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 -- 172.21.15.122:0/1715063281 >> 172.21.15.122:0/1715063281 conn(0x556c3d2c3cf0 msgr2=0x556c3d2c40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 -- 172.21.15.122:0/1715063281 shutdown_connections 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 -- 172.21.15.122:0/1715063281 wait complete. 2024-09-12T01:52:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 -- start start 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.663+0000 7f7e27790740 1 Processor -- start 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2c7680 0x556c3d2d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2d6d50 0x556c3d2d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c3d2d98c0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556c3d1b56e0 con 0x556c3d2d6d50 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c3d2d9a40 con 0x556c3d2c7680 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 10 client.0 _ll_register_callbacks cb 0x556c3d1d9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41700/0 (socket says 172.21.15.122:41700) 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 -- 172.21.15.122:0/3106123136 learned_addr learned my addr 172.21.15.122:0/3106123136 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2c7680 0x556c3d2d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2d6d50 0x556c3d2d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 -- 172.21.15.122:0/3106123136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2c7680 msgr2=0x556c3d2d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2c7680 0x556c3d2d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 -- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2d6d50 msgr2=0x556c3d2d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2d6d50 0x556c3d2d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 -- 172.21.15.122:0/3106123136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3d2e2ed0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e1effd640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d4240 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7e10010ca0 tx=0x7f7e1000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330580de108991ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e02ff5640 1 -- 172.21.15.122:0/3106123136 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10029060 con 0x556c3d2c7fa0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e02ff5640 1 -- 172.21.15.122:0/3106123136 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7e10004440 con 0x556c3d2c7fa0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.664+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e100041d0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e02ff5640 1 -- 172.21.15.122:0/3106123136 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e10003800 con 0x556c3d2c7fa0 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 10 client.16464 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c3d2e57b0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e02ff5640 1 -- 172.21.15.122:0/3106123136 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e100041d0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e02ff5640 1 -- 172.21.15.122:0/3106123136 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7e1002e070 con 0x556c3d2c7fa0 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 10 client.16464 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 10 client.16464 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 20 client.16464 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d2e6c10 0x556c3d2e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 4 client.16464 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c3d23a9d0 con 0x556c3d2e6c10 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.665+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d2e6c10 0x556c3d2e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.666+0000 7f7e02ff5640 10 client.16464 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.666+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d2e6c10 0x556c3d2e9000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2101d6917d009c server_cookie=4f124f42b9db486d in_seq=0 out_seq=0 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.666+0000 7f7e02ff5640 1 -- 172.21.15.122:0/3106123136 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c3d23a9d0 con 0x556c3d2e6c10 2024-09-12T01:52:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.666+0000 7f7e02ff5640 10 client.16464 handle_command_reply: tid=0 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.666+0000 7f7e27790740 1 client.16464 shutdown 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.666+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 msgr2=0x556c3d2d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d4240 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7e10010ca0 tx=0x7f7e1000b8e0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 shutdown_connections 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d2e6c10 0x556c3d2e9000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2d6d50 0x556c3d2d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2d4240 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 --2- 172.21.15.122:0/3106123136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2c7680 0x556c3d2d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e02ff5640 0 client.16464 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 >> 172.21.15.122:0/3106123136 conn(0x556c3d2c3cf0 msgr2=0x556c3d2c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 shutdown_connections 2024-09-12T01:52:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 1 -- 172.21.15.122:0/3106123136 wait complete. 2024-09-12T01:52:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.667+0000 7f7e27790740 20 client.16464 trim_cache size 0 max 16384 2024-09-12T01:52:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.684+0000 7f7e27790740 1 Processor -- start 2024-09-12T01:52:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.684+0000 7f7e27790740 1 -- start start 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e27790740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e27790740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2ee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e27790740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e27790740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201adaf0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e27790740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201ab3c0 con 0x556c3d2c7680 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e27790740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201a91f0 con 0x556c3d2d6d50 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2ee600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41712/0 (socket says 172.21.15.122:41712) 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 -- 172.21.15.122:0/3099927806 learned_addr learned my addr 172.21.15.122:0/3099927806 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 -- 172.21.15.122:0/3099927806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 msgr2=0x556c3d2eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 -- 172.21.15.122:0/3099927806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 msgr2=0x556c3d2d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2d45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 -- 172.21.15.122:0/3099927806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3d2bf260 con 0x556c3d2c7fa0 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e1effd640 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2ee600 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7e10011010 tx=0x7f7e10005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51bb2a7666fcfd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.685+0000 7f7e04ff9640 1 -- 172.21.15.122:0/3099927806 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10034070 con 0x556c3d2c7fa0 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e04ff9640 1 -- 172.21.15.122:0/3099927806 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e10004920 con 0x556c3d2c7fa0 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e04ff9640 1 -- 172.21.15.122:0/3099927806 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10029040 con 0x556c3d2c7fa0 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 -- 172.21.15.122:0/3099927806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 msgr2=0x556c3d2ee600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2ee600 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7e10011010 tx=0x7f7e10005c90 comp rx=0 tx=0).stop 2024-09-12T01:52:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 -- 172.21.15.122:0/3099927806 shutdown_connections 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7fa0 0x556c3d2ee600 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 --2- 172.21.15.122:0/3099927806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7680 0x556c3d2d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 -- 172.21.15.122:0/3099927806 >> 172.21.15.122:0/3099927806 conn(0x556c3d2c3cf0 msgr2=0x556c3d2c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 -- 172.21.15.122:0/3099927806 shutdown_connections 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.686+0000 7f7e27790740 1 -- 172.21.15.122:0/3099927806 wait complete. 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 -- start start 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 Processor -- start 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 0x556c3d2df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7fa0 0x556c3d2db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201a9880 con 0x556c3d2c7680 2024-09-12T01:52:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201aacf0 con 0x556c3d2c7fa0 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201a9f50 con 0x556c3d2d6d50 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e27790740 10 client.0 _ll_register_callbacks cb 0x7f7e10004090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 0x556c3d2df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 0x556c3d2df100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41722/0 (socket says 172.21.15.122:41722) 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e1e7fc640 1 -- 172.21.15.122:0/4125413859 learned_addr learned my addr 172.21.15.122:0/4125413859 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.687+0000 7f7e1effd640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7fa0 0x556c3d2db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e1e7fc640 1 -- 172.21.15.122:0/4125413859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 msgr2=0x556c3d2db980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2db980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e1e7fc640 1 -- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7fa0 msgr2=0x556c3d2db440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7fa0 0x556c3d2db440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e1e7fc640 1 -- 172.21.15.122:0/4125413859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3d2c3580 con 0x556c3d2c7680 2024-09-12T01:52:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e1e7fc640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 0x556c3d2df100 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x556c3d2e95a0 tx=0x7f7e10004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a33c58ed1100756 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e02ff5640 1 -- 172.21.15.122:0/4125413859 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14005020 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e02ff5640 1 -- 172.21.15.122:0/4125413859 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f7e140051c0 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e10004920 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e02ff5640 1 -- 172.21.15.122:0/4125413859 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e140053c0 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e27790740 10 client.16470 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c3d352050 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.688+0000 7f7e02ff5640 1 -- 172.21.15.122:0/4125413859 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e10004920 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e02ff5640 1 -- 172.21.15.122:0/4125413859 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7e14021580 con 0x556c3d2c7680 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e27790740 10 client.16470 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e27790740 10 client.16470 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e27790740 20 client.16470 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:18.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e27790740 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d352810 0x556c3d354c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e27790740 4 client.16470 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T01:52:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x556c3d234d20 con 0x556c3d352810 2024-09-12T01:52:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.689+0000 7f7e1effd640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d352810 0x556c3d354c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.690+0000 7f7e1effd640 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d352810 0x556c3d354c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf21a2a7761bb4a server_cookie=e97de970ecde0f4e in_seq=0 out_seq=0 2024-09-12T01:52:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.690+0000 7f7e02ff5640 10 client.16470 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e02ff5640 1 -- 172.21.15.122:0/4125413859 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556c3d234d20 con 0x556c3d352810 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e02ff5640 10 client.16470 handle_command_reply: tid=0 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 client.16470 shutdown 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 msgr2=0x556c3d2df100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 0x556c3d2df100 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x556c3d2e95a0 tx=0x7f7e10004000 comp rx=0 tx=0).stop 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 shutdown_connections 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556c3d352810 0x556c3d354c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c3d2d6d50 0x556c3d2db980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c3d2c7fa0 0x556c3d2db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e02ff5640 0 client.16470 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 --2- 172.21.15.122:0/4125413859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c3d2c7680 0x556c3d2df100 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.788+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 >> 172.21.15.122:0/4125413859 conn(0x556c3d2c3cf0 msgr2=0x556c3d2c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 shutdown_connections 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e27790740 1 -- 172.21.15.122:0/4125413859 wait complete. 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e27790740 20 client.16470 trim_cache size 0 max 16384 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:52:18.790 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 22, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "d0117e7f": { 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 22, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 140.1, 2024-09-12T01:52:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:18.792 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e080a2b80 msgr2=0x7f7e080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e080a2b80 0x7f7e080a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006010 tx=0x7f7e180095c0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 msgr2=0x7f7e2019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2019cd20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7e18004af0 tx=0x7f7e1802f5b0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 shutdown_connections 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f7e080a2b80 0x7f7e080a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e20102ef0 0x7f7e2019cd20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e20102570 0x7f7e20198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 --2- 172.21.15.122:0/2059461756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20100e20 0x7f7e20198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.789+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 >> 172.21.15.122:0/2059461756 conn(0x7f7e200fc7d0 msgr2=0x7f7e2006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.790+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 shutdown_connections 2024-09-12T01:52:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:18.790+0000 7f7e253b7640 1 -- 172.21.15.122:0/2059461756 wait complete. 2024-09-12T01:52:18.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 11.25 MB/sec execute 156 sec latency 141.157 ms 2024-09-12T01:52:19.575 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd0117e7f', quiesce took 4.2 seconds. Will release after: 15.900000000000002 2024-09-12T01:52:19.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 11.18 MB/sec execute 157 sec latency 1141.260 ms 2024-09-12T01:52:19.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:19 smithi151 ceph-mon[35229]: pgmap v510: 97 pgs: 97 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 346 op/s 2024-09-12T01:52:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:19 smithi183 ceph-mon[35127]: pgmap v510: 97 pgs: 97 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 346 op/s 2024-09-12T01:52:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:19 smithi122 ceph-mon[30925]: pgmap v510: 97 pgs: 97 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 346 op/s 2024-09-12T01:52:20.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 11.11 MB/sec execute 158 sec latency 2141.332 ms 2024-09-12T01:52:21.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 11.04 MB/sec execute 159 sec latency 3141.415 ms 2024-09-12T01:52:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:21 smithi183 ceph-mon[35127]: pgmap v511: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 4.3 MiB/s wr, 326 op/s 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 -- 172.21.15.122:0/3813015152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0101690 msgr2=0x7f75d0108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 --2- 172.21.15.122:0/3813015152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0101690 0x7f75d0108f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009ca0 tx=0x7f75c802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 -- 172.21.15.122:0/3813015152 shutdown_connections 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 --2- 172.21.15.122:0/3813015152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0101690 0x7f75d0108f30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 --2- 172.21.15.122:0/3813015152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d0100cd0 0x7f75d0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 --2- 172.21.15.122:0/3813015152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d0102030 0x7f75d0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 -- 172.21.15.122:0/3813015152 >> 172.21.15.122:0/3813015152 conn(0x7f75d00fb930 msgr2=0x7f75d00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 -- 172.21.15.122:0/3813015152 shutdown_connections 2024-09-12T01:52:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.953+0000 7f75d6923640 1 -- 172.21.15.122:0/3813015152 wait complete. 2024-09-12T01:52:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.954+0000 7f75d6923640 1 Processor -- start 2024-09-12T01:52:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.954+0000 7f75d6923640 1 -- start start 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 0x7f75d0100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d0101690 0x7f75d019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d0102030 0x7f75d01a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010c6c0 con 0x7f75d0100cd0 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010c540 con 0x7f75d0102030 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010c840 con 0x7f75d0101690 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 0x7f75d0100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 0x7f75d0100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45626/0 (socket says 172.21.15.122:45626) 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 -- 172.21.15.122:0/369939160 learned_addr learned my addr 172.21.15.122:0/369939160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d0101690 0x7f75d019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d0102030 0x7f75d01a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 -- 172.21.15.122:0/369939160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d0101690 msgr2=0x7f75d019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d0101690 0x7f75d019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 -- 172.21.15.122:0/369939160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d0102030 msgr2=0x7f75d01a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d0102030 0x7f75d01a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 -- 172.21.15.122:0/369939160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a1be0 con 0x7f75d0100cd0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cffff640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 0x7f75d0100730 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f75c000d950 tx=0x7f75c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cd7fa640 1 -- 172.21.15.122:0/369939160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0017020 con 0x7f75d0100cd0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cd7fa640 1 -- 172.21.15.122:0/369939160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c0009c30 con 0x7f75d0100cd0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a1e70 con 0x7f75d0100cd0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75cd7fa640 1 -- 172.21.15.122:0/369939160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000f740 con 0x7f75d0100cd0 2024-09-12T01:52:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.955+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a22d0 con 0x7f75d0100cd0 2024-09-12T01:52:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75cd7fa640 1 -- 172.21.15.122:0/369939160 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f75c001a020 con 0x7f75d0100cd0 2024-09-12T01:52:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 Processor -- start 2024-09-12T01:52:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 -- start start 2024-09-12T01:52:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e484da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e489da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e48c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d10e3e82f0 con 0x55d10e4849a0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d10e390270 con 0x55d10e48a370 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d96ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d10e3e42b0 con 0x55d10e4852e0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75d4ec0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e484da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.958+0000 7f75cd7fa640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f75bc0a2c60 0x7f75bc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75cffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e489da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e484da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45636/0 (socket says 172.21.15.122:45636) 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 -- 172.21.15.122:0/3980253297 learned_addr learned my addr 172.21.15.122:0/3980253297 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f75bc0a2c60 0x7f75bc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e48c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75cd7fa640 1 -- 172.21.15.122:0/369939160 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f75c00ce9b0 con 0x7f75d0100cd0 2024-09-12T01:52:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 -- 172.21.15.122:0/3980253297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 msgr2=0x55d10e489da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e489da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 -- 172.21.15.122:0/3980253297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 msgr2=0x55d10e48c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e48c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 -- 172.21.15.122:0/3980253297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d10e38f340 con 0x55d10e4849a0 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e484da0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f75c802ade0 tx=0x7f75c8031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32afe78aaa9fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f75bc0a2c60 0x7f75bc0a5120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75c8031c10 tx=0x7f75c8047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75c67fc640 1 -- 172.21.15.122:0/3980253297 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c802af40 con 0x55d10e4849a0 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75c67fc640 1 -- 172.21.15.122:0/3980253297 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c8036070 con 0x55d10e4849a0 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.959+0000 7f75c67fc640 1 -- 172.21.15.122:0/3980253297 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c802fdc0 con 0x55d10e4849a0 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 -- 172.21.15.122:0/3980253297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 msgr2=0x55d10e484da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e484da0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f75c802ade0 tx=0x7f75c8031870 comp rx=0 tx=0).stop 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 -- 172.21.15.122:0/3980253297 shutdown_connections 2024-09-12T01:52:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e48c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e489da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3980253297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e484da0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 -- 172.21.15.122:0/3980253297 >> 172.21.15.122:0/3980253297 conn(0x55d10e480010 msgr2=0x55d10e480410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 -- 172.21.15.122:0/3980253297 shutdown_connections 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 -- 172.21.15.122:0/3980253297 wait complete. 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.960+0000 7f75d96ba740 1 -- start start 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 Processor -- start 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e48ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e48f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e496170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d10e48d2e0 con 0x55d10e4849a0 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d10e371290 con 0x55d10e48a370 2024-09-12T01:52:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d10e48d460 con 0x55d10e4852e0 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d96ba740 10 client.0 _ll_register_callbacks cb 0x55d10e3a2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e48ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e48ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45646/0 (socket says 172.21.15.122:45646) 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 -- 172.21.15.122:0/1209996326 learned_addr learned my addr 172.21.15.122:0/1209996326 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e496170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e48f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 -- 172.21.15.122:0/1209996326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 msgr2=0x55d10e48f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e48f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 -- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 msgr2=0x55d10e496170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e496170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cffff640 1 -- 172.21.15.122:0/1209996326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10e0fb210 con 0x55d10e4849a0 2024-09-12T01:52:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.961+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e48f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75cffff640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e48ebf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f75c0097670 tx=0x7f75c009aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d18aff74b304c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75af7fe640 1 -- 172.21.15.122:0/1209996326 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c009b210 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75af7fe640 1 -- 172.21.15.122:0/1209996326 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f75c0097880 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c8036070 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75af7fe640 1 -- 172.21.15.122:0/1209996326 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75c009f5c0 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 10 client.16482 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d10e1c7620 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75af7fe640 1 -- 172.21.15.122:0/1209996326 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c8036070 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75af7fe640 1 -- 172.21.15.122:0/1209996326 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f75c009f170 con 0x55d10e4849a0 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 10 client.16482 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 20 client.16482 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e49a810 0x55d10e49cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 4 client.16482 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d10e3f5a20 con 0x55d10e49a810 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.962+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e49a810 0x55d10e49cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.963+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e49a810 0x55d10e49cc40 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07ea020f08e40d5 server_cookie=8b97fdcf20d577c in_seq=0 out_seq=0 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.963+0000 7f75af7fe640 10 client.16482 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75af7fe640 1 -- 172.21.15.122:0/1209996326 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d10e3f5a20 con 0x55d10e49a810 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75af7fe640 10 client.16482 handle_command_reply: tid=0 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 client.16482 shutdown 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 msgr2=0x55d10e48ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e48ebf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f75c0097670 tx=0x7f75c009aca0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 shutdown_connections 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e49a810 0x55d10e49cc40 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e48a370 0x55d10e496170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4852e0 0x55d10e48f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75af7fe640 0 client.16482 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 --2- 172.21.15.122:0/1209996326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e48ebf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 >> 172.21.15.122:0/1209996326 conn(0x55d10e480010 msgr2=0x55d10e481820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 shutdown_connections 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 1 -- 172.21.15.122:0/1209996326 wait complete. 2024-09-12T01:52:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.964+0000 7f75d96ba740 20 client.16482 trim_cache size 0 max 16384 2024-09-12T01:52:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 Processor -- start 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 -- start start 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4849a0 0x55d10e48f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e496100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 0x55d10e47f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01adaf0 con 0x55d10e48a370 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01ab3c0 con 0x55d10e4852e0 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d96ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a91f0 con 0x55d10e4849a0 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d4ec0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 0x55d10e47f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d4ec0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 0x55d10e47f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45662/0 (socket says 172.21.15.122:45662) 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.978+0000 7f75d4ec0640 1 -- 172.21.15.122:0/13810765 learned_addr learned my addr 172.21.15.122:0/13810765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75cffff640 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4849a0 0x55d10e48f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75d4ec0640 1 -- 172.21.15.122:0/13810765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4849a0 msgr2=0x55d10e48f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4849a0 0x55d10e48f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75d4ec0640 1 -- 172.21.15.122:0/13810765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 msgr2=0x55d10e496100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e496100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75d4ec0640 1 -- 172.21.15.122:0/13810765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d10e49d2c0 con 0x55d10e48a370 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75cffff640 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4849a0 0x55d10e48f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 0x55d10e47f0b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f75c8004980 tx=0x7f75c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0d65f6150da27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75c5ffb640 1 -- 172.21.15.122:0/13810765 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004610 con 0x55d10e48a370 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75c5ffb640 1 -- 172.21.15.122:0/13810765 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c8055070 con 0x55d10e48a370 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.979+0000 7f75c5ffb640 1 -- 172.21.15.122:0/13810765 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80305c0 con 0x55d10e48a370 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- 172.21.15.122:0/13810765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 msgr2=0x55d10e47f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 0x55d10e47f0b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f75c8004980 tx=0x7f75c8005f90 comp rx=0 tx=0).stop 2024-09-12T01:52:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- 172.21.15.122:0/13810765 shutdown_connections 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e48a370 0x55d10e47f0b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e496100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- 172.21.15.122:0/13810765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e4849a0 0x55d10e48f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- 172.21.15.122:0/13810765 >> 172.21.15.122:0/13810765 conn(0x55d10e480010 msgr2=0x55d10e47e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- 172.21.15.122:0/13810765 shutdown_connections 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- 172.21.15.122:0/13810765 wait complete. 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- start start 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 Processor -- start 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e493fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e4944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e48a370 0x55d10e495500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a9880 con 0x55d10e4849a0 2024-09-12T01:52:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01aacf0 con 0x55d10e4852e0 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a9f50 con 0x55d10e48a370 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.980+0000 7f75d96ba740 10 client.0 _ll_register_callbacks cb 0x7f75c009ade0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e493fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e493fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45670/0 (socket says 172.21.15.122:45670) 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 -- 172.21.15.122:0/3826771993 learned_addr learned my addr 172.21.15.122:0/3826771993 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75d4ec0640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e48a370 0x55d10e495500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e4944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 -- 172.21.15.122:0/3826771993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e48a370 msgr2=0x55d10e495500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e48a370 0x55d10e495500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 -- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 msgr2=0x55d10e4944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e4944f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 -- 172.21.15.122:0/3826771993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10e495b10 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75cffff640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e493fb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f75c00cedb0 tx=0x7f75c000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146478c7ced99862 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75af7fe640 1 -- 172.21.15.122:0/3826771993 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c009b210 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75af7fe640 1 -- 172.21.15.122:0/3826771993 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f75c009f170 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c8055070 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75af7fe640 1 -- 172.21.15.122:0/3826771993 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75c00a3e80 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75d96ba740 10 client.16488 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d10e505bb0 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.981+0000 7f75af7fe640 1 -- 172.21.15.122:0/3826771993 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c8055070 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75af7fe640 1 -- 172.21.15.122:0/3826771993 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f75c009a0d0 con 0x55d10e4849a0 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75d96ba740 10 client.16488 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75d96ba740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75d96ba740 20 client.16488 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e5063d0 0x55d10e5087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75d96ba740 4 client.16488 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:52:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d10e48a130 con 0x55d10e5063d0 2024-09-12T01:52:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e5063d0 0x55d10e5087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75af7fe640 10 client.16488 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.982+0000 7f75cf7fe640 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e5063d0 0x55d10e5087f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4883ab0f56997050 server_cookie=322d52dcb2818001 in_seq=0 out_seq=0 2024-09-12T01:52:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:21 smithi122 ceph-mon[30925]: pgmap v511: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 4.3 MiB/s wr, 326 op/s 2024-09-12T01:52:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75af7fe640 1 -- 172.21.15.122:0/3826771993 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d10e48a130 con 0x55d10e5063d0 2024-09-12T01:52:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75af7fe640 10 client.16488 handle_command_reply: tid=0 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 client.16488 shutdown 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 msgr2=0x55d10e493fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e493fb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f75c00cedb0 tx=0x7f75c000bf60 comp rx=0 tx=0).stop 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 shutdown_connections 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d10e5063d0 0x55d10e5087f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d10e48a370 0x55d10e495500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10e4852e0 0x55d10e4944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 --2- 172.21.15.122:0/3826771993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d10e4849a0 0x55d10e493fb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75af7fe640 0 client.16488 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 >> 172.21.15.122:0/3826771993 conn(0x55d10e480010 msgr2=0x55d10e4816e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 shutdown_connections 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 1 -- 172.21.15.122:0/3826771993 wait complete. 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.993+0000 7f75d96ba740 20 client.16488 trim_cache size 0 max 16384 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:52:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:52:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f75bc0a2c60 msgr2=0x7f75bc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f75bc0a2c60 0x7f75bc0a5120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75c8031c10 tx=0x7f75c8047000 comp rx=0 tx=0).stop 2024-09-12T01:52:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 msgr2=0x7f75d0100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 0x7f75d0100730 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f75c000d950 tx=0x7f75c000de10 comp rx=0 tx=0).stop 2024-09-12T01:52:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 shutdown_connections 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f75bc0a2c60 0x7f75bc0a5120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75d0102030 0x7f75d01a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d0101690 0x7f75d019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 --2- 172.21.15.122:0/369939160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75d0100cd0 0x7f75d0100730 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 >> 172.21.15.122:0/369939160 conn(0x7f75d00fb930 msgr2=0x7f75d00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 shutdown_connections 2024-09-12T01:52:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:21.994+0000 7f75d6923640 1 -- 172.21.15.122:0/369939160 wait complete. 2024-09-12T01:52:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:21 smithi151 ceph-mon[35229]: pgmap v511: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 4.3 MiB/s wr, 326 op/s 2024-09-12T01:52:22.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.97 MB/sec execute 160 sec latency 4141.510 ms 2024-09-12T01:52:23.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.90 MB/sec execute 161 sec latency 5141.610 ms 2024-09-12T01:52:23.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:23 smithi151 ceph-mon[35229]: pgmap v512: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-09-12T01:52:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:23 smithi183 ceph-mon[35127]: pgmap v512: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-09-12T01:52:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:23 smithi122 ceph-mon[30925]: pgmap v512: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-09-12T01:52:24.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.83 MB/sec execute 162 sec latency 6141.670 ms 2024-09-12T01:52:25.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.77 MB/sec execute 163 sec latency 7141.768 ms 2024-09-12T01:52:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:25 smithi183 ceph-mon[35127]: pgmap v513: 97 pgs: 97 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-09-12T01:52:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:25 smithi122 ceph-mon[30925]: pgmap v513: 97 pgs: 97 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-09-12T01:52:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:25 smithi151 ceph-mon[35229]: pgmap v513: 97 pgs: 97 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-09-12T01:52:26.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.70 MB/sec execute 164 sec latency 8141.829 ms 2024-09-12T01:52:27.829 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.63 MB/sec execute 165 sec latency 9141.885 ms 2024-09-12T01:52:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:27 smithi183 ceph-mon[35127]: pgmap v514: 97 pgs: 97 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 3.6 MiB/s wr, 192 op/s 2024-09-12T01:52:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:27 smithi122 ceph-mon[30925]: pgmap v514: 97 pgs: 97 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 3.6 MiB/s wr, 192 op/s 2024-09-12T01:52:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:27 smithi151 ceph-mon[35229]: pgmap v514: 97 pgs: 97 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 3.6 MiB/s wr, 192 op/s 2024-09-12T01:52:28.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.57 MB/sec execute 166 sec latency 10141.992 ms 2024-09-12T01:52:29.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.51 MB/sec execute 167 sec latency 11142.085 ms 2024-09-12T01:52:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:29 smithi183 ceph-mon[35127]: pgmap v515: 97 pgs: 97 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 3.4 MiB/s wr, 197 op/s 2024-09-12T01:52:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:29 smithi122 ceph-mon[30925]: pgmap v515: 97 pgs: 97 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 3.4 MiB/s wr, 197 op/s 2024-09-12T01:52:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:29 smithi151 ceph-mon[35229]: pgmap v515: 97 pgs: 97 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 3.4 MiB/s wr, 197 op/s 2024-09-12T01:52:30.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.44 MB/sec execute 168 sec latency 12142.181 ms 2024-09-12T01:52:31.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.38 MB/sec execute 169 sec latency 13142.261 ms 2024-09-12T01:52:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:31 smithi183 ceph-mon[35127]: pgmap v516: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.8 MiB/s wr, 125 op/s 2024-09-12T01:52:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:31 smithi122 ceph-mon[30925]: pgmap v516: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.8 MiB/s wr, 125 op/s 2024-09-12T01:52:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:31 smithi151 ceph-mon[35229]: pgmap v516: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.8 MiB/s wr, 125 op/s 2024-09-12T01:52:32.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.32 MB/sec execute 170 sec latency 14142.358 ms 2024-09-12T01:52:33.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.26 MB/sec execute 171 sec latency 15142.446 ms 2024-09-12T01:52:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:33 smithi183 ceph-mon[35127]: pgmap v517: 97 pgs: 97 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.5 MiB/s wr, 116 op/s 2024-09-12T01:52:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:33 smithi122 ceph-mon[30925]: pgmap v517: 97 pgs: 97 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.5 MiB/s wr, 116 op/s 2024-09-12T01:52:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:33 smithi151 ceph-mon[35229]: pgmap v517: 97 pgs: 97 active+clean; 180 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.5 MiB/s wr, 116 op/s 2024-09-12T01:52:34.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.20 MB/sec execute 172 sec latency 16142.549 ms 2024-09-12T01:52:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 -- 172.21.15.122:0/512106358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c0ffbb0 msgr2=0x7fb43c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 --2- 172.21.15.122:0/512106358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c0ffbb0 0x7fb43c100030 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009930 tx=0x7fb42c02a420 comp rx=0 tx=0).stop 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 -- 172.21.15.122:0/512106358 shutdown_connections 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 --2- 172.21.15.122:0/512106358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c100570 0x7fb43c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 --2- 172.21.15.122:0/512106358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c0ffbb0 0x7fb43c100030 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 --2- 172.21.15.122:0/512106358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c103140 0x7fb43c103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 -- 172.21.15.122:0/512106358 >> 172.21.15.122:0/512106358 conn(0x7fb43c0fb980 msgr2=0x7fb43c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 -- 172.21.15.122:0/512106358 shutdown_connections 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.088+0000 7fb444077640 1 -- 172.21.15.122:0/512106358 wait complete. 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 Processor -- start 2024-09-12T01:52:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 -- start start 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c0ffbb0 0x7fb43c102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 0x7fb43c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c103140 0x7fb43c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c10d7e0 con 0x7fb43c100570 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c10d660 con 0x7fb43c103140 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb444077640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c10d960 con 0x7fb43c0ffbb0 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 0x7fb43c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 0x7fb43c100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51718/0 (socket says 172.21.15.122:51718) 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 -- 172.21.15.122:0/991124320 learned_addr learned my addr 172.21.15.122:0/991124320 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb442614640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c103140 0x7fb43c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 -- 172.21.15.122:0/991124320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c0ffbb0 msgr2=0x7fb43c102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c0ffbb0 0x7fb43c102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 -- 172.21.15.122:0/991124320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c103140 msgr2=0x7fb43c1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c103140 0x7fb43c1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb441612640 1 -- 172.21.15.122:0/991124320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c101c50 con 0x7fb43c100570 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.089+0000 7fb442614640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c103140 0x7fb43c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.090+0000 7fb441612640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 0x7fb43c100e80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb42c031610 tx=0x7fb42c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.090+0000 7fb432ffd640 1 -- 172.21.15.122:0/991124320 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c03f030 con 0x7fb43c100570 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.090+0000 7fb432ffd640 1 -- 172.21.15.122:0/991124320 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c003ce0 con 0x7fb43c100570 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.090+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c1a1e70 con 0x7fb43c100570 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.090+0000 7fb432ffd640 1 -- 172.21.15.122:0/991124320 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c035070 con 0x7fb43c100570 2024-09-12T01:52:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.090+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1a22d0 con 0x7fb43c100570 2024-09-12T01:52:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.092+0000 7fb432ffd640 1 -- 172.21.15.122:0/991124320 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb42c042030 con 0x7fb43c100570 2024-09-12T01:52:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.092+0000 7fb446e0f740 1 Processor -- start 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 -- start start 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d702e0 0x5599a6d74da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d75370 0x5599a6d77830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5599a6cd32f0 con 0x5599a6d6f9a0 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5599a6c7b270 con 0x5599a6d75370 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb446e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599a6ccf2b0 con 0x5599a6d702e0 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d6fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51724/0 (socket says 172.21.15.122:51724) 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 -- 172.21.15.122:0/2372438051 learned_addr learned my addr 172.21.15.122:0/2372438051 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb432ffd640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4280a2c30 0x7fb4280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb441612640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d75370 0x5599a6d77830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb441e13640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d702e0 0x5599a6d74da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 -- 172.21.15.122:0/2372438051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d702e0 msgr2=0x5599a6d74da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d702e0 0x5599a6d74da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 -- 172.21.15.122:0/2372438051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d75370 msgr2=0x5599a6d77830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d75370 0x5599a6d77830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb442614640 1 -- 172.21.15.122:0/2372438051 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a6c7a340 con 0x5599a6d6f9a0 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb441e13640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d702e0 0x5599a6d74da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.093+0000 7fb441612640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d75370 0x5599a6d77830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.094+0000 7fb432ffd640 1 -- 172.21.15.122:0/991124320 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb42c0ef380 con 0x7fb43c100570 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.094+0000 7fb442614640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4280a2c30 0x7fb4280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.094+0000 7fb442614640 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d6fda0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d340 tx=0x7fb43400d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bc4984ba0d05c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.094+0000 7fb427fff640 1 -- 172.21.15.122:0/2372438051 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43401aa60 con 0x5599a6d6f9a0 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.094+0000 7fb427fff640 1 -- 172.21.15.122:0/2372438051 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb434011500 con 0x5599a6d6f9a0 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.094+0000 7fb427fff640 1 -- 172.21.15.122:0/2372438051 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43400e3b0 con 0x5599a6d6f9a0 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 -- 172.21.15.122:0/2372438051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 msgr2=0x5599a6d6fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb442614640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4280a2c30 0x7fb4280a50f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb43401e000 tx=0x7fb43401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d6fda0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d340 tx=0x7fb43400d800 comp rx=0 tx=0).stop 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 -- 172.21.15.122:0/2372438051 shutdown_connections 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d75370 0x5599a6d77830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d702e0 0x5599a6d74da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2372438051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d6fda0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 -- 172.21.15.122:0/2372438051 >> 172.21.15.122:0/2372438051 conn(0x5599a6d6b010 msgr2=0x5599a6d6b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 -- 172.21.15.122:0/2372438051 shutdown_connections 2024-09-12T01:52:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 -- 172.21.15.122:0/2372438051 wait complete. 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 -- start start 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.095+0000 7fb446e0f740 1 Processor -- start 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 0x5599a6d810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5599a6d78340 con 0x5599a6d6f9a0 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5599a6c5c290 con 0x5599a6d702e0 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599a6d784c0 con 0x5599a6d75370 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 10 client.0 _ll_register_callbacks cb 0x5599a6c8d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51736/0 (socket says 172.21.15.122:51736) 2024-09-12T01:52:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 -- 172.21.15.122:0/2867185230 learned_addr learned my addr 172.21.15.122:0/2867185230 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441612640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 -- 172.21.15.122:0/2867185230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 msgr2=0x5599a6d810c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 0x5599a6d810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 -- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 msgr2=0x5599a6d7a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 -- 172.21.15.122:0/2867185230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a69e6210 con 0x5599a6d6f9a0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441612640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb441e13640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79b40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e3a0 tx=0x7fb43800e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bfc4e9f7345834 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb4257fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb4257fa640 1 -- 172.21.15.122:0/2867185230 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438016070 con 0x5599a6d6f9a0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb4257fa640 1 -- 172.21.15.122:0/2867185230 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb4380108a0 con 0x5599a6d6f9a0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.096+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb434011500 con 0x5599a6d6f9a0 2024-09-12T01:52:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb4257fa640 1 -- 172.21.15.122:0/2867185230 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb438022050 con 0x5599a6d6f9a0 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 10 client.16500 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599a6ab2620 con 0x5599a6d6f9a0 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb4257fa640 1 -- 172.21.15.122:0/2867185230 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb434011500 con 0x5599a6d6f9a0 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb4257fa640 1 -- 172.21.15.122:0/2867185230 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb43800f040 con 0x5599a6d6f9a0 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 10 client.16500 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 20 client.16500 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6d857a0 0x5599a6d87c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 4 client.16500 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599a6ce0a20 con 0x5599a6d857a0 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.097+0000 7fb441612640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6d857a0 0x5599a6d87c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.098+0000 7fb4257fa640 10 client.16500 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.098+0000 7fb441612640 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6d857a0 0x5599a6d87c30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af86dbe0bf8a24c server_cookie=ae588e71367fbfb0 in_seq=0 out_seq=0 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.098+0000 7fb4257fa640 1 -- 172.21.15.122:0/2867185230 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599a6ce0a20 con 0x5599a6d857a0 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.098+0000 7fb4257fa640 10 client.16500 handle_command_reply: tid=0 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.098+0000 7fb446e0f740 1 client.16500 shutdown 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 msgr2=0x5599a6d79b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79b40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e3a0 tx=0x7fb43800e860 comp rx=0 tx=0).stop 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 shutdown_connections 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6d857a0 0x5599a6d87c30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 0x5599a6d810c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb4257fa640 0 client.16500 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2867185230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79b40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 >> 172.21.15.122:0/2867185230 conn(0x5599a6d6b010 msgr2=0x5599a6d6c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 shutdown_connections 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 1 -- 172.21.15.122:0/2867185230 wait complete. 2024-09-12T01:52:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.099+0000 7fb446e0f740 20 client.16500 trim_cache size 0 max 16384 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 Processor -- start 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 -- start start 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 0x5599a6d81080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 0x5599a6d6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ada80 con 0x5599a6d702e0 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ab350 con 0x5599a6d6f9a0 2024-09-12T01:52:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb446e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1a9180 con 0x5599a6d75370 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb441612640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 0x5599a6d81080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb441612640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 0x5599a6d81080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51752/0 (socket says 172.21.15.122:51752) 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb441612640 1 -- 172.21.15.122:0/2566955728 learned_addr learned my addr 172.21.15.122:0/2566955728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.113+0000 7fb441e13640 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb441612640 1 -- 172.21.15.122:0/2566955728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 msgr2=0x5599a6d6c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb441612640 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 0x5599a6d6c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb441612640 1 -- 172.21.15.122:0/2566955728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d6f9a0 msgr2=0x5599a6d79f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb441612640 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb441612640 1 -- 172.21.15.122:0/2566955728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a6d69650 con 0x5599a6d702e0 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb441612640 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 0x5599a6d81080 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0314e0 tx=0x7fb42c03b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15f6806d7d30e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb4277fe640 1 -- 172.21.15.122:0/2566955728 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c049040 con 0x5599a6d702e0 2024-09-12T01:52:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb4277fe640 1 -- 172.21.15.122:0/2566955728 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c03bda0 con 0x5599a6d702e0 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb4277fe640 1 -- 172.21.15.122:0/2566955728 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0c4380 con 0x5599a6d702e0 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.114+0000 7fb446e0f740 1 -- 172.21.15.122:0/2566955728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 msgr2=0x5599a6d81080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 0x5599a6d81080 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0314e0 tx=0x7fb42c03b870 comp rx=0 tx=0).stop 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- 172.21.15.122:0/2566955728 shutdown_connections 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d75370 0x5599a6d6c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d702e0 0x5599a6d81080 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- 172.21.15.122:0/2566955728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d6f9a0 0x5599a6d79f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- 172.21.15.122:0/2566955728 >> 172.21.15.122:0/2566955728 conn(0x5599a6d6b010 msgr2=0x5599a6d6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- 172.21.15.122:0/2566955728 shutdown_connections 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- 172.21.15.122:0/2566955728 wait complete. 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- start start 2024-09-12T01:52:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 Processor -- start 2024-09-12T01:52:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d6f9a0 0x5599a6d7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 0x5599a6d80050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1a9810 con 0x5599a6d75370 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1aac80 con 0x5599a6d702e0 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1a9ee0 con 0x5599a6d6f9a0 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.115+0000 7fb446e0f740 10 client.0 _ll_register_callbacks cb 0x7fb438009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 0x5599a6d80050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 0x5599a6d80050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51758/0 (socket says 172.21.15.122:51758) 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 -- 172.21.15.122:0/1297439903 learned_addr learned my addr 172.21.15.122:0/1297439903 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb441e13640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d6f9a0 0x5599a6d7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb441612640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 -- 172.21.15.122:0/1297439903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d6f9a0 msgr2=0x5599a6d7eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d6f9a0 0x5599a6d7eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 -- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 msgr2=0x5599a6d7f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 -- 172.21.15.122:0/1297439903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a6d80660 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb441612640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb442614640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 0x5599a6d80050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb434005fd0 tx=0x7fb43400e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbd669684411859 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb4257fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb4257fa640 1 -- 172.21.15.122:0/1297439903 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434008600 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb4257fa640 1 -- 172.21.15.122:0/1297439903 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb43400ec90 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c03bda0 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb4257fa640 1 -- 172.21.15.122:0/1297439903 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb434011500 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb446e0f740 10 client.16506 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599a6df08a0 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.116+0000 7fb4257fa640 1 -- 172.21.15.122:0/1297439903 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c03bda0 con 0x5599a6d75370 2024-09-12T01:52:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb4257fa640 1 -- 172.21.15.122:0/1297439903 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb43402c070 con 0x5599a6d75370 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb446e0f740 10 client.16506 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb446e0f740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb446e0f740 20 client.16506 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb446e0f740 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6df10f0 0x5599a6df3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb446e0f740 4 client.16506 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599a6cdad70 con 0x5599a6df10f0 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb441e13640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6df10f0 0x5599a6df3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb4257fa640 10 client.16506 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.117+0000 7fb441e13640 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6df10f0 0x5599a6df3510 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664b490598229d43 server_cookie=c226f6098f2d52e3 in_seq=0 out_seq=0 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb4257fa640 1 -- 172.21.15.122:0/1297439903 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599a6cdad70 con 0x5599a6df10f0 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb4257fa640 10 client.16506 handle_command_reply: tid=0 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 client.16506 shutdown 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 msgr2=0x5599a6d80050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 0x5599a6d80050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb434005fd0 tx=0x7fb43400e910 comp rx=0 tx=0).stop 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 shutdown_connections 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5599a6df10f0 0x5599a6df3510 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5599a6d75370 0x5599a6d80050 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599a6d702e0 0x5599a6d7f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb4257fa640 0 client.16506 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 --2- 172.21.15.122:0/1297439903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599a6d6f9a0 0x5599a6d7eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 >> 172.21.15.122:0/1297439903 conn(0x5599a6d6b010 msgr2=0x5599a6d6b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 shutdown_connections 2024-09-12T01:52:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 1 -- 172.21.15.122:0/1297439903 wait complete. 2024-09-12T01:52:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.128+0000 7fb446e0f740 20 client.16506 trim_cache size 0 max 16384 2024-09-12T01:52:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:52:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:52:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:52:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:52:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4280a2c30 msgr2=0x7fb4280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4280a2c30 0x7fb4280a50f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb43401e000 tx=0x7fb43401f000 comp rx=0 tx=0).stop 2024-09-12T01:52:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 msgr2=0x7fb43c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 0x7fb43c100e80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb42c031610 tx=0x7fb42c0043b0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 shutdown_connections 2024-09-12T01:52:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4280a2c30 0x7fb4280a50f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c103140 0x7fb43c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb43c100570 0x7fb43c100e80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 --2- 172.21.15.122:0/991124320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c0ffbb0 0x7fb43c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 >> 172.21.15.122:0/991124320 conn(0x7fb43c0fb980 msgr2=0x7fb43c106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 shutdown_connections 2024-09-12T01:52:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:35.129+0000 7fb444077640 1 -- 172.21.15.122:0/991124320 wait complete. 2024-09-12T01:52:35.476 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:52:35.780 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:52:35.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.14 MB/sec execute 173 sec latency 17142.633 ms 2024-09-12T01:52:35.901 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:35 smithi122 ceph-mon[30925]: pgmap v518: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-09-12T01:52:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:35 smithi183 ceph-mon[35127]: pgmap v518: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-09-12T01:52:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:35 smithi151 ceph-mon[35229]: pgmap v518: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 -- 172.21.15.122:0/3449479371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101240 msgr2=0x7f3334101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 --2- 172.21.15.122:0/3449479371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101240 0x7f3334101680 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3328009900 tx=0x7f332802a410 comp rx=0 tx=0).stop 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 -- 172.21.15.122:0/3449479371 shutdown_connections 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 --2- 172.21.15.122:0/3449479371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334101bc0 0x7f333410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 --2- 172.21.15.122:0/3449479371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101240 0x7f3334101680 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 --2- 172.21.15.122:0/3449479371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3334105a90 0x7f3334105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 -- 172.21.15.122:0/3449479371 >> 172.21.15.122:0/3449479371 conn(0x7f3334077210 msgr2=0x7f33340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 -- 172.21.15.122:0/3449479371 shutdown_connections 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.248+0000 7f33396d2640 1 -- 172.21.15.122:0/3449479371 wait complete. 2024-09-12T01:52:36.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 Processor -- start 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 -- start start 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3334101240 0x7f3334199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 0x7f33341998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334105a90 0x7f33341a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3334110f30 con 0x7f3334101bc0 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3334110db0 con 0x7f3334105a90 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33396d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33341110b0 con 0x7f3334101240 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33327fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 0x7f33341998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33327fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 0x7f33341998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51780/0 (socket says 172.21.15.122:51780) 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33327fc640 1 -- 172.21.15.122:0/3092141095 learned_addr learned my addr 172.21.15.122:0/3092141095 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:36.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.249+0000 7f33337fe640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334105a90 0x7f33341a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f3332ffd640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3334101240 0x7f3334199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33327fc640 1 -- 172.21.15.122:0/3092141095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3334101240 msgr2=0x7f3334199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33327fc640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3334101240 0x7f3334199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33327fc640 1 -- 172.21.15.122:0/3092141095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334105a90 msgr2=0x7f33341a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33327fc640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334105a90 0x7f33341a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33327fc640 1 -- 172.21.15.122:0/3092141095 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33341a4920 con 0x7f3334101bc0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33337fe640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334105a90 0x7f33341a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33327fc640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 0x7f33341998a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3328006010 tx=0x7f3328003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332803f070 con 0x7f3334101bc0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3328031c30 con 0x7f3334101bc0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328035030 con 0x7f3334101bc0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33341a4af0 con 0x7f3334101bc0 2024-09-12T01:52:36.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.250+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33341a4fa0 con 0x7f3334101bc0 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.251+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3334101680 con 0x7f3334101bc0 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.253+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3328042070 con 0x7f3334101bc0 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.254+0000 7f331bfff640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f331c0a2c00 0x7f331c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.254+0000 7f3332ffd640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f331c0a2c00 0x7f331c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.254+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f33280efc50 con 0x7f3334101bc0 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.254+0000 7f3332ffd640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f331c0a2c00 0x7f331c0a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3324005e00 tx=0x7f3324005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:36.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.257+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33280bcbb0 con 0x7f3334101bc0 2024-09-12T01:52:36.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.458+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3334100720 con 0x7f3334101bc0 2024-09-12T01:52:36.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.459+0000 7f331bfff640 1 -- 172.21.15.122:0/3092141095 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f3328031de0 con 0x7f3334101bc0 2024-09-12T01:52:36.459 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:52:36.460 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:52:36.460 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:52:36.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f331c0a2c00 msgr2=0x7f331c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:36.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f331c0a2c00 0x7f331c0a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3324005e00 tx=0x7f3324005c50 comp rx=0 tx=0).stop 2024-09-12T01:52:36.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 msgr2=0x7f33341998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:36.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 0x7f33341998a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3328006010 tx=0x7f3328003c10 comp rx=0 tx=0).stop 2024-09-12T01:52:36.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 shutdown_connections 2024-09-12T01:52:36.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f331c0a2c00 0x7f331c0a50c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3334105a90 0x7f33341a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3334101bc0 0x7f33341998a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 --2- 172.21.15.122:0/3092141095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3334101240 0x7f3334199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:36.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 >> 172.21.15.122:0/3092141095 conn(0x7f3334077210 msgr2=0x7f3334103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:36.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 shutdown_connections 2024-09-12T01:52:36.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:36.462+0000 7f33396d2640 1 -- 172.21.15.122:0/3092141095 wait complete. 2024-09-12T01:52:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:36 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3092141095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:52:36.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.08 MB/sec execute 174 sec latency 18142.702 ms 2024-09-12T01:52:36.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:36 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3092141095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:52:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:36 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3092141095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:52:37.201 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id d0117e7f --release --await' 2024-09-12T01:52:37.201 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id d0117e7f --release --await 2024-09-12T01:52:37.443 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:52:37.812 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:37 smithi122 ceph-mon[30925]: pgmap v519: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.5 MiB/s wr, 89 op/s 2024-09-12T01:52:37.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 503870 10.03 MB/sec execute 175 sec latency 19142.726 ms 2024-09-12T01:52:37.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.897+0000 7faf6b949640 1 -- 172.21.15.122:0/2385668240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64102790 msgr2=0x7faf641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.897+0000 7faf6b949640 1 --2- 172.21.15.122:0/2385668240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64102790 0x7faf641067e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faf54009900 tx=0x7faf5402a410 comp rx=0 tx=0).stop 2024-09-12T01:52:37.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.897+0000 7faf6b949640 1 -- 172.21.15.122:0/2385668240 shutdown_connections 2024-09-12T01:52:37.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.897+0000 7faf6b949640 1 --2- 172.21.15.122:0/2385668240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64106f10 0x7faf6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 --2- 172.21.15.122:0/2385668240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64102790 0x7faf641067e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 --2- 172.21.15.122:0/2385668240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf64101de0 0x7faf641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- 172.21.15.122:0/2385668240 >> 172.21.15.122:0/2385668240 conn(0x7faf640fd8c0 msgr2=0x7faf640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- 172.21.15.122:0/2385668240 shutdown_connections 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- 172.21.15.122:0/2385668240 wait complete. 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 Processor -- start 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- start start 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf64101de0 0x7faf64198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64102790 0x7faf64198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 0x7faf6419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111f90 con 0x7faf64106f10 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111e10 con 0x7faf64102790 2024-09-12T01:52:37.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.898+0000 7faf6b949640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112110 con 0x7faf64101de0 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 0x7faf6419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 0x7faf6419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51790/0 (socket says 172.21.15.122:51790) 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 -- 172.21.15.122:0/2746802374 learned_addr learned my addr 172.21.15.122:0/2746802374 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf696e5640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf64101de0 0x7faf64198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf68ee4640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64102790 0x7faf64198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 -- 172.21.15.122:0/2746802374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf64101de0 msgr2=0x7faf64198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf64101de0 0x7faf64198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 -- 172.21.15.122:0/2746802374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64102790 msgr2=0x7faf64198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64102790 0x7faf64198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 -- 172.21.15.122:0/2746802374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419d3c0 con 0x7faf64106f10 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf69ee6640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 0x7faf6419cce0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00cce0 tx=0x7faf5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf5a7fc640 1 -- 172.21.15.122:0/2746802374 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00f450 con 0x7faf64106f10 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419d6b0 con 0x7faf64106f10 2024-09-12T01:52:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf5a7fc640 1 -- 172.21.15.122:0/2746802374 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c00fad0 con 0x7faf64106f10 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf5a7fc640 1 -- 172.21.15.122:0/2746802374 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c018b60 con 0x7faf64106f10 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.899+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a4f90 con 0x7faf64106f10 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.901+0000 7faf5a7fc640 1 -- 172.21.15.122:0/2746802374 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faf5c00fc80 con 0x7faf64106f10 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 Processor -- start 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 -- start start 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d2f20 0x56455e8d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8e2650 0x56455e8e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56455e838620 con 0x56455e8d3840 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56455e7dfe50 con 0x56455e8e2650 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf6dd22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56455e8345e0 con 0x56455e8d2f20 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf5a7fc640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf500a2b80 0x7faf500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf696e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf696e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51802/0 (socket says 172.21.15.122:51802) 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf696e5640 1 -- 172.21.15.122:0/1081165852 learned_addr learned my addr 172.21.15.122:0/1081165852 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8e2650 0x56455e8e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf69ee6640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf500a2b80 0x7faf500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d2f20 0x56455e8d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.902+0000 7faf5a7fc640 1 -- 172.21.15.122:0/2746802374 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7faf5c0cf4f0 con 0x7faf64106f10 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf696e5640 1 -- 172.21.15.122:0/1081165852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d2f20 msgr2=0x56455e8d3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf696e5640 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d2f20 0x56455e8d3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf696e5640 1 -- 172.21.15.122:0/1081165852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8e2650 msgr2=0x56455e8e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf696e5640 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8e2650 0x56455e8e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf696e5640 1 -- 172.21.15.122:0/1081165852 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56455e7d0d40 con 0x56455e8d3840 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d2f20 0x56455e8d3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf69ee6640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf500a2b80 0x7faf500a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0981b0 tx=0x7faf5c066920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf696e5640 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e2110 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faf60009a70 tx=0x7faf60009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a9edaca3c05c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf4f7fe640 1 -- 172.21.15.122:0/1081165852 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60017070 con 0x56455e8d3840 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf4f7fe640 1 -- 172.21.15.122:0/1081165852 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60013470 con 0x56455e8d3840 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.903+0000 7faf4f7fe640 1 -- 172.21.15.122:0/1081165852 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60012630 con 0x56455e8d3840 2024-09-12T01:52:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 -- 172.21.15.122:0/1081165852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 msgr2=0x56455e8e2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e2110 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faf60009a70 tx=0x7faf60009f30 comp rx=0 tx=0).stop 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 -- 172.21.15.122:0/1081165852 shutdown_connections 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8e2650 0x56455e8e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e2110 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1081165852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d2f20 0x56455e8d3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 -- 172.21.15.122:0/1081165852 >> 172.21.15.122:0/1081165852 conn(0x56455e8cf5f0 msgr2=0x56455e8cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 -- 172.21.15.122:0/1081165852 shutdown_connections 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 -- 172.21.15.122:0/1081165852 wait complete. 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 -- start start 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.904+0000 7faf6dd22740 1 Processor -- start 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d3840 0x56455e8dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 0x56455e8e0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56455e8e5220 con 0x56455e8e2650 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56455e7c0f10 con 0x56455e8d2f20 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56455e8e53a0 con 0x56455e8d3840 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 10 client.0 _ll_register_callbacks cb 0x56455e7e4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 0x56455e8e0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 0x56455e8e0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51810/0 (socket says 172.21.15.122:51810) 2024-09-12T01:52:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 -- 172.21.15.122:0/1411949797 learned_addr learned my addr 172.21.15.122:0/1411949797 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf696e5640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d3840 0x56455e8dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 -- 172.21.15.122:0/1411949797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d3840 msgr2=0x56455e8dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d3840 0x56455e8dfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 -- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 msgr2=0x56455e8df5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8df5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 -- 172.21.15.122:0/1411949797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455e547580 con 0x56455e8e2650 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 0x56455e8e0030 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faf5402a930 tx=0x7faf54041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ab3acdb897d768 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf4cff9640 1 -- 172.21.15.122:0/1411949797 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54035030 con 0x56455e8e2650 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf4cff9640 1 -- 172.21.15.122:0/1411949797 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7faf540043a0 con 0x56455e8e2650 2024-09-12T01:52:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.905+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf60013470 con 0x56455e8e2650 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf4cff9640 1 -- 172.21.15.122:0/1411949797 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf54005020 con 0x56455e8e2650 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 10 client.16524 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56455e611960 con 0x56455e8e2650 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf4cff9640 1 -- 172.21.15.122:0/1411949797 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60013470 con 0x56455e8e2650 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf4cff9640 1 -- 172.21.15.122:0/1411949797 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7faf54004cc0 con 0x56455e8e2650 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 10 client.16524 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 10 client.16524 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 20 client.16524 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:37.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e8f2280 0x56455e8f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 4 client.16524 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56455e845b90 con 0x56455e8f2280 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.906+0000 7faf696e5640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e8f2280 0x56455e8f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.907+0000 7faf4cff9640 10 client.16524 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.907+0000 7faf696e5640 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e8f2280 0x56455e8f46c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2010cc07ddf29b server_cookie=2769a2b672cd17fb in_seq=0 out_seq=0 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf4cff9640 1 -- 172.21.15.122:0/1411949797 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56455e845b90 con 0x56455e8f2280 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf4cff9640 10 client.16524 handle_command_reply: tid=0 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 client.16524 shutdown 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 msgr2=0x56455e8e0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 0x56455e8e0030 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7faf5402a930 tx=0x7faf54041040 comp rx=0 tx=0).stop 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 shutdown_connections 2024-09-12T01:52:37.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e8f2280 0x56455e8f46c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8e2650 0x56455e8e0030 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8d3840 0x56455e8dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf4cff9640 0 client.16524 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1411949797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8df5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 >> 172.21.15.122:0/1411949797 conn(0x56455e8cf5f0 msgr2=0x56455e8d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 shutdown_connections 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 1 -- 172.21.15.122:0/1411949797 wait complete. 2024-09-12T01:52:37.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.908+0000 7faf6dd22740 20 client.16524 trim_cache size 0 max 16384 2024-09-12T01:52:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 Processor -- start 2024-09-12T01:52:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 -- start start 2024-09-12T01:52:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e8f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faf641adb50 con 0x56455e8d3840 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf641ab420 con 0x56455e8d2f20 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf6dd22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf641a9250 con 0x56455e8e2650 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51820/0 (socket says 172.21.15.122:51820) 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 -- 172.21.15.122:0/1518103974 learned_addr learned my addr 172.21.15.122:0/1518103974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e8f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 -- 172.21.15.122:0/1518103974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 msgr2=0x56455e8f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e8f9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 -- 172.21.15.122:0/1518103974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 msgr2=0x56455e8ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf696e5640 1 -- 172.21.15.122:0/1518103974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56455e8e14f0 con 0x56455e8d3840 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.926+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e8f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf696e5640 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8ce640 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faf60004700 tx=0x7faf60009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7b3bdc7743efb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf4effd640 1 -- 172.21.15.122:0/1518103974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60017040 con 0x56455e8d3840 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf4effd640 1 -- 172.21.15.122:0/1518103974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf600094a0 con 0x56455e8d3840 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf4effd640 1 -- 172.21.15.122:0/1518103974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6001c7d0 con 0x56455e8d3840 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 -- 172.21.15.122:0/1518103974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 msgr2=0x56455e8ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8ce640 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faf60004700 tx=0x7faf60009110 comp rx=0 tx=0).stop 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 -- 172.21.15.122:0/1518103974 shutdown_connections 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e8f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8ce640 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1518103974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.927+0000 7faf6dd22740 1 -- 172.21.15.122:0/1518103974 >> 172.21.15.122:0/1518103974 conn(0x56455e8cf5f0 msgr2=0x56455e8cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 -- 172.21.15.122:0/1518103974 shutdown_connections 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 -- 172.21.15.122:0/1518103974 wait complete. 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 -- start start 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 Processor -- start 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e955fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faf641a98e0 con 0x56455e8d3840 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf641aad50 con 0x56455e8d2f20 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf641a9fb0 con 0x56455e8e2650 2024-09-12T01:52:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf6dd22740 10 client.0 _ll_register_callbacks cb 0x7faf54031ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf696e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf696e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51822/0 (socket says 172.21.15.122:51822) 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.928+0000 7faf696e5640 1 -- 172.21.15.122:0/1435763304 learned_addr learned my addr 172.21.15.122:0/1435763304 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e955fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf696e5640 1 -- 172.21.15.122:0/1435763304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 msgr2=0x56455e955fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf696e5640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e955fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf696e5640 1 -- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 msgr2=0x56455e8e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf696e5640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf696e5640 1 -- 172.21.15.122:0/1435763304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455e9565e0 con 0x56455e8d3840 2024-09-12T01:52:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf696e5640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e7230 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7faf60025000 tx=0x7faf60013e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5be5ffd743404a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf4cff9640 1 -- 172.21.15.122:0/1435763304 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60036060 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56455e8400a0 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf4cff9640 1 -- 172.21.15.122:0/1435763304 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7faf60013880 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf4cff9640 1 -- 172.21.15.122:0/1435763304 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf60017070 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf6dd22740 10 client.16530 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56455e95d900 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.929+0000 7faf4cff9640 1 -- 172.21.15.122:0/1435763304 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56455e8400a0 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf4cff9640 1 -- 172.21.15.122:0/1435763304 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7faf60030070 con 0x56455e8d3840 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf6dd22740 10 client.16530 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf6dd22740 10 client.16530 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf6dd22740 20 client.16530 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e95dcf0 0x56455e9600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf6dd22740 4 client.16530 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d0117e7f", "release": true, "await": true}] 2024-09-12T01:52:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d0117e7f", "release": true, "await": true}) -- 0x56455e7bc5a0 con 0x56455e95dcf0 2024-09-12T01:52:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e95dcf0 0x56455e9600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf4cff9640 10 client.16530 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.930+0000 7faf68ee4640 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e95dcf0 0x56455e9600e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37697b29fe84d447 server_cookie=20d27fd3b442d29a in_seq=0 out_seq=0 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf4cff9640 1 -- 172.21.15.122:0/1435763304 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7faf5405e260 con 0x56455e95dcf0 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf4cff9640 10 client.16530 handle_command_reply: tid=0 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 client.16530 shutdown 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 msgr2=0x56455e8e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e7230 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7faf60025000 tx=0x7faf60013e50 comp rx=0 tx=0).stop 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 shutdown_connections 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56455e95dcf0 0x56455e9600e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56455e8e2650 0x56455e955fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56455e8d3840 0x56455e8e7230 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 --2- 172.21.15.122:0/1435763304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56455e8d2f20 0x56455e8e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 >> 172.21.15.122:0/1435763304 conn(0x56455e8cf5f0 msgr2=0x56455e8dfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf4cff9640 0 client.16530 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 shutdown_connections 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.939+0000 7faf6dd22740 1 -- 172.21.15.122:0/1435763304 wait complete. 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6dd22740 20 client.16530 trim_cache size 0 max 16384 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:52:37.941 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 24, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "d0117e7f": { 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 24, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 140.1, 2024-09-12T01:52:37.942 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:52:37.943 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:52:37.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf500a2b80 msgr2=0x7faf500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf500a2b80 0x7faf500a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0981b0 tx=0x7faf5c066920 comp rx=0 tx=0).stop 2024-09-12T01:52:37.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 msgr2=0x7faf6419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:37.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 0x7faf6419cce0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00cce0 tx=0x7faf5c00a620 comp rx=0 tx=0).stop 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 shutdown_connections 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7faf500a2b80 0x7faf500a5040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf64106f10 0x7faf6419cce0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64102790 0x7faf64198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 --2- 172.21.15.122:0/2746802374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf64101de0 0x7faf64198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 >> 172.21.15.122:0/2746802374 conn(0x7faf640fd8c0 msgr2=0x7faf640fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 shutdown_connections 2024-09-12T01:52:37.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:52:37.940+0000 7faf6b949640 1 -- 172.21.15.122:0/2746802374 wait complete. 2024-09-12T01:52:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:37 smithi183 ceph-mon[35127]: pgmap v519: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.5 MiB/s wr, 89 op/s 2024-09-12T01:52:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:37 smithi151 ceph-mon[35229]: pgmap v519: 97 pgs: 97 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.5 MiB/s wr, 89 op/s 2024-09-12T01:52:38.617 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd0117e7f', total seconds elapsed: 23.2 2024-09-12T01:52:38.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 505233 10.01 MB/sec execute 176 sec latency 19254.552 ms 2024-09-12T01:52:39.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 508267 10.06 MB/sec execute 177 sec latency 41.426 ms 2024-09-12T01:52:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:39 smithi183 ceph-mon[35127]: pgmap v520: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 94 op/s 2024-09-12T01:52:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:39 smithi122 ceph-mon[30925]: pgmap v520: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 94 op/s 2024-09-12T01:52:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:39 smithi151 ceph-mon[35229]: pgmap v520: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 94 op/s 2024-09-12T01:52:40.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 510784 10.06 MB/sec execute 178 sec latency 55.754 ms 2024-09-12T01:52:41.830 INFO:tasks.workunit.client.0.smithi122.stdout: 1 512722 10.06 MB/sec execute 179 sec latency 44.796 ms 2024-09-12T01:52:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:41 smithi183 ceph-mon[35127]: pgmap v521: 97 pgs: 97 active+clean; 195 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 3.3 MiB/s wr, 96 op/s 2024-09-12T01:52:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:41 smithi122 ceph-mon[30925]: pgmap v521: 97 pgs: 97 active+clean; 195 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 3.3 MiB/s wr, 96 op/s 2024-09-12T01:52:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:41 smithi151 ceph-mon[35229]: pgmap v521: 97 pgs: 97 active+clean; 195 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 3.3 MiB/s wr, 96 op/s 2024-09-12T01:52:42.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 515682 10.10 MB/sec execute 180 sec latency 11.535 ms 2024-09-12T01:52:43.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 518922 10.15 MB/sec execute 181 sec latency 12.036 ms 2024-09-12T01:52:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:43 smithi183 ceph-mon[35127]: pgmap v522: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 94 op/s 2024-09-12T01:52:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:43 smithi122 ceph-mon[30925]: pgmap v522: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 94 op/s 2024-09-12T01:52:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:43 smithi151 ceph-mon[35229]: pgmap v522: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 94 op/s 2024-09-12T01:52:44.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 521839 10.17 MB/sec execute 182 sec latency 13.533 ms 2024-09-12T01:52:45.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 524632 10.19 MB/sec execute 183 sec latency 51.957 ms 2024-09-12T01:52:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:45 smithi183 ceph-mon[35127]: pgmap v523: 97 pgs: 97 active+clean; 195 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2024-09-12T01:52:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:45 smithi122 ceph-mon[30925]: pgmap v523: 97 pgs: 97 active+clean; 195 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2024-09-12T01:52:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:45 smithi151 ceph-mon[35229]: pgmap v523: 97 pgs: 97 active+clean; 195 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2024-09-12T01:52:46.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 526587 10.20 MB/sec execute 184 sec latency 79.374 ms 2024-09-12T01:52:47.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 529605 10.24 MB/sec execute 185 sec latency 9.919 ms 2024-09-12T01:52:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:47 smithi183 ceph-mon[35127]: pgmap v524: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-12T01:52:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:47 smithi122 ceph-mon[30925]: pgmap v524: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-12T01:52:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:47 smithi151 ceph-mon[35229]: pgmap v524: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 206 op/s 2024-09-12T01:52:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.223+0000 7f4a45791640 1 -- 172.21.15.122:0/3522992420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40103130 msgr2=0x7f4a40103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.223+0000 7f4a45791640 1 --2- 172.21.15.122:0/3522992420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40103130 0x7f4a40103530 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009930 tx=0x7f4a3002a420 comp rx=0 tx=0).stop 2024-09-12T01:52:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 -- 172.21.15.122:0/3522992420 shutdown_connections 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 --2- 172.21.15.122:0/3522992420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a40100560 0x7f4a4010a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 --2- 172.21.15.122:0/3522992420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a400ffba0 0x7f4a40100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 --2- 172.21.15.122:0/3522992420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40103130 0x7f4a40103530 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 -- 172.21.15.122:0/3522992420 >> 172.21.15.122:0/3522992420 conn(0x7f4a400fb930 msgr2=0x7f4a400fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 -- 172.21.15.122:0/3522992420 shutdown_connections 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 -- 172.21.15.122:0/3522992420 wait complete. 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 Processor -- start 2024-09-12T01:52:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.224+0000 7f4a45791640 1 -- start start 2024-09-12T01:52:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a45791640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a400ffba0 0x7f4a4019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a45791640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 0x7f4a4019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a45791640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40103130 0x7f4a401a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a45791640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4010d7f0 con 0x7f4a40100560 2024-09-12T01:52:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a45791640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4010d670 con 0x7f4a400ffba0 2024-09-12T01:52:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a45791640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4010d970 con 0x7f4a40103130 2024-09-12T01:52:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 0x7f4a4019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 0x7f4a4019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38354/0 (socket says 172.21.15.122:38354) 2024-09-12T01:52:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/1154263780 learned_addr learned my addr 172.21.15.122:0/1154263780 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3effd640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a400ffba0 0x7f4a4019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40103130 0x7f4a401a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/1154263780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40103130 msgr2=0x7f4a401a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40103130 0x7f4a401a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/1154263780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a400ffba0 msgr2=0x7f4a4019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a400ffba0 0x7f4a4019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/1154263780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a401a1cb0 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 0x7f4a4019a5d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e9f0 tx=0x7f4a3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.225+0000 7f4a27fff640 1 -- 172.21.15.122:0/1154263780 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400cd80 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.226+0000 7f4a27fff640 1 -- 172.21.15.122:0/1154263780 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a34005500 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.226+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a401a1fa0 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.226+0000 7f4a27fff640 1 -- 172.21.15.122:0/1154263780 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34010690 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.226+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a401a2460 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a27fff640 1 -- 172.21.15.122:0/1154263780 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4a340046c0 con 0x7f4a40100560 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 Processor -- start 2024-09-12T01:52:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 -- start start 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff67da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff6a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f75fec62f0 con 0x55f75ff632e0 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f75fe6e270 con 0x55f75ff629a0 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a4860a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f75fec22b0 con 0x55f75ff68370 2024-09-12T01:52:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a3f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.228+0000 7f4a3e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff6a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff62da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:57710/0 (socket says 172.21.15.122:57710) 2024-09-12T01:52:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff67da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2967714457 learned_addr learned my addr 172.21.15.122:0/2967714457 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff6a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53822/0 (socket says 172.21.15.122:53822) 2024-09-12T01:52:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a27fff640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a280a2cb0 0x7f4a280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/2967714457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 msgr2=0x55f75ff62da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff62da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/2967714457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 msgr2=0x55f75ff67da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff67da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/2967714457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f75fe6d340 con 0x55f75ff68370 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a280a2cb0 0x7f4a280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a27fff640 1 -- 172.21.15.122:0/1154263780 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4a340cf440 con 0x7f4a40100560 2024-09-12T01:52:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff6a830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a340ccf10 tx=0x7f4a34021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29884d0cc0d5a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a257fa640 1 -- 172.21.15.122:0/2967714457 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3409b7e0 con 0x55f75ff68370 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a257fa640 1 -- 172.21.15.122:0/2967714457 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3409c3a0 con 0x55f75ff68370 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a280a2cb0 0x7f4a280a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4a380107f0 tx=0x7f4a38004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.229+0000 7f4a257fa640 1 -- 172.21.15.122:0/2967714457 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3407eb20 con 0x55f75ff68370 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 -- 172.21.15.122:0/2967714457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 msgr2=0x55f75ff6a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff6a830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a340ccf10 tx=0x7f4a34021000 comp rx=0 tx=0).stop 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 -- 172.21.15.122:0/2967714457 shutdown_connections 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff6a830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff67da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2967714457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 -- 172.21.15.122:0/2967714457 >> 172.21.15.122:0/2967714457 conn(0x55f75ff5e010 msgr2=0x55f75ff5e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 -- 172.21.15.122:0/2967714457 shutdown_connections 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.230+0000 7f4a4860a740 1 -- 172.21.15.122:0/2967714457 wait complete. 2024-09-12T01:52:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 -- start start 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 Processor -- start 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff74020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f75ff6b280 con 0x55f75ff632e0 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f75fe4f290 con 0x55f75ff629a0 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f75ff6b400 con 0x55f75ff68370 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a4860a740 10 client.0 _ll_register_callbacks cb 0x55f75fe80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff74020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff6cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38380/0 (socket says 172.21.15.122:38380) 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3e7fc640 1 -- 172.21.15.122:0/2331617816 learned_addr learned my addr 172.21.15.122:0/2331617816 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 msgr2=0x55f75ff6caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff6caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2331617816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 msgr2=0x55f75ff6cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff6cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2331617816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f75fbd9210 con 0x55f75ff68370 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.231+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff74020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a38013830 tx=0x7f4a38004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f58f61df5a9d6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2331617816 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38004260 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2331617816 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4a38004400 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a3409c3a0 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2331617816 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a38030050 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 10 client.25804 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f75fca5620 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2331617816 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3409c3a0 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2331617816 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4a38020400 con 0x55f75ff68370 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 10 client.25804 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 20 client.25804 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ff787e0 0x55f75ff7ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 4 client.25804 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.232+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f75fed3a20 con 0x55f75ff787e0 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.233+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ff787e0 0x55f75ff7ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.233+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ff787e0 0x55f75ff7ac10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de45cbf9f79158c0 server_cookie=5f6ee4f25537ae16 in_seq=0 out_seq=0 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.233+0000 7f4a22ff5640 10 client.25804 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2331617816 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f75fed3a20 con 0x55f75ff787e0 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a22ff5640 10 client.25804 handle_command_reply: tid=0 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 client.25804 shutdown 2024-09-12T01:52:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 msgr2=0x55f75ff74020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff74020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a38013830 tx=0x7f4a38004ab0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 shutdown_connections 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ff787e0 0x55f75ff7ac10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff68370 0x55f75ff74020 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff632e0 0x55f75ff6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2331617816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a22ff5640 0 client.25804 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 >> 172.21.15.122:0/2331617816 conn(0x55f75ff5e010 msgr2=0x55f75ff5f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 shutdown_connections 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 1 -- 172.21.15.122:0/2331617816 wait complete. 2024-09-12T01:52:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.234+0000 7f4a4860a740 20 client.25804 trim_cache size 0 max 16384 2024-09-12T01:52:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 Processor -- start 2024-09-12T01:52:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 -- start start 2024-09-12T01:52:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 0x55f75ff62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff68370 0x55f75ff67a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401adb00 con 0x55f75ff629a0 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401ab3d0 con 0x55f75ff68370 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.252+0000 7f4a4860a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401a9200 con 0x55f75ff632e0 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 0x55f75ff62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 0x55f75ff62d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38388/0 (socket says 172.21.15.122:38388) 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 -- 172.21.15.122:0/2866272336 learned_addr learned my addr 172.21.15.122:0/2866272336 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff68370 0x55f75ff67a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 -- 172.21.15.122:0/2866272336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 msgr2=0x55f75ff73fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff73fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 -- 172.21.15.122:0/2866272336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff68370 msgr2=0x55f75ff67a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff68370 0x55f75ff67a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 -- 172.21.15.122:0/2866272336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f75ff5de30 con 0x55f75ff629a0 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 0x55f75ff62d80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009900 tx=0x7f4a30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e3b106396836e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a24ff9640 1 -- 172.21.15.122:0/2866272336 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3004b070 con 0x55f75ff629a0 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a24ff9640 1 -- 172.21.15.122:0/2866272336 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a30005520 con 0x55f75ff629a0 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.253+0000 7f4a24ff9640 1 -- 172.21.15.122:0/2866272336 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30035040 con 0x55f75ff629a0 2024-09-12T01:52:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 -- 172.21.15.122:0/2866272336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 msgr2=0x55f75ff62d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 0x55f75ff62d80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009900 tx=0x7f4a30000bc0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 -- 172.21.15.122:0/2866272336 shutdown_connections 2024-09-12T01:52:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff68370 0x55f75ff67a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff73fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2866272336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff629a0 0x55f75ff62d80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 -- 172.21.15.122:0/2866272336 >> 172.21.15.122:0/2866272336 conn(0x55f75ff5e010 msgr2=0x55f75ff64450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 -- 172.21.15.122:0/2866272336 shutdown_connections 2024-09-12T01:52:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 -- 172.21.15.122:0/2866272336 wait complete. 2024-09-12T01:52:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 -- start start 2024-09-12T01:52:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.254+0000 7f4a4860a740 1 Processor -- start 2024-09-12T01:52:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff70bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 0x55f75ffe14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401a9890 con 0x55f75ff68370 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401aad00 con 0x55f75ff629a0 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401a9f60 con 0x55f75ff632e0 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 10 client.0 _ll_register_callbacks cb 0x7f4a38009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 0x55f75ffe14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 0x55f75ffe14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38392/0 (socket says 172.21.15.122:38392) 2024-09-12T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2267979811 learned_addr learned my addr 172.21.15.122:0/2267979811 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff70bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2267979811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 msgr2=0x55f75ff71130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff71130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 msgr2=0x55f75ff70bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff70bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 -- 172.21.15.122:0/2267979811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f75ff74140 con 0x55f75ff68370 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3e7fc640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff70bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a3f7fe640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 0x55f75ffe14e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4a380207a0 tx=0x7f4a38020c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54233317d55369d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:52:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2267979811 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38024020 con 0x55f75ff68370 2024-09-12T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2267979811 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4a38004470 con 0x55f75ff68370 2024-09-12T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.255+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a30005520 con 0x55f75ff68370 2024-09-12T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2267979811 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a38045050 con 0x55f75ff68370 2024-09-12T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 10 client.16548 fetch_fsmap learned FSMap version 30 2024-09-12T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f75ffe29b0 con 0x55f75ff68370 2024-09-12T01:52:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2267979811 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a30005520 con 0x55f75ff68370 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2267979811 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4a38033040 con 0x55f75ff68370 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 10 client.16548 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 20 client.16548 populate_metadata read hostname 'smithi122' 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ffe33b0 0x55f75ffe57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 4 client.16548 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f75fecdd70 con 0x55f75ffe33b0 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.256+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ffe33b0 0x55f75ffe57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.257+0000 7f4a22ff5640 10 client.16548 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.257+0000 7f4a3effd640 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ffe33b0 0x55f75ffe57d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d9e9488161413c server_cookie=c9b7f65bf241a9ff in_seq=0 out_seq=0 2024-09-12T01:52:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.266+0000 7f4a22ff5640 1 -- 172.21.15.122:0/2267979811 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f75fecdd70 con 0x55f75ffe33b0 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.266+0000 7f4a22ff5640 10 client.16548 handle_command_reply: tid=0 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.266+0000 7f4a4860a740 1 client.16548 shutdown 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 msgr2=0x55f75ffe14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 0x55f75ffe14e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4a380207a0 tx=0x7f4a38020c40 comp rx=0 tx=0).stop 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 shutdown_connections 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f75ffe33b0 0x55f75ffe57d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f75ff68370 0x55f75ffe14e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f75ff632e0 0x55f75ff71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a22ff5640 0 client.16548 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 --2- 172.21.15.122:0/2267979811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f75ff629a0 0x55f75ff70bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 >> 172.21.15.122:0/2267979811 conn(0x55f75ff5e010 msgr2=0x55f75ff5c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 shutdown_connections 2024-09-12T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 1 -- 172.21.15.122:0/2267979811 wait complete. 2024-09-12T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a4860a740 20 client.16548 trim_cache size 0 max 16384 2024-09-12T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:52:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a280a2cb0 msgr2=0x7f4a280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a280a2cb0 0x7f4a280a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4a380107f0 tx=0x7f4a38004a40 comp rx=0 tx=0).stop 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 msgr2=0x7f4a4019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 0x7f4a4019a5d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e9f0 tx=0x7f4a3400edf0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 shutdown_connections 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4a280a2cb0 0x7f4a280a5170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40103130 0x7f4a401a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a40100560 0x7f4a4019a5d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 --2- 172.21.15.122:0/1154263780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a400ffba0 0x7f4a4019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:52:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.267+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 >> 172.21.15.122:0/1154263780 conn(0x7f4a400fb930 msgr2=0x7f4a40100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:52:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.268+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 shutdown_connections 2024-09-12T01:52:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:52:48.268+0000 7f4a45791640 1 -- 172.21.15.122:0/1154263780 wait complete. 2024-09-12T01:52:48.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 532312 10.26 MB/sec execute 186 sec latency 14.694 ms 2024-09-12T01:52:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:52:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:52:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:52:48.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:48.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:52:48.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:52:48.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:52:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:52:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:52:49.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:52:49.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:52:49.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 534591 10.27 MB/sec execute 187 sec latency 13.805 ms 2024-09-12T01:52:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:49 smithi183 ceph-mon[35127]: pgmap v525: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 235 op/s 2024-09-12T01:52:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:49 smithi122 ceph-mon[30925]: pgmap v525: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 235 op/s 2024-09-12T01:52:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:49 smithi151 ceph-mon[35229]: pgmap v525: 97 pgs: 97 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 235 op/s 2024-09-12T01:52:50.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 537119 10.29 MB/sec execute 188 sec latency 11.801 ms 2024-09-12T01:52:51.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 539986 10.32 MB/sec execute 189 sec latency 40.800 ms 2024-09-12T01:52:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:51 smithi183 ceph-mon[35127]: pgmap v526: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 297 op/s 2024-09-12T01:52:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:51 smithi122 ceph-mon[30925]: pgmap v526: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 297 op/s 2024-09-12T01:52:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:51 smithi151 ceph-mon[35229]: pgmap v526: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 297 op/s 2024-09-12T01:52:52.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 542856 10.34 MB/sec execute 190 sec latency 41.963 ms 2024-09-12T01:52:53.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 545521 10.35 MB/sec execute 191 sec latency 11.812 ms 2024-09-12T01:52:53.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:53 smithi151 ceph-mon[35229]: pgmap v527: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-12T01:52:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:53 smithi183 ceph-mon[35127]: pgmap v527: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-12T01:52:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:53 smithi122 ceph-mon[30925]: pgmap v527: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-12T01:52:54.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 548014 10.37 MB/sec execute 192 sec latency 23.164 ms 2024-09-12T01:52:55.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 551083 10.41 MB/sec execute 193 sec latency 11.418 ms 2024-09-12T01:52:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:55 smithi183 ceph-mon[35127]: pgmap v528: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-12T01:52:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:55 smithi122 ceph-mon[30925]: pgmap v528: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-12T01:52:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:55 smithi151 ceph-mon[35229]: pgmap v528: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2024-09-12T01:52:56.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 554322 10.45 MB/sec execute 194 sec latency 14.826 ms 2024-09-12T01:52:57.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 557149 10.47 MB/sec execute 195 sec latency 12.037 ms 2024-09-12T01:52:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:57 smithi183 ceph-mon[35127]: pgmap v529: 97 pgs: 97 active+clean; 203 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-09-12T01:52:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:57 smithi122 ceph-mon[30925]: pgmap v529: 97 pgs: 97 active+clean; 203 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-09-12T01:52:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:57 smithi151 ceph-mon[35229]: pgmap v529: 97 pgs: 97 active+clean; 203 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-09-12T01:52:58.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 560128 10.49 MB/sec execute 196 sec latency 10.924 ms 2024-09-12T01:52:59.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 562441 10.50 MB/sec execute 197 sec latency 14.235 ms 2024-09-12T01:52:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:52:59 smithi183 ceph-mon[35127]: pgmap v530: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 301 op/s 2024-09-12T01:52:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:52:59 smithi122 ceph-mon[30925]: pgmap v530: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 301 op/s 2024-09-12T01:53:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:52:59 smithi151 ceph-mon[35229]: pgmap v530: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 301 op/s 2024-09-12T01:53:00.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 565345 10.54 MB/sec execute 198 sec latency 12.768 ms 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 -- 172.21.15.122:0/631113475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 msgr2=0x7efc70109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 --2- 172.21.15.122:0/631113475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc70109fa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efc68009ca0 tx=0x7efc6802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 -- 172.21.15.122:0/631113475 shutdown_connections 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 --2- 172.21.15.122:0/631113475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc70109fa0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 --2- 172.21.15.122:0/631113475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc70102d50 0x7efc701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 --2- 172.21.15.122:0/631113475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc70101b60 0x7efc70101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 -- 172.21.15.122:0/631113475 >> 172.21.15.122:0/631113475 conn(0x7efc700fd2e0 msgr2=0x7efc700ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 -- 172.21.15.122:0/631113475 shutdown_connections 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 -- 172.21.15.122:0/631113475 wait complete. 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 Processor -- start 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.357+0000 7efc75352640 1 -- start start 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc70101b60 0x7efc70197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc70102d50 0x7efc70198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc7019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc7010d7e0 con 0x7efc70103710 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc7010d660 con 0x7efc70102d50 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc7010d960 con 0x7efc70101b60 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc7019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc7019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48020/0 (socket says 172.21.15.122:48020) 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 -- 172.21.15.122:0/2234515873 learned_addr learned my addr 172.21.15.122:0/2234515873 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6effd640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc70101b60 0x7efc70197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc67fff640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc70102d50 0x7efc70198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 -- 172.21.15.122:0/2234515873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc70101b60 msgr2=0x7efc70197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc70101b60 0x7efc70197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 -- 172.21.15.122:0/2234515873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc70102d50 msgr2=0x7efc70198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc70102d50 0x7efc70198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 -- 172.21.15.122:0/2234515873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7019f9d0 con 0x7efc70103710 2024-09-12T01:53:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6f7fe640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc7019f330 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efc68031ae0 tx=0x7efc68031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6cff9640 1 -- 172.21.15.122:0/2234515873 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68003b70 con 0x7efc70103710 2024-09-12T01:53:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7019fc60 con 0x7efc70103710 2024-09-12T01:53:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6cff9640 1 -- 172.21.15.122:0/2234515873 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc68003d10 con 0x7efc70103710 2024-09-12T01:53:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc6cff9640 1 -- 172.21.15.122:0/2234515873 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6802fc40 con 0x7efc70103710 2024-09-12T01:53:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.358+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701a01b0 con 0x7efc70103710 2024-09-12T01:53:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.361+0000 7efc6cff9640 1 -- 172.21.15.122:0/2234515873 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efc68004050 con 0x7efc70103710 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.361+0000 7efc781cc740 1 Processor -- start 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.361+0000 7efc781cc740 1 -- start start 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc781cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562bfdb0 0x5603562c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc781cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc781cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc781cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56035621f2f0 con 0x5603562c5780 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc781cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603561c7270 con 0x5603562c06f0 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc781cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56035621b2b0 con 0x5603562bfdb0 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6cff9640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc5c0a2c30 0x7efc5c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:55268/0 (socket says 172.21.15.122:55268) 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48030/0 (socket says 172.21.15.122:48030) 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6effd640 1 -- 172.21.15.122:0/411080480 learned_addr learned my addr 172.21.15.122:0/411080480 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562bfdb0 0x5603562c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6f7fe640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc5c0a2c30 0x7efc5c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 -- 172.21.15.122:0/411080480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562bfdb0 msgr2=0x5603562c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562bfdb0 0x5603562c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 -- 172.21.15.122:0/411080480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 msgr2=0x5603562c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 -- 172.21.15.122:0/411080480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603561c6340 con 0x5603562c5780 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6cff9640 1 -- 172.21.15.122:0/2234515873 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efc680f0150 con 0x7efc70103710 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6effd640 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc67fff640 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562c7c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7efc58009730 tx=0x7efc580096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1686cc497d810281 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc6f7fe640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc5c0a2c30 0x7efc5c0a50f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc68082b40 tx=0x7efc680b4040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.362+0000 7efc657fa640 1 -- 172.21.15.122:0/411080480 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004050 con 0x5603562c5780 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc657fa640 1 -- 172.21.15.122:0/411080480 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc58005500 con 0x5603562c5780 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc657fa640 1 -- 172.21.15.122:0/411080480 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004d00 con 0x5603562c5780 2024-09-12T01:53:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 -- 172.21.15.122:0/411080480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 msgr2=0x5603562c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562c7c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7efc58009730 tx=0x7efc580096c0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 -- 172.21.15.122:0/411080480 shutdown_connections 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562c7c40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 --2- 172.21.15.122:0/411080480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562bfdb0 0x5603562c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 -- 172.21.15.122:0/411080480 >> 172.21.15.122:0/411080480 conn(0x5603562b7010 msgr2=0x5603562b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 -- 172.21.15.122:0/411080480 shutdown_connections 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.363+0000 7efc781cc740 1 -- 172.21.15.122:0/411080480 wait complete. 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 -- start start 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 Processor -- start 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603562c8750 con 0x5603562c5780 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603561a8290 con 0x5603562bfdb0 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5603562c88d0 con 0x5603562c06f0 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc781cc740 10 client.0 _ll_register_callbacks cb 0x5603561d9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48046/0 (socket says 172.21.15.122:48046) 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 -- 172.21.15.122:0/2682747515 learned_addr learned my addr 172.21.15.122:0/2682747515 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc6effd640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 -- 172.21.15.122:0/2682747515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 msgr2=0x5603562ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 -- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 msgr2=0x5603562ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.364+0000 7efc67fff640 1 -- 172.21.15.122:0/2682747515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560355f32210 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc67fff640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562d1580 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efc58004650 tx=0x7efc58011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a546ad78f13e7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc567fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc567fc640 1 -- 172.21.15.122:0/2682747515 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58005240 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc567fc640 1 -- 172.21.15.122:0/2682747515 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efc5800f790 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc58005500 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc567fc640 1 -- 172.21.15.122:0/2682747515 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc58036050 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 10 client.16560 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560355ffe620 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc567fc640 1 -- 172.21.15.122:0/2682747515 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc58005500 con 0x5603562c5780 2024-09-12T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc567fc640 1 -- 172.21.15.122:0/2682747515 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efc5801f810 con 0x5603562c5780 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 10 client.16560 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 20 client.16560 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603562d5a00 0x5603562d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 4 client.16560 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56035622ca20 con 0x5603562d5a00 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.365+0000 7efc6effd640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603562d5a00 0x5603562d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.366+0000 7efc567fc640 10 client.16560 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.366+0000 7efc6effd640 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603562d5a00 0x5603562d7e10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be3b75d36a84833 server_cookie=89388c3f9a21f1b in_seq=0 out_seq=0 2024-09-12T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.366+0000 7efc567fc640 1 -- 172.21.15.122:0/2682747515 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56035622ca20 con 0x5603562d5a00 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.366+0000 7efc567fc640 10 client.16560 handle_command_reply: tid=0 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.366+0000 7efc781cc740 1 client.16560 shutdown 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 msgr2=0x5603562d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562d1580 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efc58004650 tx=0x7efc58011550 comp rx=0 tx=0).stop 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 shutdown_connections 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603562d5a00 0x5603562d7e10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562d1580 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc567fc640 0 client.16560 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 --2- 172.21.15.122:0/2682747515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 >> 172.21.15.122:0/2682747515 conn(0x5603562b7010 msgr2=0x5603562b8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 shutdown_connections 2024-09-12T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 1 -- 172.21.15.122:0/2682747515 wait complete. 2024-09-12T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.367+0000 7efc781cc740 20 client.16560 trim_cache size 0 max 16384 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 Processor -- start 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 -- start start 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a95d0 con 0x5603562c5780 2024-09-12T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a6ea0 con 0x5603562bfdb0 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc781cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a4cd0 con 0x5603562c06f0 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48052/0 (socket says 172.21.15.122:48052) 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 -- 172.21.15.122:0/4044670600 learned_addr learned my addr 172.21.15.122:0/4044670600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc6effd640 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 -- 172.21.15.122:0/4044670600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 msgr2=0x5603562d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 -- 172.21.15.122:0/4044670600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 msgr2=0x5603562c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.382+0000 7efc67fff640 1 -- 172.21.15.122:0/4044670600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603562c1380 con 0x5603562c5780 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc67fff640 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562b7ec0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efc5800e450 tx=0x7efc580045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d6bfbd0c327e2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc64ff9640 1 -- 172.21.15.122:0/4044670600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004810 con 0x5603562c5780 2024-09-12T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc64ff9640 1 -- 172.21.15.122:0/4044670600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc58005240 con 0x5603562c5780 2024-09-12T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc64ff9640 1 -- 172.21.15.122:0/4044670600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800f790 con 0x5603562c5780 2024-09-12T01:53:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc781cc740 1 -- 172.21.15.122:0/4044670600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 msgr2=0x5603562b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc781cc740 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562b7ec0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efc5800e450 tx=0x7efc580045a0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc781cc740 1 -- 172.21.15.122:0/4044670600 shutdown_connections 2024-09-12T01:53:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc781cc740 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562c5780 0x5603562b7ec0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc781cc740 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c06f0 0x5603562d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.383+0000 7efc781cc740 1 --2- 172.21.15.122:0/4044670600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562bfdb0 0x5603562c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- 172.21.15.122:0/4044670600 >> 172.21.15.122:0/4044670600 conn(0x5603562b7010 msgr2=0x5603562c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- 172.21.15.122:0/4044670600 shutdown_connections 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- 172.21.15.122:0/4044670600 wait complete. 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- start start 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 Processor -- start 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 0x5603562ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c5780 0x56035633e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a5360 con 0x5603562bfdb0 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a67d0 con 0x5603562c06f0 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a5a30 con 0x5603562c5780 2024-09-12T01:53:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc781cc740 10 client.0 _ll_register_callbacks cb 0x7efc60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 0x5603562ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 0x5603562ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48066/0 (socket says 172.21.15.122:48066) 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 -- 172.21.15.122:0/1757746501 learned_addr learned my addr 172.21.15.122:0/1757746501 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc6effd640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c5780 0x56035633e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 -- 172.21.15.122:0/1757746501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c5780 msgr2=0x56035633e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c5780 0x56035633e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 -- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 msgr2=0x5603562cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.384+0000 7efc67fff640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc67fff640 1 -- 172.21.15.122:0/1757746501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035633ec10 con 0x5603562bfdb0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc67fff640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 0x5603562ce520 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efc5800a010 tx=0x7efc580053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4497e6118ee6ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc567fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc567fc640 1 -- 172.21.15.122:0/1757746501 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5803e030 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc567fc640 1 -- 172.21.15.122:0/1757746501 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7efc5801f440 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc58005240 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc567fc640 1 -- 172.21.15.122:0/1757746501 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc5801f640 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 10 client.16566 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56035633ff20 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc567fc640 1 -- 172.21.15.122:0/1757746501 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc58005240 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc567fc640 1 -- 172.21.15.122:0/1757746501 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efc5801f7e0 con 0x5603562bfdb0 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 10 client.16566 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 20 client.16566 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603563408c0 0x560356342ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 4 client.16566 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:53:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.385+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560356226d70 con 0x5603563408c0 2024-09-12T01:53:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.386+0000 7efc6effd640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603563408c0 0x560356342ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.386+0000 7efc567fc640 10 client.16566 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.386+0000 7efc6effd640 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603563408c0 0x560356342ce0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fd9b7557dd6e3e server_cookie=9fe569a11d609cbe in_seq=0 out_seq=0 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc567fc640 1 -- 172.21.15.122:0/1757746501 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560356226d70 con 0x5603563408c0 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc567fc640 10 client.16566 handle_command_reply: tid=0 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 client.16566 shutdown 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 msgr2=0x5603562ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 0x5603562ce520 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efc5800a010 tx=0x7efc580053f0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 shutdown_connections 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5603563408c0 0x560356342ce0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5603562c5780 0x56035633e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603562c06f0 0x5603562cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 --2- 172.21.15.122:0/1757746501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603562bfdb0 0x5603562ce520 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc567fc640 0 client.16566 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.396+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 >> 172.21.15.122:0/1757746501 conn(0x5603562b7010 msgr2=0x5603562b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 shutdown_connections 2024-09-12T01:53:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc781cc740 1 -- 172.21.15.122:0/1757746501 wait complete. 2024-09-12T01:53:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc781cc740 20 client.16566 trim_cache size 0 max 16384 2024-09-12T01:53:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:53:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:53:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:53:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc5c0a2c30 msgr2=0x7efc5c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc5c0a2c30 0x7efc5c0a50f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc68082b40 tx=0x7efc680b4040 comp rx=0 tx=0).stop 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 msgr2=0x7efc7019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc7019f330 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efc68031ae0 tx=0x7efc68031ee0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 shutdown_connections 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc5c0a2c30 0x7efc5c0a50f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc70103710 0x7efc7019f330 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc70102d50 0x7efc70198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 --2- 172.21.15.122:0/2234515873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc70101b60 0x7efc70197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 >> 172.21.15.122:0/2234515873 conn(0x7efc700fd2e0 msgr2=0x7efc70106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 shutdown_connections 2024-09-12T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:01.397+0000 7efc75352640 1 -- 172.21.15.122:0/2234515873 wait complete. 2024-09-12T01:53:01.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 568545 10.58 MB/sec execute 199 sec latency 11.284 ms 2024-09-12T01:53:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:01 smithi183 ceph-mon[35127]: pgmap v531: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-12T01:53:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:01 smithi122 ceph-mon[30925]: pgmap v531: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-12T01:53:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:01 smithi151 ceph-mon[35229]: pgmap v531: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-12T01:53:02.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 571509 10.59 MB/sec execute 200 sec latency 10.917 ms 2024-09-12T01:53:03.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 574355 10.61 MB/sec execute 201 sec latency 14.259 ms 2024-09-12T01:53:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:03 smithi183 ceph-mon[35127]: pgmap v532: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-09-12T01:53:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:03 smithi122 ceph-mon[30925]: pgmap v532: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-09-12T01:53:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:03 smithi151 ceph-mon[35229]: pgmap v532: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-09-12T01:53:04.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 576903 10.64 MB/sec execute 202 sec latency 13.021 ms 2024-09-12T01:53:05.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 579644 10.66 MB/sec execute 203 sec latency 11.568 ms 2024-09-12T01:53:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:05 smithi183 ceph-mon[35127]: pgmap v533: 97 pgs: 97 active+clean; 212 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T01:53:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:05 smithi122 ceph-mon[30925]: pgmap v533: 97 pgs: 97 active+clean; 212 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T01:53:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:05 smithi151 ceph-mon[35229]: pgmap v533: 97 pgs: 97 active+clean; 212 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T01:53:06.831 INFO:tasks.workunit.client.0.smithi122.stdout: 1 582862 10.70 MB/sec execute 204 sec latency 11.665 ms 2024-09-12T01:53:07.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 586049 10.73 MB/sec execute 205 sec latency 35.883 ms 2024-09-12T01:53:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:07 smithi183 ceph-mon[35127]: pgmap v534: 97 pgs: 97 active+clean; 215 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2024-09-12T01:53:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:07 smithi122 ceph-mon[30925]: pgmap v534: 97 pgs: 97 active+clean; 215 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2024-09-12T01:53:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:07 smithi151 ceph-mon[35229]: pgmap v534: 97 pgs: 97 active+clean; 215 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2024-09-12T01:53:08.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 588946 10.74 MB/sec execute 206 sec latency 14.834 ms 2024-09-12T01:53:09.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 591312 10.75 MB/sec execute 207 sec latency 13.025 ms 2024-09-12T01:53:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:09 smithi183 ceph-mon[35127]: pgmap v535: 97 pgs: 97 active+clean; 220 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-09-12T01:53:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:09 smithi122 ceph-mon[30925]: pgmap v535: 97 pgs: 97 active+clean; 220 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-09-12T01:53:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:09 smithi151 ceph-mon[35229]: pgmap v535: 97 pgs: 97 active+clean; 220 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-09-12T01:53:10.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 593954 10.77 MB/sec execute 208 sec latency 10.951 ms 2024-09-12T01:53:11.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 596969 10.80 MB/sec execute 209 sec latency 63.529 ms 2024-09-12T01:53:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:11 smithi183 ceph-mon[35127]: pgmap v536: 97 pgs: 97 active+clean; 222 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 277 op/s 2024-09-12T01:53:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:11 smithi122 ceph-mon[30925]: pgmap v536: 97 pgs: 97 active+clean; 222 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 277 op/s 2024-09-12T01:53:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:11 smithi151 ceph-mon[35229]: pgmap v536: 97 pgs: 97 active+clean; 222 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 277 op/s 2024-09-12T01:53:12.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 599919 10.82 MB/sec execute 210 sec latency 17.512 ms 2024-09-12T01:53:13.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 602697 10.83 MB/sec execute 211 sec latency 10.809 ms 2024-09-12T01:53:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:13 smithi183 ceph-mon[35127]: pgmap v537: 97 pgs: 97 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-12T01:53:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:13 smithi122 ceph-mon[30925]: pgmap v537: 97 pgs: 97 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-12T01:53:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:13 smithi151 ceph-mon[35229]: pgmap v537: 97 pgs: 97 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 -- 172.21.15.122:0/623297144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 msgr2=0x7fdab0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 --2- 172.21.15.122:0/623297144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab0108e80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009ca0 tx=0x7fdaa802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 -- 172.21.15.122:0/623297144 shutdown_connections 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 --2- 172.21.15.122:0/623297144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab0108e80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 --2- 172.21.15.122:0/623297144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab0101c30 0x7fdab01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 --2- 172.21.15.122:0/623297144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab0100a40 0x7fdab0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 -- 172.21.15.122:0/623297144 >> 172.21.15.122:0/623297144 conn(0x7fdab00fc1e0 msgr2=0x7fdab00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 -- 172.21.15.122:0/623297144 shutdown_connections 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.487+0000 7fdab5d11640 1 -- 172.21.15.122:0/623297144 wait complete. 2024-09-12T01:53:14.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 Processor -- start 2024-09-12T01:53:14.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 -- start start 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab0100a40 0x7fdab0197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab0101c30 0x7fdab01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010d870 con 0x7fdab01025f0 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010d6f0 con 0x7fdab0100a40 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdab5d11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010d9f0 con 0x7fdab0101c30 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38340/0 (socket says 172.21.15.122:38340) 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 -- 172.21.15.122:0/3729941836 learned_addr learned my addr 172.21.15.122:0/3729941836 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab0100a40 0x7fdab0197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab0101c30 0x7fdab01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 -- 172.21.15.122:0/3729941836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab0101c30 msgr2=0x7fdab01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab0101c30 0x7fdab01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 -- 172.21.15.122:0/3729941836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab0100a40 msgr2=0x7fdab0197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab0100a40 0x7fdab0197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.488+0000 7fdaaffff640 1 -- 172.21.15.122:0/3729941836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab019f9b0 con 0x7fdab01025f0 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.489+0000 7fdaaffff640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab019f310 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009430 tx=0x7fdaa80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.489+0000 7fdaacff9640 1 -- 172.21.15.122:0/3729941836 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8003b70 con 0x7fdab01025f0 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.489+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab019fc40 con 0x7fdab01025f0 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.489+0000 7fdaacff9640 1 -- 172.21.15.122:0/3729941836 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa8003d10 con 0x7fdab01025f0 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.489+0000 7fdaacff9640 1 -- 172.21.15.122:0/3729941836 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa802fc50 con 0x7fdab01025f0 2024-09-12T01:53:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.489+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a0190 con 0x7fdab01025f0 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.491+0000 7fdaacff9640 1 -- 172.21.15.122:0/3729941836 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdaa8004040 con 0x7fdab01025f0 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 Processor -- start 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 -- start start 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 0x560848a01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560848a02370 0x560848a04830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5608489602f0 con 0x560848a02370 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560848908270 con 0x5608489fd2e0 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdab8aad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56084895c2b0 con 0x5608489fc9a0 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 0x560848a01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560848a02370 0x560848a04830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 0x560848a01da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:54244/0 (socket says 172.21.15.122:54244) 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/613982685 learned_addr learned my addr 172.21.15.122:0/613982685 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaacff9640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fda980a2c30 0x7fda980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaffff640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fda980a2c30 0x7fda980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 -- 172.21.15.122:0/613982685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 msgr2=0x5608489fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 -- 172.21.15.122:0/613982685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 msgr2=0x560848a01da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 0x560848a01da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaeffd640 1 -- 172.21.15.122:0/613982685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560848907340 con 0x560848a02370 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 0x560848a01da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.492+0000 7fdaacff9640 1 -- 172.21.15.122:0/3729941836 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdaa80f0670 con 0x7fdab01025f0 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560848a02370 0x560848a04830 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400efb0 tx=0x7fdaa400e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb892941c5c6352b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fda95ffb640 1 -- 172.21.15.122:0/613982685 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4018030 con 0x560848a02370 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdaaffff640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fda980a2c30 0x7fda980a50f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80bce20 tx=0x7fdaa808ffe0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fda95ffb640 1 -- 172.21.15.122:0/613982685 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa4005500 con 0x560848a02370 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fda95ffb640 1 -- 172.21.15.122:0/613982685 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4012450 con 0x560848a02370 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdab8aad740 1 -- 172.21.15.122:0/613982685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560848a02370 msgr2=0x560848a04830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdab8aad740 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560848a02370 0x560848a04830 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400efb0 tx=0x7fdaa400e550 comp rx=0 tx=0).stop 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdab8aad740 1 -- 172.21.15.122:0/613982685 shutdown_connections 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdab8aad740 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560848a02370 0x560848a04830 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdab8aad740 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fd2e0 0x560848a01da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.493+0000 7fdab8aad740 1 --2- 172.21.15.122:0/613982685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- 172.21.15.122:0/613982685 >> 172.21.15.122:0/613982685 conn(0x5608489f8010 msgr2=0x5608489f8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- 172.21.15.122:0/613982685 shutdown_connections 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- 172.21.15.122:0/613982685 wait complete. 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- start start 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 Processor -- start 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 0x560848a06c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fd2e0 0x560848a071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x560848a0e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560848a05340 con 0x5608489fc9a0 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5608488e9290 con 0x560848a02370 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560848a054c0 con 0x5608489fd2e0 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdab8aad740 10 client.0 _ll_register_callbacks cb 0x56084891a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 0x560848a06c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 0x560848a06c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38366/0 (socket says 172.21.15.122:38366) 2024-09-12T01:53:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdaaeffd640 1 -- 172.21.15.122:0/4269121605 learned_addr learned my addr 172.21.15.122:0/4269121605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.494+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fd2e0 0x560848a071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x560848a0e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 -- 172.21.15.122:0/4269121605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fd2e0 msgr2=0x560848a071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fd2e0 0x560848a071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 -- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 msgr2=0x560848a0e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x560848a0e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 -- 172.21.15.122:0/4269121605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560848673210 con 0x5608489fc9a0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 0x560848a06c90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400e840 tx=0x7fdaa4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8b537baa026e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fda8effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fda8effd640 1 -- 172.21.15.122:0/4269121605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4018070 con 0x5608489fc9a0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fda8effd640 1 -- 172.21.15.122:0/4269121605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdaa4004250 con 0x5608489fc9a0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa4005500 con 0x5608489fc9a0 2024-09-12T01:53:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fda8effd640 1 -- 172.21.15.122:0/4269121605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdaa4005330 con 0x5608489fc9a0 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdab8aad740 10 client.16578 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56084873f620 con 0x5608489fc9a0 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.495+0000 7fda8effd640 1 -- 172.21.15.122:0/4269121605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa4005500 con 0x5608489fc9a0 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fda8effd640 1 -- 172.21.15.122:0/4269121605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdaa400f040 con 0x5608489fc9a0 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdab8aad740 10 client.16578 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdab8aad740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdab8aad740 20 client.16578 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdab8aad740 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a12730 0x560848a14b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdab8aad740 4 client.16578 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084896da20 con 0x560848a12730 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a12730 0x560848a14b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fda8effd640 10 client.16578 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.496+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a12730 0x560848a14b50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b076f2efe0c8425 server_cookie=52c993f694fd4445 in_seq=0 out_seq=0 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.497+0000 7fda8effd640 1 -- 172.21.15.122:0/4269121605 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56084896da20 con 0x560848a12730 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.497+0000 7fda8effd640 10 client.16578 handle_command_reply: tid=0 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.497+0000 7fdab8aad740 1 client.16578 shutdown 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 msgr2=0x560848a06c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 0x560848a06c90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400e840 tx=0x7fdaa4005850 comp rx=0 tx=0).stop 2024-09-12T01:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 shutdown_connections 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a12730 0x560848a14b50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x560848a0e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fda8effd640 0 client.16578 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fd2e0 0x560848a071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 --2- 172.21.15.122:0/4269121605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fc9a0 0x560848a06c90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 >> 172.21.15.122:0/4269121605 conn(0x5608489f8010 msgr2=0x5608489f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 shutdown_connections 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 1 -- 172.21.15.122:0/4269121605 wait complete. 2024-09-12T01:53:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.498+0000 7fdab8aad740 20 client.16578 trim_cache size 0 max 16384 2024-09-12T01:53:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 Processor -- start 2024-09-12T01:53:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 -- start start 2024-09-12T01:53:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x5608489f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ab780 con 0x5608489fd2e0 2024-09-12T01:53:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01a9050 con 0x560848a02370 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdab8aad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01a6e80 con 0x5608489fc9a0 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38374/0 (socket says 172.21.15.122:38374) 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/3894206238 learned_addr learned my addr 172.21.15.122:0/3894206238 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x5608489f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/3894206238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 msgr2=0x5608489f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.513+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/3894206238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 msgr2=0x5608489f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x5608489f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/3894206238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608489f7e30 con 0x5608489fd2e0 2024-09-12T01:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0e1a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x5608489fdd10 tx=0x7fdaa0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba123d2c1e96406b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fda957fa640 1 -- 172.21.15.122:0/3894206238 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0005220 con 0x5608489fd2e0 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fda957fa640 1 -- 172.21.15.122:0/3894206238 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa0004c80 con 0x5608489fd2e0 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fda957fa640 1 -- 172.21.15.122:0/3894206238 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa001ea10 con 0x5608489fd2e0 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdab8aad740 1 -- 172.21.15.122:0/3894206238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 msgr2=0x560848a0e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.514+0000 7fdab8aad740 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0e1a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x5608489fdd10 tx=0x7fdaa0005f90 comp rx=0 tx=0).stop 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- 172.21.15.122:0/3894206238 shutdown_connections 2024-09-12T01:53:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560848a02370 0x5608489f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0e1a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 --2- 172.21.15.122:0/3894206238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5608489fc9a0 0x5608489f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- 172.21.15.122:0/3894206238 >> 172.21.15.122:0/3894206238 conn(0x5608489f8010 msgr2=0x560848a06a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- 172.21.15.122:0/3894206238 shutdown_connections 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- 172.21.15.122:0/3894206238 wait complete. 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- start start 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 Processor -- start 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fc9a0 0x560848a0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560848a02370 0x560848a7b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01a7510 con 0x5608489fd2e0 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01a8980 con 0x5608489fc9a0 2024-09-12T01:53:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01a7be0 con 0x560848a02370 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdab8aad740 10 client.0 _ll_register_callbacks cb 0x7fdaa00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38378/0 (socket says 172.21.15.122:38378) 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.515+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/2342119498 learned_addr learned my addr 172.21.15.122:0/2342119498 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560848a02370 0x560848a7b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fc9a0 0x560848a0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/2342119498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560848a02370 msgr2=0x560848a7b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560848a02370 0x560848a7b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fc9a0 msgr2=0x560848a0af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fc9a0 0x560848a0af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaf7fe640 1 -- 172.21.15.122:0/2342119498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560848a7b920 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fc9a0 0x560848a0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdaaf7fe640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0b4c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0010480 tx=0x7fdaa00104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c0f2b451fb1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fda8effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fda8effd640 1 -- 172.21.15.122:0/2342119498 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00047c0 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fda8effd640 1 -- 172.21.15.122:0/2342119498 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdaa0004960 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa0004c80 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fda8effd640 1 -- 172.21.15.122:0/2342119498 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdaa0005000 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdab8aad740 10 client.16584 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560848a7cad0 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.516+0000 7fda8effd640 1 -- 172.21.15.122:0/2342119498 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa0004c80 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fda8effd640 1 -- 172.21.15.122:0/2342119498 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdaa0027470 con 0x5608489fd2e0 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdab8aad740 10 client.16584 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:14.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdab8aad740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdab8aad740 20 client.16584 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdab8aad740 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a7d450 0x560848a7f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdab8aad740 4 client.16584 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560848967d70 con 0x560848a7d450 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a7d450 0x560848a7f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fdaaeffd640 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a7d450 0x560848a7f870 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684f3e5933d143f7 server_cookie=917ade976d6e3f10 in_seq=0 out_seq=0 2024-09-12T01:53:14.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.517+0000 7fda8effd640 10 client.16584 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fda8effd640 1 -- 172.21.15.122:0/2342119498 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560848967d70 con 0x560848a7d450 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fda8effd640 10 client.16584 handle_command_reply: tid=0 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 client.16584 shutdown 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 msgr2=0x560848a0b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0b4c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0010480 tx=0x7fdaa00104b0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 shutdown_connections 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560848a7d450 0x560848a7f870 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560848a02370 0x560848a7b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fda8effd640 0 client.16584 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608489fd2e0 0x560848a0b4c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 --2- 172.21.15.122:0/2342119498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5608489fc9a0 0x560848a0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 >> 172.21.15.122:0/2342119498 conn(0x5608489f8010 msgr2=0x5608489f69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 shutdown_connections 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 1 -- 172.21.15.122:0/2342119498 wait complete. 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.527+0000 7fdab8aad740 20 client.16584 trim_cache size 0 max 16384 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:53:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:53:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fda980a2c30 msgr2=0x7fda980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fda980a2c30 0x7fda980a50f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80bce20 tx=0x7fdaa808ffe0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 msgr2=0x7fdab019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab019f310 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009430 tx=0x7fdaa80319f0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 shutdown_connections 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fda980a2c30 0x7fda980a50f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdab01025f0 0x7fdab019f310 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdab0101c30 0x7fdab01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 --2- 172.21.15.122:0/3729941836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab0100a40 0x7fdab0197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 >> 172.21.15.122:0/3729941836 conn(0x7fdab00fc1e0 msgr2=0x7fdab0104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 shutdown_connections 2024-09-12T01:53:14.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:14.528+0000 7fdab5d11640 1 -- 172.21.15.122:0/3729941836 wait complete. 2024-09-12T01:53:14.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 604945 10.84 MB/sec execute 212 sec latency 11.422 ms 2024-09-12T01:53:15.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 607903 10.87 MB/sec execute 213 sec latency 8.584 ms 2024-09-12T01:53:15.946 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T01:53:15.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:53:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:15 smithi183 ceph-mon[35127]: pgmap v538: 97 pgs: 97 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-09-12T01:53:15.972 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:15 smithi122 ceph-mon[30925]: pgmap v538: 97 pgs: 97 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-09-12T01:53:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:15 smithi151 ceph-mon[35229]: pgmap v538: 97 pgs: 97 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-09-12T01:53:16.211 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:16.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 -- 172.21.15.122:0/1822536395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc116ea0 msgr2=0x7fdcdc119290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 --2- 172.21.15.122:0/1822536395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc116ea0 0x7fdcdc119290 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009ca0 tx=0x7fdcd402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 -- 172.21.15.122:0/1822536395 shutdown_connections 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 --2- 172.21.15.122:0/1822536395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc116ea0 0x7fdcdc119290 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 --2- 172.21.15.122:0/1822536395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc109b10 0x7fdcdc116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 --2- 172.21.15.122:0/1822536395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 -- 172.21.15.122:0/1822536395 >> 172.21.15.122:0/1822536395 conn(0x7fdcdc06e140 msgr2=0x7fdcdc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 -- 172.21.15.122:0/1822536395 shutdown_connections 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.691+0000 7fdce44b4640 1 -- 172.21.15.122:0/1822536395 wait complete. 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.692+0000 7fdce44b4640 1 Processor -- start 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 -- start start 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc109b10 0x7fdcdc1a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc116ea0 0x7fdcdc1a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc11c9b0 con 0x7fdcdc1091f0 2024-09-12T01:53:16.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc11c830 con 0x7fdcdc109b10 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce44b4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc11cb30 con 0x7fdcdc116ea0 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce2250640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce2250640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38410/0 (socket says 172.21.15.122:38410) 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce1a4f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc109b10 0x7fdcdc1a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.693+0000 7fdce2250640 1 -- 172.21.15.122:0/3483367759 learned_addr learned my addr 172.21.15.122:0/3483367759 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2a51640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc116ea0 0x7fdcdc1a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2250640 1 -- 172.21.15.122:0/3483367759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc116ea0 msgr2=0x7fdcdc1a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2250640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc116ea0 0x7fdcdc1a76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2250640 1 -- 172.21.15.122:0/3483367759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc109b10 msgr2=0x7fdcdc1a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2250640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc109b10 0x7fdcdc1a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2250640 1 -- 172.21.15.122:0/3483367759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1a7d80 con 0x7fdcdc1091f0 2024-09-12T01:53:16.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.694+0000 7fdce2250640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1a2dd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800d950 tx=0x7fdcd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.695+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8017070 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.695+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd8009c30 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.695+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1a8070 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.695+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd800f690 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.695+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1af950 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.697+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc1095d0 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.699+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdcd80109d0 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.700+0000 7fdcd37fe640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2cd0 0x7fdcc80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.700+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdcd80989b0 con 0x7fdcdc1091f0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.700+0000 7fdce1a4f640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2cd0 0x7fdcc80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:16.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.701+0000 7fdce1a4f640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2cd0 0x7fdcc80a5190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc073810 tx=0x7fdccc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:16.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.708+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcd8009de0 con 0x7fdcdc1091f0 2024-09-12T01:53:16.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 610683 10.89 MB/sec execute 214 sec latency 77.666 ms 2024-09-12T01:53:16.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.939+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcdc1afc00 con 0x7fdcdc1091f0 2024-09-12T01:53:16.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.939+0000 7fdcd37fe640 1 -- 172.21.15.122:0/3483367759 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fdcd809b010 con 0x7fdcdc1091f0 2024-09-12T01:53:16.940 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:53:16.940 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:53:16.941 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:53:16.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2cd0 msgr2=0x7fdcc80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2cd0 0x7fdcc80a5190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc073810 tx=0x7fdccc005f90 comp rx=0 tx=0).stop 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 msgr2=0x7fdcdc1a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1a2dd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800d950 tx=0x7fdcd800de10 comp rx=0 tx=0).stop 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 shutdown_connections 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2cd0 0x7fdcc80a5190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc116ea0 0x7fdcdc1a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcdc109b10 0x7fdcdc1a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 --2- 172.21.15.122:0/3483367759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcdc1091f0 0x7fdcdc1a2dd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 >> 172.21.15.122:0/3483367759 conn(0x7fdcdc06e140 msgr2=0x7fdcdc117380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 shutdown_connections 2024-09-12T01:53:16.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:16.942+0000 7fdce44b4640 1 -- 172.21.15.122:0/3483367759 wait complete. 2024-09-12T01:53:17.755 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T01:53:17.755 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T01:53:17.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 613295 10.89 MB/sec execute 215 sec latency 10.991 ms 2024-09-12T01:53:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:17 smithi183 ceph-mon[35127]: pgmap v539: 97 pgs: 97 active+clean; 227 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-09-12T01:53:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:17 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3483367759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:53:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:17 smithi122 ceph-mon[30925]: pgmap v539: 97 pgs: 97 active+clean; 227 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-09-12T01:53:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:17 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3483367759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:53:18.079 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:17 smithi151 ceph-mon[35229]: pgmap v539: 97 pgs: 97 active+clean; 227 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-09-12T01:53:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:17 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3483367759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:53:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.628+0000 7f22996db640 1 -- 172.21.15.122:0/2314293028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22941082e0 msgr2=0x7f229410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.628+0000 7f22996db640 1 --2- 172.21.15.122:0/2314293028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22941082e0 0x7f229410ec50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2284009930 tx=0x7f228402a420 comp rx=0 tx=0).stop 2024-09-12T01:53:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 -- 172.21.15.122:0/2314293028 shutdown_connections 2024-09-12T01:53:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 --2- 172.21.15.122:0/2314293028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22941082e0 0x7f229410ec50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 --2- 172.21.15.122:0/2314293028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2294103760 0x7f2294107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 --2- 172.21.15.122:0/2314293028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294102db0 0x7f2294103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 -- 172.21.15.122:0/2314293028 >> 172.21.15.122:0/2314293028 conn(0x7f22940fc7d0 msgr2=0x7f22940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 -- 172.21.15.122:0/2314293028 shutdown_connections 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 -- 172.21.15.122:0/2314293028 wait complete. 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.629+0000 7f22996db640 1 Processor -- start 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 -- start start 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 0x7f22941985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294103760 0x7f2294198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22941082e0 0x7f229419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22941120b0 con 0x7f2294102db0 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2294111f30 con 0x7f2294103760 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22996db640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2294112230 con 0x7f22941082e0 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 0x7f22941985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 0x7f22941985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38428/0 (socket says 172.21.15.122:38428) 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f2292ffd640 1 -- 172.21.15.122:0/1918133613 learned_addr learned my addr 172.21.15.122:0/1918133613 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22927fc640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294103760 0x7f2294198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22937fe640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22941082e0 0x7f229419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22927fc640 1 -- 172.21.15.122:0/1918133613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22941082e0 msgr2=0x7f229419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22927fc640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22941082e0 0x7f229419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22927fc640 1 -- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 msgr2=0x7f22941985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22927fc640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 0x7f22941985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f22927fc640 1 -- 172.21.15.122:0/1918133613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229419d560 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.630+0000 7f2292ffd640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 0x7f22941985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.631+0000 7f22927fc640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294103760 0x7f2294198af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22940697d0 tx=0x7f227c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.631+0000 7f228bfff640 1 -- 172.21.15.122:0/1918133613 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00cc40 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.631+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229419d850 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.631+0000 7f228bfff640 1 -- 172.21.15.122:0/1918133613 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c00cde0 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.631+0000 7f228bfff640 1 -- 172.21.15.122:0/1918133613 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c01c070 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.631+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2294100940 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.633+0000 7f229bab4740 1 Processor -- start 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.633+0000 7f228bfff640 1 -- 172.21.15.122:0/1918133613 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f227c0049c0 con 0x7f2294103760 2024-09-12T01:53:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.633+0000 7f229bab4740 1 -- start start 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f229bab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f229bab4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f229bab4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f229bab4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564a052282a0 con 0x564a052cd940 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f229bab4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a051cfb10 con 0x564a052c3080 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f229bab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a05224260 con 0x564a052c2760 2024-09-12T01:53:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f22927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f22927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38432/0 (socket says 172.21.15.122:38432) 2024-09-12T01:53:18.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f22927fc640 1 -- 172.21.15.122:0/4154110525 learned_addr learned my addr 172.21.15.122:0/4154110525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:18.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f22937fe640 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 -- 172.21.15.122:0/4154110525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 msgr2=0x564a052c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 -- 172.21.15.122:0/4154110525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 msgr2=0x564a052cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 -- 172.21.15.122:0/4154110525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a04f99b90 con 0x564a052c3080 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f228bfff640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22800a2b80 0x7f22800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f2292ffd640 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052cd400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f228c011870 tx=0x7f228c011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ee971c164e7220 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.634+0000 7f22937fe640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22800a2b80 0x7f22800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f22897fa640 1 -- 172.21.15.122:0/4154110525 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c0112f0 con 0x564a052c3080 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f228bfff640 1 -- 172.21.15.122:0/1918133613 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f227c0cef50 con 0x7f2294103760 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f22897fa640 1 -- 172.21.15.122:0/4154110525 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c003e50 con 0x564a052c3080 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f22897fa640 1 -- 172.21.15.122:0/4154110525 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c004530 con 0x564a052c3080 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 -- 172.21.15.122:0/4154110525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 msgr2=0x564a052cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f22937fe640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22800a2b80 0x7f22800a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2284006010 tx=0x7f2284009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052cd400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f228c011870 tx=0x7f228c011d30 comp rx=0 tx=0).stop 2024-09-12T01:53:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 -- 172.21.15.122:0/4154110525 shutdown_connections 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052cd400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 --2- 172.21.15.122:0/4154110525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 -- 172.21.15.122:0/4154110525 >> 172.21.15.122:0/4154110525 conn(0x564a052becf0 msgr2=0x564a052bf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 -- 172.21.15.122:0/4154110525 shutdown_connections 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.635+0000 7f229bab4740 1 -- 172.21.15.122:0/4154110525 wait complete. 2024-09-12T01:53:18.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 -- start start 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 Processor -- start 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564a052d0510 con 0x564a052c2760 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a051b06e0 con 0x564a052cd940 2024-09-12T01:53:18.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a052d0690 con 0x564a052c3080 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f229bab4740 10 client.0 _ll_register_callbacks cb 0x564a051d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38436/0 (socket says 172.21.15.122:38436) 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 -- 172.21.15.122:0/1685323971 learned_addr learned my addr 172.21.15.122:0/1685323971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f22937fe640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f22927fc640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 -- 172.21.15.122:0/1685323971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 msgr2=0x564a052cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 -- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 msgr2=0x564a052cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f2292ffd640 1 -- 172.21.15.122:0/1685323971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a052d9980 con 0x564a052c2760 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.636+0000 7f22937fe640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f2292ffd640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052ca740 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f228c00f790 tx=0x7f228c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed275b6bb16315c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f227a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f227a7fc640 1 -- 172.21.15.122:0/1685323971 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c004a40 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f227a7fc640 1 -- 172.21.15.122:0/1685323971 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f228c004be0 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f228c003e50 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f227a7fc640 1 -- 172.21.15.122:0/1685323971 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f228c004de0 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 10 client.16602 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a052dc2b0 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f227a7fc640 1 -- 172.21.15.122:0/1685323971 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c003e50 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f227a7fc640 1 -- 172.21.15.122:0/1685323971 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f228c005130 con 0x564a052c2760 2024-09-12T01:53:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 10 client.16602 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 10 client.16602 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 20 client.16602 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a052dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 4 client.16602 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.637+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a052359d0 con 0x564a052dd650 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.638+0000 7f22927fc640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a052dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.638+0000 7f22927fc640 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a052dfa40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c1c0c40da9d5f4 server_cookie=8d70be71940d7def in_seq=0 out_seq=0 2024-09-12T01:53:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.638+0000 7f227a7fc640 10 client.16602 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f227a7fc640 1 -- 172.21.15.122:0/1685323971 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a052359d0 con 0x564a052dd650 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f227a7fc640 10 client.16602 handle_command_reply: tid=0 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 client.16602 shutdown 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 msgr2=0x564a052ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052ca740 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f228c00f790 tx=0x7f228c003ba0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 shutdown_connections 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a052dfa40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f227a7fc640 0 client.16602 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 --2- 172.21.15.122:0/1685323971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052ca740 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 >> 172.21.15.122:0/1685323971 conn(0x564a052becf0 msgr2=0x564a052c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 shutdown_connections 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 1 -- 172.21.15.122:0/1685323971 wait complete. 2024-09-12T01:53:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.639+0000 7f229bab4740 20 client.16602 trim_cache size 0 max 16384 2024-09-12T01:53:18.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 Processor -- start 2024-09-12T01:53:18.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 -- start start 2024-09-12T01:53:18.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052e5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22941b2190 con 0x564a052cd940 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22941afa60 con 0x564a052c3080 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.658+0000 7f229bab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22941ad890 con 0x564a052c2760 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f22937fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f22937fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38438/0 (socket says 172.21.15.122:38438) 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f22937fe640 1 -- 172.21.15.122:0/3289930618 learned_addr learned my addr 172.21.15.122:0/3289930618 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f2292ffd640 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f22927fc640 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f2292ffd640 1 -- 172.21.15.122:0/3289930618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 msgr2=0x564a052e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f2292ffd640 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052e5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f2292ffd640 1 -- 172.21.15.122:0/3289930618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 msgr2=0x564a052e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f2292ffd640 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052e5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.659+0000 7f2292ffd640 1 -- 172.21.15.122:0/3289930618 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a052beb20 con 0x564a052c2760 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.660+0000 7f22937fe640 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.660+0000 7f2292ffd640 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c9610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f228c0055d0 tx=0x7f228c00ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d360e6ac30101b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.660+0000 7f2288ff9640 1 -- 172.21.15.122:0/3289930618 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c0057c0 con 0x564a052c2760 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.661+0000 7f2288ff9640 1 -- 172.21.15.122:0/3289930618 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c005960 con 0x564a052c2760 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.661+0000 7f2288ff9640 1 -- 172.21.15.122:0/3289930618 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c01a820 con 0x564a052c2760 2024-09-12T01:53:18.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 -- 172.21.15.122:0/3289930618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 msgr2=0x564a052c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c9610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f228c0055d0 tx=0x7f228c00ff50 comp rx=0 tx=0).stop 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 -- 172.21.15.122:0/3289930618 shutdown_connections 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052cd940 0x564a052e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052c3080 0x564a052e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 --2- 172.21.15.122:0/3289930618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c2760 0x564a052c9610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.662+0000 7f229bab4740 1 -- 172.21.15.122:0/3289930618 >> 172.21.15.122:0/3289930618 conn(0x564a052becf0 msgr2=0x564a052c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.663+0000 7f229bab4740 1 -- 172.21.15.122:0/3289930618 shutdown_connections 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.663+0000 7f229bab4740 1 -- 172.21.15.122:0/3289930618 wait complete. 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.663+0000 7f229bab4740 1 -- start start 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.663+0000 7f229bab4740 1 Processor -- start 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22941adf20 con 0x564a052c2760 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22941af390 con 0x564a052cd940 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22941ae5f0 con 0x564a052c3080 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f229bab4740 10 client.0 _ll_register_callbacks cb 0x7f227c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38440/0 (socket says 172.21.15.122:38440) 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.664+0000 7f2292ffd640 1 -- 172.21.15.122:0/2843177151 learned_addr learned my addr 172.21.15.122:0/2843177151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f2292ffd640 1 -- 172.21.15.122:0/2843177151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 msgr2=0x564a052c7cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f22937fe640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f2292ffd640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052c7cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f2292ffd640 1 -- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 msgr2=0x564a052c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f2292ffd640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f2292ffd640 1 -- 172.21.15.122:0/2843177151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a052c3ce0 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f22937fe640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f2292ffd640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052c77b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f228c011790 tx=0x7f228c005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415737aa84b56c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.665+0000 7f227a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f227a7fc640 1 -- 172.21.15.122:0/2843177151 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c01a7c0 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f227a7fc640 1 -- 172.21.15.122:0/2843177151 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f228c01a960 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f228c005960 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f227a7fc640 1 -- 172.21.15.122:0/2843177151 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f228c01ab60 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f229bab4740 10 client.16608 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a05348310 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.666+0000 7f227a7fc640 1 -- 172.21.15.122:0/2843177151 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c01ad00 con 0x564a052c2760 2024-09-12T01:53:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f227a7fc640 1 -- 172.21.15.122:0/2843177151 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f228c046020 con 0x564a052c2760 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f229bab4740 10 client.16608 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f229bab4740 10 client.16608 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f229bab4740 20 client.16608 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f229bab4740 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a0534a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f229bab4740 4 client.16608 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x7f228c005960 con 0x564a052dd650 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.667+0000 7f22927fc640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a0534a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.668+0000 7f22927fc640 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a0534a9f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cfe0c2441d988 server_cookie=150a92347ddf77e9 in_seq=0 out_seq=0 2024-09-12T01:53:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.669+0000 7f227a7fc640 10 client.16608 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:18.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.757+0000 7f227a7fc640 1 -- 172.21.15.122:0/2843177151 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f228c005960 con 0x564a052dd650 2024-09-12T01:53:18.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.757+0000 7f227a7fc640 10 client.16608 handle_command_reply: tid=0 2024-09-12T01:53:18.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.757+0000 7f229bab4740 1 client.16608 shutdown 2024-09-12T01:53:18.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.757+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 msgr2=0x564a052c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.757+0000 7f229bab4740 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052c77b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f228c011790 tx=0x7f228c005890 comp rx=0 tx=0).stop 2024-09-12T01:53:18.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 shutdown_connections 2024-09-12T01:53:18.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564a052dd650 0x564a0534a9f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a052cd940 0x564a052c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a052c3080 0x564a052c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f227a7fc640 0 client.16608 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:18.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 --2- 172.21.15.122:0/2843177151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a052c2760 0x564a052c77b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 >> 172.21.15.122:0/2843177151 conn(0x564a052becf0 msgr2=0x564a052c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:18.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 shutdown_connections 2024-09-12T01:53:18.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 1 -- 172.21.15.122:0/2843177151 wait complete. 2024-09-12T01:53:18.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f229bab4740 20 client.16608 trim_cache size 0 max 16384 2024-09-12T01:53:18.762 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:53:18.762 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:53:18.763 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:53:18.763 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 26, 2024-09-12T01:53:18.763 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:53:18.763 INFO:teuthology.orchestra.run.smithi122.stdout: "87703940": { 2024-09-12T01:53:18.763 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 26, 2024-09-12T01:53:18.764 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:53:18.764 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:53:18.764 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:53:18.764 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:53:18.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:53:18.764 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:53:18.765 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 136.6, 2024-09-12T01:53:18.765 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:53:18.765 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:53:18.765 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:53:18.765 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:53:18.766 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:53:18.766 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:53:18.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:18.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:18.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:18.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:18.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:18.767 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:53:18.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22800a2b80 msgr2=0x7f22800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22800a2b80 0x7f22800a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2284006010 tx=0x7f2284009600 comp rx=0 tx=0).stop 2024-09-12T01:53:18.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294103760 msgr2=0x7f2294198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:18.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294103760 0x7f2294198af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22940697d0 tx=0x7f227c00ee50 comp rx=0 tx=0).stop 2024-09-12T01:53:18.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 shutdown_connections 2024-09-12T01:53:18.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22800a2b80 0x7f22800a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22941082e0 0x7f229419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2294103760 0x7f2294198af0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 --2- 172.21.15.122:0/1918133613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2294102db0 0x7f22941985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:18.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.758+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 >> 172.21.15.122:0/1918133613 conn(0x7f22940fc7d0 msgr2=0x7f2294105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:18.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.759+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 shutdown_connections 2024-09-12T01:53:18.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:18.759+0000 7f22996db640 1 -- 172.21.15.122:0/1918133613 wait complete. 2024-09-12T01:53:18.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.90 MB/sec execute 216 sec latency 153.342 ms 2024-09-12T01:53:19.512 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '87703940', quiesce took 3.6 seconds. Will release after: 13.000000000000002 2024-09-12T01:53:19.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:19 smithi122 ceph-mon[30925]: pgmap v540: 97 pgs: 97 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-09-12T01:53:19.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.85 MB/sec execute 217 sec latency 1153.431 ms 2024-09-12T01:53:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:19 smithi183 ceph-mon[35127]: pgmap v540: 97 pgs: 97 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-09-12T01:53:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:19 smithi151 ceph-mon[35229]: pgmap v540: 97 pgs: 97 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-09-12T01:53:20.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.80 MB/sec execute 218 sec latency 2153.503 ms 2024-09-12T01:53:21.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.75 MB/sec execute 219 sec latency 3153.565 ms 2024-09-12T01:53:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:21 smithi183 ceph-mon[35127]: pgmap v541: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-09-12T01:53:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:21 smithi122 ceph-mon[30925]: pgmap v541: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-09-12T01:53:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:21 smithi151 ceph-mon[35229]: pgmap v541: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-09-12T01:53:22.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.70 MB/sec execute 220 sec latency 4153.666 ms 2024-09-12T01:53:23.832 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.65 MB/sec execute 221 sec latency 5153.785 ms 2024-09-12T01:53:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:23 smithi183 ceph-mon[35127]: pgmap v542: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 243 op/s 2024-09-12T01:53:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:23 smithi122 ceph-mon[30925]: pgmap v542: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 243 op/s 2024-09-12T01:53:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:23 smithi151 ceph-mon[35229]: pgmap v542: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 243 op/s 2024-09-12T01:53:24.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.60 MB/sec execute 222 sec latency 6153.936 ms 2024-09-12T01:53:25.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.56 MB/sec execute 223 sec latency 7154.027 ms 2024-09-12T01:53:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:25 smithi183 ceph-mon[35127]: pgmap v543: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-09-12T01:53:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:25 smithi122 ceph-mon[30925]: pgmap v543: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-09-12T01:53:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:25 smithi151 ceph-mon[35229]: pgmap v543: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-09-12T01:53:26.836 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.51 MB/sec execute 224 sec latency 8154.088 ms 2024-09-12T01:53:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 -- 172.21.15.122:0/1205382245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 msgr2=0x7fdcc0104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/1205382245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0104a70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009ca0 tx=0x7fdcb802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 -- 172.21.15.122:0/1205382245 shutdown_connections 2024-09-12T01:53:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/1205382245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc01078e0 0x7fdcc0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/1205382245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0104fb0 0x7fdcc01073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/1205382245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0104a70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 -- 172.21.15.122:0/1205382245 >> 172.21.15.122:0/1205382245 conn(0x7fdcc00fc1e0 msgr2=0x7fdcc00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 -- 172.21.15.122:0/1205382245 shutdown_connections 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.624+0000 7fdcc7a53640 1 -- 172.21.15.122:0/1205382245 wait complete. 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 Processor -- start 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 -- start start 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc0104fb0 0x7fdcc01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc01078e0 0x7fdcc019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010d8a0 con 0x7fdcc0102680 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010d720 con 0x7fdcc01078e0 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc7a53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010da20 con 0x7fdcc0104fb0 2024-09-12T01:53:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36970/0 (socket says 172.21.15.122:36970) 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 -- 172.21.15.122:0/97376061 learned_addr learned my addr 172.21.15.122:0/97376061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc0104fb0 0x7fdcc01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc01078e0 0x7fdcc019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 -- 172.21.15.122:0/97376061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc0104fb0 msgr2=0x7fdcc01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc0104fb0 0x7fdcc01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 -- 172.21.15.122:0/97376061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc01078e0 msgr2=0x7fdcc019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc01078e0 0x7fdcc019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc57ef640 1 -- 172.21.15.122:0/97376061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc019fac0 con 0x7fdcc0102680 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.625+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc01078e0 0x7fdcc019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:53:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.626+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0197ea0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000f80 tx=0x7fdcb80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.626+0000 7fdcbe7fc640 1 -- 172.21.15.122:0/97376061 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8003c70 con 0x7fdcc0102680 2024-09-12T01:53:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.626+0000 7fdcbe7fc640 1 -- 172.21.15.122:0/97376061 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8003e10 con 0x7fdcc0102680 2024-09-12T01:53:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.626+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc019fd50 con 0x7fdcc0102680 2024-09-12T01:53:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.626+0000 7fdcbe7fc640 1 -- 172.21.15.122:0/97376061 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb802fdc0 con 0x7fdcc0102680 2024-09-12T01:53:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.626+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a01e0 con 0x7fdcc0102680 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcbe7fc640 1 -- 172.21.15.122:0/97376061 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdcb8004040 con 0x7fdcc0102680 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 Processor -- start 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 -- start start 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd972da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9732e0 0x5648cd977da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd97a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648cd8d62f0 con 0x5648cd9729a0 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648cd87e270 con 0x5648cd978370 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.629+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648cd8d22b0 con 0x5648cd9732e0 2024-09-12T01:53:27.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd972da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd972da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36976/0 (socket says 172.21.15.122:36976) 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/106300298 learned_addr learned my addr 172.21.15.122:0/106300298 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9732e0 0x5648cd977da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd97a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/106300298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9732e0 msgr2=0x5648cd977da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcbe7fc640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdca80a2c30 0x7fdca80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9732e0 0x5648cd977da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/106300298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 msgr2=0x5648cd97a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd97a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/106300298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648cd87d340 con 0x5648cd9729a0 2024-09-12T01:53:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcbe7fc640 1 -- 172.21.15.122:0/97376061 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdcb80f0800 con 0x7fdcc0102680 2024-09-12T01:53:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd972da0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000a990 tx=0x7fdcb000ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb17ea0bd1cc8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdca80a2c30 0x7fdca80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdca77fe640 1 -- 172.21.15.122:0/106300298 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0015070 con 0x5648cd9729a0 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdca77fe640 1 -- 172.21.15.122:0/106300298 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb001aa70 con 0x5648cd9729a0 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.630+0000 7fdca77fe640 1 -- 172.21.15.122:0/106300298 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0011da0 con 0x5648cd9729a0 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdca80a2c30 0x7fdca80a50f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0017000 tx=0x7fdcb0018870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 -- 172.21.15.122:0/106300298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 msgr2=0x5648cd972da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd972da0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000a990 tx=0x7fdcb000ae50 comp rx=0 tx=0).stop 2024-09-12T01:53:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 -- 172.21.15.122:0/106300298 shutdown_connections 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd97a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9732e0 0x5648cd977da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/106300298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd972da0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 -- 172.21.15.122:0/106300298 >> 172.21.15.122:0/106300298 conn(0x5648cd96e010 msgr2=0x5648cd96e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 -- 172.21.15.122:0/106300298 shutdown_connections 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 -- 172.21.15.122:0/106300298 wait complete. 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.631+0000 7fdcca7e8740 1 -- start start 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 Processor -- start 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd97cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd97d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 0x5648cd9840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648cd97b340 con 0x5648cd978370 2024-09-12T01:53:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648cd85f290 con 0x5648cd9732e0 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648cd97b4c0 con 0x5648cd9729a0 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcca7e8740 10 client.0 _ll_register_callbacks cb 0x5648cd890140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 0x5648cd9840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 0x5648cd9840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36992/0 (socket says 172.21.15.122:36992) 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/304280055 learned_addr learned my addr 172.21.15.122:0/304280055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd97cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd97d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/304280055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 msgr2=0x5648cd97cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd97cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 msgr2=0x5648cd97d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd97d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc5ff0640 1 -- 172.21.15.122:0/304280055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648cd5e9210 con 0x5648cd978370 2024-09-12T01:53:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.632+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd97cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 0x5648cd9840f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0004810 tx=0x7fdcb0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4cd4fc93367ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdca4ff9640 1 -- 172.21.15.122:0/304280055 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0015040 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdca4ff9640 1 -- 172.21.15.122:0/304280055 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdcb0011c60 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb001aa70 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdca4ff9640 1 -- 172.21.15.122:0/304280055 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcb001ae30 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 10 client.16620 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648cd6b5620 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdca4ff9640 1 -- 172.21.15.122:0/304280055 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb001aa70 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdca4ff9640 1 -- 172.21.15.122:0/304280055 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdcb00269b0 con 0x5648cd978370 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 10 client.16620 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 20 client.16620 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9884c0 0x5648cd98a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 4 client.16620 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.633+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648cd8e3a20 con 0x5648cd9884c0 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.634+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9884c0 0x5648cd98a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.634+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9884c0 0x5648cd98a8d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281a329dff7a281b server_cookie=993cebc7f845e88a in_seq=0 out_seq=0 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.634+0000 7fdca4ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdca4ff9640 1 -- 172.21.15.122:0/304280055 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648cd8e3a20 con 0x5648cd9884c0 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdca4ff9640 10 client.16620 handle_command_reply: tid=0 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 client.16620 shutdown 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 msgr2=0x5648cd9840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 0x5648cd9840f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0004810 tx=0x7fdcb0018000 comp rx=0 tx=0).stop 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 shutdown_connections 2024-09-12T01:53:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9884c0 0x5648cd98a8d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd978370 0x5648cd9840f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd97d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/304280055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd97cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdca4ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 >> 172.21.15.122:0/304280055 conn(0x5648cd96e010 msgr2=0x5648cd96f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 shutdown_connections 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 1 -- 172.21.15.122:0/304280055 wait complete. 2024-09-12T01:53:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.635+0000 7fdcca7e8740 20 client.16620 trim_cache size 0 max 16384 2024-09-12T01:53:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.649+0000 7fdcca7e8740 1 Processor -- start 2024-09-12T01:53:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.649+0000 7fdcca7e8740 1 -- start start 2024-09-12T01:53:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd972d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 0x5648cd9840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd96eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01ab9d0 con 0x5648cd9732e0 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01a92a0 con 0x5648cd978370 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01a70d0 con 0x5648cd9729a0 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 0x5648cd9840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 0x5648cd9840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37004/0 (socket says 172.21.15.122:37004) 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 -- 172.21.15.122:0/3568511384 learned_addr learned my addr 172.21.15.122:0/3568511384 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 -- 172.21.15.122:0/3568511384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 msgr2=0x5648cd972d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd972d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd96eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd972d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 -- 172.21.15.122:0/3568511384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 msgr2=0x5648cd96eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd96eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 -- 172.21.15.122:0/3568511384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648cd96eca0 con 0x5648cd9732e0 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd972d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd96eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 0x5648cd9840b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400fb80 tx=0x7fdcb40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a626a59b4dece76c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdca6ffd640 1 -- 172.21.15.122:0/3568511384 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400fe70 con 0x5648cd9732e0 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdca6ffd640 1 -- 172.21.15.122:0/3568511384 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4005020 con 0x5648cd9732e0 2024-09-12T01:53:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.650+0000 7fdca6ffd640 1 -- 172.21.15.122:0/3568511384 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4004cb0 con 0x5648cd9732e0 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 -- 172.21.15.122:0/3568511384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 msgr2=0x5648cd9840b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 0x5648cd9840b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400fb80 tx=0x7fdcb40117b0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 -- 172.21.15.122:0/3568511384 shutdown_connections 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd978370 0x5648cd96eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9732e0 0x5648cd9840b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/3568511384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd9729a0 0x5648cd972d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 -- 172.21.15.122:0/3568511384 >> 172.21.15.122:0/3568511384 conn(0x5648cd96e010 msgr2=0x5648cd97cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 -- 172.21.15.122:0/3568511384 shutdown_connections 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 -- 172.21.15.122:0/3568511384 wait complete. 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 -- start start 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.651+0000 7fdcca7e8740 1 Processor -- start 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd981550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd981a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd978370 0x5648cd982a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01a7760 con 0x5648cd9729a0 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01a8bd0 con 0x5648cd9732e0 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01a7e30 con 0x5648cd978370 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcca7e8740 10 client.0 _ll_register_callbacks cb 0x7fdcb000a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd981550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd981550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37014/0 (socket says 172.21.15.122:37014) 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 -- 172.21.15.122:0/436542129 learned_addr learned my addr 172.21.15.122:0/436542129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc5ff0640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd978370 0x5648cd982a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 -- 172.21.15.122:0/436542129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd978370 msgr2=0x5648cd982a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd981a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd978370 0x5648cd982a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 -- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 msgr2=0x5648cd981a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd981a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 -- 172.21.15.122:0/436542129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648cd983010 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd981a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdcc57ef640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd981550 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009430 tx=0x7fdcb802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433a2cc2b3b9c347 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.652+0000 7fdca4ff9640 1 -- 172.21.15.122:0/436542129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8039e10 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdca4ff9640 1 -- 172.21.15.122:0/436542129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdcb80ee410 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb4005020 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdca4ff9640 1 -- 172.21.15.122:0/436542129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcb80bd0f0 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 10 client.16626 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648cd984360 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdca4ff9640 1 -- 172.21.15.122:0/436542129 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4005020 con 0x5648cd9729a0 2024-09-12T01:53:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdca4ff9640 1 -- 172.21.15.122:0/436542129 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdcb000cae0 con 0x5648cd9729a0 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 10 client.16626 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 20 client.16626 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9f3820 0x5648cd9f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 4 client.16626 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648cd8ddd70 con 0x5648cd9f3820 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.653+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9f3820 0x5648cd9f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.654+0000 7fdcc4fee640 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9f3820 0x5648cd9f5c40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903d9f993a7dccb9 server_cookie=5f15981b8915b43a in_seq=0 out_seq=0 2024-09-12T01:53:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.654+0000 7fdca4ff9640 10 client.16626 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.665+0000 7fdca4ff9640 1 -- 172.21.15.122:0/436542129 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648cd8ddd70 con 0x5648cd9f3820 2024-09-12T01:53:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.665+0000 7fdca4ff9640 10 client.16626 handle_command_reply: tid=0 2024-09-12T01:53:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.665+0000 7fdcca7e8740 1 client.16626 shutdown 2024-09-12T01:53:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.665+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 msgr2=0x5648cd981550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.665+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd981550 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009430 tx=0x7fdcb802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 shutdown_connections 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5648cd9f3820 0x5648cd9f5c40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648cd978370 0x5648cd982a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648cd9732e0 0x5648cd981a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdca4ff9640 0 client.16626 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 --2- 172.21.15.122:0/436542129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648cd9729a0 0x5648cd981550 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 >> 172.21.15.122:0/436542129 conn(0x5648cd96e010 msgr2=0x5648cd96c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 shutdown_connections 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 1 -- 172.21.15.122:0/436542129 wait complete. 2024-09-12T01:53:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcca7e8740 20 client.16626 trim_cache size 0 max 16384 2024-09-12T01:53:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:53:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:53:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:53:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdca80a2c30 msgr2=0x7fdca80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdca80a2c30 0x7fdca80a50f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0017000 tx=0x7fdcb0018870 comp rx=0 tx=0).stop 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 msgr2=0x7fdcc0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0197ea0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000f80 tx=0x7fdcb80319f0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 shutdown_connections 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdca80a2c30 0x7fdca80a50f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc01078e0 0x7fdcc019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcc0104fb0 0x7fdcc01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 --2- 172.21.15.122:0/97376061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc0102680 0x7fdcc0197ea0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 >> 172.21.15.122:0/97376061 conn(0x7fdcc00fc1e0 msgr2=0x7fdcc0102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.666+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 shutdown_connections 2024-09-12T01:53:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:27.667+0000 7fdcc7a53640 1 -- 172.21.15.122:0/97376061 wait complete. 2024-09-12T01:53:27.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.46 MB/sec execute 225 sec latency 9154.147 ms 2024-09-12T01:53:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:27 smithi122 ceph-mon[30925]: pgmap v544: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-12T01:53:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:27 smithi151 ceph-mon[35229]: pgmap v544: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-12T01:53:28.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:27 smithi183 ceph-mon[35127]: pgmap v544: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-12T01:53:28.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.42 MB/sec execute 226 sec latency 10154.240 ms 2024-09-12T01:53:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:28 smithi122 ceph-mon[30925]: pgmap v545: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 181 op/s 2024-09-12T01:53:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:28 smithi151 ceph-mon[35229]: pgmap v545: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 181 op/s 2024-09-12T01:53:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:28 smithi183 ceph-mon[35127]: pgmap v545: 97 pgs: 97 active+clean; 231 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 181 op/s 2024-09-12T01:53:29.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.37 MB/sec execute 227 sec latency 11154.322 ms 2024-09-12T01:53:30.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.33 MB/sec execute 228 sec latency 12154.370 ms 2024-09-12T01:53:31.834 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.28 MB/sec execute 229 sec latency 13154.487 ms 2024-09-12T01:53:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:31 smithi183 ceph-mon[35127]: pgmap v546: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-09-12T01:53:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:31 smithi122 ceph-mon[30925]: pgmap v546: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-09-12T01:53:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:31 smithi151 ceph-mon[35229]: pgmap v546: 97 pgs: 97 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.4 MiB/s wr, 147 op/s 2024-09-12T01:53:32.514 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:53:32.783 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:32.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.24 MB/sec execute 230 sec latency 14154.560 ms 2024-09-12T01:53:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 -- 172.21.15.122:0/2142323979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 msgr2=0x7f452810cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 --2- 172.21.15.122:0/2142323979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452810cca0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f452000adb0 tx=0x7f452002b7a0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 -- 172.21.15.122:0/2142323979 shutdown_connections 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 --2- 172.21.15.122:0/2142323979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452810cca0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 --2- 172.21.15.122:0/2142323979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281009a0 0x7f45281049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 --2- 172.21.15.122:0/2142323979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45280ffff0 0x7f45281003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 -- 172.21.15.122:0/2142323979 >> 172.21.15.122:0/2142323979 conn(0x7f45280fba90 msgr2=0x7f45280fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 -- 172.21.15.122:0/2142323979 shutdown_connections 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.251+0000 7f4530131640 1 -- 172.21.15.122:0/2142323979 wait complete. 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 Processor -- start 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 -- start start 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45280ffff0 0x7f4528198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45281009a0 0x7f45281988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f452810fd50 con 0x7f4528105120 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f452810fbd0 con 0x7f45281009a0 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f4530131640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f452810fed0 con 0x7f45280ffff0 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41892/0 (socket says 172.21.15.122:41892) 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 -- 172.21.15.122:0/4170772885 learned_addr learned my addr 172.21.15.122:0/4170772885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452d6cc640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45281009a0 0x7f45281988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452decd640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45280ffff0 0x7f4528198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 -- 172.21.15.122:0/4170772885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45280ffff0 msgr2=0x7f4528198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45280ffff0 0x7f4528198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 -- 172.21.15.122:0/4170772885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45281009a0 msgr2=0x7f45281988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45281009a0 0x7f45281988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452e6ce640 1 -- 172.21.15.122:0/4170772885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819d340 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.252+0000 7f452decd640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45280ffff0 0x7f4528198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.253+0000 7f452e6ce640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452819cc60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4520005cc0 tx=0x7f4520032400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.253+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520040070 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.253+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4520032a00 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.253+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452819d5d0 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.253+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200047c0 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.253+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281a4f10 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.254+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45281003d0 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.255+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4520030bf0 con 0x7f4528105120 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.255+0000 7f451effd640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f45140a2c00 0x7f45140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.256+0000 7f452decd640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f45140a2c00 0x7f45140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.256+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f45200f0500 con 0x7f4528105120 2024-09-12T01:53:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.256+0000 7f452decd640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f45140a2c00 0x7f45140a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f452400abe0 tx=0x7f452400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.260+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45200bd460 con 0x7f4528105120 2024-09-12T01:53:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.459+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45281a50e0 con 0x7f4528105120 2024-09-12T01:53:33.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.460+0000 7f451effd640 1 -- 172.21.15.122:0/4170772885 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f45200bcd60 con 0x7f4528105120 2024-09-12T01:53:33.460 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:53:33.460 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:53:33.460 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:53:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f45140a2c00 msgr2=0x7f45140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f45140a2c00 0x7f45140a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f452400abe0 tx=0x7f452400a5c0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 msgr2=0x7f452819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452819cc60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4520005cc0 tx=0x7f4520032400 comp rx=0 tx=0).stop 2024-09-12T01:53:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 shutdown_connections 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f45140a2c00 0x7f45140a50c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4528105120 0x7f452819cc60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45281009a0 0x7f45281988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 --2- 172.21.15.122:0/4170772885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45280ffff0 0x7f4528198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 >> 172.21.15.122:0/4170772885 conn(0x7f45280fba90 msgr2=0x7f45280fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 shutdown_connections 2024-09-12T01:53:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:33.462+0000 7f4530131640 1 -- 172.21.15.122:0/4170772885 wait complete. 2024-09-12T01:53:33.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:33 smithi122 ceph-mon[30925]: pgmap v547: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.2 MiB/s wr, 86 op/s 2024-09-12T01:53:33.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:33 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4170772885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:53:33.833 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.19 MB/sec execute 231 sec latency 15154.650 ms 2024-09-12T01:53:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:33 smithi183 ceph-mon[35127]: pgmap v547: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.2 MiB/s wr, 86 op/s 2024-09-12T01:53:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:33 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4170772885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:53:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:33 smithi151 ceph-mon[35229]: pgmap v547: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.2 MiB/s wr, 86 op/s 2024-09-12T01:53:34.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:33 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4170772885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:53:34.264 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 87703940 --release --await' 2024-09-12T01:53:34.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 87703940 --release --await 2024-09-12T01:53:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:53:34.842 INFO:tasks.workunit.client.0.smithi122.stdout: 1 614973 10.15 MB/sec execute 232 sec latency 16163.615 ms 2024-09-12T01:53:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 -- 172.21.15.122:0/1933883244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2564102e00 msgr2=0x7f25641031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 --2- 172.21.15.122:0/1933883244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2564102e00 0x7f25641031e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f255402a480 comp rx=0 tx=0).stop 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 -- 172.21.15.122:0/1933883244 shutdown_connections 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 --2- 172.21.15.122:0/1933883244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f256410c1a0 0x7f256410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 --2- 172.21.15.122:0/1933883244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 0x7f256410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 --2- 172.21.15.122:0/1933883244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2564102e00 0x7f25641031e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 -- 172.21.15.122:0/1933883244 >> 172.21.15.122:0/1933883244 conn(0x7f25640fc7a0 msgr2=0x7f25640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 -- 172.21.15.122:0/1933883244 shutdown_connections 2024-09-12T01:53:35.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 -- 172.21.15.122:0/1933883244 wait complete. 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 Processor -- start 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.002+0000 7f256a844640 1 -- start start 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2564102e00 0x7f25641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 0x7f2564198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 0x7f256419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2564111fb0 con 0x7f256410c1a0 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2564111e30 con 0x7f2564103720 2024-09-12T01:53:35.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2564112130 con 0x7f2564102e00 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 0x7f256419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 0x7f256419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41904/0 (socket says 172.21.15.122:41904) 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 -- 172.21.15.122:0/2965586457 learned_addr learned my addr 172.21.15.122:0/2965586457 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2563fff640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2564102e00 0x7f25641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f25637fe640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 0x7f2564198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 -- 172.21.15.122:0/2965586457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2564102e00 msgr2=0x7f25641984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2564102e00 0x7f25641984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 -- 172.21.15.122:0/2965586457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 msgr2=0x7f2564198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 0x7f2564198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 -- 172.21.15.122:0/2965586457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256419d480 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f25637fe640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 0x7f2564198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f2568de1640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 0x7f256419cda0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f255c00b970 tx=0x7f255c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f25617fa640 1 -- 172.21.15.122:0/2965586457 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c019070 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.003+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256419d770 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.004+0000 7f25617fa640 1 -- 172.21.15.122:0/2965586457 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c00cb40 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.004+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25641a5050 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.004+0000 7f25617fa640 1 -- 172.21.15.122:0/2965586457 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c007610 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f25617fa640 1 -- 172.21.15.122:0/2965586457 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f255c01c070 con 0x7f256410c1a0 2024-09-12T01:53:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 Processor -- start 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 -- start start 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 0x562d9b107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b10a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562d9b062620 con 0x562d9b0fd980 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d9b009e50 con 0x562d9b108240 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f256cc1d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d9b05e5e0 con 0x562d9b0fd060 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f25617fa640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f254c0a2b80 0x7f254c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f2563fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 0x562d9b107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f2563fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 0x562d9b107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41910/0 (socket says 172.21.15.122:41910) 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f2563fff640 1 -- 172.21.15.122:0/544545198 learned_addr learned my addr 172.21.15.122:0/544545198 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f25637fe640 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f2568de1640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f254c0a2b80 0x7f254c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.006+0000 7f25637fe640 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b10a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f25617fa640 1 -- 172.21.15.122:0/2965586457 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f255c0ce5a0 con 0x7f256410c1a0 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2563fff640 1 -- 172.21.15.122:0/544545198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 msgr2=0x562d9b0fd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2563fff640 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0fd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2563fff640 1 -- 172.21.15.122:0/544545198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 msgr2=0x562d9b10a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2563fff640 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b10a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2563fff640 1 -- 172.21.15.122:0/544545198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9affad40 con 0x562d9b0fd980 2024-09-12T01:53:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2563fff640 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 0x562d9b107d00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f2554031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd4e238a54ea9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f2568de1640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f254c0a2b80 0x7f254c0a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f255c0cc0a0 tx=0x7f255c090390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f254a7fc640 1 -- 172.21.15.122:0/544545198 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554040070 con 0x562d9b0fd980 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f254a7fc640 1 -- 172.21.15.122:0/544545198 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2554005020 con 0x562d9b0fd980 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.007+0000 7f254a7fc640 1 -- 172.21.15.122:0/544545198 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554035040 con 0x562d9b0fd980 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 -- 172.21.15.122:0/544545198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 msgr2=0x562d9b107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 0x562d9b107d00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f2554031630 comp rx=0 tx=0).stop 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 -- 172.21.15.122:0/544545198 shutdown_connections 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd980 0x562d9b107d00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 --2- 172.21.15.122:0/544545198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 -- 172.21.15.122:0/544545198 >> 172.21.15.122:0/544545198 conn(0x562d9b0f95f0 msgr2=0x562d9b0f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 -- 172.21.15.122:0/544545198 shutdown_connections 2024-09-12T01:53:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 -- 172.21.15.122:0/544545198 wait complete. 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 -- start start 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.008+0000 7f256cc1d740 1 Processor -- start 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b1050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd980 0x562d9b1055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b1050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b105b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b1050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41914/0 (socket says 172.21.15.122:41914) 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 -- 172.21.15.122:0/2209637931 learned_addr learned my addr 172.21.15.122:0/2209637931 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562d9b10ae10 con 0x562d9b0fd060 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d9afeaf10 con 0x562d9b108240 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d9b10af90 con 0x562d9b0fd980 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f256cc1d740 10 client.0 _ll_register_callbacks cb 0x562d9b00edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f2563fff640 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd980 0x562d9b1055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 -- 172.21.15.122:0/2209637931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd980 msgr2=0x562d9b1055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd980 0x562d9b1055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 -- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 msgr2=0x562d9b105b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b105b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9ad71580 con 0x562d9b0fd060 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25637fe640 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b1050b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f255800cec0 tx=0x7f2558011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9795fb6191bc30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25437fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.009+0000 7f25437fe640 1 -- 172.21.15.122:0/2209637931 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558004220 con 0x562d9b0fd060 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2554005020 con 0x562d9b0fd060 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f25437fe640 1 -- 172.21.15.122:0/2209637931 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f25580043c0 con 0x562d9b0fd060 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f25437fe640 1 -- 172.21.15.122:0/2209637931 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f255800f660 con 0x562d9b0fd060 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 10 client.16644 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:35.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562d9ae3b960 con 0x562d9b0fd060 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f25437fe640 1 -- 172.21.15.122:0/2209637931 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2554005020 con 0x562d9b0fd060 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f25437fe640 1 -- 172.21.15.122:0/2209637931 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2558019030 con 0x562d9b0fd060 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 10 client.16644 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 10 client.16644 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 20 client.16644 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b117d70 0x562d9b11a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 4 client.16644 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d9b06fb90 con 0x562d9b117d70 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.010+0000 7f2563fff640 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b117d70 0x562d9b11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.011+0000 7f2563fff640 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b117d70 0x562d9b11a160 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1d511960fd945 server_cookie=811b189daf700600 in_seq=0 out_seq=0 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.011+0000 7f25437fe640 10 client.16644 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.011+0000 7f25437fe640 1 -- 172.21.15.122:0/2209637931 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d9b06fb90 con 0x562d9b117d70 2024-09-12T01:53:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.011+0000 7f25437fe640 10 client.16644 handle_command_reply: tid=0 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.011+0000 7f256cc1d740 1 client.16644 shutdown 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 msgr2=0x562d9b1050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b1050b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f255800cec0 tx=0x7f2558011810 comp rx=0 tx=0).stop 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 shutdown_connections 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b117d70 0x562d9b11a160 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b108240 0x562d9b105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd980 0x562d9b1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2209637931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b1050b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 >> 172.21.15.122:0/2209637931 conn(0x562d9b0f95f0 msgr2=0x562d9b0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f25437fe640 0 client.16644 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 shutdown_connections 2024-09-12T01:53:35.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 1 -- 172.21.15.122:0/2209637931 wait complete. 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.012+0000 7f256cc1d740 20 client.16644 trim_cache size 0 max 16384 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 Processor -- start 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 -- start start 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b0f8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 0x562d9b11fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25641aff70 con 0x562d9b108240 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25641ad840 con 0x562d9b0fd980 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.028+0000 7f256cc1d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25641ab670 con 0x562d9b0fd060 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 0x562d9b11fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 0x562d9b11fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41922/0 (socket says 172.21.15.122:41922) 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 -- 172.21.15.122:0/2111852529 learned_addr learned my addr 172.21.15.122:0/2111852529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:35.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f2563fff640 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b0f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 -- 172.21.15.122:0/2111852529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 msgr2=0x562d9b0ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 -- 172.21.15.122:0/2111852529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 msgr2=0x562d9b0f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b0f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 -- 172.21.15.122:0/2111852529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9b107a20 con 0x562d9b108240 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f25637fe640 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 0x562d9b11fa10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f25580128a0 tx=0x7f25580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d1e0d572bb9cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f2549ffb640 1 -- 172.21.15.122:0/2111852529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255800fa00 con 0x562d9b108240 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f2549ffb640 1 -- 172.21.15.122:0/2111852529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2558003830 con 0x562d9b108240 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.029+0000 7f2549ffb640 1 -- 172.21.15.122:0/2111852529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580256b0 con 0x562d9b108240 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 -- 172.21.15.122:0/2111852529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 msgr2=0x562d9b11fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 0x562d9b11fa10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f25580128a0 tx=0x7f25580046c0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 -- 172.21.15.122:0/2111852529 shutdown_connections 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b108240 0x562d9b11fa10 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b0f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2111852529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b0fd060 0x562d9b0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 -- 172.21.15.122:0/2111852529 >> 172.21.15.122:0/2111852529 conn(0x562d9b0f95f0 msgr2=0x562d9b0f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 -- 172.21.15.122:0/2111852529 shutdown_connections 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 -- 172.21.15.122:0/2111852529 wait complete. 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 -- start start 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.030+0000 7f256cc1d740 1 Processor -- start 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b10c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b10cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b108240 0x562d9b17ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25641abd00 con 0x562d9b0fd060 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25641ad170 con 0x562d9b0fd980 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25641ac3d0 con 0x562d9b108240 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 10 client.0 _ll_register_callbacks cb 0x7f25580128d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b10c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b10c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41934/0 (socket says 172.21.15.122:41934) 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 -- 172.21.15.122:0/2870379884 learned_addr learned my addr 172.21.15.122:0/2870379884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 -- 172.21.15.122:0/2870379884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b108240 msgr2=0x562d9b17ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b108240 0x562d9b17ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 -- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 msgr2=0x562d9b10cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b10cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 -- 172.21.15.122:0/2870379884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9b17bfe0 con 0x562d9b0fd060 2024-09-12T01:53:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25637fe640 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b10c7f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f255800d010 tx=0x7f255800c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348965f87ee65c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25437fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25437fe640 1 -- 172.21.15.122:0/2870379884 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580040b0 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f25437fe640 1 -- 172.21.15.122:0/2870379884 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2558036420 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.031+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2558003830 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f25437fe640 1 -- 172.21.15.122:0/2870379884 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f255800c3d0 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 10 client.16650 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562d9b1833d0 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f25437fe640 1 -- 172.21.15.122:0/2870379884 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2558003830 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f25437fe640 1 -- 172.21.15.122:0/2870379884 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2558019070 con 0x562d9b0fd060 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 10 client.16650 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 10 client.16650 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 20 client.16650 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b183ae0 0x562d9b185f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 4 client.16650 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "87703940", "release": true, "await": true}] 2024-09-12T01:53:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "87703940", "release": true, "await": true}) -- 0x562d9b06a0a0 con 0x562d9b183ae0 2024-09-12T01:53:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.032+0000 7f2563fff640 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b183ae0 0x562d9b185f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.033+0000 7f2563fff640 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b183ae0 0x562d9b185f00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e44eb55c460b89 server_cookie=8128a202a0a86c40 in_seq=0 out_seq=0 2024-09-12T01:53:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.033+0000 7f25437fe640 10 client.16650 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.041+0000 7f25437fe640 1 -- 172.21.15.122:0/2870379884 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562d9b06a0a0 con 0x562d9b183ae0 2024-09-12T01:53:35.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.041+0000 7f25437fe640 10 client.16650 handle_command_reply: tid=0 2024-09-12T01:53:35.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.041+0000 7f256cc1d740 1 client.16650 shutdown 2024-09-12T01:53:35.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 msgr2=0x562d9b10c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b10c7f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f255800d010 tx=0x7f255800c660 comp rx=0 tx=0).stop 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 shutdown_connections 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562d9b183ae0 0x562d9b185f00 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f25437fe640 0 client.16650 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9b108240 0x562d9b17ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9b0fd980 0x562d9b10cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 --2- 172.21.15.122:0/2870379884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562d9b0fd060 0x562d9b10c7f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 >> 172.21.15.122:0/2870379884 conn(0x562d9b0f95f0 msgr2=0x562d9b0f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 shutdown_connections 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 1 -- 172.21.15.122:0/2870379884 wait complete. 2024-09-12T01:53:35.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.042+0000 7f256cc1d740 20 client.16650 trim_cache size 0 max 16384 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 28, 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "87703940": { 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 28, 2024-09-12T01:53:35.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 136.6, 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:53:35.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:35.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:35.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:35.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:35.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:53:35.057 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:53:35.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f254c0a2b80 msgr2=0x7f254c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f254c0a2b80 0x7f254c0a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f255c0cc0a0 tx=0x7f255c090390 comp rx=0 tx=0).stop 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 msgr2=0x7f256419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 0x7f256419cda0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f255c00b970 tx=0x7f255c00be30 comp rx=0 tx=0).stop 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 shutdown_connections 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f254c0a2b80 0x7f254c0a5040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f256410c1a0 0x7f256419cda0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2564103720 0x7f2564198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 --2- 172.21.15.122:0/2965586457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2564102e00 0x7f25641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 >> 172.21.15.122:0/2965586457 conn(0x7f25640fc7a0 msgr2=0x7f25640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 shutdown_connections 2024-09-12T01:53:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:53:35.043+0000 7f256a844640 1 -- 172.21.15.122:0/2965586457 wait complete. 2024-09-12T01:53:35.737 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '87703940', total seconds elapsed: 19.8 2024-09-12T01:53:35.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 617336 10.15 MB/sec execute 233 sec latency 16364.800 ms 2024-09-12T01:53:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:35 smithi183 ceph-mon[35127]: pgmap v548: 97 pgs: 97 active+clean; 234 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-09-12T01:53:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:35 smithi122 ceph-mon[30925]: pgmap v548: 97 pgs: 97 active+clean; 234 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-09-12T01:53:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:35 smithi151 ceph-mon[35229]: pgmap v548: 97 pgs: 97 active+clean; 234 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.6 MiB/s wr, 97 op/s 2024-09-12T01:53:36.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 619623 10.16 MB/sec execute 234 sec latency 11.466 ms 2024-09-12T01:53:37.842 INFO:tasks.workunit.client.0.smithi122.stdout: 1 622065 10.18 MB/sec execute 235 sec latency 64.810 ms 2024-09-12T01:53:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:37 smithi183 ceph-mon[35127]: pgmap v549: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.3 MiB/s wr, 91 op/s 2024-09-12T01:53:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:37 smithi122 ceph-mon[30925]: pgmap v549: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.3 MiB/s wr, 91 op/s 2024-09-12T01:53:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:37 smithi151 ceph-mon[35229]: pgmap v549: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.3 MiB/s wr, 91 op/s 2024-09-12T01:53:38.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 625169 10.21 MB/sec execute 236 sec latency 20.194 ms 2024-09-12T01:53:39.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 627952 10.22 MB/sec execute 237 sec latency 26.998 ms 2024-09-12T01:53:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:39 smithi183 ceph-mon[35127]: pgmap v550: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-09-12T01:53:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:39 smithi122 ceph-mon[30925]: pgmap v550: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-09-12T01:53:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:39 smithi151 ceph-mon[35229]: pgmap v550: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-09-12T01:53:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 -- 172.21.15.122:0/1872838560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158101c30 msgr2=0x7f41581020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 --2- 172.21.15.122:0/1872838560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158101c30 0x7f41581020b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4148009860 tx=0x7f414802a340 comp rx=0 tx=0).stop 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 -- 172.21.15.122:0/1872838560 shutdown_connections 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 --2- 172.21.15.122:0/1872838560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41581025f0 0x7f4158108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 --2- 172.21.15.122:0/1872838560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158101c30 0x7f41581020b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 --2- 172.21.15.122:0/1872838560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158100a40 0x7f4158100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 -- 172.21.15.122:0/1872838560 >> 172.21.15.122:0/1872838560 conn(0x7f41580fc1e0 msgr2=0x7f41580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 -- 172.21.15.122:0/1872838560 shutdown_connections 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.765+0000 7f415f48c640 1 -- 172.21.15.122:0/1872838560 wait complete. 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 Processor -- start 2024-09-12T01:53:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 -- start start 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158100a40 0x7f415806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 0x7f415819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41581025f0 0x7f415819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f415810c6c0 con 0x7f4158101c30 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f415810c540 con 0x7f4158100a40 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415f48c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f415810c840 con 0x7f41581025f0 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415d228640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158100a40 0x7f415806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 0x7f415819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 0x7f415819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51344/0 (socket says 172.21.15.122:51344) 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 -- 172.21.15.122:0/3011873209 learned_addr learned my addr 172.21.15.122:0/3011873209 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415d228640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158100a40 0x7f415806a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36886/0 (socket says 172.21.15.122:36886) 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415da29640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41581025f0 0x7f415819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 -- 172.21.15.122:0/3011873209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41581025f0 msgr2=0x7f415819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41581025f0 0x7f415819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 -- 172.21.15.122:0/3011873209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158100a40 msgr2=0x7f415806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158100a40 0x7f415806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.766+0000 7f415ca27640 1 -- 172.21.15.122:0/3011873209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415819fa60 con 0x7f4158101c30 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.767+0000 7f415ca27640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 0x7f415819cb30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4148005940 tx=0x7f4148033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.767+0000 7f414e7fc640 1 -- 172.21.15.122:0/3011873209 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414803f020 con 0x7f4158101c30 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.767+0000 7f414e7fc640 1 -- 172.21.15.122:0/3011873209 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4148030850 con 0x7f4158101c30 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.767+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415819fc90 con 0x7f4158101c30 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.767+0000 7f414e7fc640 1 -- 172.21.15.122:0/3011873209 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148037040 con 0x7f4158101c30 2024-09-12T01:53:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.767+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581a0180 con 0x7f4158101c30 2024-09-12T01:53:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.769+0000 7f414e7fc640 1 -- 172.21.15.122:0/3011873209 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4148042020 con 0x7f4158101c30 2024-09-12T01:53:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 Processor -- start 2024-09-12T01:53:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 -- start start 2024-09-12T01:53:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 0x556d36a4d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4d6f0 0x556d36a521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a54c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d369ac2f0 con 0x556d36a4cdb0 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556d36954270 con 0x556d36a52780 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f4162225740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d369a82b0 con 0x556d36a4d6f0 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 0x556d36a4d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 0x556d36a4d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51352/0 (socket says 172.21.15.122:51352) 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 -- 172.21.15.122:0/3553945925 learned_addr learned my addr 172.21.15.122:0/3553945925 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415ca27640 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a54c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f414e7fc640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41440a2c30 0x7f41440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415d228640 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4d6f0 0x556d36a521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 -- 172.21.15.122:0/3553945925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4d6f0 msgr2=0x556d36a521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4d6f0 0x556d36a521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 -- 172.21.15.122:0/3553945925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 msgr2=0x556d36a54c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a54c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 -- 172.21.15.122:0/3553945925 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d36953340 con 0x556d36a4cdb0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415ca27640 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a54c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f415da29640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41440a2c30 0x7f41440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.770+0000 7f414e7fc640 1 -- 172.21.15.122:0/3011873209 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f41480ef440 con 0x7f4158101c30 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.771+0000 7f415da29640 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 0x556d36a4d1b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f41500092c0 tx=0x7f41500108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54136f623fc7bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.771+0000 7f41437fe640 1 -- 172.21.15.122:0/3553945925 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150016050 con 0x556d36a4cdb0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.771+0000 7f41437fe640 1 -- 172.21.15.122:0/3553945925 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415001adb0 con 0x556d36a4cdb0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.771+0000 7f41437fe640 1 -- 172.21.15.122:0/3553945925 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150025c40 con 0x556d36a4cdb0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.771+0000 7f415da29640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41440a2c30 0x7f41440a50f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4150022000 tx=0x7f4150023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 -- 172.21.15.122:0/3553945925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 msgr2=0x556d36a4d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 0x556d36a4d1b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f41500092c0 tx=0x7f41500108a0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 -- 172.21.15.122:0/3553945925 shutdown_connections 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a54c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4d6f0 0x556d36a521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 --2- 172.21.15.122:0/3553945925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4cdb0 0x556d36a4d1b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 -- 172.21.15.122:0/3553945925 >> 172.21.15.122:0/3553945925 conn(0x556d36a44010 msgr2=0x556d36a44410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 -- 172.21.15.122:0/3553945925 shutdown_connections 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 -- 172.21.15.122:0/3553945925 wait complete. 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 -- start start 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.772+0000 7f4162225740 1 Processor -- start 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d36a55750 con 0x556d36a4d6f0 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556d36935290 con 0x556d36a4cdb0 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d36a558d0 con 0x556d36a52780 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4162225740 10 client.0 _ll_register_callbacks cb 0x556d36966140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415d228640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415d228640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a57090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36902/0 (socket says 172.21.15.122:36902) 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51358/0 (socket says 172.21.15.122:51358) 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415d228640 1 -- 172.21.15.122:0/1408935097 learned_addr learned my addr 172.21.15.122:0/1408935097 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415da29640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 -- 172.21.15.122:0/1408935097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 msgr2=0x556d36a5e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 -- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 msgr2=0x556d36a57090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a57090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 -- 172.21.15.122:0/1408935097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d366bf210 con 0x556d36a4d6f0 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415d228640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f415ca27640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a575d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f41480f2fd0 tx=0x7f414802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2795b1b20b341f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.773+0000 7f4140ff9640 1 -- 172.21.15.122:0/1408935097 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480bb600 con 0x556d36a4d6f0 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4140ff9640 1 -- 172.21.15.122:0/1408935097 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f41480ef080 con 0x556d36a4d6f0 2024-09-12T01:53:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f415001adb0 con 0x556d36a4d6f0 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4140ff9640 1 -- 172.21.15.122:0/1408935097 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4148030a00 con 0x556d36a4d6f0 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 10 client.16662 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d3678b620 con 0x556d36a4d6f0 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4140ff9640 1 -- 172.21.15.122:0/1408935097 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415001adb0 con 0x556d36a4d6f0 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4140ff9640 1 -- 172.21.15.122:0/1408935097 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f414803a030 con 0x556d36a4d6f0 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 10 client.16662 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 20 client.16662 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36a62cb0 0x556d36a65100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 4 client.16662 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d369b9a20 con 0x556d36a62cb0 2024-09-12T01:53:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.774+0000 7f415d228640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36a62cb0 0x556d36a65100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.775+0000 7f4140ff9640 10 client.16662 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.775+0000 7f415d228640 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36a62cb0 0x556d36a65100 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31816e288ac94749 server_cookie=fc2f2f150abf2fe2 in_seq=0 out_seq=0 2024-09-12T01:53:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.775+0000 7f4140ff9640 1 -- 172.21.15.122:0/1408935097 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d369b9a20 con 0x556d36a62cb0 2024-09-12T01:53:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.775+0000 7f4140ff9640 10 client.16662 handle_command_reply: tid=0 2024-09-12T01:53:40.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.775+0000 7f4162225740 1 client.16662 shutdown 2024-09-12T01:53:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 msgr2=0x556d36a575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a575d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f41480f2fd0 tx=0x7f414802a340 comp rx=0 tx=0).stop 2024-09-12T01:53:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 shutdown_connections 2024-09-12T01:53:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36a62cb0 0x556d36a65100 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a575d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4140ff9640 0 client.16662 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 --2- 172.21.15.122:0/1408935097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 >> 172.21.15.122:0/1408935097 conn(0x556d36a44010 msgr2=0x556d36a45820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 shutdown_connections 2024-09-12T01:53:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 1 -- 172.21.15.122:0/1408935097 wait complete. 2024-09-12T01:53:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.776+0000 7f4162225740 20 client.16662 trim_cache size 0 max 16384 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 Processor -- start 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 -- start start 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4cdb0 0x556d36a430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a9620 con 0x556d36a4d6f0 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a6ef0 con 0x556d36a52780 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f4162225740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a4d20 con 0x556d36a4cdb0 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51366/0 (socket says 172.21.15.122:51366) 2024-09-12T01:53:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 -- 172.21.15.122:0/2930289050 learned_addr learned my addr 172.21.15.122:0/2930289050 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415da29640 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415d228640 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4cdb0 0x556d36a430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 -- 172.21.15.122:0/2930289050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4cdb0 msgr2=0x556d36a430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4cdb0 0x556d36a430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 -- 172.21.15.122:0/2930289050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 msgr2=0x556d36a45400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a45400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 -- 172.21.15.122:0/2930289050 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d36a65780 con 0x556d36a4d6f0 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.792+0000 7f415ca27640 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5e560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4148009830 tx=0x7f4148005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d221f6301ed6d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4142ffd640 1 -- 172.21.15.122:0/2930289050 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480035f0 con 0x556d36a4d6f0 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4142ffd640 1 -- 172.21.15.122:0/2930289050 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41480bc3a0 con 0x556d36a4d6f0 2024-09-12T01:53:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4142ffd640 1 -- 172.21.15.122:0/2930289050 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480bb0f0 con 0x556d36a4d6f0 2024-09-12T01:53:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 -- 172.21.15.122:0/2930289050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 msgr2=0x556d36a5e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5e560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4148009830 tx=0x7f4148005af0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 -- 172.21.15.122:0/2930289050 shutdown_connections 2024-09-12T01:53:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a52780 0x556d36a45400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5e560 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 --2- 172.21.15.122:0/2930289050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a4cdb0 0x556d36a430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 -- 172.21.15.122:0/2930289050 >> 172.21.15.122:0/2930289050 conn(0x556d36a44010 msgr2=0x556d36a429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 -- 172.21.15.122:0/2930289050 shutdown_connections 2024-09-12T01:53:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.793+0000 7f4162225740 1 -- 172.21.15.122:0/2930289050 wait complete. 2024-09-12T01:53:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 -- start start 2024-09-12T01:53:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 Processor -- start 2024-09-12T01:53:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a5bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a53b0 con 0x556d36a4d6f0 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a6820 con 0x556d36a4cdb0 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a5a80 con 0x556d36a52780 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f4162225740 10 client.0 _ll_register_callbacks cb 0x7f4148005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f415d228640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a5bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f415da29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f415ca27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51380/0 (socket says 172.21.15.122:51380) 2024-09-12T01:53:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.794+0000 7f415ca27640 1 -- 172.21.15.122:0/512426510 learned_addr learned my addr 172.21.15.122:0/512426510 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415d228640 1 -- 172.21.15.122:0/512426510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 msgr2=0x556d36a5d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415d228640 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415d228640 1 -- 172.21.15.122:0/512426510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 msgr2=0x556d36a5c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415d228640 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415d228640 1 -- 172.21.15.122:0/512426510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d36a5d690 con 0x556d36a4cdb0 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415ca27640 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f415d228640 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a5bb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x556d36a57c00 tx=0x7f4154012a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebd5e2fa9f1e630 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4140ff9640 1 -- 172.21.15.122:0/512426510 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154004250 con 0x556d36a4cdb0 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41480bc3a0 con 0x556d36a4cdb0 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4140ff9640 1 -- 172.21.15.122:0/512426510 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f41540043f0 con 0x556d36a4cdb0 2024-09-12T01:53:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4140ff9640 1 -- 172.21.15.122:0/512426510 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41540035f0 con 0x556d36a4cdb0 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4162225740 10 client.25916 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.795+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d36acd9e0 con 0x556d36a4cdb0 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4140ff9640 1 -- 172.21.15.122:0/512426510 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41480bc3a0 con 0x556d36a4cdb0 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4140ff9640 1 -- 172.21.15.122:0/512426510 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4154020970 con 0x556d36a4cdb0 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4162225740 10 client.25916 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4162225740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4162225740 20 client.25916 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4162225740 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36ace260 0x556d36ad0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4162225740 4 client.25916 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d369b3d70 con 0x556d36ace260 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f415ca27640 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36ace260 0x556d36ad0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:40.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f415ca27640 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36ace260 0x556d36ad0680 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e008f7cf597d827 server_cookie=12237ecb68251d95 in_seq=0 out_seq=0 2024-09-12T01:53:40.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.796+0000 7f4140ff9640 10 client.25916 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.806+0000 7f4140ff9640 1 -- 172.21.15.122:0/512426510 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d369b3d70 con 0x556d36ace260 2024-09-12T01:53:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.806+0000 7f4140ff9640 10 client.25916 handle_command_reply: tid=0 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 client.25916 shutdown 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 msgr2=0x556d36a5bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a5bb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x556d36a57c00 tx=0x7f4154012a60 comp rx=0 tx=0).stop 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 shutdown_connections 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556d36ace260 0x556d36ad0680 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d36a52780 0x556d36a5d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4140ff9640 0 client.25916 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d36a4d6f0 0x556d36a5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 --2- 172.21.15.122:0/512426510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d36a4cdb0 0x556d36a5bb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 >> 172.21.15.122:0/512426510 conn(0x556d36a44010 msgr2=0x556d36a431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 shutdown_connections 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 1 -- 172.21.15.122:0/512426510 wait complete. 2024-09-12T01:53:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f4162225740 20 client.25916 trim_cache size 0 max 16384 2024-09-12T01:53:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:53:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:53:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:53:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41440a2c30 msgr2=0x7f41440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f415f48c640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41440a2c30 0x7f41440a50f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4150022000 tx=0x7f4150023000 comp rx=0 tx=0).stop 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 msgr2=0x7f415819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.807+0000 7f415f48c640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 0x7f415819cb30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4148005940 tx=0x7f4148033d90 comp rx=0 tx=0).stop 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 shutdown_connections 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f41440a2c30 0x7f41440a50f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41581025f0 0x7f415819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4158101c30 0x7f415819cb30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 --2- 172.21.15.122:0/3011873209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4158100a40 0x7f415806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 >> 172.21.15.122:0/3011873209 conn(0x7f41580fc1e0 msgr2=0x7f4158104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 shutdown_connections 2024-09-12T01:53:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:40.808+0000 7f415f48c640 1 -- 172.21.15.122:0/3011873209 wait complete. 2024-09-12T01:53:40.842 INFO:tasks.workunit.client.0.smithi122.stdout: 1 630525 10.24 MB/sec execute 238 sec latency 10.153 ms 2024-09-12T01:53:41.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 633140 10.25 MB/sec execute 239 sec latency 13.122 ms 2024-09-12T01:53:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:41 smithi183 ceph-mon[35127]: pgmap v551: 97 pgs: 97 active+clean; 233 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-09-12T01:53:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:41 smithi122 ceph-mon[30925]: pgmap v551: 97 pgs: 97 active+clean; 233 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-09-12T01:53:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:41 smithi151 ceph-mon[35229]: pgmap v551: 97 pgs: 97 active+clean; 233 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-09-12T01:53:42.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 635952 10.29 MB/sec execute 240 sec latency 67.405 ms 2024-09-12T01:53:43.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 638882 10.30 MB/sec execute 241 sec latency 18.855 ms 2024-09-12T01:53:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:43 smithi183 ceph-mon[35127]: pgmap v552: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-12T01:53:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:43 smithi122 ceph-mon[30925]: pgmap v552: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-12T01:53:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:43 smithi151 ceph-mon[35229]: pgmap v552: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-12T01:53:44.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 641674 10.31 MB/sec execute 242 sec latency 11.558 ms 2024-09-12T01:53:45.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 644021 10.33 MB/sec execute 243 sec latency 15.117 ms 2024-09-12T01:53:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:45 smithi183 ceph-mon[35127]: pgmap v553: 97 pgs: 97 active+clean; 239 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-09-12T01:53:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:45 smithi122 ceph-mon[30925]: pgmap v553: 97 pgs: 97 active+clean; 239 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-09-12T01:53:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:45 smithi151 ceph-mon[35229]: pgmap v553: 97 pgs: 97 active+clean; 239 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-09-12T01:53:46.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 646902 10.35 MB/sec execute 244 sec latency 10.285 ms 2024-09-12T01:53:47.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 649336 10.36 MB/sec execute 245 sec latency 69.129 ms 2024-09-12T01:53:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:47 smithi183 ceph-mon[35127]: pgmap v554: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 252 op/s 2024-09-12T01:53:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:47 smithi122 ceph-mon[30925]: pgmap v554: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 252 op/s 2024-09-12T01:53:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:47 smithi151 ceph-mon[35229]: pgmap v554: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 252 op/s 2024-09-12T01:53:48.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 651667 10.37 MB/sec execute 246 sec latency 15.613 ms 2024-09-12T01:53:48.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:48 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:48 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:48 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:53:49.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 654230 10.38 MB/sec execute 247 sec latency 12.192 ms 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: pgmap v555: 97 pgs: 97 active+clean; 241 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 326 op/s 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:53:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:53:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: pgmap v555: 97 pgs: 97 active+clean; 241 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 326 op/s 2024-09-12T01:53:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:53:49.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:53:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: pgmap v555: 97 pgs: 97 active+clean; 241 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 326 op/s 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:53:50.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:53:50.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:53:50.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:53:50.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 657269 10.42 MB/sec execute 248 sec latency 9.368 ms 2024-09-12T01:53:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:50 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi122 to 175.9M 2024-09-12T01:53:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:50 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T01:53:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:50 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-12T01:53:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:50 smithi183 ceph-mon[35127]: Unable to set osd_memory_target on smithi122 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-12T01:53:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:50 smithi183 ceph-mon[35127]: Unable to set osd_memory_target on smithi183 to 184571699: error parsing value: Value '184571699' is below minimum 939524096 2024-09-12T01:53:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:50 smithi183 ceph-mon[35127]: Unable to set osd_memory_target on smithi151 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-12T01:53:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:50 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi122 to 175.9M 2024-09-12T01:53:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:50 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T01:53:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:50 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-12T01:53:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:50 smithi122 ceph-mon[30925]: Unable to set osd_memory_target on smithi122 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-12T01:53:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:50 smithi122 ceph-mon[30925]: Unable to set osd_memory_target on smithi183 to 184571699: error parsing value: Value '184571699' is below minimum 939524096 2024-09-12T01:53:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:50 smithi122 ceph-mon[30925]: Unable to set osd_memory_target on smithi151 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-12T01:53:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:50 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi122 to 175.9M 2024-09-12T01:53:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:50 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T01:53:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:50 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-12T01:53:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:50 smithi151 ceph-mon[35229]: Unable to set osd_memory_target on smithi122 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-12T01:53:51.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:50 smithi151 ceph-mon[35229]: Unable to set osd_memory_target on smithi183 to 184571699: error parsing value: Value '184571699' is below minimum 939524096 2024-09-12T01:53:51.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:50 smithi151 ceph-mon[35229]: Unable to set osd_memory_target on smithi151 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-12T01:53:51.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 660005 10.43 MB/sec execute 249 sec latency 12.480 ms 2024-09-12T01:53:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:51 smithi183 ceph-mon[35127]: pgmap v556: 97 pgs: 97 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-09-12T01:53:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:51 smithi122 ceph-mon[30925]: pgmap v556: 97 pgs: 97 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-09-12T01:53:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:51 smithi151 ceph-mon[35229]: pgmap v556: 97 pgs: 97 active+clean; 242 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-09-12T01:53:52.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 662877 10.44 MB/sec execute 250 sec latency 17.849 ms 2024-09-12T01:53:53.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 665237 10.45 MB/sec execute 251 sec latency 10.416 ms 2024-09-12T01:53:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 -- 172.21.15.122:0/1498646430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654107840 msgr2=0x7f8654109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 --2- 172.21.15.122:0/1498646430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654107840 0x7f8654109cb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f864c009ca0 tx=0x7f864c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 -- 172.21.15.122:0/1498646430 shutdown_connections 2024-09-12T01:53:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 --2- 172.21.15.122:0/1498646430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654107840 0x7f8654109cb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 --2- 172.21.15.122:0/1498646430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654104f10 0x7f8654107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 --2- 172.21.15.122:0/1498646430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86541025e0 0x7f86541049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 -- 172.21.15.122:0/1498646430 >> 172.21.15.122:0/1498646430 conn(0x7f86540fc1e0 msgr2=0x7f86540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 -- 172.21.15.122:0/1498646430 shutdown_connections 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 -- 172.21.15.122:0/1498646430 wait complete. 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 Processor -- start 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.896+0000 7f865adef640 1 -- start start 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865adef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86541025e0 0x7f8654197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865adef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 0x7f8654198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865adef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8654107840 0x7f865419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865adef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f865410d870 con 0x7f8654104f10 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865adef640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f865410d6f0 con 0x7f8654107840 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865adef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f865410d9f0 con 0x7f86541025e0 2024-09-12T01:53:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 0x7f8654198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8658b8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86541025e0 0x7f8654197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 0x7f8654198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38984/0 (socket says 172.21.15.122:38984) 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 -- 172.21.15.122:0/87044157 learned_addr learned my addr 172.21.15.122:0/87044157 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8658b8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86541025e0 0x7f8654197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:42374/0 (socket says 172.21.15.122:42374) 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f865938c640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8654107840 0x7f865419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 -- 172.21.15.122:0/87044157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86541025e0 msgr2=0x7f8654197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86541025e0 0x7f8654197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 -- 172.21.15.122:0/87044157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8654107840 msgr2=0x7f865419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8654107840 0x7f865419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 -- 172.21.15.122:0/87044157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865419fa40 con 0x7f8654104f10 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8653fff640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 0x7f8654198360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f864000c970 tx=0x7f864000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.897+0000 7f8651ffb640 1 -- 172.21.15.122:0/87044157 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640007bb0 con 0x7f8654104f10 2024-09-12T01:53:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.898+0000 7f8651ffb640 1 -- 172.21.15.122:0/87044157 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8640007d50 con 0x7f8654104f10 2024-09-12T01:53:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.898+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865419fd30 con 0x7f8654104f10 2024-09-12T01:53:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.898+0000 7f8651ffb640 1 -- 172.21.15.122:0/87044157 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400187a0 con 0x7f8654104f10 2024-09-12T01:53:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.898+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86541a0220 con 0x7f8654104f10 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f8651ffb640 1 -- 172.21.15.122:0/87044157 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f864000f450 con 0x7f8654104f10 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 Processor -- start 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 -- start start 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fafdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb00830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b92fa5c2f0 con 0x55b92faf92e0 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b92fa04270 con 0x55b92faf89a0 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.900+0000 7f865dc65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b92fa582b0 con 0x55b92fafe370 2024-09-12T01:53:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f865938c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fafdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8651ffb640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86440a2cb0 0x7f86440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f865938c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:42758/0 (socket says 172.21.15.122:42758) 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f865938c640 1 -- 172.21.15.122:0/1877865710 learned_addr learned my addr 172.21.15.122:0/1877865710 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fafdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38990/0 (socket says 172.21.15.122:38990) 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8653fff640 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb00830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f865938c640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86440a2cb0 0x7f86440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 -- 172.21.15.122:0/1877865710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 msgr2=0x55b92fb00830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb00830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8651ffb640 1 -- 172.21.15.122:0/87044157 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86400cefb0 con 0x7f8654104f10 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 -- 172.21.15.122:0/1877865710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 msgr2=0x55b92faf8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 -- 172.21.15.122:0/1877865710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b92fa03340 con 0x55b92faf92e0 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f865938c640 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8653fff640 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb00830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f8658b8b640 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fafdda0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8648009c20 tx=0x7f8648011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be009152368a096f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f865938c640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86440a2cb0 0x7f86440a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f864c005e30 tx=0x7f864c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f863effd640 1 -- 172.21.15.122:0/1877865710 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648004050 con 0x55b92faf92e0 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f863effd640 1 -- 172.21.15.122:0/1877865710 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8648005500 con 0x55b92faf92e0 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.901+0000 7f863effd640 1 -- 172.21.15.122:0/1877865710 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648004e00 con 0x55b92faf92e0 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 -- 172.21.15.122:0/1877865710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 msgr2=0x55b92fafdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fafdda0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8648009c20 tx=0x7f8648011810 comp rx=0 tx=0).stop 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 -- 172.21.15.122:0/1877865710 shutdown_connections 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb00830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fafdda0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 --2- 172.21.15.122:0/1877865710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 -- 172.21.15.122:0/1877865710 >> 172.21.15.122:0/1877865710 conn(0x55b92faf4010 msgr2=0x55b92faf4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 -- 172.21.15.122:0/1877865710 shutdown_connections 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 -- 172.21.15.122:0/1877865710 wait complete. 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.902+0000 7f865dc65740 1 -- start start 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 Processor -- start 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf89a0 0x55b92fb02bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb03100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb0a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b92fb01340 con 0x55b92faf92e0 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b92f9e5290 con 0x55b92fafe370 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b92fb014c0 con 0x55b92faf89a0 2024-09-12T01:53:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865dc65740 10 client.0 _ll_register_callbacks cb 0x55b92fa16140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb03100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb03100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38992/0 (socket says 172.21.15.122:38992) 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 -- 172.21.15.122:0/1142238585 learned_addr learned my addr 172.21.15.122:0/1142238585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f865938c640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb0a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8658b8b640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf89a0 0x55b92fb02bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 -- 172.21.15.122:0/1142238585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf89a0 msgr2=0x55b92fb02bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf89a0 0x55b92fb02bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 -- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 msgr2=0x55b92fb0a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb0a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.903+0000 7f8653fff640 1 -- 172.21.15.122:0/1142238585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92f76f210 con 0x55b92faf92e0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8653fff640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb03100 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8640007590 tx=0x7f864009b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afc51058a65eb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8633fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8633fff640 1 -- 172.21.15.122:0/1142238585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864009bed0 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8633fff640 1 -- 172.21.15.122:0/1142238585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f864000f740 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8648005500 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8633fff640 1 -- 172.21.15.122:0/1142238585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86400ba1f0 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 10 client.16680 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b92f83b620 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8633fff640 1 -- 172.21.15.122:0/1142238585 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8648005500 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f8633fff640 1 -- 172.21.15.122:0/1142238585 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8640020030 con 0x55b92faf92e0 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 10 client.16680 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 20 client.16680 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb0e7c0 0x55b92fb10bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 4 client.16680 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:53:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.904+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b92fa69a20 con 0x55b92fb0e7c0 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.905+0000 7f8658b8b640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb0e7c0 0x55b92fb10bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.905+0000 7f8633fff640 10 client.16680 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.905+0000 7f8658b8b640 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb0e7c0 0x55b92fb10bb0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c091bd243df20c2 server_cookie=aad0313d696370ae in_seq=0 out_seq=0 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.905+0000 7f8633fff640 1 -- 172.21.15.122:0/1142238585 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b92fa69a20 con 0x55b92fb0e7c0 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.905+0000 7f8633fff640 10 client.16680 handle_command_reply: tid=0 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 client.16680 shutdown 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 msgr2=0x55b92fb03100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb03100 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8640007590 tx=0x7f864009b130 comp rx=0 tx=0).stop 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 shutdown_connections 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb0e7c0 0x55b92fb10bb0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb0a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb03100 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f8633fff640 0 client.16680 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 --2- 172.21.15.122:0/1142238585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf89a0 0x55b92fb02bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 >> 172.21.15.122:0/1142238585 conn(0x55b92faf4010 msgr2=0x55b92faf5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 shutdown_connections 2024-09-12T01:53:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 1 -- 172.21.15.122:0/1142238585 wait complete. 2024-09-12T01:53:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.906+0000 7f865dc65740 20 client.16680 trim_cache size 0 max 16384 2024-09-12T01:53:53.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:53 smithi151 ceph-mon[35229]: pgmap v557: 97 pgs: 97 active+clean; 244 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 343 op/s 2024-09-12T01:53:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 Processor -- start 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 -- start start 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb15470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86541ab930 con 0x55b92faf92e0 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86541a9200 con 0x55b92faf89a0 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865dc65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86541a7030 con 0x55b92fafe370 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb0a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38998/0 (socket says 172.21.15.122:38998) 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 -- 172.21.15.122:0/3192171364 learned_addr learned my addr 172.21.15.122:0/3192171364 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8658b8b640 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f865938c640 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb15470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 -- 172.21.15.122:0/3192171364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 msgr2=0x55b92fb15470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb15470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 -- 172.21.15.122:0/3192171364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 msgr2=0x55b92faf30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.921+0000 7f8653fff640 1 -- 172.21.15.122:0/3192171364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b92faef2d0 con 0x55b92faf92e0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f8653fff640 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb0a090 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8640097c30 tx=0x7f864000fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17af5519913affd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f863e7fc640 1 -- 172.21.15.122:0/3192171364 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864009fb80 con 0x55b92faf92e0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f863e7fc640 1 -- 172.21.15.122:0/3192171364 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86400a4440 con 0x55b92faf92e0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f863e7fc640 1 -- 172.21.15.122:0/3192171364 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640097520 con 0x55b92faf92e0 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 -- 172.21.15.122:0/3192171364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 msgr2=0x55b92fb0a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb0a090 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8640097c30 tx=0x7f864000fe00 comp rx=0 tx=0).stop 2024-09-12T01:53:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 -- 172.21.15.122:0/3192171364 shutdown_connections 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92fafe370 0x55b92fb15470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf92e0 0x55b92fb0a090 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 --2- 172.21.15.122:0/3192171364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92faf89a0 0x55b92faf30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.922+0000 7f865dc65740 1 -- 172.21.15.122:0/3192171364 >> 172.21.15.122:0/3192171364 conn(0x55b92faf4010 msgr2=0x55b92faf29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 -- 172.21.15.122:0/3192171364 shutdown_connections 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 -- 172.21.15.122:0/3192171364 wait complete. 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 -- start start 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 Processor -- start 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 0x55b92fb079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf92e0 0x55b92fb07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb08f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86541a76c0 con 0x55b92faf89a0 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86541a8b30 con 0x55b92fafe370 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86541a7d90 con 0x55b92faf92e0 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865dc65740 10 client.0 _ll_register_callbacks cb 0x7f86400046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f8658b8b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 0x55b92fb079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f8658b8b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 0x55b92fb079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39000/0 (socket says 172.21.15.122:39000) 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f8658b8b640 1 -- 172.21.15.122:0/2092846086 learned_addr learned my addr 172.21.15.122:0/2092846086 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.923+0000 7f865938c640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb08f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8658b8b640 1 -- 172.21.15.122:0/2092846086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf92e0 msgr2=0x55b92fb07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8653fff640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf92e0 0x55b92fb07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8658b8b640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf92e0 0x55b92fb07f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8658b8b640 1 -- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 msgr2=0x55b92fb08f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8658b8b640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb08f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8658b8b640 1 -- 172.21.15.122:0/2092846086 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92fb09550 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8658b8b640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 0x55b92fb079f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8648004760 tx=0x7f8648009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd03ce8c15e3f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8633fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8633fff640 1 -- 172.21.15.122:0/2092846086 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648032070 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8633fff640 1 -- 172.21.15.122:0/2092846086 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8648004220 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86400a4440 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8633fff640 1 -- 172.21.15.122:0/2092846086 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f864803d050 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f865dc65740 10 client.16686 fetch_fsmap learned FSMap version 30 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b92fb79670 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8633fff640 1 -- 172.21.15.122:0/2092846086 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86400a4440 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f8633fff640 1 -- 172.21.15.122:0/2092846086 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f86480035f0 con 0x55b92faf89a0 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.924+0000 7f865dc65740 10 client.16686 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:53:53.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f865dc65740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f865dc65740 20 client.16686 populate_metadata read hostname 'smithi122' 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f865dc65740 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb79e90 0x55b92fb7c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f865dc65740 4 client.16686 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b92fa63d70 con 0x55b92fb79e90 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f8653fff640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb79e90 0x55b92fb7c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f8633fff640 10 client.16686 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.925+0000 7f8653fff640 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb79e90 0x55b92fb7c2b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8481784c8dfe3f2 server_cookie=727583c3b867204f in_seq=0 out_seq=0 2024-09-12T01:53:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f8633fff640 1 -- 172.21.15.122:0/2092846086 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b92fa63d70 con 0x55b92fb79e90 2024-09-12T01:53:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f8633fff640 10 client.16686 handle_command_reply: tid=0 2024-09-12T01:53:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 client.16686 shutdown 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 msgr2=0x55b92fb079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 0x55b92fb079f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8648004760 tx=0x7f8648009680 comp rx=0 tx=0).stop 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 shutdown_connections 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b92fb79e90 0x55b92fb7c2b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b92fafe370 0x55b92fb08f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b92faf92e0 0x55b92fb07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f8633fff640 0 client.16686 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 --2- 172.21.15.122:0/2092846086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b92faf89a0 0x55b92fb079f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.935+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 >> 172.21.15.122:0/2092846086 conn(0x55b92faf4010 msgr2=0x55b92faf29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 shutdown_connections 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865dc65740 1 -- 172.21.15.122:0/2092846086 wait complete. 2024-09-12T01:53:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865dc65740 20 client.16686 trim_cache size 0 max 16384 2024-09-12T01:53:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:53:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:53:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:53:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86440a2cb0 msgr2=0x7f86440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86440a2cb0 0x7f86440a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f864c005e30 tx=0x7f864c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 msgr2=0x7f8654198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 0x7f8654198360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f864000c970 tx=0x7f864000ce30 comp rx=0 tx=0).stop 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 shutdown_connections 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86440a2cb0 0x7f86440a5170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8654107840 0x7f865419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8654104f10 0x7f8654198360 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 --2- 172.21.15.122:0/87044157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86541025e0 0x7f8654197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:53:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 >> 172.21.15.122:0/87044157 conn(0x7f86540fc1e0 msgr2=0x7f8654102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:53:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 shutdown_connections 2024-09-12T01:53:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:53:53.936+0000 7f865adef640 1 -- 172.21.15.122:0/87044157 wait complete. 2024-09-12T01:53:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:53 smithi183 ceph-mon[35127]: pgmap v557: 97 pgs: 97 active+clean; 244 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 343 op/s 2024-09-12T01:53:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:53 smithi122 ceph-mon[30925]: pgmap v557: 97 pgs: 97 active+clean; 244 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 343 op/s 2024-09-12T01:53:54.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 668173 10.48 MB/sec execute 252 sec latency 9.212 ms 2024-09-12T01:53:55.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 671387 10.51 MB/sec execute 253 sec latency 10.274 ms 2024-09-12T01:53:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:55 smithi183 ceph-mon[35127]: pgmap v558: 97 pgs: 97 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-09-12T01:53:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:55 smithi122 ceph-mon[30925]: pgmap v558: 97 pgs: 97 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-09-12T01:53:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:55 smithi151 ceph-mon[35229]: pgmap v558: 97 pgs: 97 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 358 op/s 2024-09-12T01:53:56.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 674198 10.52 MB/sec execute 254 sec latency 17.870 ms 2024-09-12T01:53:57.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 676966 10.53 MB/sec execute 255 sec latency 11.962 ms 2024-09-12T01:53:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:57 smithi183 ceph-mon[35127]: pgmap v559: 97 pgs: 97 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 278 op/s 2024-09-12T01:53:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:57 smithi122 ceph-mon[30925]: pgmap v559: 97 pgs: 97 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 278 op/s 2024-09-12T01:53:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:57 smithi151 ceph-mon[35229]: pgmap v559: 97 pgs: 97 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 278 op/s 2024-09-12T01:53:58.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 679212 10.54 MB/sec execute 256 sec latency 11.401 ms 2024-09-12T01:53:59.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 682142 10.57 MB/sec execute 257 sec latency 43.968 ms 2024-09-12T01:53:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:53:59 smithi183 ceph-mon[35127]: pgmap v560: 97 pgs: 97 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-09-12T01:53:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:53:59 smithi122 ceph-mon[30925]: pgmap v560: 97 pgs: 97 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-09-12T01:54:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:53:59 smithi151 ceph-mon[35229]: pgmap v560: 97 pgs: 97 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2024-09-12T01:54:00.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 685032 10.58 MB/sec execute 258 sec latency 15.856 ms 2024-09-12T01:54:01.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 687802 10.59 MB/sec execute 259 sec latency 10.793 ms 2024-09-12T01:54:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:01 smithi183 ceph-mon[35127]: pgmap v561: 97 pgs: 97 active+clean; 250 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-09-12T01:54:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:01 smithi122 ceph-mon[30925]: pgmap v561: 97 pgs: 97 active+clean; 250 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-09-12T01:54:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:01 smithi151 ceph-mon[35229]: pgmap v561: 97 pgs: 97 active+clean; 250 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-09-12T01:54:02.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 689927 10.60 MB/sec execute 260 sec latency 16.108 ms 2024-09-12T01:54:03.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 692976 10.63 MB/sec execute 261 sec latency 10.745 ms 2024-09-12T01:54:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:03 smithi183 ceph-mon[35127]: pgmap v562: 97 pgs: 97 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-09-12T01:54:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:03 smithi122 ceph-mon[30925]: pgmap v562: 97 pgs: 97 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-09-12T01:54:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:03 smithi151 ceph-mon[35229]: pgmap v562: 97 pgs: 97 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-09-12T01:54:04.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 696187 10.66 MB/sec execute 262 sec latency 10.350 ms 2024-09-12T01:54:05.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 699049 10.67 MB/sec execute 263 sec latency 18.009 ms 2024-09-12T01:54:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:05 smithi183 ceph-mon[35127]: pgmap v563: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-12T01:54:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:05 smithi122 ceph-mon[30925]: pgmap v563: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-12T01:54:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:05 smithi151 ceph-mon[35229]: pgmap v563: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-12T01:54:06.566 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:54:06.567 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T01:54:06.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:54:06.839 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:06.843 INFO:tasks.workunit.client.0.smithi122.stdout: 1 701985 10.68 MB/sec execute 264 sec latency 12.843 ms 2024-09-12T01:54:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1868315746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d64104f70 msgr2=0x7f1d64107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1868315746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d64104f70 0x7f1d64107360 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009900 tx=0x7f1d5802a410 comp rx=0 tx=0).stop 2024-09-12T01:54:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1868315746 shutdown_connections 2024-09-12T01:54:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1868315746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d641078a0 0x7f1d64109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1868315746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d64104f70 0x7f1d64107360 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1868315746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64102640 0x7f1d64104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1868315746 >> 172.21.15.122:0/1868315746 conn(0x7f1d640fc1e0 msgr2=0x7f1d640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1868315746 shutdown_connections 2024-09-12T01:54:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1868315746 wait complete. 2024-09-12T01:54:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 Processor -- start 2024-09-12T01:54:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.027+0000 7f1d6b2f1640 1 -- start start 2024-09-12T01:54:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6b2f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d64102640 0x7f1d64197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6b2f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104f70 0x7f1d64198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6b2f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d641078a0 0x7f1d6419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6b2f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410d800 con 0x7f1d641078a0 2024-09-12T01:54:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6b2f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410d680 con 0x7f1d64104f70 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6b2f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410d980 con 0x7f1d64102640 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6988e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d641078a0 0x7f1d6419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104f70 0x7f1d64198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6988e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d641078a0 0x7f1d6419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40472/0 (socket says 172.21.15.122:40472) 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6988e640 1 -- 172.21.15.122:0/1547392035 learned_addr learned my addr 172.21.15.122:0/1547392035 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6908d640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d64102640 0x7f1d64197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 -- 172.21.15.122:0/1547392035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d64102640 msgr2=0x7f1d64197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d64102640 0x7f1d64197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 -- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d641078a0 msgr2=0x7f1d6419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d641078a0 0x7f1d6419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 -- 172.21.15.122:0/1547392035 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6419fa40 con 0x7f1d64104f70 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d6888c640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104f70 0x7f1d64198360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009230 tx=0x7f1d58004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.028+0000 7f1d627fc640 1 -- 172.21.15.122:0/1547392035 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5803f070 con 0x7f1d64104f70 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.029+0000 7f1d627fc640 1 -- 172.21.15.122:0/1547392035 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d58003cd0 con 0x7f1d64104f70 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.029+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6419fcd0 con 0x7f1d64104f70 2024-09-12T01:54:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.029+0000 7f1d627fc640 1 -- 172.21.15.122:0/1547392035 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58035070 con 0x7f1d64104f70 2024-09-12T01:54:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.029+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a0190 con 0x7f1d64104f70 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d627fc640 1 -- 172.21.15.122:0/1547392035 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1d5803b460 con 0x7f1d64104f70 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 Processor -- start 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 -- start start 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a467f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467fc780 0x563a467fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563a467562f0 con 0x563a467f76f0 2024-09-12T01:54:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a466fe270 con 0x563a467f6db0 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.031+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a467522b0 con 0x563a467fc780 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6908d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467fc780 0x563a467fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6908d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40476/0 (socket says 172.21.15.122:40476) 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6908d640 1 -- 172.21.15.122:0/3475995828 learned_addr learned my addr 172.21.15.122:0/3475995828 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6988e640 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a467f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d627fc640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d4c0a2cb0 0x7f1d4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 -- 172.21.15.122:0/3475995828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 msgr2=0x563a467f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a467f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 -- 172.21.15.122:0/3475995828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 msgr2=0x563a467fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 -- 172.21.15.122:0/3475995828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a466fd340 con 0x563a467fc780 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6908d640 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6988e640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d4c0a2cb0 0x7f1d4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d627fc640 1 -- 172.21.15.122:0/1547392035 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1d580ef3a0 con 0x7f1d64104f70 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d6888c640 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467fc780 0x563a467fec40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d580ece70 tx=0x7f1d58086930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a822c1d6e33fc6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.032+0000 7f1d4b7fe640 1 -- 172.21.15.122:0/3475995828 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580bb610 con 0x563a467fc780 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d4b7fe640 1 -- 172.21.15.122:0/3475995828 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5808aa40 con 0x563a467fc780 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d4b7fe640 1 -- 172.21.15.122:0/3475995828 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580dbfc0 con 0x563a467fc780 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3475995828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467fc780 msgr2=0x563a467fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467fc780 0x563a467fec40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d580ece70 tx=0x7f1d58086930 comp rx=0 tx=0).stop 2024-09-12T01:54:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6988e640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d4c0a2cb0 0x7f1d4c0a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0095e0 tx=0x7f1d5c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3475995828 shutdown_connections 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467fc780 0x563a467fec40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3475995828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a467f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3475995828 >> 172.21.15.122:0/3475995828 conn(0x563a467ee010 msgr2=0x563a467ee410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3475995828 shutdown_connections 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.033+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3475995828 wait complete. 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 -- start start 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 Processor -- start 2024-09-12T01:54:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a46800c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f76f0 0x563a46801140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46808180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563a467ff510 con 0x563a467fc780 2024-09-12T01:54:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a466df290 con 0x563a467f6db0 2024-09-12T01:54:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a467ff690 con 0x563a467f76f0 2024-09-12T01:54:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6e08b740 10 client.0 _ll_register_callbacks cb 0x563a46710140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6908d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a46800c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6988e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46808180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6888c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f76f0 0x563a46801140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6908d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a46800c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:60718/0 (socket says 172.21.15.122:60718) 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.034+0000 7f1d6908d640 1 -- 172.21.15.122:0/66355891 learned_addr learned my addr 172.21.15.122:0/66355891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6908d640 1 -- 172.21.15.122:0/66355891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f76f0 msgr2=0x563a46801140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6908d640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f76f0 0x563a46801140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6908d640 1 -- 172.21.15.122:0/66355891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 msgr2=0x563a46808180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6908d640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46808180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6908d640 1 -- 172.21.15.122:0/66355891 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a46469210 con 0x563a467f6db0 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6888c640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f76f0 0x563a46801140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6988e640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46808180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6908d640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a46800c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d54011cf0 tx=0x7f1d5400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed74ce226943032e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d48ff9640 1 -- 172.21.15.122:0/66355891 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004b20 con 0x563a467f6db0 2024-09-12T01:54:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5808aa40 con 0x563a467f6db0 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d48ff9640 1 -- 172.21.15.122:0/66355891 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1d54003970 con 0x563a467f6db0 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d48ff9640 1 -- 172.21.15.122:0/66355891 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d54003bb0 con 0x563a467f6db0 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6e08b740 10 client.25946 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a46535620 con 0x563a467f6db0 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d48ff9640 1 -- 172.21.15.122:0/66355891 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5808aa40 con 0x563a467f6db0 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.035+0000 7f1d48ff9640 1 -- 172.21.15.122:0/66355891 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1d54023020 con 0x563a467f6db0 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6e08b740 10 client.25946 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6e08b740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6e08b740 20 client.25946 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a4680c870 0x563a4680ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6e08b740 4 client.25946 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a46763a20 con 0x563a4680c870 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6888c640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a4680c870 0x563a4680ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d6888c640 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a4680c870 0x563a4680ece0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78583fd5722e77e7 server_cookie=90d9335f0414d695 in_seq=0 out_seq=0 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.036+0000 7f1d48ff9640 10 client.25946 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d48ff9640 1 -- 172.21.15.122:0/66355891 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a46763a20 con 0x563a4680c870 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d48ff9640 10 client.25946 handle_command_reply: tid=0 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 client.25946 shutdown 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 msgr2=0x563a46800c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a46800c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d54011cf0 tx=0x7f1d5400f790 comp rx=0 tx=0).stop 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 shutdown_connections 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a4680c870 0x563a4680ece0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46808180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d48ff9640 0 client.25946 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f76f0 0x563a46801140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/66355891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f6db0 0x563a46800c00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 >> 172.21.15.122:0/66355891 conn(0x563a467ee010 msgr2=0x563a467ef820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 shutdown_connections 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 1 -- 172.21.15.122:0/66355891 wait complete. 2024-09-12T01:54:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.037+0000 7f1d6e08b740 20 client.25946 trim_cache size 0 max 16384 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 Processor -- start 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 -- start start 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467fc780 0x563a468080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641a96b0 con 0x563a467f76f0 2024-09-12T01:54:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641a6f80 con 0x563a467fc780 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.051+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641a4db0 con 0x563a467f6db0 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40492/0 (socket says 172.21.15.122:40492) 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 -- 172.21.15.122:0/4004974495 learned_addr learned my addr 172.21.15.122:0/4004974495 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6908d640 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6988e640 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467fc780 0x563a468080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 -- 172.21.15.122:0/4004974495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 msgr2=0x563a468011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 -- 172.21.15.122:0/4004974495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467fc780 msgr2=0x563a468080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467fc780 0x563a468080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 -- 172.21.15.122:0/4004974495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a467ff350 con 0x563a467f76f0 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d6888c640 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467f7190 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009900 tx=0x7f1d5802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb892795e504ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d4affd640 1 -- 172.21.15.122:0/4004974495 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58048040 con 0x563a467f76f0 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d4affd640 1 -- 172.21.15.122:0/4004974495 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d580bbcb0 con 0x563a467f76f0 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.052+0000 7f1d4affd640 1 -- 172.21.15.122:0/4004974495 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580b8320 con 0x563a467f76f0 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 -- 172.21.15.122:0/4004974495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 msgr2=0x563a467f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467f7190 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009900 tx=0x7f1d5802a410 comp rx=0 tx=0).stop 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 -- 172.21.15.122:0/4004974495 shutdown_connections 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467fc780 0x563a468080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467f76f0 0x563a467f7190 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/4004974495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 -- 172.21.15.122:0/4004974495 >> 172.21.15.122:0/4004974495 conn(0x563a467ee010 msgr2=0x563a467ec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 -- 172.21.15.122:0/4004974495 shutdown_connections 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 -- 172.21.15.122:0/4004974495 wait complete. 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 -- start start 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.053+0000 7f1d6e08b740 1 Processor -- start 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f76f0 0x563a46805d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46806d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641a5440 con 0x563a467fc780 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641a68b0 con 0x563a467f76f0 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641a5b10 con 0x563a467f6db0 2024-09-12T01:54:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6e08b740 10 client.0 _ll_register_callbacks cb 0x7f1d54009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46806d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6888c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f76f0 0x563a46805d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6908d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46806d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40498/0 (socket says 172.21.15.122:40498) 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 -- 172.21.15.122:0/3409517573 learned_addr learned my addr 172.21.15.122:0/3409517573 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 -- 172.21.15.122:0/3409517573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 msgr2=0x563a468057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 -- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f76f0 msgr2=0x563a46805d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f76f0 0x563a46805d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 -- 172.21.15.122:0/3409517573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a46807320 con 0x563a467fc780 2024-09-12T01:54:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6888c640 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f76f0 0x563a46805d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d6988e640 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46806d10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01cf40 tx=0x7f1d5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39ebc4ee6424565 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.054+0000 7f1d48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d48ff9640 1 -- 172.21.15.122:0/3409517573 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c02e070 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d580bbcb0 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d48ff9640 1 -- 172.21.15.122:0/3409517573 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1d5c02a650 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d48ff9640 1 -- 172.21.15.122:0/3409517573 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d5c03a050 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 10 client.16704 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a468776f0 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d48ff9640 1 -- 172.21.15.122:0/3409517573 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d580bbcb0 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d48ff9640 1 -- 172.21.15.122:0/3409517573 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1d5c0328e0 con 0x563a467fc780 2024-09-12T01:54:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 10 client.16704 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 20 client.16704 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a468780d0 0x563a4687a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 4 client.16704 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a4675dd70 con 0x563a468780d0 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.055+0000 7f1d6908d640 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a468780d0 0x563a4687a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.056+0000 7f1d48ff9640 10 client.16704 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.056+0000 7f1d6908d640 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a468780d0 0x563a4687a4f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1242c9d37495e605 server_cookie=f7efd297641bc0b in_seq=0 out_seq=0 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.065+0000 7f1d48ff9640 1 -- 172.21.15.122:0/3409517573 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a4675dd70 con 0x563a468780d0 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.065+0000 7f1d48ff9640 10 client.16704 handle_command_reply: tid=0 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.065+0000 7f1d6e08b740 1 client.16704 shutdown 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 msgr2=0x563a46806d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46806d10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01cf40 tx=0x7f1d5c004730 comp rx=0 tx=0).stop 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 shutdown_connections 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563a468780d0 0x563a4687a4f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563a467fc780 0x563a46806d10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d48ff9640 0 client.16704 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a467f76f0 0x563a46805d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 --2- 172.21.15.122:0/3409517573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a467f6db0 0x563a468057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 >> 172.21.15.122:0/3409517573 conn(0x563a467ee010 msgr2=0x563a467ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 shutdown_connections 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 1 -- 172.21.15.122:0/3409517573 wait complete. 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6e08b740 20 client.16704 trim_cache size 0 max 16384 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:54:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:54:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d4c0a2cb0 msgr2=0x7f1d4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d4c0a2cb0 0x7f1d4c0a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0095e0 tx=0x7f1d5c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104f70 msgr2=0x7f1d64198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104f70 0x7f1d64198360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009230 tx=0x7f1d58004270 comp rx=0 tx=0).stop 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 shutdown_connections 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d4c0a2cb0 0x7f1d4c0a5170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d641078a0 0x7f1d6419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104f70 0x7f1d64198360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 --2- 172.21.15.122:0/1547392035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d64102640 0x7f1d64197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.066+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 >> 172.21.15.122:0/1547392035 conn(0x7f1d640fc1e0 msgr2=0x7f1d64102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.067+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 shutdown_connections 2024-09-12T01:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:07.067+0000 7f1d6b2f1640 1 -- 172.21.15.122:0/1547392035 wait complete. 2024-09-12T01:54:07.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 -- 172.21.15.122:0/951900880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8100cc0 msgr2=0x7fedd8106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 --2- 172.21.15.122:0/951900880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8100cc0 0x7fedd8106800 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009900 tx=0x7fedc802a410 comp rx=0 tx=0).stop 2024-09-12T01:54:07.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 -- 172.21.15.122:0/951900880 shutdown_connections 2024-09-12T01:54:07.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 --2- 172.21.15.122:0/951900880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8106f30 0x7fedd810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 --2- 172.21.15.122:0/951900880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8100cc0 0x7fedd8106800 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 --2- 172.21.15.122:0/951900880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100310 0x7fedd81006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 -- 172.21.15.122:0/951900880 >> 172.21.15.122:0/951900880 conn(0x7fedd8077220 msgr2=0x7fedd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 -- 172.21.15.122:0/951900880 shutdown_connections 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.307+0000 7feddfcf8640 1 -- 172.21.15.122:0/951900880 wait complete. 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 Processor -- start 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 -- start start 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8100310 0x7fedd81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100cc0 0x7fedd8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 0x7fedd819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8111f90 con 0x7fedd8106f30 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8111e10 con 0x7fedd8100cc0 2024-09-12T01:54:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddfcf8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112110 con 0x7fedd8100310 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7fedde295640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 0x7fedd819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7fedde295640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 0x7fedd819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40514/0 (socket says 172.21.15.122:40514) 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7fedde295640 1 -- 172.21.15.122:0/1757666232 learned_addr learned my addr 172.21.15.122:0/1757666232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddd293640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100cc0 0x7fedd8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.308+0000 7feddda94640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8100310 0x7fedd81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedde295640 1 -- 172.21.15.122:0/1757666232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8100310 msgr2=0x7fedd81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedde295640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8100310 0x7fedd81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedde295640 1 -- 172.21.15.122:0/1757666232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100cc0 msgr2=0x7fedd8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedde295640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100cc0 0x7fedd8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedde295640 1 -- 172.21.15.122:0/1757666232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd819d3a0 con 0x7fedd8106f30 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7feddda94640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8100310 0x7fedd81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7feddd293640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100cc0 0x7fedd8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedde295640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 0x7fedd819ccc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fedd000eb10 tx=0x7fedd000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0004b20 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd819d690 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a4f70 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd0021920 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.309+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00108d0 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.311+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc005290 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.312+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fedd0021440 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.312+0000 7fedceffd640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fedc40a2c00 0x7fedc40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.312+0000 7feddda94640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fedc40a2c00 0x7fedc40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.312+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fedd00cf4e0 con 0x7fedd8106f30 2024-09-12T01:54:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.313+0000 7feddda94640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fedc40a2c00 0x7fedc40a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009fc0 tx=0x7fedd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.317+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedd009c440 con 0x7fedd8106f30 2024-09-12T01:54:07.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.533+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedbc005580 con 0x7fedd8106f30 2024-09-12T01:54:07.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.533+0000 7fedceffd640 1 -- 172.21.15.122:0/1757666232 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fedd009bd40 con 0x7fedd8106f30 2024-09-12T01:54:07.534 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:54:07.534 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:54:07.534 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:54:07.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fedc40a2c00 msgr2=0x7fedc40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fedc40a2c00 0x7fedc40a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009fc0 tx=0x7fedd4005f90 comp rx=0 tx=0).stop 2024-09-12T01:54:07.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 msgr2=0x7fedd819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:07.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 0x7fedd819ccc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fedd000eb10 tx=0x7fedd000c2d0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 shutdown_connections 2024-09-12T01:54:07.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fedc40a2c00 0x7fedc40a50c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8106f30 0x7fedd819ccc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8100cc0 0x7fedd8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 --2- 172.21.15.122:0/1757666232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedd8100310 0x7fedd81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:07.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 >> 172.21.15.122:0/1757666232 conn(0x7fedd8077220 msgr2=0x7fedd81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:07.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 shutdown_connections 2024-09-12T01:54:07.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:07.536+0000 7feddfcf8640 1 -- 172.21.15.122:0/1757666232 wait complete. 2024-09-12T01:54:07.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 703931 10.69 MB/sec execute 265 sec latency 47.239 ms 2024-09-12T01:54:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:07 smithi183 ceph-mon[35127]: pgmap v564: 97 pgs: 97 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-09-12T01:54:07.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:07 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1757666232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:07 smithi122 ceph-mon[30925]: pgmap v564: 97 pgs: 97 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-09-12T01:54:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:07 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1757666232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:08.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:07 smithi151 ceph-mon[35229]: pgmap v564: 97 pgs: 97 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-09-12T01:54:08.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:07 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1757666232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:08.480 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T01:54:08.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T01:54:08.744 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:08.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 707065 10.72 MB/sec execute 266 sec latency 9.840 ms 2024-09-12T01:54:09.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 -- 172.21.15.122:0/405363779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c106f40 msgr2=0x7f0a8c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 --2- 172.21.15.122:0/405363779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c106f40 0x7f0a8c10eac0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ca0 tx=0x7f0a8402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 -- 172.21.15.122:0/405363779 shutdown_connections 2024-09-12T01:54:09.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 --2- 172.21.15.122:0/405363779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c106f40 0x7f0a8c10eac0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 --2- 172.21.15.122:0/405363779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 --2- 172.21.15.122:0/405363779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a8c101e10 0x7f0a8c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 -- 172.21.15.122:0/405363779 >> 172.21.15.122:0/405363779 conn(0x7f0a8c0fd8d0 msgr2=0x7f0a8c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 -- 172.21.15.122:0/405363779 shutdown_connections 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.238+0000 7f0a94045640 1 -- 172.21.15.122:0/405363779 wait complete. 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 Processor -- start 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 -- start start 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a8c101e10 0x7f0a8c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c106f40 0x7f0a8c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111fa0 con 0x7f0a8c1027c0 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111e20 con 0x7f0a8c106f40 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a94045640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c112120 con 0x7f0a8c101e10 2024-09-12T01:54:09.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a915e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a915e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40526/0 (socket says 172.21.15.122:40526) 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a915e0640 1 -- 172.21.15.122:0/604383555 learned_addr learned my addr 172.21.15.122:0/604383555 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a925e2640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c106f40 0x7f0a8c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a91de1640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a8c101e10 0x7f0a8c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a915e0640 1 -- 172.21.15.122:0/604383555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a8c101e10 msgr2=0x7f0a8c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.239+0000 7f0a915e0640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a8c101e10 0x7f0a8c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a915e0640 1 -- 172.21.15.122:0/604383555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c106f40 msgr2=0x7f0a8c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a915e0640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c106f40 0x7f0a8c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a915e0640 1 -- 172.21.15.122:0/604383555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c19d3b0 con 0x7f0a8c1027c0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a915e0640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c198940 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e9f0 tx=0x7f0a7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a82ffd640 1 -- 172.21.15.122:0/604383555 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00cd80 con 0x7f0a8c1027c0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c19d6a0 con 0x7f0a8c1027c0 2024-09-12T01:54:09.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a82ffd640 1 -- 172.21.15.122:0/604383555 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7c010910 con 0x7f0a8c1027c0 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a82ffd640 1 -- 172.21.15.122:0/604383555 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0188f0 con 0x7f0a8c1027c0 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.240+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1a4f80 con 0x7f0a8c1027c0 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.243+0000 7f0a82ffd640 1 -- 172.21.15.122:0/604383555 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0a7c010ac0 con 0x7f0a8c1027c0 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.243+0000 7f0a9641e740 1 Processor -- start 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 -- start start 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c0866a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866f40 0x55a5c0875810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c0878140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a5c07cc2a0 con 0x55a5c0866620 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0773b10 con 0x55a5c0866f40 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a9641e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5c07c8260 con 0x55a5c0875d50 2024-09-12T01:54:09.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a82ffd640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0a780a2b80 0x7f0a780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a925e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c0866a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a925e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c0866a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40528/0 (socket says 172.21.15.122:40528) 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a925e2640 1 -- 172.21.15.122:0/2071688500 learned_addr learned my addr 172.21.15.122:0/2071688500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a91de1640 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866f40 0x55a5c0875810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a915e0640 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c0878140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a925e2640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0a780a2b80 0x7f0a780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a91de1640 1 -- 172.21.15.122:0/2071688500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 msgr2=0x55a5c0878140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a91de1640 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c0878140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a91de1640 1 -- 172.21.15.122:0/2071688500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 msgr2=0x55a5c0866a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a91de1640 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c0866a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.244+0000 7f0a91de1640 1 -- 172.21.15.122:0/2071688500 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c053db90 con 0x55a5c0866f40 2024-09-12T01:54:09.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a925e2640 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c0866a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a82ffd640 1 -- 172.21.15.122:0/604383555 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0a7c0cf350 con 0x7f0a8c1027c0 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a91de1640 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866f40 0x55a5c0875810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009c00 tx=0x7f0a88011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc2bfaf82660da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a77fff640 1 -- 172.21.15.122:0/2071688500 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88004230 con 0x55a5c0866f40 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a925e2640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0a780a2b80 0x7f0a780a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0a84004af0 tx=0x7f0a8402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a77fff640 1 -- 172.21.15.122:0/2071688500 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a880043d0 con 0x55a5c0866f40 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.245+0000 7f0a77fff640 1 -- 172.21.15.122:0/2071688500 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88004d90 con 0x55a5c0866f40 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 -- 172.21.15.122:0/2071688500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866f40 msgr2=0x55a5c0875810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866f40 0x55a5c0875810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009c00 tx=0x7f0a88011810 comp rx=0 tx=0).stop 2024-09-12T01:54:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 -- 172.21.15.122:0/2071688500 shutdown_connections 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c0878140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866f40 0x55a5c0875810 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 --2- 172.21.15.122:0/2071688500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c0866a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 -- 172.21.15.122:0/2071688500 >> 172.21.15.122:0/2071688500 conn(0x55a5c0862cf0 msgr2=0x55a5c08630d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 -- 172.21.15.122:0/2071688500 shutdown_connections 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 -- 172.21.15.122:0/2071688500 wait complete. 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 -- start start 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.246+0000 7f0a9641e740 1 Processor -- start 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866620 0x55a5c0872c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c0873170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c08736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0878920 con 0x55a5c0866f40 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a5c07546e0 con 0x55a5c0875d50 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0878aa0 con 0x55a5c0866620 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a91de1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866620 0x55a5c0872c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a9641e740 10 client.0 _ll_register_callbacks cb 0x55a5c0778a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a925e2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c08736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a91de1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866620 0x55a5c0872c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43406/0 (socket says 172.21.15.122:43406) 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a91de1640 1 -- 172.21.15.122:0/40876174 learned_addr learned my addr 172.21.15.122:0/40876174 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a925e2640 1 -- 172.21.15.122:0/40876174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866620 msgr2=0x55a5c0872c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a925e2640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866620 0x55a5c0872c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a925e2640 1 -- 172.21.15.122:0/40876174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 msgr2=0x55a5c0873170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a915e0640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c0873170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a925e2640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c0873170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.247+0000 7f0a925e2640 1 -- 172.21.15.122:0/40876174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c0881e50 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a915e0640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c0873170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a925e2640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c08736b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ca0 tx=0x7f0a840095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fbb1876099a285e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a757fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a757fa640 1 -- 172.21.15.122:0/40876174 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8404f030 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a757fa640 1 -- 172.21.15.122:0/40876174 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0a84004720 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a880043d0 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a757fa640 1 -- 172.21.15.122:0/40876174 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a840319a0 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a9641e740 10 client.25970 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5c08847e0 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.248+0000 7f0a757fa640 1 -- 172.21.15.122:0/40876174 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a880043d0 con 0x55a5c0875d50 2024-09-12T01:54:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a757fa640 1 -- 172.21.15.122:0/40876174 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0a8404a040 con 0x55a5c0875d50 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a9641e740 10 client.25970 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a9641e740 10 client.25970 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a9641e740 20 client.25970 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a9641e740 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c0885c40 0x55a5c0888090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a9641e740 4 client.25970 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5c07d99d0 con 0x55a5c0885c40 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a91de1640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c0885c40 0x55a5c0888090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a91de1640 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c0885c40 0x55a5c0888090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7257d98601ec4c2 server_cookie=b9784aeeab20c186 in_seq=0 out_seq=0 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.249+0000 7f0a757fa640 10 client.25970 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a757fa640 1 -- 172.21.15.122:0/40876174 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5c07d99d0 con 0x55a5c0885c40 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a757fa640 10 client.25970 handle_command_reply: tid=0 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 client.25970 shutdown 2024-09-12T01:54:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 msgr2=0x55a5c08736b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c08736b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ca0 tx=0x7f0a840095c0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 shutdown_connections 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c0885c40 0x55a5c0888090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c08736b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a757fa640 0 client.25970 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c0873170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 --2- 172.21.15.122:0/40876174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866620 0x55a5c0872c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.250+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 >> 172.21.15.122:0/40876174 conn(0x55a5c0862cf0 msgr2=0x55a5c0868a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.251+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 shutdown_connections 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.251+0000 7f0a9641e740 1 -- 172.21.15.122:0/40876174 wait complete. 2024-09-12T01:54:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.251+0000 7f0a9641e740 20 client.25970 trim_cache size 0 max 16384 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 Processor -- start 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 -- start start 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866620 0x55a5c0861d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c088d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c088dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1adc80 con 0x55a5c0866f40 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1ab550 con 0x55a5c0866620 2024-09-12T01:54:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a9641e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1a9380 con 0x55a5c0875d50 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a915e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c088d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a915e0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c088d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40544/0 (socket says 172.21.15.122:40544) 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a915e0640 1 -- 172.21.15.122:0/3614050656 learned_addr learned my addr 172.21.15.122:0/3614050656 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a91de1640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866620 0x55a5c0861d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a925e2640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c088dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a915e0640 1 -- 172.21.15.122:0/3614050656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 msgr2=0x55a5c088dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.269+0000 7f0a915e0640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c088dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a915e0640 1 -- 172.21.15.122:0/3614050656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866620 msgr2=0x55a5c0861d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a915e0640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866620 0x55a5c0861d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a915e0640 1 -- 172.21.15.122:0/3614050656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c0871be0 con 0x55a5c0866f40 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a91de1640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866620 0x55a5c0861d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a925e2640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c088dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a915e0640 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c088d9a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0cce50 tx=0x7f0a7c06dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2c0c0502e4d8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a777fe640 1 -- 172.21.15.122:0/3614050656 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0ccf60 con 0x55a5c0866f40 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a777fe640 1 -- 172.21.15.122:0/3614050656 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7c0982d0 con 0x55a5c0866f40 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.270+0000 7f0a777fe640 1 -- 172.21.15.122:0/3614050656 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c097d30 con 0x55a5c0866f40 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 -- 172.21.15.122:0/3614050656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 msgr2=0x55a5c088d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c088d9a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0cce50 tx=0x7f0a7c06dd30 comp rx=0 tx=0).stop 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 -- 172.21.15.122:0/3614050656 shutdown_connections 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0875d50 0x55a5c088dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866f40 0x55a5c088d9a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 --2- 172.21.15.122:0/3614050656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0866620 0x55a5c0861d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 -- 172.21.15.122:0/3614050656 >> 172.21.15.122:0/3614050656 conn(0x55a5c0862cf0 msgr2=0x55a5c0861650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 -- 172.21.15.122:0/3614050656 shutdown_connections 2024-09-12T01:54:09.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 -- 172.21.15.122:0/3614050656 wait complete. 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 -- start start 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.271+0000 7f0a9641e740 1 Processor -- start 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c087e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866f40 0x55a5c087ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c087b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1a9a10 con 0x55a5c0866620 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1aae80 con 0x55a5c0875d50 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1aa0e0 con 0x55a5c0866f40 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a9641e740 10 client.0 _ll_register_callbacks cb 0x7f0a84003ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a91de1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c087e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a925e2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c087b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a91de1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c087e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40552/0 (socket says 172.21.15.122:40552) 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a91de1640 1 -- 172.21.15.122:0/1669833223 learned_addr learned my addr 172.21.15.122:0/1669833223 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866f40 0x55a5c087ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 -- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 msgr2=0x55a5c087b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c087b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 -- 172.21.15.122:0/1669833223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 msgr2=0x55a5c087e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c087e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 -- 172.21.15.122:0/1669833223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c087b980 con 0x55a5c0866f40 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a925e2640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c087b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a91de1640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c087e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a915e0640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866f40 0x55a5c087ad00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c098630 tx=0x7f0a7c0a48e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874723a681d0d7ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a757fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:54:09.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.272+0000 7f0a757fa640 1 -- 172.21.15.122:0/1669833223 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0b2070 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a757fa640 1 -- 172.21.15.122:0/1669833223 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0a7c010430 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a7c0982d0 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a757fa640 1 -- 172.21.15.122:0/1669833223 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a7c010630 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 10 client.25948 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5c08f1890 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a757fa640 1 -- 172.21.15.122:0/1669833223 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7c0982d0 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a757fa640 1 -- 172.21.15.122:0/1669833223 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0a7c018c90 con 0x55a5c0866f40 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 10 client.25948 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 10 client.25948 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 20 client.25948 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c08f1fc0 0x55a5c08f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 4 client.25948 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T01:54:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55a5c07d3d20 con 0x55a5c08f1fc0 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.273+0000 7f0a91de1640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c08f1fc0 0x55a5c08f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.274+0000 7f0a91de1640 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c08f1fc0 0x55a5c08f43e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4636e88264dd177 server_cookie=92c28a13f8e6c99 in_seq=0 out_seq=0 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.274+0000 7f0a757fa640 10 client.25948 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a757fa640 1 -- 172.21.15.122:0/1669833223 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a5c07d3d20 con 0x55a5c08f1fc0 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a757fa640 10 client.25948 handle_command_reply: tid=0 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 client.25948 shutdown 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866f40 msgr2=0x55a5c087ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866f40 0x55a5c087ad00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c098630 tx=0x7f0a7c0a48e0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 shutdown_connections 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a5c08f1fc0 0x55a5c08f43e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5c0875d50 0x55a5c087b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5c0866f40 0x55a5c087ad00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a757fa640 0 client.25948 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 --2- 172.21.15.122:0/1669833223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a5c0866620 0x55a5c087e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 >> 172.21.15.122:0/1669833223 conn(0x55a5c0862cf0 msgr2=0x55a5c0873410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 shutdown_connections 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.338+0000 7f0a9641e740 1 -- 172.21.15.122:0/1669833223 wait complete. 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a9641e740 20 client.25948 trim_cache size 0 max 16384 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:54:09.531 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 30, 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "7a9d734d": { 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 30, 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 139.4, 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:54:09.532 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:09.533 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:54:09.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0a780a2b80 msgr2=0x7f0a780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0a780a2b80 0x7f0a780a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0a84004af0 tx=0x7f0a8402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 msgr2=0x7f0a8c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:09.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c198940 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e9f0 tx=0x7f0a7c00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 shutdown_connections 2024-09-12T01:54:09.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0a780a2b80 0x7f0a780a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c106f40 0x7f0a8c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a8c1027c0 0x7f0a8c198940 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 --2- 172.21.15.122:0/604383555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a8c101e10 0x7f0a8c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:09.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 >> 172.21.15.122:0/604383555 conn(0x7f0a8c0fd8d0 msgr2=0x7f0a8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:09.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 shutdown_connections 2024-09-12T01:54:09.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:09.339+0000 7f0a94045640 1 -- 172.21.15.122:0/604383555 wait complete. 2024-09-12T01:54:09.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.70 MB/sec execute 267 sec latency 569.280 ms 2024-09-12T01:54:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:09 smithi183 ceph-mon[35127]: pgmap v565: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 271 op/s 2024-09-12T01:54:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:09 smithi122 ceph-mon[30925]: pgmap v565: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 271 op/s 2024-09-12T01:54:10.038 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7a9d734d', quiesce took 3.5 seconds. Will release after: 15.899999999999999 2024-09-12T01:54:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:09 smithi151 ceph-mon[35229]: pgmap v565: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 271 op/s 2024-09-12T01:54:10.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.66 MB/sec execute 268 sec latency 1569.727 ms 2024-09-12T01:54:11.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.62 MB/sec execute 269 sec latency 2569.818 ms 2024-09-12T01:54:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:11 smithi183 ceph-mon[35127]: pgmap v566: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-09-12T01:54:11.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:11 smithi122 ceph-mon[30925]: pgmap v566: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-09-12T01:54:12.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:11 smithi151 ceph-mon[35229]: pgmap v566: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-09-12T01:54:12.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.58 MB/sec execute 270 sec latency 3569.871 ms 2024-09-12T01:54:13.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.54 MB/sec execute 271 sec latency 4569.996 ms 2024-09-12T01:54:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:13 smithi183 ceph-mon[35127]: pgmap v567: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-09-12T01:54:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:13 smithi122 ceph-mon[30925]: pgmap v567: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-09-12T01:54:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:13 smithi151 ceph-mon[35229]: pgmap v567: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-09-12T01:54:14.844 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.50 MB/sec execute 272 sec latency 5570.126 ms 2024-09-12T01:54:15.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.46 MB/sec execute 273 sec latency 6570.224 ms 2024-09-12T01:54:15.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:15 smithi183 ceph-mon[35127]: pgmap v568: 97 pgs: 97 active+clean; 264 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-12T01:54:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:15 smithi122 ceph-mon[30925]: pgmap v568: 97 pgs: 97 active+clean; 264 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-12T01:54:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:15 smithi151 ceph-mon[35229]: pgmap v568: 97 pgs: 97 active+clean; 264 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-12T01:54:16.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.42 MB/sec execute 274 sec latency 7570.322 ms 2024-09-12T01:54:17.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.39 MB/sec execute 275 sec latency 8570.407 ms 2024-09-12T01:54:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:17 smithi183 ceph-mon[35127]: pgmap v569: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-12T01:54:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:17 smithi122 ceph-mon[30925]: pgmap v569: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-12T01:54:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:17 smithi151 ceph-mon[35229]: pgmap v569: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-09-12T01:54:18.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.35 MB/sec execute 276 sec latency 9570.492 ms 2024-09-12T01:54:19.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.31 MB/sec execute 277 sec latency 10570.540 ms 2024-09-12T01:54:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:19 smithi183 ceph-mon[35127]: pgmap v570: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 148 op/s 2024-09-12T01:54:19.993 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:19 smithi122 ceph-mon[30925]: pgmap v570: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 148 op/s 2024-09-12T01:54:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:19 smithi151 ceph-mon[35229]: pgmap v570: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 148 op/s 2024-09-12T01:54:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 -- 172.21.15.122:0/4153134483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102d50 msgr2=0x7f4d2c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 --2- 172.21.15.122:0/4153134483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1031d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009900 tx=0x7f4d1402a410 comp rx=0 tx=0).stop 2024-09-12T01:54:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 -- 172.21.15.122:0/4153134483 shutdown_connections 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 --2- 172.21.15.122:0/4153134483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d2c103710 0x7f4d2c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 --2- 172.21.15.122:0/4153134483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1031d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 --2- 172.21.15.122:0/4153134483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c101b60 0x7f4d2c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 -- 172.21.15.122:0/4153134483 >> 172.21.15.122:0/4153134483 conn(0x7f4d2c0fd2e0 msgr2=0x7f4d2c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 -- 172.21.15.122:0/4153134483 shutdown_connections 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 -- 172.21.15.122:0/4153134483 wait complete. 2024-09-12T01:54:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.162+0000 7f4d317c4640 1 Processor -- start 2024-09-12T01:54:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 -- start start 2024-09-12T01:54:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c101b60 0x7f4d2c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d2c102d50 0x7f4d2c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 0x7f4d2c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10d890 con 0x7f4d2c103710 2024-09-12T01:54:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10d710 con 0x7f4d2c101b60 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d317c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10da10 con 0x7f4d2c102d50 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 0x7f4d2c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 0x7f4d2c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54608/0 (socket says 172.21.15.122:54608) 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 -- 172.21.15.122:0/3648036576 learned_addr learned my addr 172.21.15.122:0/3648036576 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2affd640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c101b60 0x7f4d2c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d227fc640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d2c102d50 0x7f4d2c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 -- 172.21.15.122:0/3648036576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d2c102d50 msgr2=0x7f4d2c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d2c102d50 0x7f4d2c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 -- 172.21.15.122:0/3648036576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c101b60 msgr2=0x7f4d2c075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c101b60 0x7f4d2c075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 -- 172.21.15.122:0/3648036576 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0753b0 con 0x7f4d2c103710 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d2b7fe640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 0x7f4d2c074b20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400b970 tx=0x7f4d2400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.163+0000 7f4d28ff9640 1 -- 172.21.15.122:0/3648036576 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24019070 con 0x7f4d2c103710 2024-09-12T01:54:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.164+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c1a1fa0 con 0x7f4d2c103710 2024-09-12T01:54:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.164+0000 7f4d28ff9640 1 -- 172.21.15.122:0/3648036576 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2400cb40 con 0x7f4d2c103710 2024-09-12T01:54:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.164+0000 7f4d28ff9640 1 -- 172.21.15.122:0/3648036576 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24007610 con 0x7f4d2c103710 2024-09-12T01:54:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.164+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c1a2480 con 0x7f4d2c103710 2024-09-12T01:54:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d28ff9640 1 -- 172.21.15.122:0/3648036576 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4d2401c070 con 0x7f4d2c103710 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 Processor -- start 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 -- start start 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4855039a0 0x55e485503da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485508da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 0x55e48550b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e4854672f0 con 0x55e485509370 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e48540f270 con 0x55e4855039a0 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.166+0000 7f4d34638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4854632b0 con 0x55e4855042e0 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 0x55e48550b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 0x55e48550b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54618/0 (socket says 172.21.15.122:54618) 2024-09-12T01:54:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 -- 172.21.15.122:0/484162903 learned_addr learned my addr 172.21.15.122:0/484162903 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d28ff9640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d180a2c30 0x7f4d180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d2affd640 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485508da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4855039a0 0x55e485503da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d2b7fe640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d180a2c30 0x7f4d180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 -- 172.21.15.122:0/484162903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 msgr2=0x55e485508da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485508da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 -- 172.21.15.122:0/484162903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4855039a0 msgr2=0x55e485503da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4855039a0 0x55e485503da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d28ff9640 1 -- 172.21.15.122:0/3648036576 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4d240ce660 con 0x7f4d2c103710 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 -- 172.21.15.122:0/484162903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48540e340 con 0x55e485509370 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d2affd640 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485508da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d227fc640 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 0x55e48550b830 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4d14031440 tx=0x7f4d140314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be524d72a16f0ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d2b7fe640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d180a2c30 0x7f4d180a50f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400b940 tx=0x7f4d2400f300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d217fa640 1 -- 172.21.15.122:0/484162903 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14043030 con 0x55e485509370 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d217fa640 1 -- 172.21.15.122:0/484162903 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14005500 con 0x55e485509370 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.167+0000 7f4d217fa640 1 -- 172.21.15.122:0/484162903 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14035050 con 0x55e485509370 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 -- 172.21.15.122:0/484162903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 msgr2=0x55e48550b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 0x55e48550b830 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4d14031440 tx=0x7f4d140314e0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 -- 172.21.15.122:0/484162903 shutdown_connections 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e485509370 0x55e48550b830 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485508da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 --2- 172.21.15.122:0/484162903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4855039a0 0x55e485503da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 -- 172.21.15.122:0/484162903 >> 172.21.15.122:0/484162903 conn(0x55e4854ff010 msgr2=0x55e4854ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 -- 172.21.15.122:0/484162903 shutdown_connections 2024-09-12T01:54:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.168+0000 7f4d34638740 1 -- 172.21.15.122:0/484162903 wait complete. 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 -- start start 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 Processor -- start 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e48550dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e48550e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e4855151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e48550c2e0 con 0x55e4855042e0 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4853f0290 con 0x55e485509370 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e48550c460 con 0x55e4855039a0 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d34638740 10 client.0 _ll_register_callbacks cb 0x55e485421140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e48550e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e48550e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54634/0 (socket says 172.21.15.122:54634) 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 -- 172.21.15.122:0/9747599 learned_addr learned my addr 172.21.15.122:0/9747599 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d227fc640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e4855151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d227fc640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e48550dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 -- 172.21.15.122:0/9747599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 msgr2=0x55e48550dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e48550dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 -- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 msgr2=0x55e4855151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e4855151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d2affd640 1 -- 172.21.15.122:0/9747599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48517a210 con 0x55e4855042e0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.169+0000 7f4d227fc640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e4855151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d2affd640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e48550e170 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c010840 tx=0x7f4d1c010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bef5d3cefe985b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d127fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d127fc640 1 -- 172.21.15.122:0/9747599 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c004440 con 0x55e4855042e0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d127fc640 1 -- 172.21.15.122:0/9747599 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4d1c00e930 con 0x55e4855042e0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14005500 con 0x55e4855042e0 2024-09-12T01:54:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d127fc640 1 -- 172.21.15.122:0/9747599 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d1c00eb30 con 0x55e4855042e0 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d34638740 10 client.16737 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e485246620 con 0x55e4855042e0 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d127fc640 1 -- 172.21.15.122:0/9747599 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14005500 con 0x55e4855042e0 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.170+0000 7f4d127fc640 1 -- 172.21.15.122:0/9747599 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4d1c011440 con 0x55e4855042e0 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d34638740 10 client.16737 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d34638740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d34638740 20 client.16737 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d34638740 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e485519a10 0x55e48551be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d34638740 4 client.16737 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e485474a20 con 0x55e485519a10 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d227fc640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e485519a10 0x55e48551be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d227fc640 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e485519a10 0x55e48551be30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afaf72a01061c3a6 server_cookie=eb449a1f9cb66f11 in_seq=0 out_seq=0 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.171+0000 7f4d127fc640 10 client.16737 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d127fc640 1 -- 172.21.15.122:0/9747599 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e485474a20 con 0x55e485519a10 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d127fc640 10 client.16737 handle_command_reply: tid=0 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 client.16737 shutdown 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 msgr2=0x55e48550e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e48550e170 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c010840 tx=0x7f4d1c010d00 comp rx=0 tx=0).stop 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 shutdown_connections 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e485519a10 0x55e48551be30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e4855151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e48550e170 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d127fc640 0 client.16737 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 --2- 172.21.15.122:0/9747599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e48550dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 >> 172.21.15.122:0/9747599 conn(0x55e4854ff010 msgr2=0x55e485500820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 shutdown_connections 2024-09-12T01:54:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 1 -- 172.21.15.122:0/9747599 wait complete. 2024-09-12T01:54:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.172+0000 7f4d34638740 20 client.16737 trim_cache size 0 max 16384 2024-09-12T01:54:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.186+0000 7f4d34638740 1 Processor -- start 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.186+0000 7f4d34638740 1 -- start start 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d34638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 0x55e485500400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d34638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485515170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d34638740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485500940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d34638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1ada50 con 0x55e4855039a0 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d34638740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1ab320 con 0x55e485509370 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d34638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1a9150 con 0x55e4855042e0 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 0x55e485500400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 0x55e485500400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54640/0 (socket says 172.21.15.122:54640) 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 -- 172.21.15.122:0/4042799218 learned_addr learned my addr 172.21.15.122:0/4042799218 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 -- 172.21.15.122:0/4042799218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 msgr2=0x55e485515170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485515170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 -- 172.21.15.122:0/4042799218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 msgr2=0x55e485500940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485500940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 -- 172.21.15.122:0/4042799218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4854fee30 con 0x55e4855039a0 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d227fc640 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 0x55e485500400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4d1402a930 tx=0x7f4d14004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0101b59a2b127a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.187+0000 7f4d20ff9640 1 -- 172.21.15.122:0/4042799218 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14043070 con 0x55e4855039a0 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d20ff9640 1 -- 172.21.15.122:0/4042799218 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14050070 con 0x55e4855039a0 2024-09-12T01:54:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d20ff9640 1 -- 172.21.15.122:0/4042799218 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14035040 con 0x55e4855039a0 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 -- 172.21.15.122:0/4042799218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 msgr2=0x55e485500400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 0x55e485500400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4d1402a930 tx=0x7f4d14004330 comp rx=0 tx=0).stop 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 -- 172.21.15.122:0/4042799218 shutdown_connections 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485500940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855042e0 0x55e485515170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 --2- 172.21.15.122:0/4042799218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855039a0 0x55e485500400 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 -- 172.21.15.122:0/4042799218 >> 172.21.15.122:0/4042799218 conn(0x55e4854ff010 msgr2=0x55e4854fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 -- 172.21.15.122:0/4042799218 shutdown_connections 2024-09-12T01:54:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.188+0000 7f4d34638740 1 -- 172.21.15.122:0/4042799218 wait complete. 2024-09-12T01:54:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 -- start start 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 Processor -- start 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e485512860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e485512da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485513d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1a97e0 con 0x55e4855042e0 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1aac50 con 0x55e485509370 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1a9eb0 con 0x55e4855039a0 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d34638740 10 client.0 _ll_register_callbacks cb 0x7f4d1c00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e485512da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e485512da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54650/0 (socket says 172.21.15.122:54650) 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 -- 172.21.15.122:0/2952918506 learned_addr learned my addr 172.21.15.122:0/2952918506 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d227fc640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485513d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d227fc640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e485512860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 -- 172.21.15.122:0/2952918506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 msgr2=0x55e485512860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e485512860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 -- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 msgr2=0x55e485513d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485513d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d2affd640 1 -- 172.21.15.122:0/2952918506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e485514370 con 0x55e4855042e0 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.189+0000 7f4d227fc640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e485512860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d2affd640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e485512da0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c011aa0 tx=0x7f4d1c00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c490a22ad4cdf9e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d127fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d127fc640 1 -- 172.21.15.122:0/2952918506 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c011d30 con 0x55e4855042e0 2024-09-12T01:54:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d127fc640 1 -- 172.21.15.122:0/2952918506 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4d1c004240 con 0x55e4855042e0 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14050070 con 0x55e4855042e0 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d127fc640 1 -- 172.21.15.122:0/2952918506 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d1c00e720 con 0x55e4855042e0 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 10 client.16743 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4855847e0 con 0x55e4855042e0 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d127fc640 1 -- 172.21.15.122:0/2952918506 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14050070 con 0x55e4855042e0 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d127fc640 1 -- 172.21.15.122:0/2952918506 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4d1c0114d0 con 0x55e4855042e0 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 10 client.16743 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 20 client.16743 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4855851c0 0x55e4855875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 4 client.16743 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:54:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e48546ed70 con 0x55e4855851c0 2024-09-12T01:54:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.190+0000 7f4d227fc640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4855851c0 0x55e4855875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.191+0000 7f4d227fc640 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4855851c0 0x55e4855875e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51fac5e285c175a9 server_cookie=46ef92bf19597792 in_seq=0 out_seq=0 2024-09-12T01:54:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.191+0000 7f4d127fc640 10 client.16743 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.202+0000 7f4d127fc640 1 -- 172.21.15.122:0/2952918506 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e48546ed70 con 0x55e4855851c0 2024-09-12T01:54:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.202+0000 7f4d127fc640 10 client.16743 handle_command_reply: tid=0 2024-09-12T01:54:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.202+0000 7f4d34638740 1 client.16743 shutdown 2024-09-12T01:54:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 msgr2=0x55e485512da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e485512da0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c011aa0 tx=0x7f4d1c00ef80 comp rx=0 tx=0).stop 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 shutdown_connections 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4855851c0 0x55e4855875e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e485509370 0x55e485513d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4855042e0 0x55e485512da0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 --2- 172.21.15.122:0/2952918506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4855039a0 0x55e485512860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d127fc640 0 client.16743 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 >> 172.21.15.122:0/2952918506 conn(0x55e4854ff010 msgr2=0x55e4854fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 shutdown_connections 2024-09-12T01:54:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 1 -- 172.21.15.122:0/2952918506 wait complete. 2024-09-12T01:54:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d34638740 20 client.16743 trim_cache size 0 max 16384 2024-09-12T01:54:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:54:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:54:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:54:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d180a2c30 msgr2=0x7f4d180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d180a2c30 0x7f4d180a50f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400b940 tx=0x7f4d2400f300 comp rx=0 tx=0).stop 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 msgr2=0x7f4d2c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 0x7f4d2c074b20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400b970 tx=0x7f4d2400be30 comp rx=0 tx=0).stop 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 shutdown_connections 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d180a2c30 0x7f4d180a50f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.203+0000 7f4d317c4640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c103710 0x7f4d2c074b20 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.204+0000 7f4d317c4640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d2c102d50 0x7f4d2c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.204+0000 7f4d317c4640 1 --2- 172.21.15.122:0/3648036576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c101b60 0x7f4d2c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.204+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 >> 172.21.15.122:0/3648036576 conn(0x7f4d2c0fd2e0 msgr2=0x7f4d2c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.204+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 shutdown_connections 2024-09-12T01:54:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:20.204+0000 7f4d317c4640 1 -- 172.21.15.122:0/3648036576 wait complete. 2024-09-12T01:54:20.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.27 MB/sec execute 278 sec latency 11570.639 ms 2024-09-12T01:54:21.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.24 MB/sec execute 279 sec latency 12570.752 ms 2024-09-12T01:54:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:21 smithi183 ceph-mon[35127]: pgmap v571: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-09-12T01:54:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:21 smithi122 ceph-mon[30925]: pgmap v571: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-09-12T01:54:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:21 smithi151 ceph-mon[35229]: pgmap v571: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-09-12T01:54:22.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.20 MB/sec execute 280 sec latency 13570.817 ms 2024-09-12T01:54:23.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.16 MB/sec execute 281 sec latency 14570.865 ms 2024-09-12T01:54:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:23 smithi183 ceph-mon[35127]: pgmap v572: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-12T01:54:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:23 smithi122 ceph-mon[30925]: pgmap v572: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-12T01:54:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:23 smithi151 ceph-mon[35229]: pgmap v572: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-12T01:54:24.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.13 MB/sec execute 282 sec latency 15570.911 ms 2024-09-12T01:54:25.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.09 MB/sec execute 283 sec latency 16571.018 ms 2024-09-12T01:54:25.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:54:25.973 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:25 smithi122 ceph-mon[30925]: pgmap v573: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 2.4 MiB/s wr, 106 op/s 2024-09-12T01:54:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:25 smithi151 ceph-mon[35229]: pgmap v573: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 2.4 MiB/s wr, 106 op/s 2024-09-12T01:54:26.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:25 smithi183 ceph-mon[35127]: pgmap v573: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 2.4 MiB/s wr, 106 op/s 2024-09-12T01:54:26.210 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 -- 172.21.15.122:0/2587120997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 msgr2=0x7f1048106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 --2- 172.21.15.122:0/2587120997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f1048106e60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1044009930 tx=0x7f104402a420 comp rx=0 tx=0).stop 2024-09-12T01:54:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 -- 172.21.15.122:0/2587120997 shutdown_connections 2024-09-12T01:54:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 --2- 172.21.15.122:0/2587120997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 0x7f104810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 --2- 172.21.15.122:0/2587120997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1048100a80 0x7f1048100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 --2- 172.21.15.122:0/2587120997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f1048106e60 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.686+0000 7f1050755640 1 -- 172.21.15.122:0/2587120997 >> 172.21.15.122:0/2587120997 conn(0x7f10480fc7a0 msgr2=0x7f10480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 -- 172.21.15.122:0/2587120997 shutdown_connections 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 -- 172.21.15.122:0/2587120997 wait complete. 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 Processor -- start 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 -- start start 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1048100a80 0x7f1048198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 0x7f10481989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.687+0000 7f1050755640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f104819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f1050755640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1048110ec0 con 0x7f1048106a80 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f1050755640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1048110d40 con 0x7f1048101420 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f1050755640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1048111040 con 0x7f1048100a80 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f104819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f104819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53452/0 (socket says 172.21.15.122:53452) 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 -- 172.21.15.122:0/2542753885 learned_addr learned my addr 172.21.15.122:0/2542753885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104e4f1640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1048100a80 0x7f1048198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104dcf0640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 0x7f10481989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 -- 172.21.15.122:0/2542753885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1048100a80 msgr2=0x7f1048198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1048100a80 0x7f1048198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 -- 172.21.15.122:0/2542753885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 msgr2=0x7f10481989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 0x7f10481989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 -- 172.21.15.122:0/2542753885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104819d410 con 0x7f1048106a80 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104dcf0640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 0x7f10481989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f104ecf2640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f104819cd30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f104000b970 tx=0x7f104000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040019070 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f104000cb40 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104819d700 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040007650 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.688+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481a4fe0 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.689+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1048100ee0 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.690+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f104001c070 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.691+0000 7f103f7fe640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10340a2b80 0x7f10340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.691+0000 7f104e4f1640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10340a2b80 0x7f10340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.691+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f10400ceae0 con 0x7f1048106a80 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.692+0000 7f104e4f1640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10340a2b80 0x7f10340a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1044009900 tx=0x7f10440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.694+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f104009b3f0 con 0x7f1048106a80 2024-09-12T01:54:26.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.06 MB/sec execute 284 sec latency 17571.054 ms 2024-09-12T01:54:26.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.897+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10481997c0 con 0x7f1048106a80 2024-09-12T01:54:26.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.898+0000 7f103f7fe640 1 -- 172.21.15.122:0/2542753885 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f104009f6f0 con 0x7f1048106a80 2024-09-12T01:54:26.899 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:54:26.899 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:54:26.899 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:54:26.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10340a2b80 msgr2=0x7f10340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:26.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10340a2b80 0x7f10340a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1044009900 tx=0x7f10440058d0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 msgr2=0x7f104819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f104819cd30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f104000b970 tx=0x7f104000be30 comp rx=0 tx=0).stop 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 shutdown_connections 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10340a2b80 0x7f10340a5040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1048106a80 0x7f104819cd30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1048101420 0x7f10481989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 --2- 172.21.15.122:0/2542753885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1048100a80 0x7f1048198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 >> 172.21.15.122:0/2542753885 conn(0x7f10480fc7a0 msgr2=0x7f104806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 shutdown_connections 2024-09-12T01:54:26.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:26.901+0000 7f1050755640 1 -- 172.21.15.122:0/2542753885 wait complete. 2024-09-12T01:54:27.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 7a9d734d --release --await' 2024-09-12T01:54:27.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 7a9d734d --release --await 2024-09-12T01:54:27.845 INFO:tasks.workunit.client.0.smithi122.stdout: 1 708083 10.02 MB/sec execute 285 sec latency 18571.111 ms 2024-09-12T01:54:27.984 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:27 smithi122 ceph-mon[30925]: pgmap v574: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.3 MiB/s wr, 91 op/s 2024-09-12T01:54:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:27 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2542753885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:27 smithi151 ceph-mon[35229]: pgmap v574: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.3 MiB/s wr, 91 op/s 2024-09-12T01:54:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:27 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2542753885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:28.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:27 smithi183 ceph-mon[35127]: pgmap v574: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.3 MiB/s wr, 91 op/s 2024-09-12T01:54:28.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:27 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2542753885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 -- 172.21.15.122:0/3195372914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 msgr2=0x7fc364103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 --2- 172.21.15.122:0/3195372914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc364103110 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc358009930 tx=0x7fc35802a420 comp rx=0 tx=0).stop 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 -- 172.21.15.122:0/3195372914 shutdown_connections 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 --2- 172.21.15.122:0/3195372914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3641082a0 0x7fc36410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 --2- 172.21.15.122:0/3195372914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3641036e0 0x7fc364107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 --2- 172.21.15.122:0/3195372914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc364103110 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 -- 172.21.15.122:0/3195372914 >> 172.21.15.122:0/3195372914 conn(0x7fc3640fc7d0 msgr2=0x7fc3640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 -- 172.21.15.122:0/3195372914 shutdown_connections 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.491+0000 7fc36c799640 1 -- 172.21.15.122:0/3195372914 wait complete. 2024-09-12T01:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 Processor -- start 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 -- start start 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc3641985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3641036e0 0x7fc364198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3641082a0 0x7fc36419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641120a0 con 0x7fc364102d30 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111f20 con 0x7fc3641036e0 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36c799640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112220 con 0x7fc3641082a0 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36a535640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc3641985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36a535640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc3641985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53464/0 (socket says 172.21.15.122:53464) 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc36a535640 1 -- 172.21.15.122:0/1465915873 learned_addr learned my addr 172.21.15.122:0/1465915873 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.492+0000 7fc369d34640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3641036e0 0x7fc364198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36a535640 1 -- 172.21.15.122:0/1465915873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3641082a0 msgr2=0x7fc36419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36ad36640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3641082a0 0x7fc36419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36a535640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3641082a0 0x7fc36419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36a535640 1 -- 172.21.15.122:0/1465915873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3641036e0 msgr2=0x7fc364198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36a535640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3641036e0 0x7fc364198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36a535640 1 -- 172.21.15.122:0/1465915873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36419d550 con 0x7fc364102d30 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36a535640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc3641985a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc358009230 tx=0x7fc358004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc3577fe640 1 -- 172.21.15.122:0/1465915873 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35803f030 con 0x7fc364102d30 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc3577fe640 1 -- 172.21.15.122:0/1465915873 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc358003ad0 con 0x7fc364102d30 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36419d7e0 con 0x7fc364102d30 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc3577fe640 1 -- 172.21.15.122:0/1465915873 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358035070 con 0x7fc364102d30 2024-09-12T01:54:28.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.493+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3641a5120 con 0x7fc364102d30 2024-09-12T01:54:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.496+0000 7fc3577fe640 1 -- 172.21.15.122:0/1465915873 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc358042030 con 0x7fc364102d30 2024-09-12T01:54:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.496+0000 7fc36eb72740 1 Processor -- start 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 -- start start 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc3577fe640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3500a2b80 0x7fc3500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 0x55bb0344e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb0345d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0345d650 0x55bb0345fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bb033b3620 con 0x55bb0344df20 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bb0335ae50 con 0x55bb0345d650 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36eb72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bb033af5e0 con 0x55bb0344e840 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36ad36640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3500a2b80 0x7fc3500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36a535640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 0x55bb0344e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36a535640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 0x55bb0344e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53466/0 (socket says 172.21.15.122:53466) 2024-09-12T01:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36a535640 1 -- 172.21.15.122:0/2979596849 learned_addr learned my addr 172.21.15.122:0/2979596849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc3577fe640 1 -- 172.21.15.122:0/1465915873 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc3580ef210 con 0x7fc364102d30 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc36ad36640 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0345d650 0x55bb0345fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.497+0000 7fc369d34640 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb0345d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36a535640 1 -- 172.21.15.122:0/2979596849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 msgr2=0x55bb0345d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36a535640 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb0345d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36a535640 1 -- 172.21.15.122:0/2979596849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0345d650 msgr2=0x55bb0345fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36a535640 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0345d650 0x55bb0345fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36a535640 1 -- 172.21.15.122:0/2979596849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb0334bd40 con 0x55bb0344df20 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36ad36640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3500a2b80 0x7fc3500a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c014950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc36a535640 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 0x55bb0344e300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc358005e70 tx=0x7fc358009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e39cee8f198c3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc354ff9640 1 -- 172.21.15.122:0/2979596849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358049040 con 0x55bb0344df20 2024-09-12T01:54:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc354ff9640 1 -- 172.21.15.122:0/2979596849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3580bfde0 con 0x55bb0344df20 2024-09-12T01:54:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.498+0000 7fc354ff9640 1 -- 172.21.15.122:0/2979596849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3580b80a0 con 0x55bb0344df20 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 -- 172.21.15.122:0/2979596849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 msgr2=0x55bb0344e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 0x55bb0344e300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc358005e70 tx=0x7fc358009560 comp rx=0 tx=0).stop 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 -- 172.21.15.122:0/2979596849 shutdown_connections 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0345d650 0x55bb0345fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb0345d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2979596849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0344df20 0x55bb0344e300 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 -- 172.21.15.122:0/2979596849 >> 172.21.15.122:0/2979596849 conn(0x55bb0344a5f0 msgr2=0x55bb0344a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 -- 172.21.15.122:0/2979596849 shutdown_connections 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 -- 172.21.15.122:0/2979596849 wait complete. 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 -- start start 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.499+0000 7fc36eb72740 1 Processor -- start 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb0345a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb0345ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb0345b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bb0345ffe0 con 0x55bb0345d650 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bb0333bf10 con 0x55bb0344e840 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bb03460160 con 0x55bb0344df20 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36eb72740 10 client.0 _ll_register_callbacks cb 0x55bb0335fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb0345b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb0345b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53482/0 (socket says 172.21.15.122:53482) 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 -- 172.21.15.122:0/3888477667 learned_addr learned my addr 172.21.15.122:0/3888477667 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36a535640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb0345a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc369d34640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb0345ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 -- 172.21.15.122:0/3888477667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 msgr2=0x55bb0345a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb0345a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 -- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 msgr2=0x55bb0345ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb0345ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 -- 172.21.15.122:0/3888477667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb030c2580 con 0x55bb0345d650 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc369d34640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb0345ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc36ad36640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb0345b0b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011b90 tx=0x7fc35c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568eb2818a993d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc34dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.500+0000 7fc34dffb640 1 -- 172.21.15.122:0/3888477667 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c023070 con 0x55bb0345d650 2024-09-12T01:54:28.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc34dffb640 1 -- 172.21.15.122:0/3888477667 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc35c004b20 con 0x55bb0345d650 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3580bfde0 con 0x55bb0345d650 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc34dffb640 1 -- 172.21.15.122:0/3888477667 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc35c004d60 con 0x55bb0345d650 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 10 client.16761 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb0318c960 con 0x55bb0345d650 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc34dffb640 1 -- 172.21.15.122:0/3888477667 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3580bfde0 con 0x55bb0345d650 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc34dffb640 1 -- 172.21.15.122:0/3888477667 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc35c01e070 con 0x55bb0345d650 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 10 client.16761 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 10 client.16761 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 20 client.16761 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb0346d030 0x55bb0346f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 4 client.16761 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.501+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb033c0b90 con 0x55bb0346d030 2024-09-12T01:54:28.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.502+0000 7fc36a535640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb0346d030 0x55bb0346f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.502+0000 7fc34dffb640 10 client.16761 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.502+0000 7fc36a535640 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb0346d030 0x55bb0346f4b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb05bf55a0956ff4 server_cookie=553d89f5a163df7f in_seq=0 out_seq=0 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc34dffb640 1 -- 172.21.15.122:0/3888477667 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb033c0b90 con 0x55bb0346d030 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc34dffb640 10 client.16761 handle_command_reply: tid=0 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 client.16761 shutdown 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 msgr2=0x55bb0345b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb0345b0b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011b90 tx=0x7fc35c004ab0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 shutdown_connections 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb0346d030 0x55bb0346f4b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb0345b0b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb0345ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 --2- 172.21.15.122:0/3888477667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb0345a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 >> 172.21.15.122:0/3888477667 conn(0x55bb0344a5f0 msgr2=0x55bb03449750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:28.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc34dffb640 0 client.16761 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:28.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 shutdown_connections 2024-09-12T01:54:28.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 1 -- 172.21.15.122:0/3888477667 wait complete. 2024-09-12T01:54:28.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.503+0000 7fc36eb72740 20 client.16761 trim_cache size 0 max 16384 2024-09-12T01:54:28.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 Processor -- start 2024-09-12T01:54:28.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 -- start start 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb03474d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb034752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb034653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641add40 con 0x55bb0345d650 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641ab610 con 0x55bb0344e840 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36eb72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641a9440 con 0x55bb0344df20 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36ad36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb034653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36ad36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb034653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53498/0 (socket says 172.21.15.122:53498) 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc36ad36640 1 -- 172.21.15.122:0/2383851973 learned_addr learned my addr 172.21.15.122:0/2383851973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.520+0000 7fc369d34640 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb034752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36a535640 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb03474d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36ad36640 1 -- 172.21.15.122:0/2383851973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 msgr2=0x55bb03474d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36ad36640 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb03474d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36ad36640 1 -- 172.21.15.122:0/2383851973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 msgr2=0x55bb034752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36ad36640 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb034752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36ad36640 1 -- 172.21.15.122:0/2383851973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb034595e0 con 0x55bb0345d650 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc36ad36640 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb034653a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011fd0 tx=0x7fc35c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3ffff5747b5f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc34ffff640 1 -- 172.21.15.122:0/2383851973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c023070 con 0x55bb0345d650 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc34ffff640 1 -- 172.21.15.122:0/2383851973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c008e80 con 0x55bb0345d650 2024-09-12T01:54:28.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.521+0000 7fc34ffff640 1 -- 172.21.15.122:0/2383851973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c02e030 con 0x55bb0345d650 2024-09-12T01:54:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 -- 172.21.15.122:0/2383851973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 msgr2=0x55bb034653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb034653a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011fd0 tx=0x7fc35c004a20 comp rx=0 tx=0).stop 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 -- 172.21.15.122:0/2383851973 shutdown_connections 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb034653a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344e840 0x55bb034752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 --2- 172.21.15.122:0/2383851973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344df20 0x55bb03474d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 -- 172.21.15.122:0/2383851973 >> 172.21.15.122:0/2383851973 conn(0x55bb0344a5f0 msgr2=0x55bb0345a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 -- 172.21.15.122:0/2383851973 shutdown_connections 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 -- 172.21.15.122:0/2383851973 wait complete. 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 -- start start 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.522+0000 7fc36eb72740 1 Processor -- start 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344df20 0x55bb03465b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb03461f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb03462470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641a9ad0 con 0x55bb0345d650 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641aaf40 con 0x55bb0344df20 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3641aa1a0 con 0x55bb0344e840 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36eb72740 10 client.0 _ll_register_callbacks cb 0x7fc35c011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36ad36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb03462470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36ad36640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb03462470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53512/0 (socket says 172.21.15.122:53512) 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36ad36640 1 -- 172.21.15.122:0/1315806193 learned_addr learned my addr 172.21.15.122:0/1315806193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344df20 0x55bb03465b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc369d34640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb03461f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 -- 172.21.15.122:0/1315806193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 msgr2=0x55bb03461f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb03461f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 -- 172.21.15.122:0/1315806193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 msgr2=0x55bb03462470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb03462470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 -- 172.21.15.122:0/1315806193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb03455420 con 0x55bb0344df20 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36ad36640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb03462470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc36a535640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344df20 0x55bb03465b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc3580bb9c0 tx=0x7fc3580bbbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1003b0d7848fb6df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc34dffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.523+0000 7fc34dffb640 1 -- 172.21.15.122:0/1315806193 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358049040 con 0x55bb0344df20 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc34dffb640 1 -- 172.21.15.122:0/1315806193 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc358003c80 con 0x55bb0344df20 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb033bb0a0 con 0x55bb0344df20 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc34dffb640 1 -- 172.21.15.122:0/1315806193 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3580b80a0 con 0x55bb0344df20 2024-09-12T01:54:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 10 client.26006 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb034d8c50 con 0x55bb0344df20 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc34dffb640 1 -- 172.21.15.122:0/1315806193 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bb033bb0a0 con 0x55bb0344df20 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc34dffb640 1 -- 172.21.15.122:0/1315806193 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc35804c060 con 0x55bb0344df20 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 10 client.26006 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 10 client.26006 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 20 client.26006 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb034d91e0 0x55bb034db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 4 client.26006 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7a9d734d", "release": true, "await": true}] 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7a9d734d", "release": true, "await": true}) -- 0x55bb033375a0 con 0x55bb034d91e0 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.524+0000 7fc369d34640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb034d91e0 0x55bb034db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.525+0000 7fc34dffb640 10 client.26006 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.525+0000 7fc369d34640 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb034d91e0 0x55bb034db5d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49aff8d959364be8 server_cookie=a31355ba0878b0e in_seq=0 out_seq=0 2024-09-12T01:54:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc34dffb640 1 -- 172.21.15.122:0/1315806193 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bb033375a0 con 0x55bb034d91e0 2024-09-12T01:54:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc34dffb640 10 client.26006 handle_command_reply: tid=0 2024-09-12T01:54:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 client.26006 shutdown 2024-09-12T01:54:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344df20 msgr2=0x55bb03465b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344df20 0x55bb03465b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc3580bb9c0 tx=0x7fc3580bbbb0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 shutdown_connections 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bb034d91e0 0x55bb034db5d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bb0345d650 0x55bb03462470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb0344e840 0x55bb03461f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc34dffb640 0 client.26006 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 --2- 172.21.15.122:0/1315806193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb0344df20 0x55bb03465b80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 >> 172.21.15.122:0/1315806193 conn(0x55bb0344a5f0 msgr2=0x55bb03448f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 shutdown_connections 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 1 -- 172.21.15.122:0/1315806193 wait complete. 2024-09-12T01:54:28.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.532+0000 7fc36eb72740 20 client.26006 trim_cache size 0 max 16384 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 32, 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "7a9d734d": { 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 32, 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 139.4, 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:28.537 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:54:28.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3500a2b80 msgr2=0x7fc3500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3500a2b80 0x7fc3500a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c014950 comp rx=0 tx=0).stop 2024-09-12T01:54:28.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 msgr2=0x7fc3641985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:28.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc3641985a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc358009230 tx=0x7fc358004270 comp rx=0 tx=0).stop 2024-09-12T01:54:28.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 shutdown_connections 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc3500a2b80 0x7fc3500a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3641082a0 0x7fc36419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3641036e0 0x7fc364198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 --2- 172.21.15.122:0/1465915873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc364102d30 0x7fc3641985a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 >> 172.21.15.122:0/1465915873 conn(0x7fc3640fc7d0 msgr2=0x7fc364105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 shutdown_connections 2024-09-12T01:54:28.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:28.533+0000 7fc36c799640 1 -- 172.21.15.122:0/1465915873 wait complete. 2024-09-12T01:54:28.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 709081 9.99 MB/sec execute 286 sec latency 19258.731 ms 2024-09-12T01:54:29.239 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7a9d734d', total seconds elapsed: 22.7 2024-09-12T01:54:29.239 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T01:54:29.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:54:29.515 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:29.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 711204 10.00 MB/sec execute 287 sec latency 16.799 ms 2024-09-12T01:54:29.913 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:29 smithi122 ceph-mon[30925]: pgmap v575: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 -- 172.21.15.122:0/1076606648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 msgr2=0x7f356810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 --2- 172.21.15.122:0/1076606648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356810da40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3560009ca0 tx=0x7f356002a950 comp rx=0 tx=0).stop 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 -- 172.21.15.122:0/1076606648 shutdown_connections 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 --2- 172.21.15.122:0/1076606648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356810da40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 --2- 172.21.15.122:0/1076606648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3568102570 0x7f35681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 --2- 172.21.15.122:0/1076606648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568100e20 0x7f3568101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 -- 172.21.15.122:0/1076606648 >> 172.21.15.122:0/1076606648 conn(0x7f35680fc7d0 msgr2=0x7f35680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 -- 172.21.15.122:0/1076606648 shutdown_connections 2024-09-12T01:54:30.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:29.999+0000 7f356cc88640 1 -- 172.21.15.122:0/1076606648 wait complete. 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 Processor -- start 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 -- start start 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3568100e20 0x7f3568197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568102570 0x7f35681978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3568110f80 con 0x7f3568102ef0 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3568110e00 con 0x7f3568102570 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356cc88640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3568111100 con 0x7f3568100e20 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f3566d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f3566d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53528/0 (socket says 172.21.15.122:53528) 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f3566d9d640 1 -- 172.21.15.122:0/1380890278 learned_addr learned my addr 172.21.15.122:0/1380890278 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f3565d9b640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568102570 0x7f35681978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:30.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f356659c640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3568100e20 0x7f3568197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f3566d9d640 1 -- 172.21.15.122:0/1380890278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3568100e20 msgr2=0x7f3568197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.000+0000 7f3566d9d640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3568100e20 0x7f3568197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f3566d9d640 1 -- 172.21.15.122:0/1380890278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568102570 msgr2=0x7f35681978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f3566d9d640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568102570 0x7f35681978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f3566d9d640 1 -- 172.21.15.122:0/1380890278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356819c310 con 0x7f3568102ef0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f3565d9b640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568102570 0x7f35681978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f3566d9d640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356819bc30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3560004af0 tx=0x7f356002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560034040 con 0x7f3568102ef0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356819c5a0 con 0x7f3568102ef0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3560030920 con 0x7f3568102ef0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560004820 con 0x7f3568102ef0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.001+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35681a3ee0 con 0x7f3568102ef0 2024-09-12T01:54:30.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.002+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3568101200 con 0x7f3568102ef0 2024-09-12T01:54:30.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.003+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3560037020 con 0x7f3568102ef0 2024-09-12T01:54:30.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.004+0000 7f35577fe640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35500a2b80 0x7f35500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:30.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.004+0000 7f356659c640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35500a2b80 0x7f35500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:30.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.005+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f35600f01b0 con 0x7f3568102ef0 2024-09-12T01:54:30.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.005+0000 7f356659c640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35500a2b80 0x7f35500a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3558009b60 tx=0x7f3558005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:30.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.008+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35600811b0 con 0x7f3568102ef0 2024-09-12T01:54:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:29 smithi151 ceph-mon[35229]: pgmap v575: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:54:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:29 smithi183 ceph-mon[35127]: pgmap v575: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:54:30.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.229+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35681a40b0 con 0x7f3568102ef0 2024-09-12T01:54:30.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.230+0000 7f35577fe640 1 -- 172.21.15.122:0/1380890278 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f35600bd110 con 0x7f3568102ef0 2024-09-12T01:54:30.231 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:54:30.231 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:54:30.231 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:54:30.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35500a2b80 msgr2=0x7f35500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35500a2b80 0x7f35500a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3558009b60 tx=0x7f3558005f90 comp rx=0 tx=0).stop 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 msgr2=0x7f356819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356819bc30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3560004af0 tx=0x7f356002f5b0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 shutdown_connections 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f35500a2b80 0x7f35500a5040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3568102ef0 0x7f356819bc30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3568102570 0x7f35681978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 --2- 172.21.15.122:0/1380890278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3568100e20 0x7f3568197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:30.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 >> 172.21.15.122:0/1380890278 conn(0x7f35680fc7d0 msgr2=0x7f3568106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:30.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 shutdown_connections 2024-09-12T01:54:30.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:30.232+0000 7f356cc88640 1 -- 172.21.15.122:0/1380890278 wait complete. 2024-09-12T01:54:30.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 714273 10.03 MB/sec execute 288 sec latency 10.409 ms 2024-09-12T01:54:30.903 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:30 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1380890278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:30.904 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T01:54:30.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T01:54:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:30 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1380890278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:31.168 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:30 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1380890278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:31.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 -- 172.21.15.122:0/4035462683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480100e20 msgr2=0x7ff480101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 --2- 172.21.15.122:0/4035462683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480100e20 0x7ff480101200 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff470009900 tx=0x7ff47002a480 comp rx=0 tx=0).stop 2024-09-12T01:54:31.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 -- 172.21.15.122:0/4035462683 shutdown_connections 2024-09-12T01:54:31.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 --2- 172.21.15.122:0/4035462683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 0x7ff48010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 --2- 172.21.15.122:0/4035462683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480102570 0x7ff4801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 --2- 172.21.15.122:0/4035462683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480100e20 0x7ff480101200 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 -- 172.21.15.122:0/4035462683 >> 172.21.15.122:0/4035462683 conn(0x7ff4800fc7d0 msgr2=0x7ff4800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:31.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 -- 172.21.15.122:0/4035462683 shutdown_connections 2024-09-12T01:54:31.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.649+0000 7ff485211640 1 -- 172.21.15.122:0/4035462683 wait complete. 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.650+0000 7ff485211640 1 Processor -- start 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.650+0000 7ff485211640 1 -- start start 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff485211640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 0x7ff4801993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff485211640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480102570 0x7ff480199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff485211640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 0x7ff4801a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff485211640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff480110f50 con 0x7ff480102570 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff485211640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff480110dd0 con 0x7ff480100e20 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff485211640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801110d0 con 0x7ff480102ef0 2024-09-12T01:54:31.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480102570 0x7ff480199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 0x7ff4801993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 0x7ff4801a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480102570 0x7ff480199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45080/0 (socket says 172.21.15.122:45080) 2024-09-12T01:54:31.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 0x7ff4801993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45686/0 (socket says 172.21.15.122:45686) 2024-09-12T01:54:31.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47e59c640 1 -- 172.21.15.122:0/1561187060 learned_addr learned my addr 172.21.15.122:0/1561187060 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:31.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 -- 172.21.15.122:0/1561187060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 msgr2=0x7ff4801993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 0x7ff4801993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 -- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480102570 msgr2=0x7ff480199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480102570 0x7ff480199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 -- 172.21.15.122:0/1561187060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4801a4980 con 0x7ff480102ef0 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 0x7ff4801993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.651+0000 7ff47f59e640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 0x7ff4801a4440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff478009fd0 tx=0x7ff47800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.652+0000 7ff467fff640 1 -- 172.21.15.122:0/1561187060 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47800ce20 con 0x7ff480102ef0 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.652+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4801a4b50 con 0x7ff480102ef0 2024-09-12T01:54:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.652+0000 7ff467fff640 1 -- 172.21.15.122:0/1561187060 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff478005020 con 0x7ff480102ef0 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.652+0000 7ff467fff640 1 -- 172.21.15.122:0/1561187060 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478010720 con 0x7ff480102ef0 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.652+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4801a5010 con 0x7ff480102ef0 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff467fff640 1 -- 172.21.15.122:0/1561187060 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff478005540 con 0x7ff480102ef0 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 Processor -- start 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 -- start start 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96426f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b96431810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96434140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560b963882a0 con 0x560b96431d50 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b9632fb10 con 0x560b96427490 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.654+0000 7ff4875ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560b96384260 con 0x560b96426b70 2024-09-12T01:54:31.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96426f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b96431810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b96431810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45688/0 (socket says 172.21.15.122:45688) 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1673098303 learned_addr learned my addr 172.21.15.122:0/1673098303 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96426f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43702/0 (socket says 172.21.15.122:43702) 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47e59c640 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96434140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff467fff640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4680a2b80 0x7ff4680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47f59e640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4680a2b80 0x7ff4680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1673098303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 msgr2=0x560b96426f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96426f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1673098303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 msgr2=0x560b96434140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96434140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1673098303 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b960f9b90 con 0x560b96427490 2024-09-12T01:54:31.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff467fff640 1 -- 172.21.15.122:0/1561187060 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff4780cf6f0 con 0x7ff480102ef0 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b96431810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff47002a9a0 tx=0x7ff470005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4c4cdecd65bcc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff47f59e640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4680a2b80 0x7ff4680a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4780cf9e0 tx=0x7ff4780bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff4657fa640 1 -- 172.21.15.122:0/1673098303 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470035030 con 0x560b96427490 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff4657fa640 1 -- 172.21.15.122:0/1673098303 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff470040070 con 0x560b96427490 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.655+0000 7ff4657fa640 1 -- 172.21.15.122:0/1673098303 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470031b50 con 0x560b96427490 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 -- 172.21.15.122:0/1673098303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 msgr2=0x560b96431810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b96431810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff47002a9a0 tx=0x7ff470005dd0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 -- 172.21.15.122:0/1673098303 shutdown_connections 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96434140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b96431810 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1673098303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96426f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 -- 172.21.15.122:0/1673098303 >> 172.21.15.122:0/1673098303 conn(0x560b96423100 msgr2=0x560b964234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 -- 172.21.15.122:0/1673098303 shutdown_connections 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 -- 172.21.15.122:0/1673098303 wait complete. 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.656+0000 7ff4875ea740 1 -- start start 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 Processor -- start 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b9642ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 0x560b9642f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96431d50 0x560b9642f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560b96434920 con 0x560b96427490 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b963106e0 con 0x560b96431d50 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560b96434aa0 con 0x560b96426b70 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff4875ea740 10 client.0 _ll_register_callbacks cb 0x560b96334a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 0x560b9642f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b9642ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 0x560b9642f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45096/0 (socket says 172.21.15.122:45096) 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1554356867 learned_addr learned my addr 172.21.15.122:0/1554356867 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:31.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47e59c640 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96431d50 0x560b9642f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1554356867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 msgr2=0x560b9642ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b9642ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96431d50 msgr2=0x560b9642f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96431d50 0x560b9642f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 -- 172.21.15.122:0/1554356867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9643de10 con 0x560b96427490 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 0x560b9642f130 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff470005940 tx=0x7ff47002ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ff67554fb711f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.657+0000 7ff462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff462ff5640 1 -- 172.21.15.122:0/1554356867 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47004d070 con 0x560b96427490 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff462ff5640 1 -- 172.21.15.122:0/1554356867 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff470005220 con 0x560b96427490 2024-09-12T01:54:31.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff470040070 con 0x560b96427490 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff462ff5640 1 -- 172.21.15.122:0/1554356867 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff470058050 con 0x560b96427490 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 10 client.16785 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b96440780 con 0x560b96427490 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff462ff5640 1 -- 172.21.15.122:0/1554356867 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff470040070 con 0x560b96427490 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff462ff5640 1 -- 172.21.15.122:0/1554356867 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff4700035f0 con 0x560b96427490 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 10 client.16785 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 10 client.16785 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:31.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 20 client.16785 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b96441ad0 0x560b96443ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 4 client.16785 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.658+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b963959d0 con 0x560b96441ad0 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.659+0000 7ff47e59c640 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b96441ad0 0x560b96443ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.659+0000 7ff47e59c640 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b96441ad0 0x560b96443ec0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be21f3acd584c73a server_cookie=a0079319a2794336 in_seq=0 out_seq=0 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.659+0000 7ff462ff5640 10 client.16785 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff462ff5640 1 -- 172.21.15.122:0/1554356867 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b963959d0 con 0x560b96441ad0 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff462ff5640 10 client.16785 handle_command_reply: tid=0 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 client.16785 shutdown 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 msgr2=0x560b9642f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 0x560b9642f130 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff470005940 tx=0x7ff47002ac30 comp rx=0 tx=0).stop 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 shutdown_connections 2024-09-12T01:54:31.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b96441ad0 0x560b96443ec0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96431d50 0x560b9642f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96427490 0x560b9642f130 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 --2- 172.21.15.122:0/1554356867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b9642ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 >> 172.21.15.122:0/1554356867 conn(0x560b96423100 msgr2=0x560b9641de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff462ff5640 0 client.16785 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 shutdown_connections 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 1 -- 172.21.15.122:0/1554356867 wait complete. 2024-09-12T01:54:31.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.660+0000 7ff4875ea740 20 client.16785 trim_cache size 0 max 16384 2024-09-12T01:54:31.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.677+0000 7ff4875ea740 1 Processor -- start 2024-09-12T01:54:31.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 -- start start 2024-09-12T01:54:31.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 0x560b96430ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9642f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96431d50 0x560b964494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801adc30 con 0x560b96426b70 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801ab500 con 0x560b96427490 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff4875ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801a9330 con 0x560b96431d50 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 0x560b96430ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 0x560b96430ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45102/0 (socket says 172.21.15.122:45102) 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 -- 172.21.15.122:0/2161141985 learned_addr learned my addr 172.21.15.122:0/2161141985 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9642f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96431d50 0x560b964494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 -- 172.21.15.122:0/2161141985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96431d50 msgr2=0x560b964494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96431d50 0x560b964494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 -- 172.21.15.122:0/2161141985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 msgr2=0x560b9642f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9642f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 -- 172.21.15.122:0/2161141985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b96422f30 con 0x560b96426b70 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9642f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff47e59c640 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 0x560b96430ac0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff4740094b0 tx=0x7ff474005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd2b824272667f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.678+0000 7ff464ff9640 1 -- 172.21.15.122:0/2161141985 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474012e20 con 0x560b96426b70 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff464ff9640 1 -- 172.21.15.122:0/2161141985 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff474005510 con 0x560b96426b70 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff464ff9640 1 -- 172.21.15.122:0/2161141985 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474022aa0 con 0x560b96426b70 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 -- 172.21.15.122:0/2161141985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 msgr2=0x560b96430ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 0x560b96430ac0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff4740094b0 tx=0x7ff474005f90 comp rx=0 tx=0).stop 2024-09-12T01:54:31.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 -- 172.21.15.122:0/2161141985 shutdown_connections 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96431d50 0x560b964494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9642f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 --2- 172.21.15.122:0/2161141985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96426b70 0x560b96430ac0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 -- 172.21.15.122:0/2161141985 >> 172.21.15.122:0/2161141985 conn(0x560b96423100 msgr2=0x560b9641de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 -- 172.21.15.122:0/2161141985 shutdown_connections 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.679+0000 7ff4875ea740 1 -- 172.21.15.122:0/2161141985 wait complete. 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 -- start start 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 Processor -- start 2024-09-12T01:54:31.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96439df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9643a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96436230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801a99c0 con 0x560b96431d50 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801aae30 con 0x560b96427490 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4801aa090 con 0x560b96426b70 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff4875ea740 10 client.0 _ll_register_callbacks cb 0x7ff470009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96436230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9643a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96436230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45110/0 (socket says 172.21.15.122:45110) 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47e59c640 1 -- 172.21.15.122:0/706705611 learned_addr learned my addr 172.21.15.122:0/706705611 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47e59c640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96439df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47ed9d640 1 -- 172.21.15.122:0/706705611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 msgr2=0x560b96439df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96439df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47ed9d640 1 -- 172.21.15.122:0/706705611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 msgr2=0x560b96436230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96436230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47ed9d640 1 -- 172.21.15.122:0/706705611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9641a260 con 0x560b96427490 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.680+0000 7ff47e59c640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96439df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff47ed9d640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9643a330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff470005ec0 tx=0x7ff470003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62ac09848ed21c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff462ff5640 1 -- 172.21.15.122:0/706705611 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47004d040 con 0x560b96427490 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff474005510 con 0x560b96427490 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff462ff5640 1 -- 172.21.15.122:0/706705611 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ff470032270 con 0x560b96427490 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff462ff5640 1 -- 172.21.15.122:0/706705611 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff470035070 con 0x560b96427490 2024-09-12T01:54:31.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff4875ea740 10 client.26030 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b964acd20 con 0x560b96427490 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff462ff5640 1 -- 172.21.15.122:0/706705611 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff474005510 con 0x560b96427490 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff462ff5640 1 -- 172.21.15.122:0/706705611 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff470032470 con 0x560b96427490 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff4875ea740 10 client.26030 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.681+0000 7ff4875ea740 10 client.26030 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff4875ea740 20 client.26030 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff4875ea740 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b964ad400 0x560b964af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff4875ea740 4 client.26030 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T01:54:31.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x560b9638fd20 con 0x560b964ad400 2024-09-12T01:54:31.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff47e59c640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b964ad400 0x560b964af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:31.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff47e59c640 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b964ad400 0x560b964af7f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e171e8067861f9 server_cookie=30455744efb43dc2 in_seq=0 out_seq=0 2024-09-12T01:54:31.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.682+0000 7ff462ff5640 10 client.26030 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff462ff5640 1 -- 172.21.15.122:0/706705611 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560b9638fd20 con 0x560b964ad400 2024-09-12T01:54:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff462ff5640 10 client.26030 handle_command_reply: tid=0 2024-09-12T01:54:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 client.26030 shutdown 2024-09-12T01:54:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 msgr2=0x560b9643a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9643a330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff470005ec0 tx=0x7ff470003820 comp rx=0 tx=0).stop 2024-09-12T01:54:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 shutdown_connections 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x560b964ad400 0x560b964af7f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b96431d50 0x560b96436230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b96427490 0x560b9643a330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 --2- 172.21.15.122:0/706705611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b96426b70 0x560b96439df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff462ff5640 0 client.26030 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 >> 172.21.15.122:0/706705611 conn(0x560b96423100 msgr2=0x560b9643cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 shutdown_connections 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.767+0000 7ff4875ea740 1 -- 172.21.15.122:0/706705611 wait complete. 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff4875ea740 20 client.26030 trim_cache size 0 max 16384 2024-09-12T01:54:31.770 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 34, 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "dd7f595c": { 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 34, 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:54:31.771 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 139.4, 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:31.772 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:54:31.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4680a2b80 msgr2=0x7ff4680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4680a2b80 0x7ff4680a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4780cf9e0 tx=0x7ff4780bc040 comp rx=0 tx=0).stop 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 msgr2=0x7ff4801a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 0x7ff4801a4440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff478009fd0 tx=0x7ff47800ef10 comp rx=0 tx=0).stop 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 shutdown_connections 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4680a2b80 0x7ff4680a5040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff480102ef0 0x7ff4801a4440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff480102570 0x7ff480199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 --2- 172.21.15.122:0/1561187060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff480100e20 0x7ff4801993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 >> 172.21.15.122:0/1561187060 conn(0x7ff4800fc7d0 msgr2=0x7ff4800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 shutdown_connections 2024-09-12T01:54:31.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:31.768+0000 7ff485211640 1 -- 172.21.15.122:0/1561187060 wait complete. 2024-09-12T01:54:31.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 10.05 MB/sec execute 289 sec latency 80.165 ms 2024-09-12T01:54:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:31 smithi122 ceph-mon[30925]: pgmap v576: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 2.0 MiB/s wr, 91 op/s 2024-09-12T01:54:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:31 smithi151 ceph-mon[35229]: pgmap v576: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 2.0 MiB/s wr, 91 op/s 2024-09-12T01:54:32.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:31 smithi183 ceph-mon[35127]: pgmap v576: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 2.0 MiB/s wr, 91 op/s 2024-09-12T01:54:32.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 10.01 MB/sec execute 290 sec latency 1080.221 ms 2024-09-12T01:54:32.914 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:32 smithi122 ceph-mon[30925]: pgmap v577: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.4 MiB/s wr, 108 op/s 2024-09-12T01:54:32.948 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dd7f595c', quiesce took 3.7 seconds. Will release after: 15.7 2024-09-12T01:54:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:32 smithi151 ceph-mon[35229]: pgmap v577: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.4 MiB/s wr, 108 op/s 2024-09-12T01:54:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:32 smithi183 ceph-mon[35127]: pgmap v577: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.4 MiB/s wr, 108 op/s 2024-09-12T01:54:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.299+0000 7fa6ba253640 1 -- 172.21.15.122:0/2914182941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 msgr2=0x7fa6b4074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.299+0000 7fa6ba253640 1 --2- 172.21.15.122:0/2914182941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 0x7fa6b4074600 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009ca0 tx=0x7fa6a402a950 comp rx=0 tx=0).stop 2024-09-12T01:54:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.307+0000 7fa6ba253640 1 -- 172.21.15.122:0/2914182941 shutdown_connections 2024-09-12T01:54:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.307+0000 7fa6ba253640 1 --2- 172.21.15.122:0/2914182941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b4109a10 0x7fa6b410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.307+0000 7fa6ba253640 1 --2- 172.21.15.122:0/2914182941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4074bd0 0x7fa6b4075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.307+0000 7fa6ba253640 1 --2- 172.21.15.122:0/2914182941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 0x7fa6b4074600 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.307+0000 7fa6ba253640 1 -- 172.21.15.122:0/2914182941 >> 172.21.15.122:0/2914182941 conn(0x7fa6b40fd2e0 msgr2=0x7fa6b40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.312+0000 7fa6ba253640 1 -- 172.21.15.122:0/2914182941 shutdown_connections 2024-09-12T01:54:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.314+0000 7fa6ba253640 1 -- 172.21.15.122:0/2914182941 wait complete. 2024-09-12T01:54:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.314+0000 7fa6ba253640 1 Processor -- start 2024-09-12T01:54:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.315+0000 7fa6ba253640 1 -- start start 2024-09-12T01:54:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.315+0000 7fa6ba253640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b4074bd0 0x7fa6b4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.315+0000 7fa6ba253640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 0x7fa6b419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6ba253640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 0x7fa6b41a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6ba253640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410fab0 con 0x7fa6b40761b0 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6ba253640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410f930 con 0x7fa6b4074bd0 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6ba253640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410fc30 con 0x7fa6b4109a10 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6b3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 0x7fa6b41a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6b3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 0x7fa6b41a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43746/0 (socket says 172.21.15.122:43746) 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.316+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1102778561 learned_addr learned my addr 172.21.15.122:0/1102778561 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.318+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1102778561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b4074bd0 msgr2=0x7fa6b4199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.318+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b4074bd0 0x7fa6b4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.318+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 msgr2=0x7fa6b419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.318+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 0x7fa6b419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.318+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1102778561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b41a1aa0 con 0x7fa6b4109a10 2024-09-12T01:54:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.319+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 0x7fa6b41a1430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009fd0 tx=0x7fa6ac00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.320+0000 7fa6b0ff9640 1 -- 172.21.15.122:0/1102778561 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011830 con 0x7fa6b4109a10 2024-09-12T01:54:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.320+0000 7fa6b0ff9640 1 -- 172.21.15.122:0/1102778561 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ac016070 con 0x7fa6b4109a10 2024-09-12T01:54:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.320+0000 7fa6b0ff9640 1 -- 172.21.15.122:0/1102778561 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00f2e0 con 0x7fa6b4109a10 2024-09-12T01:54:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.322+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b41a1d90 con 0x7fa6b4109a10 2024-09-12T01:54:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.322+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a2310 con 0x7fa6b4109a10 2024-09-12T01:54:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.326+0000 7fa6b0ff9640 1 -- 172.21.15.122:0/1102778561 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa6ac01b460 con 0x7fa6b4109a10 2024-09-12T01:54:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.329+0000 7fa6bcfe9740 1 Processor -- start 2024-09-12T01:54:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 -- start start 2024-09-12T01:54:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 0x555d670695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d6706c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555d66fbf2f0 con 0x555d670641b0 2024-09-12T01:54:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d66f67270 con 0x555d67064af0 2024-09-12T01:54:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.330+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d66fbb2b0 con 0x555d67069b80 2024-09-12T01:54:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 0x555d670695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 0x555d670695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:45730/0 (socket says 172.21.15.122:45730) 2024-09-12T01:54:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2900146379 learned_addr learned my addr 172.21.15.122:0/2900146379 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2900146379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 msgr2=0x555d6706c040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:54:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d6706c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2900146379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 msgr2=0x555d670645b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T01:54:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670645b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.331+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2900146379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d66f66340 con 0x555d67064af0 2024-09-12T01:54:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.332+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 0x555d670695b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402af40 tx=0x7fa6a4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a73bc203ebb1acf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.332+0000 7fa6b0ff9640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa69c0a2e20 0x7fa69c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.332+0000 7fa699ffb640 1 -- 172.21.15.122:0/2900146379 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4034040 con 0x555d67064af0 2024-09-12T01:54:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.333+0000 7fa6b0ff9640 1 -- 172.21.15.122:0/1102778561 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa6ac0ce6c0 con 0x7fa6b4109a10 2024-09-12T01:54:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.333+0000 7fa699ffb640 1 -- 172.21.15.122:0/2900146379 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4003e50 con 0x555d67064af0 2024-09-12T01:54:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2900146379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 msgr2=0x555d670695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 0x555d670695b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402af40 tx=0x7fa6a4009a20 comp rx=0 tx=0).stop 2024-09-12T01:54:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2900146379 shutdown_connections 2024-09-12T01:54:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d6706c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67064af0 0x555d670695b0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402af40 tx=0x7fa6a4009a20 comp rx=0 tx=0).stop 2024-09-12T01:54:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2900146379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.334+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2900146379 >> 172.21.15.122:0/2900146379 conn(0x555d6705f820 msgr2=0x555d6705fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.335+0000 7fa6b2ffd640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa69c0a2e20 0x7fa69c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.335+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2900146379 shutdown_connections 2024-09-12T01:54:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.336+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2900146379 wait complete. 2024-09-12T01:54:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.336+0000 7fa6b2ffd640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa69c0a2e20 0x7fa69c0a52e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80058c0 tx=0x7fa6a8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.337+0000 7fa6bcfe9740 1 -- start start 2024-09-12T01:54:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.337+0000 7fa6bcfe9740 1 Processor -- start 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.337+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67058940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d67069b80 0x555d67058e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d6705e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555d67056de0 con 0x555d67069b80 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d66f48290 con 0x555d670641b0 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d67056f60 con 0x555d6705deb0 2024-09-12T01:54:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6bcfe9740 10 client.0 _ll_register_callbacks cb 0x555d66f79140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d6705e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d6705e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43752/0 (socket says 172.21.15.122:43752) 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1204739890 learned_addr learned my addr 172.21.15.122:0/1204739890 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67058940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 msgr2=0x555d67058940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67058940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1204739890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d67069b80 msgr2=0x555d67058e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d67069b80 0x555d67058e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.338+0000 7fa6b3fff640 1 -- 172.21.15.122:0/1204739890 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d66cd2210 con 0x555d6705deb0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.339+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d6705e310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac097340 tx=0x7fa6ac09a900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c292c7055d9ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.339+0000 7fa692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.339+0000 7fa692ffd640 1 -- 172.21.15.122:0/1204739890 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac09aee0 con 0x555d6705deb0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.339+0000 7fa692ffd640 1 -- 172.21.15.122:0/1204739890 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa6ac097550 con 0x555d6705deb0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.339+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4003e50 con 0x555d6705deb0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa692ffd640 1 -- 172.21.15.122:0/1204739890 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6ac09f290 con 0x555d6705deb0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 10 client.26008 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d66d9e620 con 0x555d6705deb0 2024-09-12T01:54:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa692ffd640 1 -- 172.21.15.122:0/1204739890 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4003e50 con 0x555d6705deb0 2024-09-12T01:54:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa692ffd640 1 -- 172.21.15.122:0/1204739890 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa6ac096c30 con 0x555d6705deb0 2024-09-12T01:54:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 10 client.26008 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 10 client.26008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 20 client.26008 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d67082d90 0x555d67085200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 4 client.26008 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.340+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d66fcca20 con 0x555d67082d90 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.341+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d67082d90 0x555d67085200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.341+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d67082d90 0x555d67085200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c6f1f51152f5c4 server_cookie=8ed481c007d76e7 in_seq=0 out_seq=0 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.341+0000 7fa692ffd640 10 client.26008 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.342+0000 7fa692ffd640 1 -- 172.21.15.122:0/1204739890 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d66fcca20 con 0x555d67082d90 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.342+0000 7fa692ffd640 10 client.26008 handle_command_reply: tid=0 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.342+0000 7fa6bcfe9740 1 client.26008 shutdown 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.342+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 msgr2=0x555d6705e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.342+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d6705e310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac097340 tx=0x7fa6ac09a900 comp rx=0 tx=0).stop 2024-09-12T01:54:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 shutdown_connections 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d67082d90 0x555d67085200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d6705e310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d67069b80 0x555d67058e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1204739890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67058940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 >> 172.21.15.122:0/1204739890 conn(0x555d6705f820 msgr2=0x555d67061030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa692ffd640 0 client.26008 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 shutdown_connections 2024-09-12T01:54:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1204739890 wait complete. 2024-09-12T01:54:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.343+0000 7fa6bcfe9740 20 client.26008 trim_cache size 0 max 16384 2024-09-12T01:54:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 Processor -- start 2024-09-12T01:54:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 -- start start 2024-09-12T01:54:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 0x555d6705fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67060140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d67060680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a95a0 con 0x555d6705deb0 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a6e70 con 0x555d670641b0 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a4cc0 con 0x555d67069b80 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 0x555d6705fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 0x555d6705fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45148/0 (socket says 172.21.15.122:45148) 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2552511239 learned_addr learned my addr 172.21.15.122:0/2552511239 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b2ffd640 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67060140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d67060680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2552511239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 msgr2=0x555d67060680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d67060680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2552511239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 msgr2=0x555d67060140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67060140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 -- 172.21.15.122:0/2552511239 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d6705f640 con 0x555d6705deb0 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 0x555d6705fc00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402f790 tx=0x7fa6a4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a1d82990676c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6997fa640 1 -- 172.21.15.122:0/2552511239 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4034040 con 0x555d6705deb0 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6997fa640 1 -- 172.21.15.122:0/2552511239 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4003e50 con 0x555d6705deb0 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.358+0000 7fa6997fa640 1 -- 172.21.15.122:0/2552511239 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4048040 con 0x555d6705deb0 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2552511239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 msgr2=0x555d6705fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 0x555d6705fc00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402f790 tx=0x7fa6a4005ef0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2552511239 shutdown_connections 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d67069b80 0x555d67060680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d670641b0 0x555d67060140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/2552511239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d6705deb0 0x555d6705fc00 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2552511239 >> 172.21.15.122:0/2552511239 conn(0x555d6705f820 msgr2=0x555d67065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2552511239 shutdown_connections 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/2552511239 wait complete. 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 -- start start 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.359+0000 7fa6bcfe9740 1 Processor -- start 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d67094180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67069b80 0x555d670956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a5330 con 0x555d670641b0 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a67a0 con 0x555d67069b80 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41a5a00 con 0x555d6705deb0 2024-09-12T01:54:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6bcfe9740 10 client.0 _ll_register_callbacks cb 0x7fa6ac010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670946c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45162/0 (socket says 172.21.15.122:45162) 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 -- 172.21.15.122:0/1850704002 learned_addr learned my addr 172.21.15.122:0/1850704002 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b3fff640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67069b80 0x555d670956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d67094180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 -- 172.21.15.122:0/1850704002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 msgr2=0x555d67094180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d67094180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 -- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67069b80 msgr2=0x555d670956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67069b80 0x555d670956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 -- 172.21.15.122:0/1850704002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d67095ce0 con 0x555d670641b0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa6b2ffd640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670946c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x555d67059e20 tx=0x7fa6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bca81cd48f61c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.360+0000 7fa692ffd640 1 -- 172.21.15.122:0/1850704002 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8010840 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa692ffd640 1 -- 172.21.15.122:0/1850704002 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fa6a80109e0 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4003e50 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa692ffd640 1 -- 172.21.15.122:0/1850704002 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6a8010be0 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 10 client.16800 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d670970f0 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa692ffd640 1 -- 172.21.15.122:0/1850704002 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4003e50 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa692ffd640 1 -- 172.21.15.122:0/1850704002 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa6a80263a0 con 0x555d670641b0 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 10 client.16800 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 20 client.16800 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d670eddf0 0x555d670f0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 4 client.16800 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:54:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d66fc6d70 con 0x555d670eddf0 2024-09-12T01:54:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.361+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d670eddf0 0x555d670f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.362+0000 7fa692ffd640 10 client.16800 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.362+0000 7fa6b37fe640 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d670eddf0 0x555d670f0210 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88775c1286a06c5f server_cookie=b4103e985cbeb31 in_seq=0 out_seq=0 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.373+0000 7fa692ffd640 1 -- 172.21.15.122:0/1850704002 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d66fc6d70 con 0x555d670eddf0 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa692ffd640 10 client.16800 handle_command_reply: tid=0 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 client.16800 shutdown 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 msgr2=0x555d670946c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670946c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x555d67059e20 tx=0x7fa6a80046c0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 shutdown_connections 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d670eddf0 0x555d670f0210 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d67069b80 0x555d670956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa692ffd640 0 client.16800 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d670641b0 0x555d670946c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 --2- 172.21.15.122:0/1850704002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6705deb0 0x555d67094180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 >> 172.21.15.122:0/1850704002 conn(0x555d6705f820 msgr2=0x555d670559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 shutdown_connections 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 1 -- 172.21.15.122:0/1850704002 wait complete. 2024-09-12T01:54:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6bcfe9740 20 client.16800 trim_cache size 0 max 16384 2024-09-12T01:54:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:54:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:54:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:54:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:54:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa69c0a2e20 msgr2=0x7fa69c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6ba253640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa69c0a2e20 0x7fa69c0a52e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80058c0 tx=0x7fa6a8005850 comp rx=0 tx=0).stop 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 msgr2=0x7fa6b41a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.374+0000 7fa6ba253640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 0x7fa6b41a1430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009fd0 tx=0x7fa6ac00b8e0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 shutdown_connections 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa69c0a2e20 0x7fa69c0a52e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6b4109a10 0x7fa6b41a1430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6b40761b0 0x7fa6b419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 --2- 172.21.15.122:0/1102778561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b4074bd0 0x7fa6b4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 >> 172.21.15.122:0/1102778561 conn(0x7fa6b40fd2e0 msgr2=0x7fa6b41059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 shutdown_connections 2024-09-12T01:54:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:33.375+0000 7fa6ba253640 1 -- 172.21.15.122:0/1102778561 wait complete. 2024-09-12T01:54:33.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.98 MB/sec execute 291 sec latency 2080.327 ms 2024-09-12T01:54:34.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.94 MB/sec execute 292 sec latency 3080.376 ms 2024-09-12T01:54:35.846 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.91 MB/sec execute 293 sec latency 4080.469 ms 2024-09-12T01:54:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:35 smithi183 ceph-mon[35127]: pgmap v578: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 143 op/s 2024-09-12T01:54:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:35 smithi122 ceph-mon[30925]: pgmap v578: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 143 op/s 2024-09-12T01:54:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:35 smithi151 ceph-mon[35229]: pgmap v578: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 143 op/s 2024-09-12T01:54:36.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.88 MB/sec execute 294 sec latency 5080.566 ms 2024-09-12T01:54:37.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.84 MB/sec execute 295 sec latency 6080.666 ms 2024-09-12T01:54:37.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:37 smithi183 ceph-mon[35127]: pgmap v579: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:54:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:37 smithi122 ceph-mon[30925]: pgmap v579: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:54:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:37 smithi151 ceph-mon[35229]: pgmap v579: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 154 op/s 2024-09-12T01:54:38.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.81 MB/sec execute 296 sec latency 7080.749 ms 2024-09-12T01:54:39.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.78 MB/sec execute 297 sec latency 8080.797 ms 2024-09-12T01:54:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:39 smithi183 ceph-mon[35127]: pgmap v580: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-12T01:54:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:39 smithi122 ceph-mon[30925]: pgmap v580: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-12T01:54:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:39 smithi151 ceph-mon[35229]: pgmap v580: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-12T01:54:40.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.74 MB/sec execute 298 sec latency 9080.917 ms 2024-09-12T01:54:41.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.71 MB/sec execute 299 sec latency 10081.010 ms 2024-09-12T01:54:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:41 smithi183 ceph-mon[35127]: pgmap v581: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 154 op/s 2024-09-12T01:54:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:41 smithi122 ceph-mon[30925]: pgmap v581: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 154 op/s 2024-09-12T01:54:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:41 smithi151 ceph-mon[35229]: pgmap v581: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 154 op/s 2024-09-12T01:54:42.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.68 MB/sec execute 300 sec latency 11081.069 ms 2024-09-12T01:54:43.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.65 MB/sec execute 301 sec latency 12081.178 ms 2024-09-12T01:54:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:43 smithi183 ceph-mon[35127]: pgmap v582: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 156 op/s 2024-09-12T01:54:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:43 smithi122 ceph-mon[30925]: pgmap v582: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 156 op/s 2024-09-12T01:54:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:43 smithi151 ceph-mon[35229]: pgmap v582: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 156 op/s 2024-09-12T01:54:44.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.62 MB/sec execute 302 sec latency 13081.278 ms 2024-09-12T01:54:45.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.58 MB/sec execute 303 sec latency 14081.368 ms 2024-09-12T01:54:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:45 smithi183 ceph-mon[35127]: pgmap v583: 97 pgs: 97 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.5 MiB/s wr, 141 op/s 2024-09-12T01:54:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:45 smithi122 ceph-mon[30925]: pgmap v583: 97 pgs: 97 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.5 MiB/s wr, 141 op/s 2024-09-12T01:54:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:45 smithi151 ceph-mon[35229]: pgmap v583: 97 pgs: 97 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.5 MiB/s wr, 141 op/s 2024-09-12T01:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 -- 172.21.15.122:0/344443470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8102600 msgr2=0x7f89d8108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 --2- 172.21.15.122:0/344443470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8102600 0x7f89d8108e90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009900 tx=0x7f89d002a410 comp rx=0 tx=0).stop 2024-09-12T01:54:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 -- 172.21.15.122:0/344443470 shutdown_connections 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 --2- 172.21.15.122:0/344443470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8102600 0x7f89d8108e90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 --2- 172.21.15.122:0/344443470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d8101c40 0x7f89d81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 --2- 172.21.15.122:0/344443470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d8100a50 0x7f89d8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 -- 172.21.15.122:0/344443470 >> 172.21.15.122:0/344443470 conn(0x7f89d80fc1b0 msgr2=0x7f89d80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 -- 172.21.15.122:0/344443470 shutdown_connections 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 -- 172.21.15.122:0/344443470 wait complete. 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.470+0000 7f89dde9b640 1 Processor -- start 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 -- start start 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 0x7f89d8197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d8101c40 0x7f89d8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d8102600 0x7f89d819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89d810c6d0 con 0x7f89d8100a50 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89d810c550 con 0x7f89d8101c40 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89dde9b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89d810c850 con 0x7f89d8102600 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 0x7f89d8197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 0x7f89d8197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43332/0 (socket says 172.21.15.122:43332) 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 -- 172.21.15.122:0/2801298088 learned_addr learned my addr 172.21.15.122:0/2801298088 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d7fff640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d8102600 0x7f89d819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 -- 172.21.15.122:0/2801298088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d8102600 msgr2=0x7f89d819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d8102600 0x7f89d819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d8101c40 0x7f89d8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 -- 172.21.15.122:0/2801298088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d8101c40 msgr2=0x7f89d8198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d8101c40 0x7f89d8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 -- 172.21.15.122:0/2801298088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d819f950 con 0x7f89d8100a50 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.471+0000 7f89d77fe640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 0x7f89d8197d40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f89c800ccb0 tx=0x7f89c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.472+0000 7f89d4ff9640 1 -- 172.21.15.122:0/2801298088 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8003990 con 0x7f89d8100a50 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.472+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d819fc40 con 0x7f89d8100a50 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.472+0000 7f89d4ff9640 1 -- 172.21.15.122:0/2801298088 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c8003b30 con 0x7f89d8100a50 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.472+0000 7f89d4ff9640 1 -- 172.21.15.122:0/2801298088 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800f7b0 con 0x7f89d8100a50 2024-09-12T01:54:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.472+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81a0190 con 0x7f89d8100a50 2024-09-12T01:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89d4ff9640 1 -- 172.21.15.122:0/2801298088 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f89c800f9b0 con 0x7f89d8100a50 2024-09-12T01:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 Processor -- start 2024-09-12T01:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 -- start start 2024-09-12T01:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a69a0 0x55eda05a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eda050a2f0 con 0x55eda05a72e0 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eda04b2270 con 0x55eda05ac370 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89e0c35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eda05062b0 con 0x55eda05a69a0 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.474+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a69a0 0x55eda05a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43342/0 (socket says 172.21.15.122:43342) 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 -- 172.21.15.122:0/3954470251 learned_addr learned my addr 172.21.15.122:0/3954470251 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d4ff9640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89c00a2c30 0x7f89c00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d7fff640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89c00a2c30 0x7f89c00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d4ff9640 1 -- 172.21.15.122:0/2801298088 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89c80cf260 con 0x7f89d8100a50 2024-09-12T01:54:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 -- 172.21.15.122:0/3954470251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a69a0 msgr2=0x55eda05a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a69a0 0x55eda05a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 -- 172.21.15.122:0/3954470251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 msgr2=0x55eda05ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 -- 172.21.15.122:0/3954470251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda04b1340 con 0x55eda05a72e0 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d77fe640 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05abda0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b7c0 tx=0x7f89c800af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77469fd11e33e801 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89bdffb640 1 -- 172.21.15.122:0/3954470251 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c801d070 con 0x55eda05a72e0 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89d7fff640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89c00a2c30 0x7f89c00a50f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009230 tx=0x7f89d002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89bdffb640 1 -- 172.21.15.122:0/3954470251 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c809c1c0 con 0x55eda05a72e0 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.475+0000 7f89bdffb640 1 -- 172.21.15.122:0/3954470251 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80cce70 con 0x55eda05a72e0 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 -- 172.21.15.122:0/3954470251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 msgr2=0x55eda05abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05abda0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b7c0 tx=0x7f89c800af30 comp rx=0 tx=0).stop 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 -- 172.21.15.122:0/3954470251 shutdown_connections 2024-09-12T01:54:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05abda0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 --2- 172.21.15.122:0/3954470251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a69a0 0x55eda05a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 -- 172.21.15.122:0/3954470251 >> 172.21.15.122:0/3954470251 conn(0x55eda05a2010 msgr2=0x55eda05a2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 -- 172.21.15.122:0/3954470251 shutdown_connections 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 -- 172.21.15.122:0/3954470251 wait complete. 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.476+0000 7f89e0c35740 1 -- start start 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 Processor -- start 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a69a0 0x55eda05b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05b8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eda05af280 con 0x55eda05a72e0 2024-09-12T01:54:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eda0493290 con 0x55eda05a69a0 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eda05af400 con 0x55eda05ac370 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89e0c35740 10 client.0 _ll_register_callbacks cb 0x55eda04c4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43356/0 (socket says 172.21.15.122:43356) 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 -- 172.21.15.122:0/156092734 learned_addr learned my addr 172.21.15.122:0/156092734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d7fff640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d77fe640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a69a0 0x55eda05b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 -- 172.21.15.122:0/156092734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 msgr2=0x55eda05b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 -- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a69a0 msgr2=0x55eda05b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a69a0 0x55eda05b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d6ffd640 1 -- 172.21.15.122:0/156092734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda021d210 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d7fff640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.477+0000 7f89d77fe640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a69a0 0x55eda05b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05b1020 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0106b0 tx=0x7f89cc010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bbaa25fe5145dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89b6ffd640 1 -- 172.21.15.122:0/156092734 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc01d060 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89b6ffd640 1 -- 172.21.15.122:0/156092734 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89cc00eda0 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c809c1c0 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89b6ffd640 1 -- 172.21.15.122:0/156092734 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f89cc018430 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 10 client.16812 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eda02e9620 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89b6ffd640 1 -- 172.21.15.122:0/156092734 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c809c1c0 con 0x55eda05a72e0 2024-09-12T01:54:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89b6ffd640 1 -- 172.21.15.122:0/156092734 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f89cc022070 con 0x55eda05a72e0 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 10 client.16812 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 20 client.16812 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda05bc7d0 0x55eda05bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 4 client.16812 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.478+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eda0517a20 con 0x55eda05bc7d0 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.479+0000 7f89d77fe640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda05bc7d0 0x55eda05bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.479+0000 7f89d77fe640 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda05bc7d0 0x55eda05bebc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8903428a463dff4 server_cookie=de992be739c19c3d in_seq=0 out_seq=0 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.479+0000 7f89b6ffd640 10 client.16812 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89b6ffd640 1 -- 172.21.15.122:0/156092734 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eda0517a20 con 0x55eda05bc7d0 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89b6ffd640 10 client.16812 handle_command_reply: tid=0 2024-09-12T01:54:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 client.16812 shutdown 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 msgr2=0x55eda05b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05b1020 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0106b0 tx=0x7f89cc010b70 comp rx=0 tx=0).stop 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 shutdown_connections 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda05bc7d0 0x55eda05bebc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a72e0 0x55eda05b1020 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 --2- 172.21.15.122:0/156092734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a69a0 0x55eda05b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89b6ffd640 0 client.16812 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 >> 172.21.15.122:0/156092734 conn(0x55eda05a2010 msgr2=0x55eda05a3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 shutdown_connections 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 1 -- 172.21.15.122:0/156092734 wait complete. 2024-09-12T01:54:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.480+0000 7f89e0c35740 20 client.16812 trim_cache size 0 max 16384 2024-09-12T01:54:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 Processor -- start 2024-09-12T01:54:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 -- start start 2024-09-12T01:54:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a72e0 0x55eda05b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81ab820 con 0x55eda05a69a0 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81a90f0 con 0x55eda05a72e0 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.494+0000 7f89e0c35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81a6f20 con 0x55eda05ac370 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43364/0 (socket says 172.21.15.122:43364) 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 -- 172.21.15.122:0/1644812965 learned_addr learned my addr 172.21.15.122:0/1644812965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d7fff640 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 -- 172.21.15.122:0/1644812965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 msgr2=0x55eda05a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 -- 172.21.15.122:0/1644812965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a72e0 msgr2=0x55eda05b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a72e0 0x55eda05b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 -- 172.21.15.122:0/1644812965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda05a1e30 con 0x55eda05a69a0 2024-09-12T01:54:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d7fff640 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89d77fe640 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05a6d80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c80d2870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29c3854b1fede09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89bd7fa640 1 -- 172.21.15.122:0/1644812965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c801d040 con 0x55eda05a69a0 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89bd7fa640 1 -- 172.21.15.122:0/1644812965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c80cce70 con 0x55eda05a69a0 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.495+0000 7f89bd7fa640 1 -- 172.21.15.122:0/1644812965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800fca0 con 0x55eda05a69a0 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 -- 172.21.15.122:0/1644812965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 msgr2=0x55eda05a6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05a6d80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c80d2870 comp rx=0 tx=0).stop 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 -- 172.21.15.122:0/1644812965 shutdown_connections 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05ac370 0x55eda05a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05a72e0 0x55eda05b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 --2- 172.21.15.122:0/1644812965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05a6d80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 -- 172.21.15.122:0/1644812965 >> 172.21.15.122:0/1644812965 conn(0x55eda05a2010 msgr2=0x55eda05a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 -- 172.21.15.122:0/1644812965 shutdown_connections 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 -- 172.21.15.122:0/1644812965 wait complete. 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 -- start start 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 Processor -- start 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.496+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a72e0 0x55eda05b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81a75b0 con 0x55eda05a69a0 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81a8a20 con 0x55eda05ac370 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81a7c80 con 0x55eda05a72e0 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 10 client.0 _ll_register_callbacks cb 0x7f89cc00b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43380/0 (socket says 172.21.15.122:43380) 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 -- 172.21.15.122:0/123983614 learned_addr learned my addr 172.21.15.122:0/123983614 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a72e0 0x55eda05b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 -- 172.21.15.122:0/123983614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a72e0 msgr2=0x55eda05b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a72e0 0x55eda05b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 -- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 msgr2=0x55eda05b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 -- 172.21.15.122:0/123983614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda05b7540 con 0x55eda05a69a0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89d77fe640 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05b59e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b7c0 tx=0x7f89c809bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5227bed59625c515 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89b6ffd640 1 -- 172.21.15.122:0/123983614 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c801d070 con 0x55eda05a69a0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89b6ffd640 1 -- 172.21.15.122:0/123983614 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89c8099dc0 con 0x55eda05a69a0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c80cce70 con 0x55eda05a69a0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89b6ffd640 1 -- 172.21.15.122:0/123983614 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f89c809c1c0 con 0x55eda05a69a0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 10 client.16818 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eda0627700 con 0x55eda05a69a0 2024-09-12T01:54:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.497+0000 7f89b6ffd640 1 -- 172.21.15.122:0/123983614 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c80cce70 con 0x55eda05a69a0 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89b6ffd640 1 -- 172.21.15.122:0/123983614 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f89c809af10 con 0x55eda05a69a0 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89e0c35740 10 client.16818 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89e0c35740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89e0c35740 20 client.16818 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89e0c35740 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda0627af0 0x55eda0629f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89e0c35740 4 client.16818 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eda0511d70 con 0x55eda0627af0 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda0627af0 0x55eda0629f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89b6ffd640 10 client.16818 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.498+0000 7f89d6ffd640 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda0627af0 0x55eda0629f10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3002272bfaab8b43 server_cookie=979aedfa9a9260f6 in_seq=0 out_seq=0 2024-09-12T01:54:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.510+0000 7f89b6ffd640 1 -- 172.21.15.122:0/123983614 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eda0511d70 con 0x55eda0627af0 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.510+0000 7f89b6ffd640 10 client.16818 handle_command_reply: tid=0 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.510+0000 7f89e0c35740 1 client.16818 shutdown 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 msgr2=0x55eda05b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05b59e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b7c0 tx=0x7f89c809bac0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 shutdown_connections 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55eda0627af0 0x55eda0629f10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eda05ac370 0x55eda05b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eda05a72e0 0x55eda05b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89b6ffd640 0 client.16818 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 --2- 172.21.15.122:0/123983614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eda05a69a0 0x55eda05b59e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 >> 172.21.15.122:0/123983614 conn(0x55eda05a2010 msgr2=0x55eda05a11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 shutdown_connections 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 1 -- 172.21.15.122:0/123983614 wait complete. 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89e0c35740 20 client.16818 trim_cache size 0 max 16384 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:54:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:54:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89c00a2c30 msgr2=0x7f89c00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89c00a2c30 0x7f89c00a50f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009230 tx=0x7f89d002a410 comp rx=0 tx=0).stop 2024-09-12T01:54:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 msgr2=0x7f89d8197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 0x7f89d8197d40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f89c800ccb0 tx=0x7f89c800a620 comp rx=0 tx=0).stop 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 shutdown_connections 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89c00a2c30 0x7f89c00a50f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89d8102600 0x7f89d819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89d8101c40 0x7f89d8198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 --2- 172.21.15.122:0/2801298088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89d8100a50 0x7f89d8197d40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.511+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 >> 172.21.15.122:0/2801298088 conn(0x7f89d80fc1b0 msgr2=0x7f89d80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.512+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 shutdown_connections 2024-09-12T01:54:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:46.512+0000 7f89dde9b640 1 -- 172.21.15.122:0/2801298088 wait complete. 2024-09-12T01:54:46.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.55 MB/sec execute 304 sec latency 15081.471 ms 2024-09-12T01:54:47.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.52 MB/sec execute 305 sec latency 16081.562 ms 2024-09-12T01:54:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:47 smithi183 ceph-mon[35127]: pgmap v584: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-12T01:54:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:47 smithi122 ceph-mon[30925]: pgmap v584: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-12T01:54:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:47 smithi151 ceph-mon[35229]: pgmap v584: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 2.0 MiB/s wr, 101 op/s 2024-09-12T01:54:48.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:54:48.847 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.49 MB/sec execute 306 sec latency 17081.612 ms 2024-09-12T01:54:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:49.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 -- 172.21.15.122:0/1843799678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 msgr2=0x7f77081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:49.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 --2- 172.21.15.122:0/1843799678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f77081067e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009900 tx=0x7f76f802a410 comp rx=0 tx=0).stop 2024-09-12T01:54:49.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 -- 172.21.15.122:0/1843799678 shutdown_connections 2024-09-12T01:54:49.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 --2- 172.21.15.122:0/1843799678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f770810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 --2- 172.21.15.122:0/1843799678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f77081067e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 --2- 172.21.15.122:0/1843799678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 -- 172.21.15.122:0/1843799678 >> 172.21.15.122:0/1843799678 conn(0x7f77080fd8e0 msgr2=0x7f77080ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:49.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 -- 172.21.15.122:0/1843799678 shutdown_connections 2024-09-12T01:54:49.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 -- 172.21.15.122:0/1843799678 wait complete. 2024-09-12T01:54:49.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 Processor -- start 2024-09-12T01:54:49.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.373+0000 7f77106b3640 1 -- start start 2024-09-12T01:54:49.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f77106b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:49.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f77106b3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:49.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f77106b3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f7708074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:49.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f77106b3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111f70 con 0x7f7708101de0 2024-09-12T01:54:49.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f77106b3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111df0 con 0x7f7708106f10 2024-09-12T01:54:49.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f77106b3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77081120f0 con 0x7f7708102790 2024-09-12T01:54:49.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:49.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43398/0 (socket says 172.21.15.122:43398) 2024-09-12T01:54:49.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 -- 172.21.15.122:0/1713688957 learned_addr learned my addr 172.21.15.122:0/1713688957 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:49.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770ec50640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f7708074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:49.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770dc4e640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:49.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 -- 172.21.15.122:0/1713688957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 msgr2=0x7f7708074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:49.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 -- 172.21.15.122:0/1713688957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 msgr2=0x7f7708074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:49.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f7708074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770e44f640 1 -- 172.21.15.122:0/1713688957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77081a6970 con 0x7f7708101de0 2024-09-12T01:54:49.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.374+0000 7f770ec50640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f7708074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:49.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.375+0000 7f770e44f640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77080744f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f770400e9f0 tx=0x7f770400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:49.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.375+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400cd80 con 0x7f7708101de0 2024-09-12T01:54:49.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.375+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704010910 con 0x7f7708101de0 2024-09-12T01:54:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.375+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704018990 con 0x7f7708101de0 2024-09-12T01:54:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.375+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77081a6c60 con 0x7f7708101de0 2024-09-12T01:54:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.375+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081a71c0 con 0x7f7708101de0 2024-09-12T01:54:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.376+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77081021c0 con 0x7f7708101de0 2024-09-12T01:54:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.377+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7704010ac0 con 0x7f7708101de0 2024-09-12T01:54:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.378+0000 7f76ff7fe640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c00 0x7f76f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:49.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.378+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f77040cf2b0 con 0x7f7708101de0 2024-09-12T01:54:49.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.378+0000 7f770dc4e640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c00 0x7f76f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:49.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.379+0000 7f770dc4e640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c00 0x7f76f40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009230 tx=0x7f76f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:49.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.382+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f770409bbc0 con 0x7f7708101de0 2024-09-12T01:54:49.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.608+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7708075a20 con 0x7f7708101de0 2024-09-12T01:54:49.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.609+0000 7f76ff7fe640 1 -- 172.21.15.122:0/1713688957 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f770409fec0 con 0x7f7708101de0 2024-09-12T01:54:49.610 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:54:49.610 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:54:49.610 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:54:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c00 msgr2=0x7f76f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c00 0x7f76f40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009230 tx=0x7f76f80058d0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 msgr2=0x7f77080744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77080744f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f770400e9f0 tx=0x7f770400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 shutdown_connections 2024-09-12T01:54:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f76f40a2c00 0x7f76f40a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708106f10 0x7f7708074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7708102790 0x7f7708074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 --2- 172.21.15.122:0/1713688957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7708101de0 0x7f77080744f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 >> 172.21.15.122:0/1713688957 conn(0x7f77080fd8e0 msgr2=0x7f77080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 shutdown_connections 2024-09-12T01:54:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:49.612+0000 7f77106b3640 1 -- 172.21.15.122:0/1713688957 wait complete. 2024-09-12T01:54:49.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.46 MB/sec execute 307 sec latency 18081.667 ms 2024-09-12T01:54:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:49 smithi183 ceph-mon[35127]: pgmap v585: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 71 op/s 2024-09-12T01:54:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:54:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:49 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1713688957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:49 smithi122 ceph-mon[30925]: pgmap v585: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 71 op/s 2024-09-12T01:54:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:54:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:49 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1713688957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:49 smithi151 ceph-mon[35229]: pgmap v585: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 71 op/s 2024-09-12T01:54:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:54:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:54:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:54:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:54:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:49 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1713688957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:54:50.253 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id dd7f595c --release --await' 2024-09-12T01:54:50.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id dd7f595c --release --await 2024-09-12T01:54:50.508 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:54:50.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 717106 9.43 MB/sec execute 308 sec latency 19082.005 ms 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 -- 172.21.15.122:0/2749350695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 msgr2=0x7f69f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 --2- 172.21.15.122:0/2749350695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0100ee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009930 tx=0x7f69ec02a420 comp rx=0 tx=0).stop 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 -- 172.21.15.122:0/2749350695 shutdown_connections 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 --2- 172.21.15.122:0/2749350695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f0101420 0x7f69f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 --2- 172.21.15.122:0/2749350695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0100ee0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 --2- 172.21.15.122:0/2749350695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0106a80 0x7f69f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 -- 172.21.15.122:0/2749350695 >> 172.21.15.122:0/2749350695 conn(0x7f69f00fc7a0 msgr2=0x7f69f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 -- 172.21.15.122:0/2749350695 shutdown_connections 2024-09-12T01:54:50.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.942+0000 7f69f8152640 1 -- 172.21.15.122:0/2749350695 wait complete. 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 Processor -- start 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 -- start start 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 0x7f69f0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f0106a80 0x7f69f019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0110f10 con 0x7f69f0100a80 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0110d90 con 0x7f69f0106a80 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.943+0000 7f69f8152640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0111090 con 0x7f69f0101420 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 0x7f69f0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 0x7f69f0197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53334/0 (socket says 172.21.15.122:53334) 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 -- 172.21.15.122:0/2457019842 learned_addr learned my addr 172.21.15.122:0/2457019842 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f5eee640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f66ef640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f0106a80 0x7f69f019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 -- 172.21.15.122:0/2457019842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f0106a80 msgr2=0x7f69f019bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f0106a80 0x7f69f019bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 -- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 msgr2=0x7f69f0197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 -- 172.21.15.122:0/2457019842 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f019c3c0 con 0x7f69f0101420 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f5eee640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f56ed640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 0x7f69f0197950 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69ec02a940 tx=0x7f69ec003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69e2ffd640 1 -- 172.21.15.122:0/2457019842 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec03f070 con 0x7f69f0101420 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f019c650 con 0x7f69f0101420 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69e2ffd640 1 -- 172.21.15.122:0/2457019842 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ec03b460 con 0x7f69f0101420 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.944+0000 7f69e2ffd640 1 -- 172.21.15.122:0/2457019842 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec035030 con 0x7f69f0101420 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.945+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f01a3f90 con 0x7f69f0101420 2024-09-12T01:54:50.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.947+0000 7f69e2ffd640 1 -- 172.21.15.122:0/2457019842 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f69ec042070 con 0x7f69f0101420 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.947+0000 7f69fa52b740 1 Processor -- start 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.947+0000 7f69fa52b740 1 -- start start 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.947+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a370150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.947+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 0x55b69a370af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a37d300 0x55b69a37f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69e2ffd640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69dc0a2b80 0x7f69dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69fa52b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b69a2d3620 con 0x55b69a370690 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69fa52b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b69a27ae50 con 0x55b69a3718c0 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69fa52b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b69a2cf5e0 con 0x55b69a37d300 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 0x55b69a370af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 0x55b69a370af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44416/0 (socket says 172.21.15.122:44416) 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 -- 172.21.15.122:0/2773154325 learned_addr learned my addr 172.21.15.122:0/2773154325 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:50.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f66ef640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69dc0a2b80 0x7f69dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69e2ffd640 1 -- 172.21.15.122:0/2457019842 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f69ec0ef840 con 0x7f69f0101420 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f56ed640 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a37d300 0x55b69a37f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f66ef640 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a370150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 -- 172.21.15.122:0/2773154325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a37d300 msgr2=0x55b69a37f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a37d300 0x55b69a37f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 -- 172.21.15.122:0/2773154325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 msgr2=0x55b69a370150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a370150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 -- 172.21.15.122:0/2773154325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b69a26bd40 con 0x55b69a370690 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f66ef640 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a370150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f5eee640 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 0x55b69a370af0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f69e400afd0 tx=0x7f69e400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be74cf04d3b5b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69f66ef640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69dc0a2b80 0x7f69dc0a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009440 tx=0x7f69e8009390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69dbfff640 1 -- 172.21.15.122:0/2773154325 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e401d070 con 0x55b69a370690 2024-09-12T01:54:50.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69dbfff640 1 -- 172.21.15.122:0/2773154325 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e4010de0 con 0x55b69a370690 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.948+0000 7f69dbfff640 1 -- 172.21.15.122:0/2773154325 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4016340 con 0x55b69a370690 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 -- 172.21.15.122:0/2773154325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 msgr2=0x55b69a370af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 0x55b69a370af0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f69e400afd0 tx=0x7f69e400f850 comp rx=0 tx=0).stop 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 -- 172.21.15.122:0/2773154325 shutdown_connections 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a37d300 0x55b69a37f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a370690 0x55b69a370af0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 --2- 172.21.15.122:0/2773154325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a370150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 -- 172.21.15.122:0/2773154325 >> 172.21.15.122:0/2773154325 conn(0x55b69a36c800 msgr2=0x55b69a36cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 -- 172.21.15.122:0/2773154325 shutdown_connections 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.949+0000 7f69fa52b740 1 -- 172.21.15.122:0/2773154325 wait complete. 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 -- start start 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 Processor -- start 2024-09-12T01:54:50.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a37a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a37a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a37ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b69a380210 con 0x55b69a37d300 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b69a25bf10 con 0x55b69a3718c0 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b69a380390 con 0x55b69a370690 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69fa52b740 10 client.0 _ll_register_callbacks cb 0x55b69a27fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69f66ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a37ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69f66ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a37ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44426/0 (socket says 172.21.15.122:44426) 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69f66ef640 1 -- 172.21.15.122:0/1512994488 learned_addr learned my addr 172.21.15.122:0/1512994488 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.950+0000 7f69f56ed640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a37a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a37a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f66ef640 1 -- 172.21.15.122:0/1512994488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 msgr2=0x55b69a37a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f66ef640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a37a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f66ef640 1 -- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 msgr2=0x55b69a37a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f66ef640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a37a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f66ef640 1 -- 172.21.15.122:0/1512994488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b699fe2580 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69f66ef640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a37ad30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f69e80196b0 tx=0x7f69e8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3929638cb3e8abb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69d97fa640 1 -- 172.21.15.122:0/1512994488 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8029060 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e4010de0 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69d97fa640 1 -- 172.21.15.122:0/1512994488 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f69e8018570 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69d97fa640 1 -- 172.21.15.122:0/1512994488 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69e800be40 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69fa52b740 10 client.16836 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b69a0ac960 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69d97fa640 1 -- 172.21.15.122:0/1512994488 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e4010de0 con 0x55b69a37d300 2024-09-12T01:54:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69d97fa640 1 -- 172.21.15.122:0/1512994488 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f69e802e070 con 0x55b69a37d300 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.951+0000 7f69fa52b740 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69fa52b740 10 client.16836 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69fa52b740 20 client.16836 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a38d300 0x55b69a38f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69fa52b740 4 client.16836 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b69a2e0b90 con 0x55b69a38d300 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a38d300 0x55b69a38f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69d97fa640 10 client.16836 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.952+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a38d300 0x55b69a38f770 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc182dd33e7347c4 server_cookie=bbd38aaf8b20b315 in_seq=0 out_seq=0 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69d97fa640 1 -- 172.21.15.122:0/1512994488 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b69a2e0b90 con 0x55b69a38d300 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69d97fa640 10 client.16836 handle_command_reply: tid=0 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 client.16836 shutdown 2024-09-12T01:54:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 msgr2=0x55b69a37ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a37ad30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f69e80196b0 tx=0x7f69e8009650 comp rx=0 tx=0).stop 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 shutdown_connections 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a38d300 0x55b69a38f770 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a37ad30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a37a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1512994488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a37a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69d97fa640 0 client.16836 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 >> 172.21.15.122:0/1512994488 conn(0x55b69a36c800 msgr2=0x55b69a372560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 shutdown_connections 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 1 -- 172.21.15.122:0/1512994488 wait complete. 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.953+0000 7f69fa52b740 20 client.16836 trim_cache size 0 max 16384 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 Processor -- start 2024-09-12T01:54:50.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 -- start start 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a370690 0x55b69a37c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a3718c0 0x55b69a395040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a395580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01adcb0 con 0x55b69a37d300 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01ab580 con 0x55b69a370690 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69fa52b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01a93b0 con 0x55b69a3718c0 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f66ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a395580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f66ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a395580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44434/0 (socket says 172.21.15.122:44434) 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f66ef640 1 -- 172.21.15.122:0/537879171 learned_addr learned my addr 172.21.15.122:0/537879171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f66ef640 1 -- 172.21.15.122:0/537879171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a3718c0 msgr2=0x55b69a395040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f66ef640 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a3718c0 0x55b69a395040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f66ef640 1 -- 172.21.15.122:0/537879171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a370690 msgr2=0x55b69a37c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.970+0000 7f69f5eee640 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a370690 0x55b69a37c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69f66ef640 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a370690 0x55b69a37c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69f66ef640 1 -- 172.21.15.122:0/537879171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b69a38fdf0 con 0x55b69a37d300 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69f5eee640 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a370690 0x55b69a37c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69f66ef640 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a395580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009ab0 tx=0x7f69e8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f4d05b2d631e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69db7fe640 1 -- 172.21.15.122:0/537879171 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8034070 con 0x55b69a37d300 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69db7fe640 1 -- 172.21.15.122:0/537879171 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e800b590 con 0x55b69a37d300 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69db7fe640 1 -- 172.21.15.122:0/537879171 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8018cb0 con 0x55b69a37d300 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 -- 172.21.15.122:0/537879171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 msgr2=0x55b69a395580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a395580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009ab0 tx=0x7f69e8004b50 comp rx=0 tx=0).stop 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 -- 172.21.15.122:0/537879171 shutdown_connections 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a395580 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a3718c0 0x55b69a395040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 --2- 172.21.15.122:0/537879171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a370690 0x55b69a37c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.971+0000 7f69fa52b740 1 -- 172.21.15.122:0/537879171 >> 172.21.15.122:0/537879171 conn(0x55b69a36c800 msgr2=0x55b69a37bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 -- 172.21.15.122:0/537879171 shutdown_connections 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 -- 172.21.15.122:0/537879171 wait complete. 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 -- start start 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 Processor -- start 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a3f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a3f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a382d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01a9a40 con 0x55b69a37d300 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01aaeb0 con 0x55b69a3718c0 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01aa110 con 0x55b69a370690 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69fa52b740 10 client.0 _ll_register_callbacks cb 0x7f69e8008dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69f66ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a382d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69f66ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a382d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44436/0 (socket says 172.21.15.122:44436) 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.972+0000 7f69f66ef640 1 -- 172.21.15.122:0/1561173076 learned_addr learned my addr 172.21.15.122:0/1561173076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a3f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f56ed640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a3f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f66ef640 1 -- 172.21.15.122:0/1561173076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 msgr2=0x55b69a3f13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f66ef640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a3f13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f66ef640 1 -- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 msgr2=0x55b69a3f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f66ef640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a3f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f66ef640 1 -- 172.21.15.122:0/1561173076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b69a383530 con 0x55b69a37d300 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a3f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69f66ef640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a382d00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f69e8018850 tx=0x7f69e802f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608c2882213fe6db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69d97fa640 1 -- 172.21.15.122:0/1561173076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8034040 con 0x55b69a37d300 2024-09-12T01:54:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69d97fa640 1 -- 172.21.15.122:0/1561173076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f69e8009650 con 0x55b69a37d300 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e800b590 con 0x55b69a37d300 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69d97fa640 1 -- 172.21.15.122:0/1561173076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69e8029040 con 0x55b69a37d300 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69fa52b740 10 client.16842 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.973+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b69a3f9b70 con 0x55b69a37d300 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69d97fa640 1 -- 172.21.15.122:0/1561173076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e800b590 con 0x55b69a37d300 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69d97fa640 1 -- 172.21.15.122:0/1561173076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f69e802e040 con 0x55b69a37d300 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69fa52b740 10 client.16842 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69fa52b740 10 client.16842 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69fa52b740 20 client.16842 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a3fa230 0x55b69a3fc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69fa52b740 4 client.16842 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dd7f595c", "release": true, "await": true}] 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dd7f595c", "release": true, "await": true}) -- 0x55b69a2db0a0 con 0x55b69a3fa230 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.974+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a3fa230 0x55b69a3fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.975+0000 7f69d97fa640 10 client.16842 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:50.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.975+0000 7f69f5eee640 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a3fa230 0x55b69a3fc650 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2783098579f7df6 server_cookie=1bbdda412b90f98b in_seq=0 out_seq=0 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69d97fa640 1 -- 172.21.15.122:0/1561173076 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b69a2db0a0 con 0x55b69a3fa230 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69d97fa640 10 client.16842 handle_command_reply: tid=0 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 client.16842 shutdown 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 msgr2=0x55b69a382d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a382d00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f69e8018850 tx=0x7f69e802f3e0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 shutdown_connections 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b69a3fa230 0x55b69a3fc650 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b69a37d300 0x55b69a382d00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b69a3718c0 0x55b69a3f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69d97fa640 0 client.16842 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 --2- 172.21.15.122:0/1561173076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b69a370690 0x55b69a3f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 >> 172.21.15.122:0/1561173076 conn(0x55b69a36c800 msgr2=0x55b69a37c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:50.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 shutdown_connections 2024-09-12T01:54:50.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 1 -- 172.21.15.122:0/1561173076 wait complete. 2024-09-12T01:54:50.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.984+0000 7f69fa52b740 20 client.16842 trim_cache size 0 max 16384 2024-09-12T01:54:50.993 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:54:50.993 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 36, 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "dd7f595c": { 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 36, 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:54:50.994 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 139.4, 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:50.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:50.996 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:54:50.996 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:54:50.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69dc0a2b80 msgr2=0x7f69dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69dc0a2b80 0x7f69dc0a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009440 tx=0x7f69e8009390 comp rx=0 tx=0).stop 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 msgr2=0x7f69f0197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 0x7f69f0197950 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69ec02a940 tx=0x7f69ec003c00 comp rx=0 tx=0).stop 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 shutdown_connections 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f69dc0a2b80 0x7f69dc0a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f0106a80 0x7f69f019bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f0101420 0x7f69f0197950 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 --2- 172.21.15.122:0/2457019842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f0100a80 0x7f69f0197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 >> 172.21.15.122:0/2457019842 conn(0x7f69f00fc7a0 msgr2=0x7f69f00fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 shutdown_connections 2024-09-12T01:54:50.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:54:50.985+0000 7f69f8152640 1 -- 172.21.15.122:0/2457019842 wait complete. 2024-09-12T01:54:51.654 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dd7f595c', total seconds elapsed: 22.4 2024-09-12T01:54:51.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 719228 9.43 MB/sec execute 309 sec latency 19221.060 ms 2024-09-12T01:54:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:51 smithi183 ceph-mon[35127]: pgmap v586: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-09-12T01:54:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:51 smithi122 ceph-mon[30925]: pgmap v586: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-09-12T01:54:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:51 smithi151 ceph-mon[35229]: pgmap v586: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-09-12T01:54:52.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 721836 9.45 MB/sec execute 310 sec latency 10.317 ms 2024-09-12T01:54:53.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 724872 9.48 MB/sec execute 311 sec latency 10.113 ms 2024-09-12T01:54:53.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:53 smithi183 ceph-mon[35127]: pgmap v587: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 837 KiB/s wr, 45 op/s 2024-09-12T01:54:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:53 smithi122 ceph-mon[30925]: pgmap v587: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 837 KiB/s wr, 45 op/s 2024-09-12T01:54:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:53 smithi151 ceph-mon[35229]: pgmap v587: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 837 KiB/s wr, 45 op/s 2024-09-12T01:54:54.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 728037 9.50 MB/sec execute 312 sec latency 14.494 ms 2024-09-12T01:54:55.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 730840 9.51 MB/sec execute 313 sec latency 11.792 ms 2024-09-12T01:54:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:55 smithi183 ceph-mon[35127]: pgmap v588: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 454 KiB/s wr, 36 op/s 2024-09-12T01:54:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:55 smithi122 ceph-mon[30925]: pgmap v588: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 454 KiB/s wr, 36 op/s 2024-09-12T01:54:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:55 smithi151 ceph-mon[35229]: pgmap v588: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 454 KiB/s wr, 36 op/s 2024-09-12T01:54:56.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 733342 9.52 MB/sec execute 314 sec latency 19.686 ms 2024-09-12T01:54:57.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 735921 9.54 MB/sec execute 315 sec latency 15.040 ms 2024-09-12T01:54:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:57 smithi183 ceph-mon[35127]: pgmap v589: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 576 KiB/s wr, 96 op/s 2024-09-12T01:54:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:57 smithi122 ceph-mon[30925]: pgmap v589: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 576 KiB/s wr, 96 op/s 2024-09-12T01:54:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:57 smithi151 ceph-mon[35229]: pgmap v589: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 576 KiB/s wr, 96 op/s 2024-09-12T01:54:58.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 738973 9.57 MB/sec execute 316 sec latency 11.528 ms 2024-09-12T01:54:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 -- 172.21.15.122:0/3901568101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec1025c0 msgr2=0x7fc4ec1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/3901568101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec1025c0 0x7fc4ec1049b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009900 tx=0x7fc4dc02a410 comp rx=0 tx=0).stop 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 -- 172.21.15.122:0/3901568101 shutdown_connections 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/3901568101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4ec107820 0x7fc4ec109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/3901568101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/3901568101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec1025c0 0x7fc4ec1049b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 -- 172.21.15.122:0/3901568101 >> 172.21.15.122:0/3901568101 conn(0x7fc4ec0fc1e0 msgr2=0x7fc4ec0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 -- 172.21.15.122:0/3901568101 shutdown_connections 2024-09-12T01:54:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.600+0000 7fc4f11c0640 1 -- 172.21.15.122:0/3901568101 wait complete. 2024-09-12T01:54:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 Processor -- start 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 -- start start 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec1025c0 0x7fc4ec197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4ec107820 0x7fc4ec19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10d820 con 0x7fc4ec104ef0 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10d6a0 con 0x7fc4ec107820 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.601+0000 7fc4f11c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10d9a0 con 0x7fc4ec1025c0 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44442/0 (socket says 172.21.15.122:44442) 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 -- 172.21.15.122:0/2050213536 learned_addr learned my addr 172.21.15.122:0/2050213536 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec1025c0 0x7fc4ec197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4ec107820 0x7fc4ec19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 -- 172.21.15.122:0/2050213536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec1025c0 msgr2=0x7fc4ec197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec1025c0 0x7fc4ec197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 -- 172.21.15.122:0/2050213536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4ec107820 msgr2=0x7fc4ec19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4ec107820 0x7fc4ec19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 -- 172.21.15.122:0/2050213536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19faa0 con 0x7fc4ec104ef0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1983c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e9f0 tx=0x7fc4e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4d3fff640 1 -- 172.21.15.122:0/2050213536 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000ccc0 con 0x7fc4ec104ef0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec19fd90 con 0x7fc4ec104ef0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4d3fff640 1 -- 172.21.15.122:0/2050213536 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e0005500 con 0x7fc4ec104ef0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4d3fff640 1 -- 172.21.15.122:0/2050213536 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0010690 con 0x7fc4ec104ef0 2024-09-12T01:54:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.602+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec1a0230 con 0x7fc4ec104ef0 2024-09-12T01:54:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4d3fff640 1 -- 172.21.15.122:0/2050213536 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc4e0010830 con 0x7fc4ec104ef0 2024-09-12T01:54:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 Processor -- start 2024-09-12T01:54:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 -- start start 2024-09-12T01:54:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b613da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b618da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b61b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b5772f0 con 0x55ad8b6142e0 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b51f270 con 0x55ad8b619370 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4f402f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b5732b0 con 0x55ad8b6139a0 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b618da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4eb59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b613da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b618da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44452/0 (socket says 172.21.15.122:44452) 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 -- 172.21.15.122:0/943187436 learned_addr learned my addr 172.21.15.122:0/943187436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b61b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 -- 172.21.15.122:0/943187436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 msgr2=0x55ad8b613da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b613da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 -- 172.21.15.122:0/943187436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 msgr2=0x55ad8b61b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b61b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ead9d640 1 -- 172.21.15.122:0/943187436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8b51e340 con 0x55ad8b6142e0 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b613da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.605+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b61b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4d3fff640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4d40a2c30 0x7fc4d40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b618da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02a930 tx=0x7fc4dc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5ebdae6274d966 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4d17fa640 1 -- 172.21.15.122:0/943187436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc040070 con 0x55ad8b6142e0 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4d3fff640 1 -- 172.21.15.122:0/2050213536 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc4e00cf400 con 0x7fc4ec104ef0 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4d40a2c30 0x7fc4d40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4d17fa640 1 -- 172.21.15.122:0/943187436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4dc004030 con 0x55ad8b6142e0 2024-09-12T01:54:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4d17fa640 1 -- 172.21.15.122:0/943187436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc035040 con 0x55ad8b6142e0 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4f402f740 1 -- 172.21.15.122:0/943187436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 msgr2=0x55ad8b618da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4d40a2c30 0x7fc4d40a50f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009fd0 tx=0x7fc4e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.606+0000 7fc4f402f740 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b618da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02a930 tx=0x7fc4dc005de0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- 172.21.15.122:0/943187436 shutdown_connections 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b61b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b618da0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 --2- 172.21.15.122:0/943187436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b613da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- 172.21.15.122:0/943187436 >> 172.21.15.122:0/943187436 conn(0x55ad8b60f010 msgr2=0x55ad8b60f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- 172.21.15.122:0/943187436 shutdown_connections 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- 172.21.15.122:0/943187436 wait complete. 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- start start 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 Processor -- start 2024-09-12T01:54:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 0x55ad8b61db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6142e0 0x55ad8b61e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b619370 0x55ad8b6250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b61c2e0 con 0x55ad8b6139a0 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b500290 con 0x55ad8b6142e0 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b61c460 con 0x55ad8b619370 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4f402f740 10 client.0 _ll_register_callbacks cb 0x55ad8b531140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 0x55ad8b61db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 0x55ad8b61db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44458/0 (socket says 172.21.15.122:44458) 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4ead9d640 1 -- 172.21.15.122:0/4204508065 learned_addr learned my addr 172.21.15.122:0/4204508065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6142e0 0x55ad8b61e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b619370 0x55ad8b6250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4ead9d640 1 -- 172.21.15.122:0/4204508065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b619370 msgr2=0x55ad8b6250e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.607+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b619370 0x55ad8b6250e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ead9d640 1 -- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6142e0 msgr2=0x55ad8b61e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6142e0 0x55ad8b61e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ead9d640 1 -- 172.21.15.122:0/4204508065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8b28a210 con 0x55ad8b6139a0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 0x55ad8b61db60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0314e0 tx=0x7fc4dc032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddd8edae226b2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ceff5640 1 -- 172.21.15.122:0/4204508065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc04c020 con 0x55ad8b6139a0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ceff5640 1 -- 172.21.15.122:0/4204508065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc4dc0316b0 con 0x55ad8b6139a0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4dc004030 con 0x55ad8b6139a0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ceff5640 1 -- 172.21.15.122:0/4204508065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4dc040070 con 0x55ad8b6139a0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad8b356620 con 0x55ad8b6139a0 2024-09-12T01:54:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ceff5640 1 -- 172.21.15.122:0/4204508065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4dc004030 con 0x55ad8b6139a0 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4ceff5640 1 -- 172.21.15.122:0/4204508065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc4dc004780 con 0x55ad8b6139a0 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 20 client.16854 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b6293e0 0x55ad8b62b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 4 client.16854 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.608+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad8b584a20 con 0x55ad8b6293e0 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.609+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b6293e0 0x55ad8b62b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.609+0000 7fc4ceff5640 10 client.16854 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.609+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b6293e0 0x55ad8b62b7d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f9de304af0798c server_cookie=2ad7a5e93c3cdf98 in_seq=0 out_seq=0 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4ceff5640 1 -- 172.21.15.122:0/4204508065 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad8b584a20 con 0x55ad8b6293e0 2024-09-12T01:54:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4ceff5640 10 client.16854 handle_command_reply: tid=0 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 client.16854 shutdown 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 msgr2=0x55ad8b61db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 0x55ad8b61db60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0314e0 tx=0x7fc4dc032f60 comp rx=0 tx=0).stop 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 shutdown_connections 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b6293e0 0x55ad8b62b7d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b619370 0x55ad8b6250e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6142e0 0x55ad8b61e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4ceff5640 0 client.16854 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 --2- 172.21.15.122:0/4204508065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6139a0 0x55ad8b61db60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 >> 172.21.15.122:0/4204508065 conn(0x55ad8b60f010 msgr2=0x55ad8b610820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 shutdown_connections 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 1 -- 172.21.15.122:0/4204508065 wait complete. 2024-09-12T01:54:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.610+0000 7fc4f402f740 20 client.16854 trim_cache size 0 max 16384 2024-09-12T01:54:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 Processor -- start 2024-09-12T01:54:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 -- start start 2024-09-12T01:54:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b60e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b6250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b610400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1ab8d0 con 0x55ad8b6142e0 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1a91a0 con 0x55ad8b619370 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4f402f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1a6fd0 con 0x55ad8b6139a0 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4ea59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b6250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4ea59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b6250a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44472/0 (socket says 172.21.15.122:44472) 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4ea59c640 1 -- 172.21.15.122:0/222306271 learned_addr learned my addr 172.21.15.122:0/222306271 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b610400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4ea59c640 1 -- 172.21.15.122:0/222306271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 msgr2=0x55ad8b60e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.625+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b60e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4ea59c640 1 -- 172.21.15.122:0/222306271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 msgr2=0x55ad8b610400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b610400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4ea59c640 1 -- 172.21.15.122:0/222306271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8b60ee30 con 0x55ad8b6142e0 2024-09-12T01:54:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b610400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b6250a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc4e009b960 tx=0x7fc4e0004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a045b810015a1637 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4d0ff9640 1 -- 172.21.15.122:0/222306271 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00b4e60 con 0x55ad8b6142e0 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4d0ff9640 1 -- 172.21.15.122:0/222306271 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e00b1070 con 0x55ad8b6142e0 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4d0ff9640 1 -- 172.21.15.122:0/222306271 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00a48d0 con 0x55ad8b6142e0 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 -- 172.21.15.122:0/222306271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 msgr2=0x55ad8b6250a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b6250a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc4e009b960 tx=0x7fc4e0004ba0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 -- 172.21.15.122:0/222306271 shutdown_connections 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b619370 0x55ad8b610400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b6142e0 0x55ad8b6250a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 --2- 172.21.15.122:0/222306271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6139a0 0x55ad8b60e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 -- 172.21.15.122:0/222306271 >> 172.21.15.122:0/222306271 conn(0x55ad8b60f010 msgr2=0x55ad8b60d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 -- 172.21.15.122:0/222306271 shutdown_connections 2024-09-12T01:54:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.626+0000 7fc4f402f740 1 -- 172.21.15.122:0/222306271 wait complete. 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 -- start start 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 Processor -- start 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6139a0 0x55ad8b622b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6142e0 0x55ad8b623060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 0x55ad8b624020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1a7660 con 0x55ad8b619370 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1a8ad0 con 0x55ad8b6139a0 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1a7d30 con 0x55ad8b6142e0 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4f402f740 10 client.0 _ll_register_callbacks cb 0x7fc4e4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 0x55ad8b624020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 0x55ad8b624020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44478/0 (socket says 172.21.15.122:44478) 2024-09-12T01:54:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 -- 172.21.15.122:0/2032500646 learned_addr learned my addr 172.21.15.122:0/2032500646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4ea59c640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6142e0 0x55ad8b623060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6139a0 0x55ad8b622b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 -- 172.21.15.122:0/2032500646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6142e0 msgr2=0x55ad8b623060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6142e0 0x55ad8b623060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 -- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6139a0 msgr2=0x55ad8b622b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6139a0 0x55ad8b622b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.627+0000 7fc4eb59e640 1 -- 172.21.15.122:0/2032500646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8b624630 con 0x55ad8b619370 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4eb59e640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 0x55ad8b624020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4011f20 tx=0x7fc4e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26e31fc1771bec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ceff5640 1 -- 172.21.15.122:0/2032500646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400fe60 con 0x55ad8b619370 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ceff5640 1 -- 172.21.15.122:0/2032500646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc4e4003b80 con 0x55ad8b619370 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e00b1070 con 0x55ad8b619370 2024-09-12T01:54:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ceff5640 1 -- 172.21.15.122:0/2032500646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4e4003d80 con 0x55ad8b619370 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 10 client.16860 fetch_fsmap learned FSMap version 30 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad8b6944a0 con 0x55ad8b619370 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ceff5640 1 -- 172.21.15.122:0/2032500646 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e00b1070 con 0x55ad8b619370 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ceff5640 1 -- 172.21.15.122:0/2032500646 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc4e4004650 con 0x55ad8b619370 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 20 client.16860 populate_metadata read hostname 'smithi122' 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b694ed0 0x55ad8b6972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 4 client.16860 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad8b57ed70 con 0x55ad8b694ed0 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.628+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b694ed0 0x55ad8b6972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.629+0000 7fc4ead9d640 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b694ed0 0x55ad8b6972f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fa715d4c6a03f0 server_cookie=2eea11f1d6d8b960 in_seq=0 out_seq=0 2024-09-12T01:54:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.629+0000 7fc4ceff5640 10 client.16860 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4ceff5640 1 -- 172.21.15.122:0/2032500646 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad8b57ed70 con 0x55ad8b694ed0 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4ceff5640 10 client.16860 handle_command_reply: tid=0 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 client.16860 shutdown 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 msgr2=0x55ad8b624020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 0x55ad8b624020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4011f20 tx=0x7fc4e4004a40 comp rx=0 tx=0).stop 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 shutdown_connections 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ad8b694ed0 0x55ad8b6972f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad8b619370 0x55ad8b624020 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4ceff5640 0 client.16860 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad8b6142e0 0x55ad8b623060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 --2- 172.21.15.122:0/2032500646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad8b6139a0 0x55ad8b622b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 >> 172.21.15.122:0/2032500646 conn(0x55ad8b60f010 msgr2=0x55ad8b60e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 shutdown_connections 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 1 -- 172.21.15.122:0/2032500646 wait complete. 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.640+0000 7fc4f402f740 20 client.16860 trim_cache size 0 max 16384 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:54:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:54:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:54:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4d40a2c30 msgr2=0x7fc4d40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4d40a2c30 0x7fc4d40a50f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009fd0 tx=0x7fc4e4009500 comp rx=0 tx=0).stop 2024-09-12T01:54:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 msgr2=0x7fc4ec1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1983c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e9f0 tx=0x7fc4e000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 shutdown_connections 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc4d40a2c30 0x7fc4d40a50f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4ec107820 0x7fc4ec19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4ec104ef0 0x7fc4ec1983c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 --2- 172.21.15.122:0/2050213536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4ec1025c0 0x7fc4ec197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 >> 172.21.15.122:0/2050213536 conn(0x7fc4ec0fc1e0 msgr2=0x7fc4ec102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 shutdown_connections 2024-09-12T01:54:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:54:59.641+0000 7fc4f11c0640 1 -- 172.21.15.122:0/2050213536 wait complete. 2024-09-12T01:54:59.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 741795 9.58 MB/sec execute 317 sec latency 13.985 ms 2024-09-12T01:54:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:54:59 smithi183 ceph-mon[35127]: pgmap v590: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 717 KiB/s wr, 105 op/s 2024-09-12T01:54:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:54:59 smithi122 ceph-mon[30925]: pgmap v590: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 717 KiB/s wr, 105 op/s 2024-09-12T01:55:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:54:59 smithi151 ceph-mon[35229]: pgmap v590: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 717 KiB/s wr, 105 op/s 2024-09-12T01:55:00.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 744567 9.59 MB/sec execute 318 sec latency 10.726 ms 2024-09-12T01:55:01.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:01 smithi183 ceph-mon[35127]: pgmap v591: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-09-12T01:55:01.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:01 smithi122 ceph-mon[30925]: pgmap v591: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-09-12T01:55:01.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:01 smithi151 ceph-mon[35229]: pgmap v591: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-09-12T01:55:01.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 746836 9.60 MB/sec execute 319 sec latency 11.658 ms 2024-09-12T01:55:02.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 749694 9.63 MB/sec execute 320 sec latency 14.308 ms 2024-09-12T01:55:03.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 752826 9.65 MB/sec execute 321 sec latency 13.023 ms 2024-09-12T01:55:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:03 smithi183 ceph-mon[35127]: pgmap v592: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-12T01:55:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:03 smithi122 ceph-mon[30925]: pgmap v592: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-12T01:55:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:03 smithi151 ceph-mon[35229]: pgmap v592: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-12T01:55:04.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 755245 9.65 MB/sec execute 322 sec latency 55.776 ms 2024-09-12T01:55:05.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 757770 9.67 MB/sec execute 323 sec latency 17.586 ms 2024-09-12T01:55:05.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:05 smithi183 ceph-mon[35127]: pgmap v593: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 221 op/s 2024-09-12T01:55:06.034 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:05 smithi122 ceph-mon[30925]: pgmap v593: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 221 op/s 2024-09-12T01:55:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:05 smithi151 ceph-mon[35229]: pgmap v593: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 221 op/s 2024-09-12T01:55:06.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 760577 9.69 MB/sec execute 324 sec latency 13.083 ms 2024-09-12T01:55:07.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 763713 9.71 MB/sec execute 325 sec latency 10.502 ms 2024-09-12T01:55:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:07 smithi183 ceph-mon[35127]: pgmap v594: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-09-12T01:55:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:07 smithi122 ceph-mon[30925]: pgmap v594: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-09-12T01:55:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:07 smithi151 ceph-mon[35229]: pgmap v594: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2024-09-12T01:55:08.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 766770 9.73 MB/sec execute 326 sec latency 10.662 ms 2024-09-12T01:55:09.848 INFO:tasks.workunit.client.0.smithi122.stdout: 1 769375 9.74 MB/sec execute 327 sec latency 62.311 ms 2024-09-12T01:55:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:09 smithi183 ceph-mon[35127]: pgmap v595: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-12T01:55:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:09 smithi122 ceph-mon[30925]: pgmap v595: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-12T01:55:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:09 smithi151 ceph-mon[35229]: pgmap v595: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-12T01:55:10.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 771968 9.76 MB/sec execute 328 sec latency 13.189 ms 2024-09-12T01:55:11.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 774770 9.77 MB/sec execute 329 sec latency 17.025 ms 2024-09-12T01:55:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:11 smithi183 ceph-mon[35127]: pgmap v596: 97 pgs: 97 active+clean; 276 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-09-12T01:55:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:11 smithi122 ceph-mon[30925]: pgmap v596: 97 pgs: 97 active+clean; 276 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-09-12T01:55:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:11 smithi151 ceph-mon[35229]: pgmap v596: 97 pgs: 97 active+clean; 276 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 -- 172.21.15.122:0/1811773019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 msgr2=0x7f02d0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 --2- 172.21.15.122:0/1811773019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d0100e40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009900 tx=0x7f02c802a410 comp rx=0 tx=0).stop 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 -- 172.21.15.122:0/1811773019 shutdown_connections 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 --2- 172.21.15.122:0/1811773019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d01025f0 0x7f02d0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 --2- 172.21.15.122:0/1811773019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02d0101c30 0x7f02d01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 --2- 172.21.15.122:0/1811773019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d0100e40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 -- 172.21.15.122:0/1811773019 >> 172.21.15.122:0/1811773019 conn(0x7f02d00fc1e0 msgr2=0x7f02d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 -- 172.21.15.122:0/1811773019 shutdown_connections 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.734+0000 7f02d70f1640 1 -- 172.21.15.122:0/1811773019 wait complete. 2024-09-12T01:55:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 Processor -- start 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 -- start start 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d0101c30 0x7f02d019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02d01025f0 0x7f02d01a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010c6c0 con 0x7f02d0100a40 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010c540 con 0x7f02d01025f0 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d70f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010c840 con 0x7f02d0101c30 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48636/0 (socket says 172.21.15.122:48636) 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 -- 172.21.15.122:0/498497802 learned_addr learned my addr 172.21.15.122:0/498497802 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02cffff640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d0101c30 0x7f02d019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d568e640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02d01025f0 0x7f02d01a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 -- 172.21.15.122:0/498497802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d0101c30 msgr2=0x7f02d019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d0101c30 0x7f02d019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 -- 172.21.15.122:0/498497802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02d01025f0 msgr2=0x7f02d01a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02d01025f0 0x7f02d01a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 -- 172.21.15.122:0/498497802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d01a1c70 con 0x7f02d0100a40 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02cffff640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d0101c30 0x7f02d019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.735+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d006a410 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f02c8031610 tx=0x7f02c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.736+0000 7f02cdffb640 1 -- 172.21.15.122:0/498497802 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c803f030 con 0x7f02d0100a40 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.736+0000 7f02cdffb640 1 -- 172.21.15.122:0/498497802 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02c8003ad0 con 0x7f02d0100a40 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.736+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d01a1ea0 con 0x7f02d0100a40 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.736+0000 7f02cdffb640 1 -- 172.21.15.122:0/498497802 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8035070 con 0x7f02d0100a40 2024-09-12T01:55:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.736+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d01a2300 con 0x7f02d0100a40 2024-09-12T01:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.738+0000 7f02cdffb640 1 -- 172.21.15.122:0/498497802 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f02c8042030 con 0x7f02d0100a40 2024-09-12T01:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.738+0000 7f02d9e8f740 1 Processor -- start 2024-09-12T01:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.738+0000 7f02d9e8f740 1 -- start start 2024-09-12T01:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fea143a2f0 con 0x55fea14d72e0 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cdffb640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f02bc0a2c60 0x7f02bc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fea13e2270 con 0x55fea14d69a0 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fea14362b0 con 0x55fea14dc370 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d4e8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d568e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d568e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:41248/0 (socket says 172.21.15.122:41248) 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d568e640 1 -- 172.21.15.122:0/3412571707 learned_addr learned my addr 172.21.15.122:0/3412571707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d568e640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f02bc0a2c60 0x7f02bc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cdffb640 1 -- 172.21.15.122:0/498497802 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f02c80ef5f0 con 0x7f02d0100a40 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 -- 172.21.15.122:0/3412571707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 msgr2=0x55fea14d6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 -- 172.21.15.122:0/3412571707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 msgr2=0x55fea14dbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14dbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 -- 172.21.15.122:0/3412571707 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea13e1340 con 0x55fea14dc370 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d568e640 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14dbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02cffff640 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14de830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02d006a180 tx=0x7f02c0011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31090a69ccf1f63d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02d568e640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f02bc0a2c60 0x7f02bc0a5120 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f02b8004650 tx=0x7f02b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.739+0000 7f02c6ffd640 1 -- 172.21.15.122:0/3412571707 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0003c30 con 0x55fea14dc370 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02c6ffd640 1 -- 172.21.15.122:0/3412571707 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02c000fe50 con 0x55fea14dc370 2024-09-12T01:55:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02c6ffd640 1 -- 172.21.15.122:0/3412571707 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c00193e0 con 0x55fea14dc370 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 -- 172.21.15.122:0/3412571707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 msgr2=0x55fea14de830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14de830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02d006a180 tx=0x7f02c0011c70 comp rx=0 tx=0).stop 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 -- 172.21.15.122:0/3412571707 shutdown_connections 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14de830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14dbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/3412571707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 -- 172.21.15.122:0/3412571707 >> 172.21.15.122:0/3412571707 conn(0x55fea14d2010 msgr2=0x55fea14d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 -- 172.21.15.122:0/3412571707 shutdown_connections 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.740+0000 7f02d9e8f740 1 -- 172.21.15.122:0/3412571707 wait complete. 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 -- start start 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 Processor -- start 2024-09-12T01:55:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fea14df340 con 0x55fea14d72e0 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fea13c3290 con 0x55fea14d69a0 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fea14df4c0 con 0x55fea14dc370 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d9e8f740 10 client.0 _ll_register_callbacks cb 0x55fea13f4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d4e8d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48652/0 (socket says 172.21.15.122:48652) 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02cffff640 1 -- 172.21.15.122:0/4111515424 learned_addr learned my addr 172.21.15.122:0/4111515424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.741+0000 7f02d568e640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02cffff640 1 -- 172.21.15.122:0/4111515424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 msgr2=0x55fea14e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02cffff640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02cffff640 1 -- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 msgr2=0x55fea14e0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02cffff640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02cffff640 1 -- 172.21.15.122:0/4111515424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea114d210 con 0x55fea14d72e0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02d568e640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02cffff640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e10a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f02c0005f40 tx=0x7f02c0004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2ddc163ae3f34c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02affff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02affff640 1 -- 172.21.15.122:0/4111515424 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c001f040 con 0x55fea14d72e0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02affff640 1 -- 172.21.15.122:0/4111515424 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f02c000fbf0 con 0x55fea14d72e0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c000fe50 con 0x55fea14d72e0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02affff640 1 -- 172.21.15.122:0/4111515424 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02c0011040 con 0x55fea14d72e0 2024-09-12T01:55:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02d9e8f740 10 client.16872 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fea1219620 con 0x55fea14d72e0 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02affff640 1 -- 172.21.15.122:0/4111515424 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02c000fe50 con 0x55fea14d72e0 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.742+0000 7f02affff640 1 -- 172.21.15.122:0/4111515424 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f02c0018070 con 0x55fea14d72e0 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d9e8f740 10 client.16872 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d9e8f740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d9e8f740 20 client.16872 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea14ec5b0 0x55fea14ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d9e8f740 4 client.16872 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea1447a20 con 0x55fea14ec5b0 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea14ec5b0 0x55fea14ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02affff640 10 client.16872 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.743+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea14ec5b0 0x55fea14ee9a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d856e192fe8edbe4 server_cookie=e0a05a6a7ca2192f in_seq=0 out_seq=0 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02affff640 1 -- 172.21.15.122:0/4111515424 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fea1447a20 con 0x55fea14ec5b0 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02affff640 10 client.16872 handle_command_reply: tid=0 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 client.16872 shutdown 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 msgr2=0x55fea14e10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e10a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f02c0005f40 tx=0x7f02c0004480 comp rx=0 tx=0).stop 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 shutdown_connections 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea14ec5b0 0x55fea14ee9a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea14e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e10a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/4111515424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02affff640 0 client.16872 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 >> 172.21.15.122:0/4111515424 conn(0x55fea14d2010 msgr2=0x55fea14d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 shutdown_connections 2024-09-12T01:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 1 -- 172.21.15.122:0/4111515424 wait complete. 2024-09-12T01:55:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.744+0000 7f02d9e8f740 20 client.16872 trim_cache size 0 max 16384 2024-09-12T01:55:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 Processor -- start 2024-09-12T01:55:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 -- start start 2024-09-12T01:55:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14d72e0 0x55fea14e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 0x55fea14d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01ada50 con 0x55fea14dc370 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01ab320 con 0x55fea14d69a0 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01a9150 con 0x55fea14d72e0 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d568e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 0x55fea14d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d568e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 0x55fea14d3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48654/0 (socket says 172.21.15.122:48654) 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.759+0000 7f02d568e640 1 -- 172.21.15.122:0/1350330737 learned_addr learned my addr 172.21.15.122:0/1350330737 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02cffff640 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14d72e0 0x55fea14e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d568e640 1 -- 172.21.15.122:0/1350330737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14d72e0 msgr2=0x55fea14e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d568e640 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14d72e0 0x55fea14e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d568e640 1 -- 172.21.15.122:0/1350330737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 msgr2=0x55fea14d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d568e640 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d568e640 1 -- 172.21.15.122:0/1350330737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea14d1e30 con 0x55fea14dc370 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02d568e640 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 0x55fea14d3400 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f02b800fe30 tx=0x7f02b8008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef11c3b13276cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02c67fc640 1 -- 172.21.15.122:0/1350330737 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b801ec50 con 0x55fea14dc370 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02c67fc640 1 -- 172.21.15.122:0/1350330737 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b801edf0 con 0x55fea14dc370 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.760+0000 7f02c67fc640 1 -- 172.21.15.122:0/1350330737 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b802a480 con 0x55fea14dc370 2024-09-12T01:55:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 -- 172.21.15.122:0/1350330737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 msgr2=0x55fea14d3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 0x55fea14d3400 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f02b800fe30 tx=0x7f02b8008000 comp rx=0 tx=0).stop 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 -- 172.21.15.122:0/1350330737 shutdown_connections 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14dc370 0x55fea14d3400 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14d72e0 0x55fea14e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/1350330737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 -- 172.21.15.122:0/1350330737 >> 172.21.15.122:0/1350330737 conn(0x55fea14d2010 msgr2=0x55fea14d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 -- 172.21.15.122:0/1350330737 shutdown_connections 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 -- 172.21.15.122:0/1350330737 wait complete. 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 -- start start 2024-09-12T01:55:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.761+0000 7f02d9e8f740 1 Processor -- start 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea15551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01a97e0 con 0x55fea14d72e0 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01aac50 con 0x55fea14d69a0 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01a9eb0 con 0x55fea14dc370 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 10 client.0 _ll_register_callbacks cb 0x7f02c802ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d4e8d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48664/0 (socket says 172.21.15.122:48664) 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 -- 172.21.15.122:0/2291600735 learned_addr learned my addr 172.21.15.122:0/2291600735 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d568e640 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea15551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 -- 172.21.15.122:0/2291600735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 msgr2=0x55fea15551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea15551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 -- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 msgr2=0x55fea14e4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 -- 172.21.15.122:0/2291600735 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea1555720 con 0x55fea14d72e0 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02cffff640 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e5420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f02c00051a0 tx=0x7f02c0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6efff6d11273b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02affff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02affff640 1 -- 172.21.15.122:0/2291600735 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c00056a0 con 0x55fea14d72e0 2024-09-12T01:55:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02affff640 1 -- 172.21.15.122:0/2291600735 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f02c0005840 con 0x55fea14d72e0 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b801edf0 con 0x55fea14d72e0 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.762+0000 7f02affff640 1 -- 172.21.15.122:0/2291600735 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02c000f910 con 0x55fea14d72e0 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 10 client.16878 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fea1556980 con 0x55fea14d72e0 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02affff640 1 -- 172.21.15.122:0/2291600735 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b801edf0 con 0x55fea14d72e0 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02affff640 1 -- 172.21.15.122:0/2291600735 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f02c0041020 con 0x55fea14d72e0 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 10 client.16878 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 20 client.16878 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea1557360 0x55fea1559780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 4 client.16878 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fea1441d70 con 0x55fea1557360 2024-09-12T01:55:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea1557360 0x55fea1559780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02affff640 10 client.16878 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.763+0000 7f02d4e8d640 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea1557360 0x55fea1559780 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435d5069b92086b1 server_cookie=2534ec40d631f59b in_seq=0 out_seq=0 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02affff640 1 -- 172.21.15.122:0/2291600735 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fea1441d70 con 0x55fea1557360 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02affff640 10 client.16878 handle_command_reply: tid=0 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 client.16878 shutdown 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 msgr2=0x55fea14e5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e5420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f02c00051a0 tx=0x7f02c0005bd0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 shutdown_connections 2024-09-12T01:55:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fea1557360 0x55fea1559780 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fea14dc370 0x55fea15551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea14d72e0 0x55fea14e5420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02affff640 0 client.16878 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 --2- 172.21.15.122:0/2291600735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fea14d69a0 0x55fea14e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 >> 172.21.15.122:0/2291600735 conn(0x55fea14d2010 msgr2=0x55fea14d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 shutdown_connections 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 1 -- 172.21.15.122:0/2291600735 wait complete. 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.774+0000 7f02d9e8f740 20 client.16878 trim_cache size 0 max 16384 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:55:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:55:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f02bc0a2c60 msgr2=0x7f02bc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f02bc0a2c60 0x7f02bc0a5120 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f02b8004650 tx=0x7f02b8005f90 comp rx=0 tx=0).stop 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 msgr2=0x7f02d006a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d006a410 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f02c8031610 tx=0x7f02c8004270 comp rx=0 tx=0).stop 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 shutdown_connections 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f02bc0a2c60 0x7f02bc0a5120 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02d01025f0 0x7f02d01a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02d0101c30 0x7f02d019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 --2- 172.21.15.122:0/498497802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d0100a40 0x7f02d006a410 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 >> 172.21.15.122:0/498497802 conn(0x7f02d00fc1e0 msgr2=0x7f02d0104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 shutdown_connections 2024-09-12T01:55:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:12.775+0000 7f02d70f1640 1 -- 172.21.15.122:0/498497802 wait complete. 2024-09-12T01:55:12.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 777912 9.80 MB/sec execute 330 sec latency 12.655 ms 2024-09-12T01:55:13.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 781133 9.82 MB/sec execute 331 sec latency 11.013 ms 2024-09-12T01:55:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:13 smithi183 ceph-mon[35127]: pgmap v597: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-12T01:55:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:13 smithi122 ceph-mon[30925]: pgmap v597: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-12T01:55:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:13 smithi151 ceph-mon[35229]: pgmap v597: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-12T01:55:14.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 783734 9.82 MB/sec execute 332 sec latency 17.529 ms 2024-09-12T01:55:15.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 786211 9.84 MB/sec execute 333 sec latency 13.156 ms 2024-09-12T01:55:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:15 smithi183 ceph-mon[35127]: pgmap v598: 97 pgs: 97 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-09-12T01:55:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:15 smithi122 ceph-mon[30925]: pgmap v598: 97 pgs: 97 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-09-12T01:55:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:15 smithi151 ceph-mon[35229]: pgmap v598: 97 pgs: 97 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-09-12T01:55:16.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 789019 9.85 MB/sec execute 334 sec latency 13.304 ms 2024-09-12T01:55:17.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 792175 9.88 MB/sec execute 335 sec latency 11.073 ms 2024-09-12T01:55:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:17 smithi183 ceph-mon[35127]: pgmap v599: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 289 op/s 2024-09-12T01:55:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:17 smithi122 ceph-mon[30925]: pgmap v599: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 289 op/s 2024-09-12T01:55:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:17 smithi151 ceph-mon[35229]: pgmap v599: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 289 op/s 2024-09-12T01:55:18.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 795322 9.90 MB/sec execute 336 sec latency 15.596 ms 2024-09-12T01:55:19.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 798087 9.91 MB/sec execute 337 sec latency 10.284 ms 2024-09-12T01:55:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:19 smithi183 ceph-mon[35127]: pgmap v600: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-12T01:55:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:19 smithi122 ceph-mon[30925]: pgmap v600: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-12T01:55:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:19 smithi151 ceph-mon[35229]: pgmap v600: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-12T01:55:20.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 800779 9.92 MB/sec execute 338 sec latency 10.607 ms 2024-09-12T01:55:21.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 803449 9.94 MB/sec execute 339 sec latency 15.901 ms 2024-09-12T01:55:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:21 smithi183 ceph-mon[35127]: pgmap v601: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-12T01:55:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:21 smithi122 ceph-mon[30925]: pgmap v601: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-12T01:55:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:21 smithi151 ceph-mon[35229]: pgmap v601: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-12T01:55:22.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 806458 9.96 MB/sec execute 340 sec latency 15.162 ms 2024-09-12T01:55:23.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 809637 9.98 MB/sec execute 341 sec latency 16.751 ms 2024-09-12T01:55:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:23 smithi183 ceph-mon[35127]: pgmap v602: 97 pgs: 97 active+clean; 291 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-09-12T01:55:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:23 smithi122 ceph-mon[30925]: pgmap v602: 97 pgs: 97 active+clean; 291 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-09-12T01:55:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:23 smithi151 ceph-mon[35229]: pgmap v602: 97 pgs: 97 active+clean; 291 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-09-12T01:55:24.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 812396 9.99 MB/sec execute 342 sec latency 77.048 ms 2024-09-12T01:55:25.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 814736 10.00 MB/sec execute 343 sec latency 22.504 ms 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 -- 172.21.15.122:0/2261816595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0100a40 msgr2=0x7f40b0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 --2- 172.21.15.122:0/2261816595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0100a40 0x7f40b0100e40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009980 tx=0x7f40ac02c620 comp rx=0 tx=0).stop 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 -- 172.21.15.122:0/2261816595 shutdown_connections 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 --2- 172.21.15.122:0/2261816595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40b01025f0 0x7f40b0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 --2- 172.21.15.122:0/2261816595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b0101c30 0x7f40b01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 --2- 172.21.15.122:0/2261816595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0100a40 0x7f40b0100e40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 -- 172.21.15.122:0/2261816595 >> 172.21.15.122:0/2261816595 conn(0x7f40b00fc1e0 msgr2=0x7f40b00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.868+0000 7f40b6d98640 1 -- 172.21.15.122:0/2261816595 shutdown_connections 2024-09-12T01:55:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 -- 172.21.15.122:0/2261816595 wait complete. 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 Processor -- start 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 -- start start 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 0x7f40b019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b01025f0 0x7f40b019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40b019f780 0x7f40b01a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010b600 con 0x7f40b0101c30 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010b480 con 0x7f40b019f780 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b6d98640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010b780 con 0x7f40b01025f0 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b4b34640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 0x7f40b019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b4b34640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 0x7f40b019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60776/0 (socket says 172.21.15.122:60776) 2024-09-12T01:55:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b4b34640 1 -- 172.21.15.122:0/3649411027 learned_addr learned my addr 172.21.15.122:0/3649411027 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40b5335640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40b019f780 0x7f40b01a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.869+0000 7f40a7fff640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b01025f0 0x7f40b019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b4b34640 1 -- 172.21.15.122:0/3649411027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b01025f0 msgr2=0x7f40b019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b4b34640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b01025f0 0x7f40b019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b4b34640 1 -- 172.21.15.122:0/3649411027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40b019f780 msgr2=0x7f40b01a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b4b34640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40b019f780 0x7f40b01a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b4b34640 1 -- 172.21.15.122:0/3649411027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b01a21e0 con 0x7f40b0101c30 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b4b34640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 0x7f40b019a1e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009230 tx=0x7f40ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40a5ffb640 1 -- 172.21.15.122:0/3649411027 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac041070 con 0x7f40b0101c30 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b01a2470 con 0x7f40b0101c30 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40a5ffb640 1 -- 172.21.15.122:0/3649411027 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ac033a20 con 0x7f40b0101c30 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40a5ffb640 1 -- 172.21.15.122:0/3649411027 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac037050 con 0x7f40b0101c30 2024-09-12T01:55:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.870+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b01a2960 con 0x7f40b0101c30 2024-09-12T01:55:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.873+0000 7f40a5ffb640 1 -- 172.21.15.122:0/3649411027 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f40ac044070 con 0x7f40b0101c30 2024-09-12T01:55:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.873+0000 7f40b9c0f740 1 Processor -- start 2024-09-12T01:55:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.873+0000 7f40b9c0f740 1 -- start start 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b470322f0 con 0x557b470d4370 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b46fda270 con 0x557b470cf2e0 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b4702e2b0 con 0x557b470ce9a0 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b4b34640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a5ffb640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f40980a2c30 0x7f40980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470d6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60782/0 (socket says 172.21.15.122:60782) 2024-09-12T01:55:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 -- 172.21.15.122:0/1286388362 learned_addr learned my addr 172.21.15.122:0/1286388362 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b5335640 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b5335640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f40980a2c30 0x7f40980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 -- 172.21.15.122:0/1286388362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 msgr2=0x557b470ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 -- 172.21.15.122:0/1286388362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 msgr2=0x557b470d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 -- 172.21.15.122:0/1286388362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b46fd9340 con 0x557b470d4370 2024-09-12T01:55:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a5ffb640 1 -- 172.21.15.122:0/3649411027 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f40ac0ef530 con 0x7f40b0101c30 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40a7fff640 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470d6830 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ca0 tx=0x7f40a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96ea55a389ceda7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.874+0000 7f40b5335640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f40980a2c30 0x7f40980a50f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005c40 tx=0x7f40a8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f4096ffd640 1 -- 172.21.15.122:0/1286388362 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a001e070 con 0x557b470d4370 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f4096ffd640 1 -- 172.21.15.122:0/1286388362 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a0019b10 con 0x557b470d4370 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f4096ffd640 1 -- 172.21.15.122:0/1286388362 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a000eaa0 con 0x557b470d4370 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 -- 172.21.15.122:0/1286388362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 msgr2=0x557b470d6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470d6830 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ca0 tx=0x7f40a0011810 comp rx=0 tx=0).stop 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 -- 172.21.15.122:0/1286388362 shutdown_connections 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470d6830 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/1286388362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 -- 172.21.15.122:0/1286388362 >> 172.21.15.122:0/1286388362 conn(0x557b470ca010 msgr2=0x557b470ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 -- 172.21.15.122:0/1286388362 shutdown_connections 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.875+0000 7f40b9c0f740 1 -- 172.21.15.122:0/1286388362 wait complete. 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 -- start start 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 Processor -- start 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470d8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b470d7340 con 0x557b470d4370 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b46fbb290 con 0x557b470cf2e0 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b470d74c0 con 0x557b470ce9a0 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b9c0f740 10 client.0 _ll_register_callbacks cb 0x557b46fec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b4b34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470d8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b5335640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b4b34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470d8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43626/0 (socket says 172.21.15.122:43626) 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40b4b34640 1 -- 172.21.15.122:0/484773816 learned_addr learned my addr 172.21.15.122:0/484773816 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.876+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b5335640 1 -- 172.21.15.122:0/484773816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 msgr2=0x557b470d8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b5335640 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470d8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b5335640 1 -- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 msgr2=0x557b470d90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b5335640 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b5335640 1 -- 172.21.15.122:0/484773816 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b46d45210 con 0x557b470d4370 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40a7fff640 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b5335640 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470e0120 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009fd0 tx=0x7f40a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703b47ee46f391cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f408ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f408ffff640 1 -- 172.21.15.122:0/484773816 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8025030 con 0x557b470d4370 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f408ffff640 1 -- 172.21.15.122:0/484773816 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f40a80045d0 con 0x557b470d4370 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a0019b10 con 0x557b470d4370 2024-09-12T01:55:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f408ffff640 1 -- 172.21.15.122:0/484773816 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40a8003e40 con 0x557b470d4370 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b9c0f740 10 client.16890 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b46e11620 con 0x557b470d4370 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f408ffff640 1 -- 172.21.15.122:0/484773816 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a0019b10 con 0x557b470d4370 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.877+0000 7f408ffff640 1 -- 172.21.15.122:0/484773816 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f40a801f040 con 0x557b470d4370 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b9c0f740 10 client.16890 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b9c0f740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b9c0f740 20 client.16890 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b470e4890 0x557b470e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b9c0f740 4 client.16890 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:55:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4703fa20 con 0x557b470e4890 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b4b34640 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b470e4890 0x557b470e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f408ffff640 10 client.16890 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.878+0000 7f40b4b34640 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b470e4890 0x557b470e6c80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c159b99ccdee138e server_cookie=a787da99722475e8 in_seq=0 out_seq=0 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f408ffff640 1 -- 172.21.15.122:0/484773816 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b4703fa20 con 0x557b470e4890 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f408ffff640 10 client.16890 handle_command_reply: tid=0 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 client.16890 shutdown 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 msgr2=0x557b470e0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470e0120 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009fd0 tx=0x7f40a8004a40 comp rx=0 tx=0).stop 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 shutdown_connections 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b470e4890 0x557b470e6c80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470d4370 0x557b470e0120 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470cf2e0 0x557b470d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/484773816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470d8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f408ffff640 0 client.16890 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 >> 172.21.15.122:0/484773816 conn(0x557b470ca010 msgr2=0x557b470cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 shutdown_connections 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 1 -- 172.21.15.122:0/484773816 wait complete. 2024-09-12T01:55:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.879+0000 7f40b9c0f740 20 client.16890 trim_cache size 0 max 16384 2024-09-12T01:55:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 Processor -- start 2024-09-12T01:55:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 -- start start 2024-09-12T01:55:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470d4370 0x557b470cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ae030 con 0x557b470cf2e0 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ab900 con 0x557b470d4370 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a9730 con 0x557b470ce9a0 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470e00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60806/0 (socket says 172.21.15.122:60806) 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40a7fff640 1 -- 172.21.15.122:0/3452137025 learned_addr learned my addr 172.21.15.122:0/3452137025 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b5335640 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470d4370 0x557b470cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.894+0000 7f40b4b34640 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40a7fff640 1 -- 172.21.15.122:0/3452137025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 msgr2=0x557b470caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40a7fff640 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40a7fff640 1 -- 172.21.15.122:0/3452137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470d4370 msgr2=0x557b470cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40a7fff640 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470d4370 0x557b470cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40a7fff640 1 -- 172.21.15.122:0/3452137025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b470c9e30 con 0x557b470cf2e0 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40a7fff640 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470e00e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x557b470cae90 tx=0x7f40a000e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93482ff9036cab98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40967fc640 1 -- 172.21.15.122:0/3452137025 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0028030 con 0x557b470cf2e0 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40967fc640 1 -- 172.21.15.122:0/3452137025 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a001f040 con 0x557b470cf2e0 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.895+0000 7f40967fc640 1 -- 172.21.15.122:0/3452137025 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a001e040 con 0x557b470cf2e0 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- 172.21.15.122:0/3452137025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 msgr2=0x557b470e00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470e00e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x557b470cae90 tx=0x7f40a000e540 comp rx=0 tx=0).stop 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- 172.21.15.122:0/3452137025 shutdown_connections 2024-09-12T01:55:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470d4370 0x557b470cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470e00e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/3452137025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470ce9a0 0x557b470caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- 172.21.15.122:0/3452137025 >> 172.21.15.122:0/3452137025 conn(0x557b470ca010 msgr2=0x557b470d8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- 172.21.15.122:0/3452137025 shutdown_connections 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- 172.21.15.122:0/3452137025 wait complete. 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- start start 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 Processor -- start 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470ce9a0 0x557b470dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470d4370 0x557b470deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01a9dc0 con 0x557b470cf2e0 2024-09-12T01:55:25.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ab230 con 0x557b470ce9a0 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01aa490 con 0x557b470d4370 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b9c0f740 10 client.0 _ll_register_callbacks cb 0x7f40a8003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470ddb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60820/0 (socket says 172.21.15.122:60820) 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40a7fff640 1 -- 172.21.15.122:0/983663407 learned_addr learned my addr 172.21.15.122:0/983663407 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.896+0000 7f40b4b34640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470ce9a0 0x557b470dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40b5335640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470d4370 0x557b470deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40a7fff640 1 -- 172.21.15.122:0/983663407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470d4370 msgr2=0x557b470deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40a7fff640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470d4370 0x557b470deb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40a7fff640 1 -- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470ce9a0 msgr2=0x557b470dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40a7fff640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470ce9a0 0x557b470dd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40a7fff640 1 -- 172.21.15.122:0/983663407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b470df120 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40a7fff640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470ddb00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009f50 tx=0x7f40a000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2a07e3e43358e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f408ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f408ffff640 1 -- 172.21.15.122:0/983663407 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0028040 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f408ffff640 1 -- 172.21.15.122:0/983663407 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f40a0019740 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a001f040 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f408ffff640 1 -- 172.21.15.122:0/983663407 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40a001e040 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40b9c0f740 10 client.16896 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b470e0430 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.897+0000 7f408ffff640 1 -- 172.21.15.122:0/983663407 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a001f040 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f408ffff640 1 -- 172.21.15.122:0/983663407 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f40a0003680 con 0x557b470cf2e0 2024-09-12T01:55:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b9c0f740 10 client.16896 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b9c0f740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b9c0f740 20 client.16896 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b4714f980 0x557b47151da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b9c0f740 4 client.16896 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b47039d70 con 0x557b4714f980 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b4b34640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b4714f980 0x557b47151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f40b4b34640 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b4714f980 0x557b47151da0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0596deebd2794d server_cookie=de36ebe6d69ada33 in_seq=0 out_seq=0 2024-09-12T01:55:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.898+0000 7f408ffff640 10 client.16896 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f408ffff640 1 -- 172.21.15.122:0/983663407 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b47039d70 con 0x557b4714f980 2024-09-12T01:55:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f408ffff640 10 client.16896 handle_command_reply: tid=0 2024-09-12T01:55:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 client.16896 shutdown 2024-09-12T01:55:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 msgr2=0x557b470ddb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470ddb00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009f50 tx=0x7f40a000e3b0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 shutdown_connections 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b4714f980 0x557b47151da0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b470d4370 0x557b470deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b470cf2e0 0x557b470ddb00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f408ffff640 0 client.16896 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 --2- 172.21.15.122:0/983663407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b470ce9a0 0x557b470dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 >> 172.21.15.122:0/983663407 conn(0x557b470ca010 msgr2=0x557b470c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 shutdown_connections 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 1 -- 172.21.15.122:0/983663407 wait complete. 2024-09-12T01:55:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.910+0000 7f40b9c0f740 20 client.16896 trim_cache size 0 max 16384 2024-09-12T01:55:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:55:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:55:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:55:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f40980a2c30 msgr2=0x7f40980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f40980a2c30 0x7f40980a50f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005c40 tx=0x7f40a8004ab0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 msgr2=0x7f40b019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 0x7f40b019a1e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009230 tx=0x7f40ac0042f0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 shutdown_connections 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f40980a2c30 0x7f40980a50f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40b019f780 0x7f40b01a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b01025f0 0x7f40b019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 --2- 172.21.15.122:0/3649411027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b0101c30 0x7f40b019a1e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 >> 172.21.15.122:0/3649411027 conn(0x7f40b00fc1e0 msgr2=0x7f40b01050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 shutdown_connections 2024-09-12T01:55:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:25.911+0000 7f40b6d98640 1 -- 172.21.15.122:0/3649411027 wait complete. 2024-09-12T01:55:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:25 smithi183 ceph-mon[35127]: pgmap v603: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T01:55:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:25 smithi122 ceph-mon[30925]: pgmap v603: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T01:55:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:25 smithi151 ceph-mon[35229]: pgmap v603: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T01:55:26.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 817507 10.01 MB/sec execute 344 sec latency 10.657 ms 2024-09-12T01:55:27.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 820619 10.04 MB/sec execute 345 sec latency 8.013 ms 2024-09-12T01:55:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:27 smithi183 ceph-mon[35127]: pgmap v604: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 233 op/s 2024-09-12T01:55:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:27 smithi122 ceph-mon[30925]: pgmap v604: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 233 op/s 2024-09-12T01:55:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:27 smithi151 ceph-mon[35229]: pgmap v604: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 233 op/s 2024-09-12T01:55:28.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 823921 10.06 MB/sec execute 346 sec latency 15.390 ms 2024-09-12T01:55:29.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 826461 10.07 MB/sec execute 347 sec latency 55.380 ms 2024-09-12T01:55:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:29 smithi183 ceph-mon[35127]: pgmap v605: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-09-12T01:55:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:29 smithi122 ceph-mon[30925]: pgmap v605: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-09-12T01:55:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:29 smithi151 ceph-mon[35229]: pgmap v605: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-09-12T01:55:30.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 828715 10.08 MB/sec execute 348 sec latency 30.875 ms 2024-09-12T01:55:31.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 831367 10.09 MB/sec execute 349 sec latency 16.662 ms 2024-09-12T01:55:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:31 smithi183 ceph-mon[35127]: pgmap v606: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.4 MiB/s wr, 293 op/s 2024-09-12T01:55:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:31 smithi122 ceph-mon[30925]: pgmap v606: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.4 MiB/s wr, 293 op/s 2024-09-12T01:55:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:31 smithi151 ceph-mon[35229]: pgmap v606: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.4 MiB/s wr, 293 op/s 2024-09-12T01:55:32.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 834478 10.11 MB/sec execute 350 sec latency 10.630 ms 2024-09-12T01:55:33.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 837287 10.12 MB/sec execute 351 sec latency 11.208 ms 2024-09-12T01:55:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:33 smithi183 ceph-mon[35127]: pgmap v607: 97 pgs: 97 active+clean; 290 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-12T01:55:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:33 smithi122 ceph-mon[30925]: pgmap v607: 97 pgs: 97 active+clean; 290 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-12T01:55:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:33 smithi151 ceph-mon[35229]: pgmap v607: 97 pgs: 97 active+clean; 290 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-09-12T01:55:34.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 839317 10.12 MB/sec execute 352 sec latency 37.980 ms 2024-09-12T01:55:35.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 842109 10.14 MB/sec execute 353 sec latency 31.374 ms 2024-09-12T01:55:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:35 smithi183 ceph-mon[35127]: pgmap v608: 97 pgs: 97 active+clean; 290 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 349 op/s 2024-09-12T01:55:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:35 smithi122 ceph-mon[30925]: pgmap v608: 97 pgs: 97 active+clean; 290 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 349 op/s 2024-09-12T01:55:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:35 smithi151 ceph-mon[35229]: pgmap v608: 97 pgs: 97 active+clean; 290 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 349 op/s 2024-09-12T01:55:36.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 845349 10.16 MB/sec execute 354 sec latency 7.852 ms 2024-09-12T01:55:37.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 848561 10.18 MB/sec execute 355 sec latency 9.966 ms 2024-09-12T01:55:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:37 smithi183 ceph-mon[35127]: pgmap v609: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 315 op/s 2024-09-12T01:55:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:37 smithi122 ceph-mon[30925]: pgmap v609: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 315 op/s 2024-09-12T01:55:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:37 smithi151 ceph-mon[35229]: pgmap v609: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 315 op/s 2024-09-12T01:55:38.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 851322 10.19 MB/sec execute 356 sec latency 11.055 ms 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 -- 172.21.15.122:0/1729653724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8104ef0 msgr2=0x7fabc81072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 --2- 172.21.15.122:0/1729653724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8104ef0 0x7fabc81072e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009900 tx=0x7fabb402a410 comp rx=0 tx=0).stop 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 -- 172.21.15.122:0/1729653724 shutdown_connections 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 --2- 172.21.15.122:0/1729653724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabc8107820 0x7fabc8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 --2- 172.21.15.122:0/1729653724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8104ef0 0x7fabc81072e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 --2- 172.21.15.122:0/1729653724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc81049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 -- 172.21.15.122:0/1729653724 >> 172.21.15.122:0/1729653724 conn(0x7fabc80fc1e0 msgr2=0x7fabc80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 -- 172.21.15.122:0/1729653724 shutdown_connections 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.003+0000 7fabcf19f640 1 -- 172.21.15.122:0/1729653724 wait complete. 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 Processor -- start 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 -- start start 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabc8104ef0 0x7fabc81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 0x7fabc819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabc810d7d0 con 0x7fabc8107820 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabc810d650 con 0x7fabc81025c0 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcf19f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabc810d950 con 0x7fabc8104ef0 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 0x7fabc819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabccf3b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 0x7fabc819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54832/0 (socket says 172.21.15.122:54832) 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabccf3b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc8197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36518/0 (socket says 172.21.15.122:36518) 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 -- 172.21.15.122:0/4075168981 learned_addr learned my addr 172.21.15.122:0/4075168981 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabc7fff640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabc8104ef0 0x7fabc81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 -- 172.21.15.122:0/4075168981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabc8104ef0 msgr2=0x7fabc81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabc8104ef0 0x7fabc81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 -- 172.21.15.122:0/4075168981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 msgr2=0x7fabc8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.004+0000 7fabcd73c640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabcd73c640 1 -- 172.21.15.122:0/4075168981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc819fa80 con 0x7fabc8107820 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabccf3b640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabcd73c640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 0x7fabc819f410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fabc000dac0 tx=0x7fabc000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabc5ffb640 1 -- 172.21.15.122:0/4075168981 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0017070 con 0x7fabc8107820 2024-09-12T01:55:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabc5ffb640 1 -- 172.21.15.122:0/4075168981 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc0003e50 con 0x7fabc8107820 2024-09-12T01:55:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabc5ffb640 1 -- 172.21.15.122:0/4075168981 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc00106c0 con 0x7fabc8107820 2024-09-12T01:55:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc819fd70 con 0x7fabc8107820 2024-09-12T01:55:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.005+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc81a0230 con 0x7fabc8107820 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.007+0000 7fabc5ffb640 1 -- 172.21.15.122:0/4075168981 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fabc0003970 con 0x7fabc8107820 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.007+0000 7fabd2017740 1 Processor -- start 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.007+0000 7fabd2017740 1 -- start start 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabd2017740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c92eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabd2017740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c933da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabd2017740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c934370 0x55f65c936830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabd2017740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f65c8922f0 con 0x55f65c934370 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabd2017740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f65c83a270 con 0x55f65c92f2e0 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabc5ffb640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fabb80a2cb0 0x7fabb80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabd2017740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f65c88e2b0 con 0x55f65c92e9a0 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c92eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabcd73c640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fabb80a2cb0 0x7fabb80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c933da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c933da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36528/0 (socket says 172.21.15.122:36528) 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c92eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53174/0 (socket says 172.21.15.122:53174) 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 -- 172.21.15.122:0/3918726008 learned_addr learned my addr 172.21.15.122:0/3918726008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabc7fff640 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c934370 0x55f65c936830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 -- 172.21.15.122:0/3918726008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 msgr2=0x55f65c92eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c92eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 -- 172.21.15.122:0/3918726008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c934370 msgr2=0x55f65c936830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabc5ffb640 1 -- 172.21.15.122:0/4075168981 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fabc00ce8d0 con 0x7fabc8107820 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c934370 0x55f65c936830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 -- 172.21.15.122:0/3918726008 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65c839340 con 0x55f65c92f2e0 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabcd73c640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fabb80a2cb0 0x7fabb80a5170 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fabc00975f0 tx=0x7fabc009f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabccf3b640 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c933da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0115f0 tx=0x7fabbc011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722ee55e00ef248a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.008+0000 7fabb2ffd640 1 -- 172.21.15.122:0/3918726008 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc01e070 con 0x55f65c92f2e0 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabb2ffd640 1 -- 172.21.15.122:0/3918726008 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc00fb20 con 0x55f65c92f2e0 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabb2ffd640 1 -- 172.21.15.122:0/3918726008 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc019bd0 con 0x55f65c92f2e0 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 -- 172.21.15.122:0/3918726008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 msgr2=0x55f65c933da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c933da0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0115f0 tx=0x7fabbc011ab0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 -- 172.21.15.122:0/3918726008 shutdown_connections 2024-09-12T01:55:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c934370 0x55f65c936830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c933da0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 --2- 172.21.15.122:0/3918726008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c92eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 -- 172.21.15.122:0/3918726008 >> 172.21.15.122:0/3918726008 conn(0x55f65c92a010 msgr2=0x55f65c92a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 -- 172.21.15.122:0/3918726008 shutdown_connections 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.009+0000 7fabd2017740 1 -- 172.21.15.122:0/3918726008 wait complete. 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 -- start start 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 Processor -- start 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c938c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c939160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c934370 0x55f65c940130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f65c937340 con 0x55f65c92f2e0 2024-09-12T01:55:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f65c81b290 con 0x55f65c92e9a0 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f65c9374c0 con 0x55f65c934370 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabd2017740 10 client.0 _ll_register_callbacks cb 0x55f65c84c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c939160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c939160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54842/0 (socket says 172.21.15.122:54842) 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 -- 172.21.15.122:0/671825520 learned_addr learned my addr 172.21.15.122:0/671825520 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabc7fff640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c938c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabc7fff640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c934370 0x55f65c940130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 -- 172.21.15.122:0/671825520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c934370 msgr2=0x55f65c940130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c934370 0x55f65c940130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 -- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 msgr2=0x55f65c938c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c938c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.010+0000 7fabccf3b640 1 -- 172.21.15.122:0/671825520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65c5a5210 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabccf3b640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c939160 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005890 tx=0x7fabbc00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986e4d2801602d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7faba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7faba7fff640 1 -- 172.21.15.122:0/671825520 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc01e040 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7faba7fff640 1 -- 172.21.15.122:0/671825520 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fabbc004040 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabbc00fb20 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7faba7fff640 1 -- 172.21.15.122:0/671825520 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fabbc004240 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 10 client.16908 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f65c671620 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7faba7fff640 1 -- 172.21.15.122:0/671825520 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc00fb20 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7faba7fff640 1 -- 172.21.15.122:0/671825520 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fabbc02a030 con 0x55f65c92f2e0 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 10 client.16908 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 20 client.16908 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c944600 0x55f65c9469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 4 client.16908 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.011+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f65c89fa20 con 0x55f65c944600 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.012+0000 7fabc7fff640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c944600 0x55f65c9469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.012+0000 7faba7fff640 10 client.16908 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.012+0000 7fabc7fff640 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c944600 0x55f65c9469f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2add44b79e2072cf server_cookie=e3443049166f0b5b in_seq=0 out_seq=0 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7faba7fff640 1 -- 172.21.15.122:0/671825520 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f65c89fa20 con 0x55f65c944600 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7faba7fff640 10 client.16908 handle_command_reply: tid=0 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 client.16908 shutdown 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 msgr2=0x55f65c939160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c939160 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005890 tx=0x7fabbc00f8f0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 shutdown_connections 2024-09-12T01:55:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c944600 0x55f65c9469f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c934370 0x55f65c940130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c939160 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7faba7fff640 0 client.16908 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 --2- 172.21.15.122:0/671825520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c938c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 >> 172.21.15.122:0/671825520 conn(0x55f65c92a010 msgr2=0x55f65c92b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 shutdown_connections 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 1 -- 172.21.15.122:0/671825520 wait complete. 2024-09-12T01:55:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.013+0000 7fabd2017740 20 client.16908 trim_cache size 0 max 16384 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.027+0000 7fabd2017740 1 Processor -- start 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.027+0000 7fabd2017740 1 -- start start 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabd2017740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92a010 0x55f65c92b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabd2017740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c9400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabd2017740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c92b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabd2017740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81ab900 con 0x55f65c92f2e0 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabd2017740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81a91d0 con 0x55f65c92e9a0 2024-09-12T01:55:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabd2017740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81a7000 con 0x55f65c92a010 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c92b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c92b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54852/0 (socket says 172.21.15.122:54852) 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 -- 172.21.15.122:0/3155147134 learned_addr learned my addr 172.21.15.122:0/3155147134 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92a010 0x55f65c92b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabccf3b640 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c9400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 -- 172.21.15.122:0/3155147134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92a010 msgr2=0x55f65c92b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92a010 0x55f65c92b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 -- 172.21.15.122:0/3155147134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 msgr2=0x55f65c9400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c9400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 -- 172.21.15.122:0/3155147134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65c937220 con 0x55f65c92f2e0 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabc7fff640 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c92b940 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fabc806a400 tx=0x7fabb402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf99efb1c830693 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabb27fc640 1 -- 172.21.15.122:0/3155147134 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4052040 con 0x55f65c92f2e0 2024-09-12T01:55:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabb27fc640 1 -- 172.21.15.122:0/3155147134 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb4005460 con 0x55f65c92f2e0 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.028+0000 7fabb27fc640 1 -- 172.21.15.122:0/3155147134 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4035040 con 0x55f65c92f2e0 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 -- 172.21.15.122:0/3155147134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 msgr2=0x55f65c92b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c92b940 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fabc806a400 tx=0x7fabb402a410 comp rx=0 tx=0).stop 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 -- 172.21.15.122:0/3155147134 shutdown_connections 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92f2e0 0x55f65c92b940 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92e9a0 0x55f65c9400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 --2- 172.21.15.122:0/3155147134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92a010 0x55f65c92b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 -- 172.21.15.122:0/3155147134 >> 172.21.15.122:0/3155147134 conn(0x55f65c933e90 msgr2=0x55f65c9291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 -- 172.21.15.122:0/3155147134 shutdown_connections 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 -- 172.21.15.122:0/3155147134 wait complete. 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 -- start start 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.029+0000 7fabd2017740 1 Processor -- start 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 0x55f65c93d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c93d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c93e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81a7690 con 0x55f65c92a010 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81a8b00 con 0x55f65c92f2e0 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81a7d60 con 0x55f65c92e9a0 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 10 client.0 _ll_register_callbacks cb 0x7fabbc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 0x55f65c93d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 0x55f65c93d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54854/0 (socket says 172.21.15.122:54854) 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 -- 172.21.15.122:0/2679189935 learned_addr learned my addr 172.21.15.122:0/2679189935 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabccf3b640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c93d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c93e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 -- 172.21.15.122:0/2679189935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 msgr2=0x55f65c93d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c93d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 -- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 msgr2=0x55f65c93e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c93e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 -- 172.21.15.122:0/2679189935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65c93ef90 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabc7fff640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 0x55f65c93d430 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fabb4001eb0 tx=0x7fabb402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95a29ab37669ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7faba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7faba7fff640 1 -- 172.21.15.122:0/2679189935 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4035040 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7faba7fff640 1 -- 172.21.15.122:0/2679189935 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fabb4004bd0 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.030+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb4005460 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7faba7fff640 1 -- 172.21.15.122:0/2679189935 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fabb4004dd0 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 10 client.16914 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f65c940310 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7faba7fff640 1 -- 172.21.15.122:0/2679189935 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb4005460 con 0x55f65c92a010 2024-09-12T01:55:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7faba7fff640 1 -- 172.21.15.122:0/2679189935 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fabb4003b80 con 0x55f65c92a010 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 10 client.16914 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 20 client.16914 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c934370 0x55f65c9b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 4 client.16914 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f65c899d70 con 0x55f65c934370 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.031+0000 7fabccf3b640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c934370 0x55f65c9b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.032+0000 7faba7fff640 10 client.16914 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.032+0000 7fabccf3b640 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c934370 0x55f65c9b1940 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9df6463eaf5f2f0 server_cookie=59187325c3ff1cf9 in_seq=0 out_seq=0 2024-09-12T01:55:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.042+0000 7faba7fff640 1 -- 172.21.15.122:0/2679189935 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f65c899d70 con 0x55f65c934370 2024-09-12T01:55:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.042+0000 7faba7fff640 10 client.16914 handle_command_reply: tid=0 2024-09-12T01:55:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.042+0000 7fabd2017740 1 client.16914 shutdown 2024-09-12T01:55:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 msgr2=0x55f65c93d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 0x55f65c93d430 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fabb4001eb0 tx=0x7fabb402ab10 comp rx=0 tx=0).stop 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 shutdown_connections 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f65c934370 0x55f65c9b1940 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f65c92f2e0 0x55f65c93e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f65c92e9a0 0x55f65c93d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 --2- 172.21.15.122:0/2679189935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f65c92a010 0x55f65c93d430 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7faba7fff640 0 client.16914 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 >> 172.21.15.122:0/2679189935 conn(0x55f65c933e90 msgr2=0x55f65c9291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 shutdown_connections 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 1 -- 172.21.15.122:0/2679189935 wait complete. 2024-09-12T01:55:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabd2017740 20 client.16914 trim_cache size 0 max 16384 2024-09-12T01:55:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:55:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:55:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:55:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:55:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fabb80a2cb0 msgr2=0x7fabb80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabcf19f640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fabb80a2cb0 0x7fabb80a5170 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fabc00975f0 tx=0x7fabc009f040 comp rx=0 tx=0).stop 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 msgr2=0x7fabc819f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.043+0000 7fabcf19f640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 0x7fabc819f410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fabc000dac0 tx=0x7fabc000df80 comp rx=0 tx=0).stop 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 shutdown_connections 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fabb80a2cb0 0x7fabb80a5170 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabc8107820 0x7fabc819f410 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabc8104ef0 0x7fabc81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 --2- 172.21.15.122:0/4075168981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc81025c0 0x7fabc8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 >> 172.21.15.122:0/4075168981 conn(0x7fabc80fc1e0 msgr2=0x7fabc8102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 shutdown_connections 2024-09-12T01:55:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:39.044+0000 7fabcf19f640 1 -- 172.21.15.122:0/4075168981 wait complete. 2024-09-12T01:55:39.849 INFO:tasks.workunit.client.0.smithi122.stdout: 1 853617 10.20 MB/sec execute 357 sec latency 28.428 ms 2024-09-12T01:55:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:39 smithi183 ceph-mon[35127]: pgmap v610: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 334 op/s 2024-09-12T01:55:39.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:39 smithi122 ceph-mon[30925]: pgmap v610: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 334 op/s 2024-09-12T01:55:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:39 smithi151 ceph-mon[35229]: pgmap v610: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 334 op/s 2024-09-12T01:55:40.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 856048 10.21 MB/sec execute 358 sec latency 50.149 ms 2024-09-12T01:55:41.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 858840 10.22 MB/sec execute 359 sec latency 21.564 ms 2024-09-12T01:55:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:41 smithi183 ceph-mon[35127]: pgmap v611: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 267 op/s 2024-09-12T01:55:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:41 smithi122 ceph-mon[30925]: pgmap v611: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 267 op/s 2024-09-12T01:55:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:41 smithi151 ceph-mon[35229]: pgmap v611: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 267 op/s 2024-09-12T01:55:42.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 861632 10.22 MB/sec execute 360 sec latency 10.684 ms 2024-09-12T01:55:43.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 863923 10.23 MB/sec execute 361 sec latency 11.117 ms 2024-09-12T01:55:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:43 smithi183 ceph-mon[35127]: pgmap v612: 97 pgs: 97 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 337 op/s 2024-09-12T01:55:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:43 smithi122 ceph-mon[30925]: pgmap v612: 97 pgs: 97 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 337 op/s 2024-09-12T01:55:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:43 smithi151 ceph-mon[35229]: pgmap v612: 97 pgs: 97 active+clean; 289 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 337 op/s 2024-09-12T01:55:44.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 866829 10.25 MB/sec execute 362 sec latency 11.784 ms 2024-09-12T01:55:45.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 869667 10.26 MB/sec execute 363 sec latency 60.399 ms 2024-09-12T01:55:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:45 smithi183 ceph-mon[35127]: pgmap v613: 97 pgs: 97 active+clean; 292 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-09-12T01:55:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:45 smithi122 ceph-mon[30925]: pgmap v613: 97 pgs: 97 active+clean; 292 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-09-12T01:55:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:45 smithi151 ceph-mon[35229]: pgmap v613: 97 pgs: 97 active+clean; 292 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-09-12T01:55:46.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 872356 10.27 MB/sec execute 364 sec latency 16.924 ms 2024-09-12T01:55:47.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 874856 10.29 MB/sec execute 365 sec latency 16.795 ms 2024-09-12T01:55:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:47 smithi183 ceph-mon[35127]: pgmap v614: 97 pgs: 97 active+clean; 294 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-12T01:55:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:47 smithi122 ceph-mon[30925]: pgmap v614: 97 pgs: 97 active+clean; 294 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-12T01:55:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:47 smithi151 ceph-mon[35229]: pgmap v614: 97 pgs: 97 active+clean; 294 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-12T01:55:48.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 877611 10.30 MB/sec execute 366 sec latency 13.592 ms 2024-09-12T01:55:49.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 880731 10.32 MB/sec execute 367 sec latency 11.980 ms 2024-09-12T01:55:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:49 smithi183 ceph-mon[35127]: pgmap v615: 97 pgs: 97 active+clean; 297 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.2 MiB/s wr, 337 op/s 2024-09-12T01:55:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:49 smithi122 ceph-mon[30925]: pgmap v615: 97 pgs: 97 active+clean; 297 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.2 MiB/s wr, 337 op/s 2024-09-12T01:55:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:49 smithi151 ceph-mon[35229]: pgmap v615: 97 pgs: 97 active+clean; 297 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.2 MiB/s wr, 337 op/s 2024-09-12T01:55:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:55:50.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 883353 10.32 MB/sec execute 368 sec latency 98.444 ms 2024-09-12T01:55:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:55:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:55:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:55:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:55:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:55:51.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 885443 10.33 MB/sec execute 369 sec latency 35.478 ms 2024-09-12T01:55:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:51 smithi183 ceph-mon[35127]: pgmap v616: 97 pgs: 97 active+clean; 296 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-09-12T01:55:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:51 smithi122 ceph-mon[30925]: pgmap v616: 97 pgs: 97 active+clean; 296 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-09-12T01:55:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:51 smithi151 ceph-mon[35229]: pgmap v616: 97 pgs: 97 active+clean; 296 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2024-09-12T01:55:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 -- 172.21.15.122:0/1058301944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 msgr2=0x7f2954108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 --2- 172.21.15.122:0/1058301944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954108eb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2944009900 tx=0x7f294402a410 comp rx=0 tx=0).stop 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 -- 172.21.15.122:0/1058301944 shutdown_connections 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 --2- 172.21.15.122:0/1058301944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954108eb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 --2- 172.21.15.122:0/1058301944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954101c60 0x7f29541020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 --2- 172.21.15.122:0/1058301944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954100a70 0x7f2954100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 -- 172.21.15.122:0/1058301944 >> 172.21.15.122:0/1058301944 conn(0x7f29540fc1f0 msgr2=0x7f29540fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 -- 172.21.15.122:0/1058301944 shutdown_connections 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.136+0000 7f295a0f1640 1 -- 172.21.15.122:0/1058301944 wait complete. 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 Processor -- start 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 -- start start 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954100a70 0x7f295410b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954101c60 0x7f2954109a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954109f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f295410d840 con 0x7f2954102620 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f295410d6c0 con 0x7f2954100a70 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f295a0f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f295410d9c0 con 0x7f2954101c60 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f2953fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f2953fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954109f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47630/0 (socket says 172.21.15.122:47630) 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f29537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954100a70 0x7f295410b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954101c60 0x7f2954109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f2953fff640 1 -- 172.21.15.122:0/3334786181 learned_addr learned my addr 172.21.15.122:0/3334786181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f29537fe640 1 -- 172.21.15.122:0/3334786181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954101c60 msgr2=0x7f2954109a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f29537fe640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954101c60 0x7f2954109a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f29537fe640 1 -- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 msgr2=0x7f2954109f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f29537fe640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954109f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f29537fe640 1 -- 172.21.15.122:0/3334786181 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295410a800 con 0x7f2954100a70 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.137+0000 7f2953fff640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.138+0000 7f29537fe640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954100a70 0x7f295410b3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e9c0 tx=0x7f294c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.138+0000 7f2950ff9640 1 -- 172.21.15.122:0/3334786181 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00ccc0 con 0x7f2954100a70 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.138+0000 7f2950ff9640 1 -- 172.21.15.122:0/3334786181 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0035f0 con 0x7f2954100a70 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.138+0000 7f2950ff9640 1 -- 172.21.15.122:0/3334786181 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00ce60 con 0x7f2954100a70 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.138+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29541a1ee0 con 0x7f2954100a70 2024-09-12T01:55:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.138+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a22f0 con 0x7f2954100a70 2024-09-12T01:55:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.140+0000 7f2950ff9640 1 -- 172.21.15.122:0/3334786181 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f294c0037a0 con 0x7f2954100a70 2024-09-12T01:55:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.140+0000 7f295ce92740 1 Processor -- start 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 -- start start 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4c99a0 0x5600fd4c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5600fd42d2f0 con 0x5600fd4ca2e0 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5600fd3d5270 con 0x5600fd4cf370 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f295ce92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600fd4292b0 con 0x5600fd4c99a0 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f2953fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4c99a0 0x5600fd4c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47640/0 (socket says 172.21.15.122:47640) 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 -- 172.21.15.122:0/2614535309 learned_addr learned my addr 172.21.15.122:0/2614535309 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f2952ffd640 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 -- 172.21.15.122:0/2614535309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4c99a0 msgr2=0x5600fd4c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4c99a0 0x5600fd4c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 -- 172.21.15.122:0/2614535309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 msgr2=0x5600fd4d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 -- 172.21.15.122:0/2614535309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fd3d4340 con 0x5600fd4ca2e0 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f2953fff640 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4c99a0 0x5600fd4c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f2952ffd640 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f2950ff9640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f293c0a2cb0 0x7f293c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f29537fe640 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4ceda0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f294c0d7440 tx=0x7f294c0d7900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd0da800a305fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.141+0000 7f2953fff640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f293c0a2cb0 0x7f293c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f2950ff9640 1 -- 172.21.15.122:0/3334786181 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f294c0cf490 con 0x7f2954100a70 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f2939ffb640 1 -- 172.21.15.122:0/2614535309 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c097f90 con 0x5600fd4ca2e0 2024-09-12T01:55:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f2939ffb640 1 -- 172.21.15.122:0/2614535309 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c09c3f0 con 0x5600fd4ca2e0 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f2939ffb640 1 -- 172.21.15.122:0/2614535309 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c09b650 con 0x5600fd4ca2e0 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f295ce92740 1 -- 172.21.15.122:0/2614535309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 msgr2=0x5600fd4ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f295ce92740 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4ceda0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f294c0d7440 tx=0x7f294c0d7900 comp rx=0 tx=0).stop 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f2953fff640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f293c0a2cb0 0x7f293c0a5170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29440314e0 tx=0x7f29440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f295ce92740 1 -- 172.21.15.122:0/2614535309 shutdown_connections 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.142+0000 7f295ce92740 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4ceda0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 --2- 172.21.15.122:0/2614535309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4c99a0 0x5600fd4c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- 172.21.15.122:0/2614535309 >> 172.21.15.122:0/2614535309 conn(0x5600fd4c5010 msgr2=0x5600fd4c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- 172.21.15.122:0/2614535309 shutdown_connections 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- 172.21.15.122:0/2614535309 wait complete. 2024-09-12T01:55:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- start start 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 Processor -- start 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5600fd4d2340 con 0x5600fd4ca2e0 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5600fd3b6290 con 0x5600fd4c99a0 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600fd4d24c0 con 0x5600fd4cf370 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f295ce92740 10 client.0 _ll_register_callbacks cb 0x5600fd3e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.143+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47642/0 (socket says 172.21.15.122:47642) 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2952ffd640 1 -- 172.21.15.122:0/1786840838 learned_addr learned my addr 172.21.15.122:0/1786840838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2953fff640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 -- 172.21.15.122:0/1786840838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 msgr2=0x5600fd4db170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4db170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 -- 172.21.15.122:0/1786840838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 msgr2=0x5600fd4d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 -- 172.21.15.122:0/1786840838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fd140210 con 0x5600fd4c99a0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2952ffd640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f29537fe640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4d3bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f294c0d7440 tx=0x7f294c00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e84cd28c76405e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2932ffd640 1 -- 172.21.15.122:0/1786840838 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c09b830 con 0x5600fd4c99a0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2932ffd640 1 -- 172.21.15.122:0/1786840838 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f294c0cf0c0 con 0x5600fd4c99a0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c09c3f0 con 0x5600fd4c99a0 2024-09-12T01:55:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f2932ffd640 1 -- 172.21.15.122:0/1786840838 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f294c09bac0 con 0x5600fd4c99a0 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f295ce92740 10 client.26150 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.144+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600fd20c620 con 0x5600fd4c99a0 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f2932ffd640 1 -- 172.21.15.122:0/1786840838 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c09c3f0 con 0x5600fd4c99a0 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f2932ffd640 1 -- 172.21.15.122:0/1786840838 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f294c0a44d0 con 0x5600fd4c99a0 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f295ce92740 10 client.26150 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f295ce92740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f295ce92740 20 client.26150 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f295ce92740 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd4df470 0x5600fd4e1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f295ce92740 4 client.26150 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600fd43aa20 con 0x5600fd4df470 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f2952ffd640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd4df470 0x5600fd4e1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f2932ffd640 10 client.26150 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.145+0000 7f2952ffd640 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd4df470 0x5600fd4e1860 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77c9130c851db29 server_cookie=d2dbaa8e5bc9a178 in_seq=0 out_seq=0 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f2932ffd640 1 -- 172.21.15.122:0/1786840838 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600fd43aa20 con 0x5600fd4df470 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f2932ffd640 10 client.26150 handle_command_reply: tid=0 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 client.26150 shutdown 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 msgr2=0x5600fd4d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4d3bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f294c0d7440 tx=0x7f294c00c7d0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 shutdown_connections 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd4df470 0x5600fd4e1860 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4db170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f2932ffd640 0 client.26150 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 --2- 172.21.15.122:0/1786840838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4d3bf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 >> 172.21.15.122:0/1786840838 conn(0x5600fd4c5010 msgr2=0x5600fd4c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 shutdown_connections 2024-09-12T01:55:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 1 -- 172.21.15.122:0/1786840838 wait complete. 2024-09-12T01:55:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.146+0000 7f295ce92740 20 client.26150 trim_cache size 0 max 16384 2024-09-12T01:55:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 Processor -- start 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 -- start start 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4c6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29541ada30 con 0x5600fd4ca2e0 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29541ab300 con 0x5600fd4c99a0 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f295ce92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29541a9130 con 0x5600fd4cf370 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f29537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4db130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47658/0 (socket says 172.21.15.122:47658) 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f2952ffd640 1 -- 172.21.15.122:0/1758780342 learned_addr learned my addr 172.21.15.122:0/1758780342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.162+0000 7f2953fff640 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29537fe640 1 -- 172.21.15.122:0/1758780342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 msgr2=0x5600fd4c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29537fe640 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29537fe640 1 -- 172.21.15.122:0/1758780342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 msgr2=0x5600fd4db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29537fe640 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4db130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29537fe640 1 -- 172.21.15.122:0/1758780342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fd4d2bc0 con 0x5600fd4c99a0 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f2952ffd640 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29537fe640 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4c48d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c6a0 tx=0x7f294c00c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c5791e041328db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29397fa640 1 -- 172.21.15.122:0/1758780342 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c09b830 con 0x5600fd4c99a0 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29397fa640 1 -- 172.21.15.122:0/1758780342 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0d2ce0 con 0x5600fd4c99a0 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.163+0000 7f29397fa640 1 -- 172.21.15.122:0/1758780342 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c097f90 con 0x5600fd4c99a0 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- 172.21.15.122:0/1758780342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 msgr2=0x5600fd4c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4c48d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c6a0 tx=0x7f294c00c840 comp rx=0 tx=0).stop 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- 172.21.15.122:0/1758780342 shutdown_connections 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4cf370 0x5600fd4c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4ca2e0 0x5600fd4db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- 172.21.15.122:0/1758780342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4c99a0 0x5600fd4c48d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- 172.21.15.122:0/1758780342 >> 172.21.15.122:0/1758780342 conn(0x5600fd4c5010 msgr2=0x5600fd4c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- 172.21.15.122:0/1758780342 shutdown_connections 2024-09-12T01:55:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- 172.21.15.122:0/1758780342 wait complete. 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- start start 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 Processor -- start 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 0x5600fd4d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.164+0000 7f295ce92740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29541a97c0 con 0x5600fd4c99a0 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f295ce92740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29541aac30 con 0x5600fd4cf370 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f295ce92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29541a9e90 con 0x5600fd4ca2e0 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f295ce92740 10 client.0 _ll_register_callbacks cb 0x7f294c09f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 0x5600fd4d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 0x5600fd4d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47664/0 (socket says 172.21.15.122:47664) 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2952ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 -- 172.21.15.122:0/974031588 learned_addr learned my addr 172.21.15.122:0/974031588 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2953fff640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 -- 172.21.15.122:0/974031588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4ca2e0 msgr2=0x5600fd4d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 -- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 msgr2=0x5600fd4d9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 -- 172.21.15.122:0/974031588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fd4da300 con 0x5600fd4c99a0 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2952ffd640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f29537fe640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 0x5600fd4d87f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f294c098150 tx=0x7f294c0d2f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a41f24c562002f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2932ffd640 1 -- 172.21.15.122:0/974031588 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c097f90 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2932ffd640 1 -- 172.21.15.122:0/974031588 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f294c09b140 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c0d2ce0 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f2932ffd640 1 -- 172.21.15.122:0/974031588 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f294c09b830 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f295ce92740 10 client.16932 fetch_fsmap learned FSMap version 30 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.165+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600fd4da300 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f2932ffd640 1 -- 172.21.15.122:0/974031588 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0d2ce0 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f2932ffd640 1 -- 172.21.15.122:0/974031588 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f294c097a40 con 0x5600fd4c99a0 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f295ce92740 10 client.16932 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f295ce92740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:55:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f295ce92740 20 client.16932 populate_metadata read hostname 'smithi122' 2024-09-12T01:55:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f295ce92740 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd54a2d0 0x5600fd54c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:55:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f295ce92740 4 client.16932 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:55:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600fd434d70 con 0x5600fd54a2d0 2024-09-12T01:55:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f2952ffd640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd54a2d0 0x5600fd54c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:55:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f2932ffd640 10 client.16932 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.166+0000 7f2952ffd640 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd54a2d0 0x5600fd54c6f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321c6631048153dd server_cookie=f19283ba4dc64d57 in_seq=0 out_seq=0 2024-09-12T01:55:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f2932ffd640 1 -- 172.21.15.122:0/974031588 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600fd434d70 con 0x5600fd54a2d0 2024-09-12T01:55:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f2932ffd640 10 client.16932 handle_command_reply: tid=0 2024-09-12T01:55:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 client.16932 shutdown 2024-09-12T01:55:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 msgr2=0x5600fd4d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 0x5600fd4d87f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f294c098150 tx=0x7f294c0d2f10 comp rx=0 tx=0).stop 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 shutdown_connections 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5600fd54a2d0 0x5600fd54c6f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600fd4cf370 0x5600fd4d9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f2932ffd640 0 client.16932 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600fd4ca2e0 0x5600fd4d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 --2- 172.21.15.122:0/974031588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600fd4c99a0 0x5600fd4d87f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 >> 172.21.15.122:0/974031588 conn(0x5600fd4c5010 msgr2=0x5600fd4c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 shutdown_connections 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 1 -- 172.21.15.122:0/974031588 wait complete. 2024-09-12T01:55:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.178+0000 7f295ce92740 20 client.16932 trim_cache size 0 max 16384 2024-09-12T01:55:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:55:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:55:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:55:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:55:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f293c0a2cb0 msgr2=0x7f293c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f293c0a2cb0 0x7f293c0a5170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29440314e0 tx=0x7f29440058d0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954100a70 msgr2=0x7f295410b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:55:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954100a70 0x7f295410b3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e9c0 tx=0x7f294c00ee80 comp rx=0 tx=0).stop 2024-09-12T01:55:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 shutdown_connections 2024-09-12T01:55:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f293c0a2cb0 0x7f293c0a5170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102620 0x7f2954109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954101c60 0x7f2954109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 --2- 172.21.15.122:0/3334786181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2954100a70 0x7f295410b3f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:55:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 >> 172.21.15.122:0/3334786181 conn(0x7f29540fc1f0 msgr2=0x7f2954104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:55:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 shutdown_connections 2024-09-12T01:55:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:55:52.179+0000 7f295a0f1640 1 -- 172.21.15.122:0/3334786181 wait complete. 2024-09-12T01:55:52.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 888035 10.34 MB/sec execute 370 sec latency 12.641 ms 2024-09-12T01:55:53.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 891182 10.36 MB/sec execute 371 sec latency 10.121 ms 2024-09-12T01:55:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:53 smithi122 ceph-mon[30925]: pgmap v617: 97 pgs: 97 active+clean; 299 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-12T01:55:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:53 smithi151 ceph-mon[35229]: pgmap v617: 97 pgs: 97 active+clean; 299 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-12T01:55:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:53 smithi183 ceph-mon[35127]: pgmap v617: 97 pgs: 97 active+clean; 299 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-12T01:55:54.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 893827 10.37 MB/sec execute 372 sec latency 13.152 ms 2024-09-12T01:55:55.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 895698 10.37 MB/sec execute 373 sec latency 61.030 ms 2024-09-12T01:55:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:55 smithi122 ceph-mon[30925]: pgmap v618: 97 pgs: 97 active+clean; 299 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 318 op/s 2024-09-12T01:55:56.004 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:55 smithi183 ceph-mon[35127]: pgmap v618: 97 pgs: 97 active+clean; 299 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 318 op/s 2024-09-12T01:55:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:55 smithi151 ceph-mon[35229]: pgmap v618: 97 pgs: 97 active+clean; 299 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 318 op/s 2024-09-12T01:55:56.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 898574 10.38 MB/sec execute 374 sec latency 18.494 ms 2024-09-12T01:55:57.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 901502 10.40 MB/sec execute 375 sec latency 13.471 ms 2024-09-12T01:55:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:57 smithi122 ceph-mon[30925]: pgmap v619: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-09-12T01:55:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:57 smithi151 ceph-mon[35229]: pgmap v619: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-09-12T01:55:58.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:57 smithi183 ceph-mon[35127]: pgmap v619: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-09-12T01:55:58.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 904145 10.40 MB/sec execute 376 sec latency 18.130 ms 2024-09-12T01:55:59.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 906522 10.41 MB/sec execute 377 sec latency 14.286 ms 2024-09-12T01:55:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:55:59 smithi122 ceph-mon[30925]: pgmap v620: 97 pgs: 97 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2024-09-12T01:56:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:55:59 smithi151 ceph-mon[35229]: pgmap v620: 97 pgs: 97 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2024-09-12T01:56:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:55:59 smithi183 ceph-mon[35127]: pgmap v620: 97 pgs: 97 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2024-09-12T01:56:00.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 909107 10.43 MB/sec execute 378 sec latency 106.239 ms 2024-09-12T01:56:01.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 912033 10.43 MB/sec execute 379 sec latency 22.023 ms 2024-09-12T01:56:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:01 smithi122 ceph-mon[30925]: pgmap v621: 97 pgs: 97 active+clean; 304 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-12T01:56:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:01 smithi151 ceph-mon[35229]: pgmap v621: 97 pgs: 97 active+clean; 304 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-12T01:56:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:01 smithi183 ceph-mon[35127]: pgmap v621: 97 pgs: 97 active+clean; 304 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 303 op/s 2024-09-12T01:56:02.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 914861 10.44 MB/sec execute 380 sec latency 15.597 ms 2024-09-12T01:56:03.712 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:56:03.712 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T01:56:03.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:56:03.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 916895 10.47 MB/sec execute 381 sec latency 47.441 ms 2024-09-12T01:56:03.967 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:03 smithi122 ceph-mon[30925]: pgmap v622: 97 pgs: 97 active+clean; 305 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-09-12T01:56:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:03 smithi151 ceph-mon[35229]: pgmap v622: 97 pgs: 97 active+clean; 305 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-09-12T01:56:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:03 smithi183 ceph-mon[35127]: pgmap v622: 97 pgs: 97 active+clean; 305 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 -- 172.21.15.122:0/2646475229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594102ef0 msgr2=0x7fc59410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 --2- 172.21.15.122:0/2646475229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594102ef0 0x7fc59410da40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ca0 tx=0x7fc58c02a950 comp rx=0 tx=0).stop 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 -- 172.21.15.122:0/2646475229 shutdown_connections 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 --2- 172.21.15.122:0/2646475229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594102ef0 0x7fc59410da40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 --2- 172.21.15.122:0/2646475229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594102570 0x7fc5941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 --2- 172.21.15.122:0/2646475229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594100e20 0x7fc594101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 -- 172.21.15.122:0/2646475229 >> 172.21.15.122:0/2646475229 conn(0x7fc5940fc7d0 msgr2=0x7fc5940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 -- 172.21.15.122:0/2646475229 shutdown_connections 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.487+0000 7fc59a91e640 1 -- 172.21.15.122:0/2646475229 wait complete. 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 Processor -- start 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 -- start start 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 0x7fc5941992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594102570 0x7fc594199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594102ef0 0x7fc5941a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110f80 con 0x7fc594100e20 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110e00 con 0x7fc594102570 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc59a91e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc594111100 con 0x7fc594102ef0 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc593fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 0x7fc5941992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc593fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 0x7fc5941992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51786/0 (socket says 172.21.15.122:51786) 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc593fff640 1 -- 172.21.15.122:0/76807325 learned_addr learned my addr 172.21.15.122:0/76807325 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.488+0000 7fc5937fe640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594102570 0x7fc594199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc598ebb640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594102ef0 0x7fc5941a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc593fff640 1 -- 172.21.15.122:0/76807325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594102ef0 msgr2=0x7fc5941a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc593fff640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594102ef0 0x7fc5941a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc593fff640 1 -- 172.21.15.122:0/76807325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594102570 msgr2=0x7fc594199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc593fff640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594102570 0x7fc594199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc593fff640 1 -- 172.21.15.122:0/76807325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5941a4890 con 0x7fc594100e20 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc593fff640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 0x7fc5941992d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc58400e9f0 tx=0x7fc58400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58400cd80 con 0x7fc594100e20 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc584010910 con 0x7fc594100e20 2024-09-12T01:56:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5941a4a60 con 0x7fc594100e20 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584018990 con 0x7fc594100e20 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.489+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5941a4fa0 con 0x7fc594100e20 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.490+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc594101200 con 0x7fc594100e20 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.491+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc584010ac0 con 0x7fc594100e20 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.492+0000 7fc5917fa640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc57c0a2c00 0x7fc57c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.492+0000 7fc5937fe640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc57c0a2c00 0x7fc57c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.492+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc5840cf2b0 con 0x7fc594100e20 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.492+0000 7fc5937fe640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc57c0a2c00 0x7fc57c0a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc59406aac0 tx=0x7fc58800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.496+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc58409bbc0 con 0x7fc594100e20 2024-09-12T01:56:04.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.711+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5941a5290 con 0x7fc594100e20 2024-09-12T01:56:04.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.712+0000 7fc5917fa640 1 -- 172.21.15.122:0/76807325 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fc58409fec0 con 0x7fc594100e20 2024-09-12T01:56:04.713 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:56:04.713 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:56:04.713 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc57c0a2c00 msgr2=0x7fc57c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc57c0a2c00 0x7fc57c0a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc59406aac0 tx=0x7fc58800b410 comp rx=0 tx=0).stop 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 msgr2=0x7fc5941992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 0x7fc5941992d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc58400e9f0 tx=0x7fc58400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 shutdown_connections 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc57c0a2c00 0x7fc57c0a50c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc594102ef0 0x7fc5941a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594102570 0x7fc594199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 --2- 172.21.15.122:0/76807325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc594100e20 0x7fc5941992d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 >> 172.21.15.122:0/76807325 conn(0x7fc5940fc7d0 msgr2=0x7fc594106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 shutdown_connections 2024-09-12T01:56:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:04.715+0000 7fc59a91e640 1 -- 172.21.15.122:0/76807325 wait complete. 2024-09-12T01:56:04.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 920305 10.52 MB/sec execute 382 sec latency 46.981 ms 2024-09-12T01:56:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:04 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/76807325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:04 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/76807325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:04 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/76807325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 -- 172.21.15.122:0/1105919310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 msgr2=0x7ffa1c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 --2- 172.21.15.122:0/1105919310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c108e80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009ca0 tx=0x7ffa1402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 -- 172.21.15.122:0/1105919310 shutdown_connections 2024-09-12T01:56:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 --2- 172.21.15.122:0/1105919310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c108e80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 --2- 172.21.15.122:0/1105919310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa1c101c30 0x7ffa1c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 --2- 172.21.15.122:0/1105919310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1c100a40 0x7ffa1c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 -- 172.21.15.122:0/1105919310 >> 172.21.15.122:0/1105919310 conn(0x7ffa1c0fc1e0 msgr2=0x7ffa1c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 -- 172.21.15.122:0/1105919310 shutdown_connections 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 -- 172.21.15.122:0/1105919310 wait complete. 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.272+0000 7ffa21de4640 1 Processor -- start 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 -- start start 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa1c100a40 0x7ffa1c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c10c6c0 con 0x7ffa1c1025f0 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c10c540 con 0x7ffa1c101c30 2024-09-12T01:56:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c10c840 con 0x7ffa1c100a40 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51804/0 (socket says 172.21.15.122:51804) 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 -- 172.21.15.122:0/3718130949 learned_addr learned my addr 172.21.15.122:0/3718130949 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1affd640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa1c100a40 0x7ffa1c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 -- 172.21.15.122:0/3718130949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa1c100a40 msgr2=0x7ffa1c06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa1c100a40 0x7ffa1c06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 -- 172.21.15.122:0/3718130949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1c101c30 msgr2=0x7ffa1c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 -- 172.21.15.122:0/3718130949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c19fa00 con 0x7ffa1c1025f0 2024-09-12T01:56:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa1bfff640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c19f0f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffa14031ae0 tx=0x7ffa14031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa18ff9640 1 -- 172.21.15.122:0/3718130949 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14003b70 con 0x7ffa1c1025f0 2024-09-12T01:56:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c19fc30 con 0x7ffa1c1025f0 2024-09-12T01:56:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa18ff9640 1 -- 172.21.15.122:0/3718130949 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa14003d10 con 0x7ffa1c1025f0 2024-09-12T01:56:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.273+0000 7ffa18ff9640 1 -- 172.21.15.122:0/3718130949 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1402fc40 con 0x7ffa1c1025f0 2024-09-12T01:56:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.274+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c1a0180 con 0x7ffa1c1025f0 2024-09-12T01:56:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa18ff9640 1 -- 172.21.15.122:0/3718130949 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ffa14004050 con 0x7ffa1c1025f0 2024-09-12T01:56:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 Processor -- start 2024-09-12T01:56:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 -- start start 2024-09-12T01:56:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe07eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe083da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe086830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564bfdfe22f0 con 0x564bfe07e9a0 2024-09-12T01:56:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bfdf8a270 con 0x564bfe084370 2024-09-12T01:56:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa24c59740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bfdfde2b0 con 0x564bfe07f2e0 2024-09-12T01:56:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa18ff9640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa040a2c30 0x7ffa040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe07eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe07eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51816/0 (socket says 172.21.15.122:51816) 2024-09-12T01:56:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa1affd640 1 -- 172.21.15.122:0/994346498 learned_addr learned my addr 172.21.15.122:0/994346498 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe083da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.276+0000 7ffa1affd640 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe086830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1bfff640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa040a2c30 0x7ffa040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa18ff9640 1 -- 172.21.15.122:0/3718130949 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffa140f0270 con 0x7ffa1c1025f0 2024-09-12T01:56:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1affd640 1 -- 172.21.15.122:0/994346498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 msgr2=0x564bfe083da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1affd640 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe083da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1affd640 1 -- 172.21.15.122:0/994346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 msgr2=0x564bfe086830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1affd640 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe086830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1affd640 1 -- 172.21.15.122:0/994346498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfdf89340 con 0x564bfe07e9a0 2024-09-12T01:56:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1affd640 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe07eda0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009710 tx=0x7ffa10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb78b23ef44516 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa1bfff640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa040a2c30 0x7ffa040a50f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffa140bc7d0 tx=0x7ffa1402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa01ffb640 1 -- 172.21.15.122:0/994346498 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10013e50 con 0x564bfe07e9a0 2024-09-12T01:56:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa01ffb640 1 -- 172.21.15.122:0/994346498 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa10013120 con 0x564bfe07e9a0 2024-09-12T01:56:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.277+0000 7ffa01ffb640 1 -- 172.21.15.122:0/994346498 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100035f0 con 0x564bfe07e9a0 2024-09-12T01:56:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 -- 172.21.15.122:0/994346498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 msgr2=0x564bfe07eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe07eda0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009710 tx=0x7ffa10009780 comp rx=0 tx=0).stop 2024-09-12T01:56:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 -- 172.21.15.122:0/994346498 shutdown_connections 2024-09-12T01:56:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe086830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe083da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 --2- 172.21.15.122:0/994346498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe07eda0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 -- 172.21.15.122:0/994346498 >> 172.21.15.122:0/994346498 conn(0x564bfe07a010 msgr2=0x564bfe07a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 -- 172.21.15.122:0/994346498 shutdown_connections 2024-09-12T01:56:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 -- 172.21.15.122:0/994346498 wait complete. 2024-09-12T01:56:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 -- start start 2024-09-12T01:56:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.278+0000 7ffa24c59740 1 Processor -- start 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07e9a0 0x564bfe088b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe0890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 0x564bfe090110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564bfe0872e0 con 0x564bfe084370 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bfdf6b290 con 0x564bfe07e9a0 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bfe087460 con 0x564bfe07f2e0 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 10 client.0 _ll_register_callbacks cb 0x564bfdf9c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 0x564bfe090110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 0x564bfe090110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51818/0 (socket says 172.21.15.122:51818) 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 -- 172.21.15.122:0/3185968778 learned_addr learned my addr 172.21.15.122:0/3185968778 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07e9a0 0x564bfe088b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe0890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 -- 172.21.15.122:0/3185968778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 msgr2=0x564bfe0890d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe0890d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 -- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07e9a0 msgr2=0x564bfe088b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07e9a0 0x564bfe088b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 -- 172.21.15.122:0/3185968778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfdcf5210 con 0x564bfe084370 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa1affd640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 0x564bfe090110 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ffa100096e0 tx=0x7ffa10015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf1622a2f053ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ff9faffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ff9faffd640 1 -- 172.21.15.122:0/3185968778 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10004260 con 0x564bfe084370 2024-09-12T01:56:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ff9faffd640 1 -- 172.21.15.122:0/3185968778 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffa10004400 con 0x564bfe084370 2024-09-12T01:56:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.279+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa10013120 con 0x564bfe084370 2024-09-12T01:56:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ff9faffd640 1 -- 172.21.15.122:0/3185968778 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa10035050 con 0x564bfe084370 2024-09-12T01:56:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 10 client.16950 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564bfddc1620 con 0x564bfe084370 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ff9faffd640 1 -- 172.21.15.122:0/3185968778 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa10013120 con 0x564bfe084370 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ff9faffd640 1 -- 172.21.15.122:0/3185968778 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ffa10003760 con 0x564bfe084370 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 10 client.16950 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 10 client.16950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 20 client.16950 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe094600 0x564bfe096a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 4 client.16950 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bfdfefa20 con 0x564bfe094600 2024-09-12T01:56:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.280+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe094600 0x564bfe096a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.281+0000 7ff9faffd640 10 client.16950 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.281+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe094600 0x564bfe096a20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f884b5c2bf63246c server_cookie=9a7f363b418889b8 in_seq=0 out_seq=0 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.281+0000 7ff9faffd640 1 -- 172.21.15.122:0/3185968778 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bfdfefa20 con 0x564bfe094600 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.281+0000 7ff9faffd640 10 client.16950 handle_command_reply: tid=0 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.281+0000 7ffa24c59740 1 client.16950 shutdown 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 msgr2=0x564bfe090110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 0x564bfe090110 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ffa100096e0 tx=0x7ffa10015040 comp rx=0 tx=0).stop 2024-09-12T01:56:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 shutdown_connections 2024-09-12T01:56:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe094600 0x564bfe096a20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe084370 0x564bfe090110 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07f2e0 0x564bfe0890d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ff9faffd640 0 client.16950 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 --2- 172.21.15.122:0/3185968778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07e9a0 0x564bfe088b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 >> 172.21.15.122:0/3185968778 conn(0x564bfe07a010 msgr2=0x564bfe07b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 shutdown_connections 2024-09-12T01:56:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 1 -- 172.21.15.122:0/3185968778 wait complete. 2024-09-12T01:56:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.282+0000 7ffa24c59740 20 client.16950 trim_cache size 0 max 16384 2024-09-12T01:56:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 Processor -- start 2024-09-12T01:56:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 -- start start 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe087f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07f2e0 0x564bfe0900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe084370 0x564bfe07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1ab840 con 0x564bfe07e9a0 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1a9110 con 0x564bfe07f2e0 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa24c59740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1a6f40 con 0x564bfe084370 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe087f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe087f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51828/0 (socket says 172.21.15.122:51828) 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.297+0000 7ffa1affd640 1 -- 172.21.15.122:0/1486805462 learned_addr learned my addr 172.21.15.122:0/1486805462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07f2e0 0x564bfe0900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe084370 0x564bfe07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 -- 172.21.15.122:0/1486805462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe084370 msgr2=0x564bfe07aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe084370 0x564bfe07aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 -- 172.21.15.122:0/1486805462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07f2e0 msgr2=0x564bfe0900a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07f2e0 0x564bfe0900a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 -- 172.21.15.122:0/1486805462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfe07aca0 con 0x564bfe07e9a0 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe087f70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ffa100059f0 tx=0x7ffa10009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ba665ffe467ace server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa017fa640 1 -- 172.21.15.122:0/1486805462 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10013120 con 0x564bfe07e9a0 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa017fa640 1 -- 172.21.15.122:0/1486805462 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa10003da0 con 0x564bfe07e9a0 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.298+0000 7ffa017fa640 1 -- 172.21.15.122:0/1486805462 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100055f0 con 0x564bfe07e9a0 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- 172.21.15.122:0/1486805462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 msgr2=0x564bfe087f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe087f70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ffa100059f0 tx=0x7ffa10009640 comp rx=0 tx=0).stop 2024-09-12T01:56:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- 172.21.15.122:0/1486805462 shutdown_connections 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe084370 0x564bfe07aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe07f2e0 0x564bfe0900a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1486805462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07e9a0 0x564bfe087f70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- 172.21.15.122:0/1486805462 >> 172.21.15.122:0/1486805462 conn(0x564bfe07a010 msgr2=0x564bfe087860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- 172.21.15.122:0/1486805462 shutdown_connections 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- 172.21.15.122:0/1486805462 wait complete. 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- start start 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 Processor -- start 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07e9a0 0x564bfe08d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 0x564bfe08d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe08e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.299+0000 7ffa24c59740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1a75d0 con 0x564bfe07f2e0 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa24c59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1a8a40 con 0x564bfe084370 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa24c59740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1a7ca0 con 0x564bfe07e9a0 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa24c59740 10 client.0 _ll_register_callbacks cb 0x7ffa10009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 0x564bfe08d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 0x564bfe08d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51844/0 (socket says 172.21.15.122:51844) 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 -- 172.21.15.122:0/1909175823 learned_addr learned my addr 172.21.15.122:0/1909175823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07e9a0 0x564bfe08d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe08e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 -- 172.21.15.122:0/1909175823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07e9a0 msgr2=0x564bfe08d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07e9a0 0x564bfe08d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 -- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 msgr2=0x564bfe08e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe08e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 -- 172.21.15.122:0/1909175823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfe08ef60 con 0x564bfe07f2e0 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa1b7fe640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 0x564bfe08d940 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009d10 tx=0x7ffa0c010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919818626a73d4dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ff9faffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ff9faffd640 1 -- 172.21.15.122:0/1909175823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c01dc40 con 0x564bfe07f2e0 2024-09-12T01:56:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ff9faffd640 1 -- 172.21.15.122:0/1909175823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7ffa0c01dde0 con 0x564bfe07f2e0 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa10003da0 con 0x564bfe07f2e0 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ff9faffd640 1 -- 172.21.15.122:0/1909175823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c010e50 con 0x564bfe07f2e0 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa24c59740 10 client.16956 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.300+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564bfe090340 con 0x564bfe07f2e0 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ff9faffd640 1 -- 172.21.15.122:0/1909175823 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa10003da0 con 0x564bfe07f2e0 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ff9faffd640 1 -- 172.21.15.122:0/1909175823 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ffa0c02f050 con 0x564bfe07f2e0 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa24c59740 10 client.16956 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa24c59740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa24c59740 20 client.16956 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe0ff870 0x564bfe101c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa24c59740 4 client.16956 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:56:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bfdfe9d70 con 0x564bfe0ff870 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe0ff870 0x564bfe101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ff9faffd640 10 client.16956 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.301+0000 7ffa1affd640 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe0ff870 0x564bfe101c90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f368d66882093f85 server_cookie=8a0d6a93456c60a4 in_seq=0 out_seq=0 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ff9faffd640 1 -- 172.21.15.122:0/1909175823 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bfdfe9d70 con 0x564bfe0ff870 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ff9faffd640 10 client.16956 handle_command_reply: tid=0 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 client.16956 shutdown 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 msgr2=0x564bfe08d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 0x564bfe08d940 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009d10 tx=0x7ffa0c010ba0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 shutdown_connections 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564bfe0ff870 0x564bfe101c90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bfe084370 0x564bfe08e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ff9faffd640 0 client.16956 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bfe07f2e0 0x564bfe08d940 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.312+0000 7ffa24c59740 1 --2- 172.21.15.122:0/1909175823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bfe07e9a0 0x564bfe08d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 >> 172.21.15.122:0/1909175823 conn(0x564bfe07a010 msgr2=0x564bfe0789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 shutdown_connections 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa24c59740 1 -- 172.21.15.122:0/1909175823 wait complete. 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa24c59740 20 client.16956 trim_cache size 0 max 16384 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:56:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:56:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa040a2c30 msgr2=0x7ffa040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa040a2c30 0x7ffa040a50f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffa140bc7d0 tx=0x7ffa1402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 msgr2=0x7ffa1c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c19f0f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffa14031ae0 tx=0x7ffa14031ee0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 shutdown_connections 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ffa040a2c30 0x7ffa040a50f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1c1025f0 0x7ffa1c19f0f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1c101c30 0x7ffa1c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 --2- 172.21.15.122:0/3718130949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa1c100a40 0x7ffa1c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.313+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 >> 172.21.15.122:0/3718130949 conn(0x7ffa1c0fc1e0 msgr2=0x7ffa1c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.314+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 shutdown_connections 2024-09-12T01:56:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:05.314+0000 7ffa21de4640 1 -- 172.21.15.122:0/3718130949 wait complete. 2024-09-12T01:56:05.440 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T01:56:05.440 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T01:56:05.721 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:05.850 INFO:tasks.workunit.client.0.smithi122.stdout: 1 923012 10.52 MB/sec execute 383 sec latency 61.347 ms 2024-09-12T01:56:05.976 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:05 smithi122 ceph-mon[30925]: pgmap v623: 97 pgs: 97 active+clean; 309 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-09-12T01:56:06.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:05 smithi151 ceph-mon[35229]: pgmap v623: 97 pgs: 97 active+clean; 309 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-09-12T01:56:06.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:05 smithi183 ceph-mon[35127]: pgmap v623: 97 pgs: 97 active+clean; 309 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 293 op/s 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 -- 172.21.15.122:0/2048602310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 msgr2=0x7eff281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 --2- 172.21.15.122:0/2048602310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 0x7eff281021c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009860 tx=0x7eff1c02a340 comp rx=0 tx=0).stop 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 -- 172.21.15.122:0/2048602310 shutdown_connections 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 --2- 172.21.15.122:0/2048602310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff28106f10 0x7eff2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 --2- 172.21.15.122:0/2048602310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 --2- 172.21.15.122:0/2048602310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 0x7eff281021c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.219+0000 7eff2e73b640 1 -- 172.21.15.122:0/2048602310 >> 172.21.15.122:0/2048602310 conn(0x7eff280fd8c0 msgr2=0x7eff280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.220+0000 7eff2e73b640 1 -- 172.21.15.122:0/2048602310 shutdown_connections 2024-09-12T01:56:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.220+0000 7eff2e73b640 1 -- 172.21.15.122:0/2048602310 wait complete. 2024-09-12T01:56:06.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.221+0000 7eff2e73b640 1 Processor -- start 2024-09-12T01:56:06.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.221+0000 7eff2e73b640 1 -- start start 2024-09-12T01:56:06.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.221+0000 7eff2e73b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 0x7eff281962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff2e73b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff2e73b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff28106f10 0x7eff2819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff2e73b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810fd80 con 0x7eff28102790 2024-09-12T01:56:06.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff2e73b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810fc00 con 0x7eff28101de0 2024-09-12T01:56:06.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff2e73b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff2810ff00 con 0x7eff28106f10 2024-09-12T01:56:06.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff27fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 0x7eff281962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff277fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff277fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51866/0 (socket says 172.21.15.122:51866) 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff277fe640 1 -- 172.21.15.122:0/4216385313 learned_addr learned my addr 172.21.15.122:0/4216385313 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.222+0000 7eff2ccd8640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff28106f10 0x7eff2819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff277fe640 1 -- 172.21.15.122:0/4216385313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff28106f10 msgr2=0x7eff2819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff277fe640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff28106f10 0x7eff2819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff277fe640 1 -- 172.21.15.122:0/4216385313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 msgr2=0x7eff281962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff277fe640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 0x7eff281962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff277fe640 1 -- 172.21.15.122:0/4216385313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2819b250 con 0x7eff28102790 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff277fe640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281967e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7eff2000ed80 tx=0x7eff2000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff257fa640 1 -- 172.21.15.122:0/4216385313 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2001c070 con 0x7eff28102790 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff257fa640 1 -- 172.21.15.122:0/4216385313 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff20017b20 con 0x7eff28102790 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.223+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2819b540 con 0x7eff28102790 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.224+0000 7eff257fa640 1 -- 172.21.15.122:0/4216385313 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2000fd00 con 0x7eff28102790 2024-09-12T01:56:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.224+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281a2e20 con 0x7eff28102790 2024-09-12T01:56:06.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.229+0000 7eff257fa640 1 -- 172.21.15.122:0/4216385313 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7eff2001f070 con 0x7eff28102790 2024-09-12T01:56:06.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.230+0000 7eff257fa640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff100a7160 0x7eff100a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.230+0000 7eff257fa640 1 -- 172.21.15.122:0/4216385313 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7eff200d01c0 con 0x7eff28102790 2024-09-12T01:56:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.231+0000 7eff27fff640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff100a7160 0x7eff100a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.231+0000 7eff30b14740 1 Processor -- start 2024-09-12T01:56:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.231+0000 7eff30b14740 1 -- start start 2024-09-12T01:56:06.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff30b14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 0x55f833a8fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff30b14740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff30b14740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a9ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff30b14740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f8339f4160 con 0x55f833a90170 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff30b14740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f83399b720 con 0x55f833a9aa30 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff30b14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f8339f0120 con 0x55f833a8f850 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff27fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 0x55f833a8fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff27fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 0x55f833a8fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:38098/0 (socket says 172.21.15.122:38098) 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff27fff640 1 -- 172.21.15.122:0/3605255491 learned_addr learned my addr 172.21.15.122:0/3605255491 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:06.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.232+0000 7eff2ccd8640 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a9ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff277fe640 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff100a7160 0x7eff100a9620 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009830 tx=0x7eff1c004040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 -- 172.21.15.122:0/3605255491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 msgr2=0x55f833a9ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a9ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 -- 172.21.15.122:0/3605255491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 msgr2=0x55f833a9a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a9a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 -- 172.21.15.122:0/3605255491 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f833765ba0 con 0x55f833a8f850 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff27fff640 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 0x55f833a8fc30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009860 tx=0x7eff1c046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77b9c082a96f1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.233+0000 7eff0e7fc640 1 -- 172.21.15.122:0/3605255491 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c037030 con 0x55f833a8f850 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.234+0000 7eff0e7fc640 1 -- 172.21.15.122:0/3605255491 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1c005520 con 0x55f833a8f850 2024-09-12T01:56:06.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.234+0000 7eff0e7fc640 1 -- 172.21.15.122:0/3605255491 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0035f0 con 0x55f833a8f850 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 -- 172.21.15.122:0/3605255491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 msgr2=0x55f833a8fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 0x55f833a8fc30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009860 tx=0x7eff1c046000 comp rx=0 tx=0).stop 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 -- 172.21.15.122:0/3605255491 shutdown_connections 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a9ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a9a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 --2- 172.21.15.122:0/3605255491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a8f850 0x55f833a8fc30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.235+0000 7eff30b14740 1 -- 172.21.15.122:0/3605255491 >> 172.21.15.122:0/3605255491 conn(0x55f833a8bde0 msgr2=0x55f833a8c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.236+0000 7eff30b14740 1 -- 172.21.15.122:0/3605255491 shutdown_connections 2024-09-12T01:56:06.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.236+0000 7eff30b14740 1 -- 172.21.15.122:0/3605255491 wait complete. 2024-09-12T01:56:06.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.236+0000 7eff30b14740 1 -- start start 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 Processor -- start 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a8f850 0x55f833a978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833a97e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a98340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f833a9d5a0 con 0x55f833a8f850 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f83397c900 con 0x55f833a9aa30 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f833a9d720 con 0x55f833a90170 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff27fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a8f850 0x55f833a978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff27fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a8f850 0x55f833a978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51890/0 (socket says 172.21.15.122:51890) 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff27fff640 1 -- 172.21.15.122:0/3448110884 learned_addr learned my addr 172.21.15.122:0/3448110884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.237+0000 7eff30b14740 10 client.0 _ll_register_callbacks cb 0x55f83397a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:06.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833a97e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff2ccd8640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a98340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 -- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 msgr2=0x55f833a98340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a98340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 -- 172.21.15.122:0/3448110884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a8f850 msgr2=0x55f833a978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a8f850 0x55f833a978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 -- 172.21.15.122:0/3448110884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8339bcc50 con 0x55f833a90170 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.238+0000 7eff277fe640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833a97e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7eff20098e40 tx=0x7eff2009d1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ed19b75eb2c026 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.239+0000 7eff077fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.239+0000 7eff077fe640 1 -- 172.21.15.122:0/3448110884 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200b5030 con 0x55f833a90170 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.239+0000 7eff077fe640 1 -- 172.21.15.122:0/3448110884 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7eff200a51c0 con 0x55f833a90170 2024-09-12T01:56:06.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.239+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c005520 con 0x55f833a90170 2024-09-12T01:56:06.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.240+0000 7eff077fe640 1 -- 172.21.15.122:0/3448110884 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff200a5650 con 0x55f833a90170 2024-09-12T01:56:06.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.240+0000 7eff30b14740 10 client.26158 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:06.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.240+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f833aa9280 con 0x55f833a90170 2024-09-12T01:56:06.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff077fe640 1 -- 172.21.15.122:0/3448110884 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff200a53c0 con 0x55f833a90170 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff077fe640 1 -- 172.21.15.122:0/3448110884 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7eff2000b030 con 0x55f833a90170 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff30b14740 10 client.26158 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff30b14740 10 client.26158 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff30b14740 20 client.26158 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff30b14740 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833aaa740 0x55f833aacb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff30b14740 4 client.26158 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7eff1c005520 con 0x55f833aaa740 2024-09-12T01:56:06.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.241+0000 7eff27fff640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833aaa740 0x55f833aacb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.242+0000 7eff27fff640 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833aaa740 0x55f833aacb30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512fb9c9e3df6c44 server_cookie=52e2211c2c5041b5 in_seq=0 out_seq=0 2024-09-12T01:56:06.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.242+0000 7eff077fe640 10 client.26158 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:06.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.243+0000 7eff077fe640 1 -- 172.21.15.122:0/3448110884 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7eff1c005520 con 0x55f833aaa740 2024-09-12T01:56:06.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.243+0000 7eff077fe640 10 client.26158 handle_command_reply: tid=0 2024-09-12T01:56:06.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.243+0000 7eff30b14740 1 client.26158 shutdown 2024-09-12T01:56:06.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.243+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 msgr2=0x55f833a97e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.243+0000 7eff30b14740 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833a97e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7eff20098e40 tx=0x7eff2009d1a0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 shutdown_connections 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833aaa740 0x55f833aacb30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a9aa30 0x55f833a98340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833a97e00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 --2- 172.21.15.122:0/3448110884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a8f850 0x55f833a978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 >> 172.21.15.122:0/3448110884 conn(0x55f833a8bde0 msgr2=0x55f833a91b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff077fe640 0 client.26158 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 shutdown_connections 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.244+0000 7eff30b14740 1 -- 172.21.15.122:0/3448110884 wait complete. 2024-09-12T01:56:06.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.245+0000 7eff30b14740 20 client.26158 trim_cache size 0 max 16384 2024-09-12T01:56:06.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.266+0000 7eff30b14740 1 Processor -- start 2024-09-12T01:56:06.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.266+0000 7eff30b14740 1 -- start start 2024-09-12T01:56:06.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.266+0000 7eff30b14740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833a8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.266+0000 7eff30b14740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff30b14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a9aa30 0x55f833a99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff30b14740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff14010d60 con 0x55f833a90170 2024-09-12T01:56:06.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff30b14740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400e630 con 0x55f833a8f850 2024-09-12T01:56:06.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff30b14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400c460 con 0x55f833a9aa30 2024-09-12T01:56:06.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff27fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833a8b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a922b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51894/0 (socket says 172.21.15.122:51894) 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff27fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833a8b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:51600/0 (socket says 172.21.15.122:51600) 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 -- 172.21.15.122:0/337932962 learned_addr learned my addr 172.21.15.122:0/337932962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff2ccd8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a9aa30 0x55f833a99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 -- 172.21.15.122:0/337932962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a9aa30 msgr2=0x55f833a99830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a9aa30 0x55f833a99830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 -- 172.21.15.122:0/337932962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 msgr2=0x55f833a8b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833a8b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 -- 172.21.15.122:0/337932962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f833a934e0 con 0x55f833a90170 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.267+0000 7eff277fe640 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a922b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7eff2009be50 tx=0x7eff200a0f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c30c8b69cd215a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.268+0000 7eff0dffb640 1 -- 172.21.15.122:0/337932962 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200b5070 con 0x55f833a90170 2024-09-12T01:56:06.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.268+0000 7eff0dffb640 1 -- 172.21.15.122:0/337932962 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff200bb2c0 con 0x55f833a90170 2024-09-12T01:56:06.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.268+0000 7eff0dffb640 1 -- 172.21.15.122:0/337932962 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200a51c0 con 0x55f833a90170 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 -- 172.21.15.122:0/337932962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 msgr2=0x55f833a922b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a922b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7eff2009be50 tx=0x7eff200a0f10 comp rx=0 tx=0).stop 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 -- 172.21.15.122:0/337932962 shutdown_connections 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a9aa30 0x55f833a99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a90170 0x55f833a922b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 --2- 172.21.15.122:0/337932962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833a8b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 -- 172.21.15.122:0/337932962 >> 172.21.15.122:0/337932962 conn(0x55f833a8bde0 msgr2=0x55f833a99120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 -- 172.21.15.122:0/337932962 shutdown_connections 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 -- 172.21.15.122:0/337932962 wait complete. 2024-09-12T01:56:06.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 -- start start 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.269+0000 7eff30b14740 1 Processor -- start 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833aa2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833aa30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 0x55f833a9f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400caf0 con 0x55f833a9aa30 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400df60 con 0x55f833a8f850 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400d1c0 con 0x55f833a90170 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 0x55f833a9f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 0x55f833a9f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51908/0 (socket says 172.21.15.122:51908) 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 -- 172.21.15.122:0/260105965 learned_addr learned my addr 172.21.15.122:0/260105965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff27fff640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833aa2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff30b14740 10 client.0 _ll_register_callbacks cb 0x7eff20098c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff277fe640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833aa30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 -- 172.21.15.122:0/260105965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 msgr2=0x55f833aa30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833aa30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 -- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 msgr2=0x55f833aa2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833aa2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 -- 172.21.15.122:0/260105965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f833aacea0 con 0x55f833a9aa30 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff2ccd8640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 0x55f833a9f9a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7eff18009cb0 tx=0x7eff180167a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cc6216bb498757 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff077fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff077fe640 1 -- 172.21.15.122:0/260105965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18017440 con 0x55f833a9aa30 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.270+0000 7eff077fe640 1 -- 172.21.15.122:0/260105965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7eff180175e0 con 0x55f833a9aa30 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff200bb2c0 con 0x55f833a9aa30 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff077fe640 1 -- 172.21.15.122:0/260105965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff180177e0 con 0x55f833a9aa30 2024-09-12T01:56:06.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 10 client.16974 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f833b1e460 con 0x55f833a9aa30 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff077fe640 1 -- 172.21.15.122:0/260105965 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1802a400 con 0x55f833a9aa30 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff077fe640 1 -- 172.21.15.122:0/260105965 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7eff1801f3d0 con 0x55f833a9aa30 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 10 client.16974 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 10 client.16974 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 20 client.16974 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833b1ec80 0x55f833b210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 4 client.16974 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x7eff200bb2c0 con 0x55f833b1ec80 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.271+0000 7eff27fff640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833b1ec80 0x55f833b210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.272+0000 7eff077fe640 10 client.16974 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:06.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.272+0000 7eff27fff640 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833b1ec80 0x55f833b210a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fa9051404ba85 server_cookie=9b00c277b1be093a in_seq=0 out_seq=0 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff077fe640 1 -- 172.21.15.122:0/260105965 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7eff200bb2c0 con 0x55f833b1ec80 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff077fe640 10 client.16974 handle_command_reply: tid=0 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 client.16974 shutdown 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 msgr2=0x55f833a9f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 0x55f833a9f9a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7eff18009cb0 tx=0x7eff180167a0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 shutdown_connections 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f833b1ec80 0x55f833b210a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f833a9aa30 0x55f833a9f9a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff077fe640 0 client.16974 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:06.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f833a90170 0x55f833aa30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 --2- 172.21.15.122:0/260105965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f833a8f850 0x55f833aa2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 >> 172.21.15.122:0/260105965 conn(0x55f833a8bde0 msgr2=0x55f833a99940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 shutdown_connections 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.389+0000 7eff30b14740 1 -- 172.21.15.122:0/260105965 wait complete. 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff30b14740 20 client.16974 trim_cache size 0 max 16384 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:56:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 38, 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "9e7b9264": { 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 38, 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 142.6, 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:56:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:06.393 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff100a7160 msgr2=0x7eff100a9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff100a7160 0x7eff100a9620 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009830 tx=0x7eff1c004040 comp rx=0 tx=0).stop 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 msgr2=0x7eff281967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281967e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7eff2000ed80 tx=0x7eff2000c6e0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 shutdown_connections 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7eff100a7160 0x7eff100a9620 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff28106f10 0x7eff2819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff28102790 0x7eff281967e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 --2- 172.21.15.122:0/4216385313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff28101de0 0x7eff281962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:06.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 >> 172.21.15.122:0/4216385313 conn(0x7eff280fd8c0 msgr2=0x7eff280fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:06.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 shutdown_connections 2024-09-12T01:56:06.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:06.390+0000 7eff2e73b640 1 -- 172.21.15.122:0/4216385313 wait complete. 2024-09-12T01:56:06.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.52 MB/sec execute 384 sec latency 574.960 ms 2024-09-12T01:56:06.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:06 smithi122 ceph-mon[30925]: pgmap v624: 97 pgs: 97 active+clean; 310 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 313 op/s 2024-09-12T01:56:07.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:06 smithi151 ceph-mon[35229]: pgmap v624: 97 pgs: 97 active+clean; 310 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 313 op/s 2024-09-12T01:56:07.175 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9e7b9264', quiesce took 3.5 seconds. Will release after: 19.05 2024-09-12T01:56:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:06 smithi183 ceph-mon[35127]: pgmap v624: 97 pgs: 97 active+clean; 310 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 313 op/s 2024-09-12T01:56:07.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.49 MB/sec execute 385 sec latency 1575.059 ms 2024-09-12T01:56:08.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.46 MB/sec execute 386 sec latency 2575.158 ms 2024-09-12T01:56:09.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.44 MB/sec execute 387 sec latency 3575.272 ms 2024-09-12T01:56:09.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:09 smithi183 ceph-mon[35127]: pgmap v625: 97 pgs: 97 active+clean; 291 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-09-12T01:56:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:09 smithi122 ceph-mon[30925]: pgmap v625: 97 pgs: 97 active+clean; 291 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-09-12T01:56:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:09 smithi151 ceph-mon[35229]: pgmap v625: 97 pgs: 97 active+clean; 291 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 304 op/s 2024-09-12T01:56:10.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.41 MB/sec execute 388 sec latency 4575.353 ms 2024-09-12T01:56:11.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.38 MB/sec execute 389 sec latency 5575.455 ms 2024-09-12T01:56:11.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:11 smithi183 ceph-mon[35127]: pgmap v626: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-12T01:56:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:11 smithi122 ceph-mon[30925]: pgmap v626: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-12T01:56:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:11 smithi151 ceph-mon[35229]: pgmap v626: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-12T01:56:12.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.35 MB/sec execute 390 sec latency 6575.544 ms 2024-09-12T01:56:13.851 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.33 MB/sec execute 391 sec latency 7575.708 ms 2024-09-12T01:56:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:13 smithi183 ceph-mon[35127]: pgmap v627: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 245 op/s 2024-09-12T01:56:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:13 smithi122 ceph-mon[30925]: pgmap v627: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 245 op/s 2024-09-12T01:56:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:13 smithi151 ceph-mon[35229]: pgmap v627: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 245 op/s 2024-09-12T01:56:14.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.30 MB/sec execute 392 sec latency 8575.792 ms 2024-09-12T01:56:15.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.28 MB/sec execute 393 sec latency 9575.889 ms 2024-09-12T01:56:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:15 smithi183 ceph-mon[35127]: pgmap v628: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.9 MiB/s wr, 203 op/s 2024-09-12T01:56:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:15 smithi122 ceph-mon[30925]: pgmap v628: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.9 MiB/s wr, 203 op/s 2024-09-12T01:56:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:15 smithi151 ceph-mon[35229]: pgmap v628: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.9 MiB/s wr, 203 op/s 2024-09-12T01:56:16.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.25 MB/sec execute 394 sec latency 10575.983 ms 2024-09-12T01:56:17.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.22 MB/sec execute 395 sec latency 11576.093 ms 2024-09-12T01:56:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:17 smithi183 ceph-mon[35127]: pgmap v629: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.7 MiB/s wr, 146 op/s 2024-09-12T01:56:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:17 smithi122 ceph-mon[30925]: pgmap v629: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.7 MiB/s wr, 146 op/s 2024-09-12T01:56:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:17 smithi151 ceph-mon[35229]: pgmap v629: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.7 MiB/s wr, 146 op/s 2024-09-12T01:56:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- 172.21.15.122:0/1436618845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44104fa0 msgr2=0x7f2c44107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1436618845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44104fa0 0x7f2c44107390 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c3402a410 comp rx=0 tx=0).stop 2024-09-12T01:56:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- 172.21.15.122:0/1436618845 shutdown_connections 2024-09-12T01:56:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1436618845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c441078d0 0x7f2c44109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1436618845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44104fa0 0x7f2c44107390 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1436618845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44102670 0x7f2c44104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- 172.21.15.122:0/1436618845 >> 172.21.15.122:0/1436618845 conn(0x7f2c440fc1b0 msgr2=0x7f2c440fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- 172.21.15.122:0/1436618845 shutdown_connections 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- 172.21.15.122:0/1436618845 wait complete. 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 Processor -- start 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- start start 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c44102670 0x7f2c44197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44104fa0 0x7f2c44198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 0x7f2c4419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410d7c0 con 0x7f2c441078d0 2024-09-12T01:56:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410d640 con 0x7f2c44104fa0 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.410+0000 7f2c4b918640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410d940 con 0x7f2c44102670 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 0x7f2c4419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 0x7f2c4419f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59296/0 (socket says 172.21.15.122:59296) 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 -- 172.21.15.122:0/1437111080 learned_addr learned my addr 172.21.15.122:0/1437111080 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c496b4640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c44102670 0x7f2c44197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44104fa0 0x7f2c44198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 -- 172.21.15.122:0/1437111080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c44102670 msgr2=0x7f2c44197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c44102670 0x7f2c44197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 -- 172.21.15.122:0/1437111080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44104fa0 msgr2=0x7f2c44198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44104fa0 0x7f2c44198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 -- 172.21.15.122:0/1437111080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4419f960 con 0x7f2c441078d0 2024-09-12T01:56:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44104fa0 0x7f2c44198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:56:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c49eb5640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 0x7f2c4419f2c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00ccb0 tx=0x7f2c3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c3a7fc640 1 -- 172.21.15.122:0/1437111080 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c003970 con 0x7f2c441078d0 2024-09-12T01:56:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4419fc50 con 0x7f2c441078d0 2024-09-12T01:56:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c3a7fc640 1 -- 172.21.15.122:0/1437111080 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3c004030 con 0x7f2c441078d0 2024-09-12T01:56:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c3a7fc640 1 -- 172.21.15.122:0/1437111080 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00f6b0 con 0x7f2c441078d0 2024-09-12T01:56:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.411+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441a0150 con 0x7f2c441078d0 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.413+0000 7f2c3a7fc640 1 -- 172.21.15.122:0/1437111080 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2c3c0041e0 con 0x7f2c441078d0 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.414+0000 7f2c4e6b2740 1 Processor -- start 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.414+0000 7f2c4e6b2740 1 -- start start 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c088510da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 0x55c088515da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c088516370 0x55c088518830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c0884742f0 con 0x55c0885112e0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c08841c270 con 0x55c088516370 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0884702b0 con 0x55c0885109a0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c3a7fc640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c300a2c30 0x7f2c300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 0x55c088515da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 0x55c088515da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59308/0 (socket says 172.21.15.122:59308) 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 -- 172.21.15.122:0/2791935056 learned_addr learned my addr 172.21.15.122:0/2791935056 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c49eb5640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c300a2c30 0x7f2c300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c088510da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c3a7fc640 1 -- 172.21.15.122:0/1437111080 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2c3c0cf180 con 0x7f2c441078d0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c088516370 0x55c088518830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 -- 172.21.15.122:0/2791935056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 msgr2=0x55c088510da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c088510da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 -- 172.21.15.122:0/2791935056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c088516370 msgr2=0x55c088518830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c088516370 0x55c088518830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 -- 172.21.15.122:0/2791935056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08841b340 con 0x55c0885112e0 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c088510da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c496b4640 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 0x55c088515da0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000ac80 tx=0x7f2c4000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e032e7b6dc7cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.415+0000 7f2c49eb5640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c300a2c30 0x7f2c300a50f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005fd0 tx=0x7f2c3c0c6570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c2f7fe640 1 -- 172.21.15.122:0/2791935056 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4001d070 con 0x55c0885112e0 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c2f7fe640 1 -- 172.21.15.122:0/2791935056 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c40018b20 con 0x55c0885112e0 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c2f7fe640 1 -- 172.21.15.122:0/2791935056 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40010c40 con 0x55c0885112e0 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2791935056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 msgr2=0x55c088515da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 0x55c088515da0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000ac80 tx=0x7f2c4000f790 comp rx=0 tx=0).stop 2024-09-12T01:56:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2791935056 shutdown_connections 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c088516370 0x55c088518830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0885112e0 0x55c088515da0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2791935056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c088510da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2791935056 >> 172.21.15.122:0/2791935056 conn(0x55c08850c010 msgr2=0x55c08850c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2791935056 shutdown_connections 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.416+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2791935056 wait complete. 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 -- start start 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 Processor -- start 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c08851f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885112e0 0x55c08851f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088526590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c088519340 con 0x55c088516370 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0883fd290 con 0x55c0885112e0 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0885194c0 con 0x55c0885109a0 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c4e6b2740 10 client.0 _ll_register_callbacks cb 0x55c08842e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c48eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088526590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c48eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088526590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59322/0 (socket says 172.21.15.122:59322) 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c48eb3640 1 -- 172.21.15.122:0/1055800798 learned_addr learned my addr 172.21.15.122:0/1055800798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.417+0000 7f2c496b4640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885112e0 0x55c08851f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c08851f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 -- 172.21.15.122:0/1055800798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 msgr2=0x55c08851f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c08851f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 -- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885112e0 msgr2=0x55c08851f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885112e0 0x55c08851f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 -- 172.21.15.122:0/1055800798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c088187210 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088526590 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005ab0 tx=0x7f2c34009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f60e31d863b170c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c2cff9640 1 -- 172.21.15.122:0/1055800798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c2cff9640 1 -- 172.21.15.122:0/1055800798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2c340037f0 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c40018b20 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c2cff9640 1 -- 172.21.15.122:0/1055800798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c34051050 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c4e6b2740 10 client.16986 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c088253620 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c2cff9640 1 -- 172.21.15.122:0/1055800798 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c40018b20 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.418+0000 7f2c2cff9640 1 -- 172.21.15.122:0/1055800798 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2c3403a040 con 0x55c088516370 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c4e6b2740 10 client.16986 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c4e6b2740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c4e6b2740 20 client.16986 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c08852ac10 0x55c08852d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c4e6b2740 4 client.16986 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c088481a20 con 0x55c08852ac10 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c496b4640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c08852ac10 0x55c08852d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c496b4640 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c08852ac10 0x55c08852d080 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a039ae00d4b93f2f server_cookie=ed38dd4cabb1e314 in_seq=0 out_seq=0 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.419+0000 7f2c2cff9640 10 client.16986 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c2cff9640 1 -- 172.21.15.122:0/1055800798 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c088481a20 con 0x55c08852ac10 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c2cff9640 10 client.16986 handle_command_reply: tid=0 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 client.16986 shutdown 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 msgr2=0x55c088526590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088526590 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005ab0 tx=0x7f2c34009260 comp rx=0 tx=0).stop 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 shutdown_connections 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c08852ac10 0x55c08852d080 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088526590 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885112e0 0x55c08851f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c2cff9640 0 client.16986 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/1055800798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885109a0 0x55c08851f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 >> 172.21.15.122:0/1055800798 conn(0x55c08850c010 msgr2=0x55c08850d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 shutdown_connections 2024-09-12T01:56:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/1055800798 wait complete. 2024-09-12T01:56:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.420+0000 7f2c4e6b2740 20 client.16986 trim_cache size 0 max 16384 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.434+0000 7f2c4e6b2740 1 Processor -- start 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.434+0000 7f2c4e6b2740 1 -- start start 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c08850cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c088526550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c08850d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441a9620 con 0x55c088516370 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441a6ef0 con 0x55c0885109a0 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441a4d20 con 0x55c0885112e0 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c08850d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c08850d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59326/0 (socket says 172.21.15.122:59326) 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 -- 172.21.15.122:0/2987059285 learned_addr learned my addr 172.21.15.122:0/2987059285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c08850cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 -- 172.21.15.122:0/2987059285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 msgr2=0x55c088526550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c496b4640 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c088526550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c088526550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 -- 172.21.15.122:0/2987059285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 msgr2=0x55c08850cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c08850cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 -- 172.21.15.122:0/2987059285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0885072d0 con 0x55c088516370 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c08850d400 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x55c0885075d0 tx=0x7f2c3402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b38b57fe3465b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c2effd640 1 -- 172.21.15.122:0/2987059285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x55c088516370 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c2effd640 1 -- 172.21.15.122:0/2987059285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3403cd50 con 0x55c088516370 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.435+0000 7f2c2effd640 1 -- 172.21.15.122:0/2987059285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340053f0 con 0x55c088516370 2024-09-12T01:56:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2987059285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 msgr2=0x55c08850d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c08850d400 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x55c0885075d0 tx=0x7f2c3402a410 comp rx=0 tx=0).stop 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2987059285 shutdown_connections 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c08850d400 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c088526550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/2987059285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c08850cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2987059285 >> 172.21.15.122:0/2987059285 conn(0x55c08850c010 msgr2=0x55c08851f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2987059285 shutdown_connections 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/2987059285 wait complete. 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.436+0000 7f2c4e6b2740 1 -- start start 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 Processor -- start 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c088521fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c0885224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088525500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441a53b0 con 0x55c088516370 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441a6820 con 0x55c0885109a0 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441a5a80 con 0x55c0885112e0 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c4e6b2740 10 client.0 _ll_register_callbacks cb 0x7f2c34009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088525500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088525500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59340/0 (socket says 172.21.15.122:59340) 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 -- 172.21.15.122:0/935592693 learned_addr learned my addr 172.21.15.122:0/935592693 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c088521fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 -- 172.21.15.122:0/935592693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 msgr2=0x55c0885224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c0885224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 -- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 msgr2=0x55c088521fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c088521fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.437+0000 7f2c48eb3640 1 -- 172.21.15.122:0/935592693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c088525b10 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c48eb3640 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088525500 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005ab0 tx=0x7f2c34005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3cbd521289112 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c2cff9640 1 -- 172.21.15.122:0/935592693 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c2cff9640 1 -- 172.21.15.122:0/935592693 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f2c34031c60 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c3403cd50 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c2cff9640 1 -- 172.21.15.122:0/935592693 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c3402aa90 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 10 client.16992 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c088595cc0 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c2cff9640 1 -- 172.21.15.122:0/935592693 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3403cd50 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c2cff9640 1 -- 172.21.15.122:0/935592693 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2c34003b40 con 0x55c088516370 2024-09-12T01:56:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 10 client.16992 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 20 client.16992 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c088596640 0x55c088598a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 4 client.16992 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.438+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c08847bd70 con 0x55c088596640 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.439+0000 7f2c496b4640 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c088596640 0x55c088598a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.439+0000 7f2c496b4640 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c088596640 0x55c088598a60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee8f602453c1e3c server_cookie=402a5b1338da54e8 in_seq=0 out_seq=0 2024-09-12T01:56:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.439+0000 7f2c2cff9640 10 client.16992 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c2cff9640 1 -- 172.21.15.122:0/935592693 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c08847bd70 con 0x55c088596640 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c2cff9640 10 client.16992 handle_command_reply: tid=0 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 client.16992 shutdown 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 msgr2=0x55c088525500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088525500 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005ab0 tx=0x7f2c34005fd0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 shutdown_connections 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c088596640 0x55c088598a60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c088516370 0x55c088525500 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0885112e0 0x55c0885224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c2cff9640 0 client.16992 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 --2- 172.21.15.122:0/935592693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0885109a0 0x55c088521fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 >> 172.21.15.122:0/935592693 conn(0x55c08850c010 msgr2=0x55c08850a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 shutdown_connections 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 1 -- 172.21.15.122:0/935592693 wait complete. 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.452+0000 7f2c4e6b2740 20 client.16992 trim_cache size 0 max 16384 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:56:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:56:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c300a2c30 msgr2=0x7f2c300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c300a2c30 0x7f2c300a50f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005fd0 tx=0x7f2c3c0c6570 comp rx=0 tx=0).stop 2024-09-12T01:56:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 msgr2=0x7f2c4419f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 0x7f2c4419f2c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00ccb0 tx=0x7f2c3c00a620 comp rx=0 tx=0).stop 2024-09-12T01:56:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 shutdown_connections 2024-09-12T01:56:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2c300a2c30 0x7f2c300a50f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c441078d0 0x7f2c4419f2c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c44104fa0 0x7f2c44198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 --2- 172.21.15.122:0/1437111080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c44102670 0x7f2c44197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 >> 172.21.15.122:0/1437111080 conn(0x7f2c440fc1b0 msgr2=0x7f2c44102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 shutdown_connections 2024-09-12T01:56:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:18.453+0000 7f2c4b918640 1 -- 172.21.15.122:0/1437111080 wait complete. 2024-09-12T01:56:18.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.20 MB/sec execute 396 sec latency 12576.153 ms 2024-09-12T01:56:19.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.17 MB/sec execute 397 sec latency 13576.241 ms 2024-09-12T01:56:19.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:19 smithi183 ceph-mon[35127]: pgmap v630: 97 pgs: 97 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 2.6 MiB/s wr, 132 op/s 2024-09-12T01:56:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:19 smithi122 ceph-mon[30925]: pgmap v630: 97 pgs: 97 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 2.6 MiB/s wr, 132 op/s 2024-09-12T01:56:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:19 smithi151 ceph-mon[35229]: pgmap v630: 97 pgs: 97 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 2.6 MiB/s wr, 132 op/s 2024-09-12T01:56:20.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.15 MB/sec execute 398 sec latency 14576.293 ms 2024-09-12T01:56:21.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.12 MB/sec execute 399 sec latency 15576.368 ms 2024-09-12T01:56:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:21 smithi183 ceph-mon[35127]: pgmap v631: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-12T01:56:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:21 smithi122 ceph-mon[30925]: pgmap v631: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-12T01:56:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:21 smithi151 ceph-mon[35229]: pgmap v631: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-09-12T01:56:22.852 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.10 MB/sec execute 400 sec latency 16576.481 ms 2024-09-12T01:56:23.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.07 MB/sec execute 401 sec latency 17576.589 ms 2024-09-12T01:56:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:23 smithi183 ceph-mon[35127]: pgmap v632: 97 pgs: 97 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-09-12T01:56:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:23 smithi122 ceph-mon[30925]: pgmap v632: 97 pgs: 97 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-09-12T01:56:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:23 smithi151 ceph-mon[35229]: pgmap v632: 97 pgs: 97 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-09-12T01:56:24.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.05 MB/sec execute 402 sec latency 18576.704 ms 2024-09-12T01:56:25.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.02 MB/sec execute 403 sec latency 19576.822 ms 2024-09-12T01:56:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:25 smithi183 ceph-mon[35127]: pgmap v633: 97 pgs: 97 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-12T01:56:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:25 smithi122 ceph-mon[30925]: pgmap v633: 97 pgs: 97 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-12T01:56:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:25 smithi151 ceph-mon[35229]: pgmap v633: 97 pgs: 97 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-12T01:56:26.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:56:26.472 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:26.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 10.00 MB/sec execute 404 sec latency 20576.953 ms 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.955+0000 7f1d9c024640 1 -- 172.21.15.122:0/3093536935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94101de0 msgr2=0x7f1d941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.955+0000 7f1d9c024640 1 --2- 172.21.15.122:0/3093536935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94101de0 0x7f1d941021c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009900 tx=0x7f1d9002a410 comp rx=0 tx=0).stop 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.955+0000 7f1d9c024640 1 -- 172.21.15.122:0/3093536935 shutdown_connections 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.955+0000 7f1d9c024640 1 --2- 172.21.15.122:0/3093536935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94106f10 0x7f1d9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 --2- 172.21.15.122:0/3093536935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d94102790 0x7f1d941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 --2- 172.21.15.122:0/3093536935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94101de0 0x7f1d941021c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- 172.21.15.122:0/3093536935 >> 172.21.15.122:0/3093536935 conn(0x7f1d940fd8c0 msgr2=0x7f1d940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- 172.21.15.122:0/3093536935 shutdown_connections 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- 172.21.15.122:0/3093536935 wait complete. 2024-09-12T01:56:26.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 Processor -- start 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- start start 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d94101de0 0x7f1d941984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94102790 0x7f1d94198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 0x7f1d9419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d94111f70 con 0x7f1d94106f10 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d94111df0 con 0x7f1d94102790 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.956+0000 7f1d9c024640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d941120f0 con 0x7f1d94101de0 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 0x7f1d9419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 0x7f1d9419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48042/0 (socket says 172.21.15.122:48042) 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 -- 172.21.15.122:0/2536719402 learned_addr learned my addr 172.21.15.122:0/2536719402 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d99dc0640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d94101de0 0x7f1d941984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:26.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d995bf640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94102790 0x7f1d94198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 -- 172.21.15.122:0/2536719402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d94101de0 msgr2=0x7f1d941984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d94101de0 0x7f1d941984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 -- 172.21.15.122:0/2536719402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94102790 msgr2=0x7f1d94198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94102790 0x7f1d94198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 -- 172.21.15.122:0/2536719402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9419d480 con 0x7f1d94106f10 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d995bf640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94102790 0x7f1d94198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9a5c1640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 0x7f1d9419cda0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00cad0 tx=0x7f1d8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00ae20 con 0x7f1d94106f10 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d8c005020 con 0x7f1d94106f10 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9419d770 con 0x7f1d94106f10 2024-09-12T01:56:26.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00f6f0 con 0x7f1d94106f10 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.957+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941a5050 con 0x7f1d94106f10 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.958+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d941021c0 con 0x7f1d94106f10 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.959+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1d8c0051d0 con 0x7f1d94106f10 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.960+0000 7f1d8affd640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d800a2c00 0x7f1d800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.960+0000 7f1d99dc0640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d800a2c00 0x7f1d800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.960+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f1d8c0cf370 con 0x7f1d94106f10 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.961+0000 7f1d99dc0640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d800a2c00 0x7f1d800a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000c00 tx=0x7f1d900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:26.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:26.963+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d8c014030 con 0x7f1d94106f10 2024-09-12T01:56:27.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.163+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d94199830 con 0x7f1d94106f10 2024-09-12T01:56:27.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.164+0000 7f1d8affd640 1 -- 172.21.15.122:0/2536719402 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f1d94199830 con 0x7f1d94106f10 2024-09-12T01:56:27.164 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:56:27.164 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:56:27.164 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:56:27.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d800a2c00 msgr2=0x7f1d800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:27.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d800a2c00 0x7f1d800a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000c00 tx=0x7f1d900058d0 comp rx=0 tx=0).stop 2024-09-12T01:56:27.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 msgr2=0x7f1d9419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:27.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 0x7f1d9419cda0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00cad0 tx=0x7f1d8c00cf90 comp rx=0 tx=0).stop 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 shutdown_connections 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1d800a2c00 0x7f1d800a50c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d94106f10 0x7f1d9419cda0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94102790 0x7f1d94198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 --2- 172.21.15.122:0/2536719402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d94101de0 0x7f1d941984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 >> 172.21.15.122:0/2536719402 conn(0x7f1d940fd8c0 msgr2=0x7f1d940fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 shutdown_connections 2024-09-12T01:56:27.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:27.166+0000 7f1d9c024640 1 -- 172.21.15.122:0/2536719402 wait complete. 2024-09-12T01:56:27.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:27 smithi122 ceph-mon[30925]: pgmap v634: 97 pgs: 97 active+clean; 269 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-09-12T01:56:27.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:27 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2536719402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:27.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924017 9.97 MB/sec execute 405 sec latency 21577.006 ms 2024-09-12T01:56:27.946 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 9e7b9264 --release --await' 2024-09-12T01:56:27.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 9e7b9264 --release --await 2024-09-12T01:56:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:27 smithi151 ceph-mon[35229]: pgmap v634: 97 pgs: 97 active+clean; 269 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-09-12T01:56:28.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:27 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2536719402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:28.193 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:28.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:27 smithi183 ceph-mon[35127]: pgmap v634: 97 pgs: 97 active+clean; 269 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.0 MiB/s wr, 88 op/s 2024-09-12T01:56:28.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:27 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2536719402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:28.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 -- 172.21.15.122:0/4164760551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610106a60 msgr2=0x7f5610106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 --2- 172.21.15.122:0/4164760551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610106a60 0x7f5610106e40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5604009930 tx=0x7f560402a420 comp rx=0 tx=0).stop 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 -- 172.21.15.122:0/4164760551 shutdown_connections 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 --2- 172.21.15.122:0/4164760551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5610101400 0x7f561010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 --2- 172.21.15.122:0/4164760551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610100a60 0x7f5610100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 --2- 172.21.15.122:0/4164760551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610106a60 0x7f5610106e40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 -- 172.21.15.122:0/4164760551 >> 172.21.15.122:0/4164760551 conn(0x7f56100fc780 msgr2=0x7f56100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 -- 172.21.15.122:0/4164760551 shutdown_connections 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 -- 172.21.15.122:0/4164760551 wait complete. 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.707+0000 7f5617105640 1 Processor -- start 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 -- start start 2024-09-12T01:56:28.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5610100a60 0x7f5610199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 0x7f5610199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610106a60 0x7f56101a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5610110e60 con 0x7f5610101400 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5610110ce0 con 0x7f5610100a60 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5610110fe0 con 0x7f5610106a60 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 0x7f5610199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 0x7f5610199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48048/0 (socket says 172.21.15.122:48048) 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 -- 172.21.15.122:0/3752132650 learned_addr learned my addr 172.21.15.122:0/3752132650 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5610100a60 0x7f5610199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f56156a2640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610106a60 0x7f56101a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 -- 172.21.15.122:0/3752132650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610106a60 msgr2=0x7f56101a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610106a60 0x7f56101a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 -- 172.21.15.122:0/3752132650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5610100a60 msgr2=0x7f5610199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5610100a60 0x7f5610199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 -- 172.21.15.122:0/3752132650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56101a4900 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f56156a2640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610106a60 0x7f56101a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560ffff640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 0x7f5610199880 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d770 tx=0x7f55fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560dffb640 1 -- 172.21.15.122:0/3752132650 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc017070 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f560dffb640 1 -- 172.21.15.122:0/3752132650 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55fc005500 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.708+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56101a4ad0 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.709+0000 7f560dffb640 1 -- 172.21.15.122:0/3752132650 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc004e00 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.709+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56101a4f90 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.711+0000 7f560dffb640 1 -- 172.21.15.122:0/3752132650 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f55fc004fa0 con 0x7f5610101400 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.711+0000 7f56194de740 1 Processor -- start 2024-09-12T01:56:28.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.711+0000 7f56194de740 1 -- start start 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.711+0000 7f560dffb640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56000a2b80 0x7f56000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56156a2640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56000a2b80 0x7f56000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f560dffb640 1 -- 172.21.15.122:0/3752132650 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f55fc0ceb00 con 0x7f5610101400 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56194de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bd32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56194de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802be2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56194de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802be4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56194de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b802b38620 con 0x55b802bd2f00 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56194de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b802adfe50 con 0x55b802bd3820 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56194de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b802b345e0 con 0x55b802be2650 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bd32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bd32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48050/0 (socket says 172.21.15.122:48050) 2024-09-12T01:56:28.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 -- 172.21.15.122:0/1739512881 learned_addr learned my addr 172.21.15.122:0/1739512881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56156a2640 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802be4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f560ffff640 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802be2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 -- 172.21.15.122:0/1739512881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 msgr2=0x55b802be4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802be4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 -- 172.21.15.122:0/1739512881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 msgr2=0x55b802be2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802be2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 -- 172.21.15.122:0/1739512881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b802ad0d40 con 0x55b802bd2f00 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f56156a2640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56000a2b80 0x7f56000a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5608005fd0 tx=0x7f56080107a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f5614ea1640 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bd32e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f560402a940 tx=0x7f5604005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba08bc6c3f0b185 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f55faffd640 1 -- 172.21.15.122:0/1739512881 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604040070 con 0x55b802bd2f00 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f55faffd640 1 -- 172.21.15.122:0/1739512881 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56040318f0 con 0x55b802bd2f00 2024-09-12T01:56:28.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.712+0000 7f55faffd640 1 -- 172.21.15.122:0/1739512881 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604035040 con 0x55b802bd2f00 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 -- 172.21.15.122:0/1739512881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 msgr2=0x55b802bd32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bd32e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f560402a940 tx=0x7f5604005c00 comp rx=0 tx=0).stop 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 -- 172.21.15.122:0/1739512881 shutdown_connections 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802be4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802be2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 --2- 172.21.15.122:0/1739512881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bd32e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 -- 172.21.15.122:0/1739512881 >> 172.21.15.122:0/1739512881 conn(0x55b802bcf5f0 msgr2=0x55b802bcf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 -- 172.21.15.122:0/1739512881 shutdown_connections 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 -- 172.21.15.122:0/1739512881 wait complete. 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.713+0000 7f56194de740 1 -- start start 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 Processor -- start 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bdf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bdf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b802be5220 con 0x55b802bd2f00 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b802ac0f10 con 0x55b802bd3820 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b802be53a0 con 0x55b802be2650 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56194de740 10 client.0 _ll_register_callbacks cb 0x55b802ae4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f5614ea1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bdf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f560ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bdf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f56156a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f5614ea1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bdf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48066/0 (socket says 172.21.15.122:48066) 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f560ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bdf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:44552/0 (socket says 172.21.15.122:44552) 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.714+0000 7f5614ea1640 1 -- 172.21.15.122:0/3004769534 learned_addr learned my addr 172.21.15.122:0/3004769534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f5614ea1640 1 -- 172.21.15.122:0/3004769534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 msgr2=0x55b802bdff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f5614ea1640 1 -- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 msgr2=0x55b802bdf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bdf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f5614ea1640 1 -- 172.21.15.122:0/3004769534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b802847580 con 0x55b802bd2f00 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f56156a2640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f560ffff640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bdf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bdf4b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f56040052d0 tx=0x7f5604005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bd41170b0440af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f55effff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f55effff640 1 -- 172.21.15.122:0/3004769534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604040030 con 0x55b802bd2f00 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f55effff640 1 -- 172.21.15.122:0/3004769534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f5604004260 con 0x55b802bd2f00 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56040318f0 con 0x55b802bd2f00 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f55effff640 1 -- 172.21.15.122:0/3004769534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5604035040 con 0x55b802bd2f00 2024-09-12T01:56:28.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f56194de740 10 client.17010 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b802911960 con 0x55b802bd2f00 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.715+0000 7f55effff640 1 -- 172.21.15.122:0/3004769534 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56040318f0 con 0x55b802bd2f00 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f55effff640 1 -- 172.21.15.122:0/3004769534 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f56040039c0 con 0x55b802bd2f00 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f56194de740 10 client.17010 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f56194de740 10 client.17010 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f56194de740 20 client.17010 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f56194de740 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802bf2070 0x55b802bf4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f56194de740 4 client.17010 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b802b45b90 con 0x55b802bf2070 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f560ffff640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802bf2070 0x55b802bf4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f55effff640 10 client.17010 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.716+0000 7f560ffff640 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802bf2070 0x55b802bf4460 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494323628886bf5 server_cookie=1d2b041d718baf4 in_seq=0 out_seq=0 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f55effff640 1 -- 172.21.15.122:0/3004769534 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b802b45b90 con 0x55b802bf2070 2024-09-12T01:56:28.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f55effff640 10 client.17010 handle_command_reply: tid=0 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 client.17010 shutdown 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 msgr2=0x55b802bdf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bdf4b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f56040052d0 tx=0x7f5604005300 comp rx=0 tx=0).stop 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 shutdown_connections 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802bf2070 0x55b802bf4460 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f55effff640 0 client.17010 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bdf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 --2- 172.21.15.122:0/3004769534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bdf4b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 >> 172.21.15.122:0/3004769534 conn(0x55b802bcf5f0 msgr2=0x55b802bd5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 shutdown_connections 2024-09-12T01:56:28.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 1 -- 172.21.15.122:0/3004769534 wait complete. 2024-09-12T01:56:28.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.717+0000 7f56194de740 20 client.17010 trim_cache size 0 max 16384 2024-09-12T01:56:28.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.734+0000 7f56194de740 1 Processor -- start 2024-09-12T01:56:28.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.734+0000 7f56194de740 1 -- start start 2024-09-12T01:56:28.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.734+0000 7f56194de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bf9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.734+0000 7f56194de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bfa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.734+0000 7f56194de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.734+0000 7f56194de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56101adbe0 con 0x55b802bd2f00 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f56194de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56101ab4b0 con 0x55b802bd3820 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f56194de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56101a92e0 con 0x55b802be2650 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bf9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bf9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48072/0 (socket says 172.21.15.122:48072) 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 -- 172.21.15.122:0/1273616474 learned_addr learned my addr 172.21.15.122:0/1273616474 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 -- 172.21.15.122:0/1273616474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 msgr2=0x55b802bdc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 -- 172.21.15.122:0/1273616474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 msgr2=0x55b802bfa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bfa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 -- 172.21.15.122:0/1273616474 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b802bd3650 con 0x55b802bd2f00 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f5614ea1640 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bf9ac0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5604003e80 tx=0x7f5604005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c54c40e10d68398 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:28.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f55fa7fc640 1 -- 172.21.15.122:0/1273616474 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604040030 con 0x55b802bd2f00 2024-09-12T01:56:28.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f55fa7fc640 1 -- 172.21.15.122:0/1273616474 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f560404d070 con 0x55b802bd2f00 2024-09-12T01:56:28.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.735+0000 7f55fa7fc640 1 -- 172.21.15.122:0/1273616474 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604004260 con 0x55b802bd2f00 2024-09-12T01:56:28.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 -- 172.21.15.122:0/1273616474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 msgr2=0x55b802bf9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bf9ac0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5604003e80 tx=0x7f5604005c00 comp rx=0 tx=0).stop 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 -- 172.21.15.122:0/1273616474 shutdown_connections 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802be2650 0x55b802bdc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd3820 0x55b802bfa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 --2- 172.21.15.122:0/1273616474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802bd2f00 0x55b802bf9ac0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 -- 172.21.15.122:0/1273616474 >> 172.21.15.122:0/1273616474 conn(0x55b802bcf5f0 msgr2=0x55b802bde490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 -- 172.21.15.122:0/1273616474 shutdown_connections 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 -- 172.21.15.122:0/1273616474 wait complete. 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 -- start start 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.736+0000 7f56194de740 1 Processor -- start 2024-09-12T01:56:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd2f00 0x55b802bea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802bd3820 0x55b802beadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 0x55b802be7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56101a9970 con 0x55b802be2650 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56101aade0 con 0x55b802bd2f00 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56101aa040 con 0x55b802bd3820 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 10 client.0 _ll_register_callbacks cb 0x7f5604031aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 0x55b802be7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 0x55b802be7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48086/0 (socket says 172.21.15.122:48086) 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 -- 172.21.15.122:0/3460020610 learned_addr learned my addr 172.21.15.122:0/3460020610 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f560ffff640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802bd3820 0x55b802beadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd2f00 0x55b802bea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 -- 172.21.15.122:0/3460020610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802bd3820 msgr2=0x55b802beadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802bd3820 0x55b802beadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 -- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd2f00 msgr2=0x55b802bea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd2f00 0x55b802bea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 -- 172.21.15.122:0/3460020610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b802be7d70 con 0x55b802be2650 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f560ffff640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802bd3820 0x55b802beadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56156a2640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 0x55b802be7660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f560801f450 tx=0x7f5608004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28633c3bf482ca12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f55effff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f55effff640 1 -- 172.21.15.122:0/3460020610 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560801e650 con 0x55b802be2650 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f55effff640 1 -- 172.21.15.122:0/3460020610 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f560801e7f0 con 0x55b802be2650 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.737+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f560404d070 con 0x55b802be2650 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f55effff640 1 -- 172.21.15.122:0/3460020610 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f560801e9f0 con 0x55b802be2650 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 10 client.17016 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b802c5d540 con 0x55b802be2650 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f55effff640 1 -- 172.21.15.122:0/3460020610 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f560404d070 con 0x55b802be2650 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f55effff640 1 -- 172.21.15.122:0/3460020610 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5608028410 con 0x55b802be2650 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 10 client.17016 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 10 client.17016 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 20 client.17016 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802c5dcf0 0x55b802c60110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 4 client.17016 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9e7b9264", "release": true, "await": true}] 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9e7b9264", "release": true, "await": true}) -- 0x55b802b400a0 con 0x55b802c5dcf0 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.738+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802c5dcf0 0x55b802c60110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.739+0000 7f5614ea1640 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802c5dcf0 0x55b802c60110 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df02fdc0fee18ca server_cookie=da9261ca965e965f in_seq=0 out_seq=0 2024-09-12T01:56:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.739+0000 7f55effff640 10 client.17016 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:28.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f55effff640 1 -- 172.21.15.122:0/3460020610 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b802b400a0 con 0x55b802c5dcf0 2024-09-12T01:56:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f55effff640 10 client.17016 handle_command_reply: tid=0 2024-09-12T01:56:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 client.17016 shutdown 2024-09-12T01:56:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 msgr2=0x55b802be7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 0x55b802be7660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f560801f450 tx=0x7f5608004ab0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 shutdown_connections 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b802c5dcf0 0x55b802c60110 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b802be2650 0x55b802be7660 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f55effff640 0 client.17016 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b802bd3820 0x55b802beadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 --2- 172.21.15.122:0/3460020610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b802bd2f00 0x55b802bea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 >> 172.21.15.122:0/3460020610 conn(0x55b802bcf5f0 msgr2=0x55b802bddc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.756+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 shutdown_connections 2024-09-12T01:56:28.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f56194de740 1 -- 172.21.15.122:0/3460020610 wait complete. 2024-09-12T01:56:28.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f56194de740 20 client.17016 trim_cache size 0 max 16384 2024-09-12T01:56:28.760 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:56:28.760 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:56:28.760 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:56:28.760 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 40, 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "9e7b9264": { 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 40, 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 142.6, 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:56:28.761 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:28.762 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56000a2b80 msgr2=0x7f56000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56000a2b80 0x7f56000a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5608005fd0 tx=0x7f56080107a0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 msgr2=0x7f5610199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 0x7f5610199880 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d770 tx=0x7f55fc00dc30 comp rx=0 tx=0).stop 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 shutdown_connections 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56000a2b80 0x7f56000a5040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5610106a60 0x7f56101a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5610101400 0x7f5610199880 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 --2- 172.21.15.122:0/3752132650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5610100a60 0x7f5610199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 >> 172.21.15.122:0/3752132650 conn(0x7f56100fc780 msgr2=0x7f56100fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:28.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 shutdown_connections 2024-09-12T01:56:28.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:28.757+0000 7f5617105640 1 -- 172.21.15.122:0/3752132650 wait complete. 2024-09-12T01:56:28.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 924068 9.95 MB/sec execute 406 sec latency 22481.786 ms 2024-09-12T01:56:29.457 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9e7b9264', total seconds elapsed: 25.7 2024-09-12T01:56:29.457 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T01:56:29.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:56:29.731 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:29.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 926794 9.95 MB/sec execute 407 sec latency 36.681 ms 2024-09-12T01:56:29.979 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:29 smithi122 ceph-mon[30925]: pgmap v635: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-12T01:56:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:29 smithi151 ceph-mon[35229]: pgmap v635: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-12T01:56:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:29 smithi183 ceph-mon[35127]: pgmap v635: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-12T01:56:30.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 -- 172.21.15.122:0/786699127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c102570 msgr2=0x7fb13c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 --2- 172.21.15.122:0/786699127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c102570 0x7fb13c1029b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb130009900 tx=0x7fb13002a480 comp rx=0 tx=0).stop 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 -- 172.21.15.122:0/786699127 shutdown_connections 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 --2- 172.21.15.122:0/786699127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102ef0 0x7fb13c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 --2- 172.21.15.122:0/786699127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c102570 0x7fb13c1029b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 --2- 172.21.15.122:0/786699127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c100e20 0x7fb13c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 -- 172.21.15.122:0/786699127 >> 172.21.15.122:0/786699127 conn(0x7fb13c0fc7d0 msgr2=0x7fb13c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 -- 172.21.15.122:0/786699127 shutdown_connections 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.204+0000 7fb14266e640 1 -- 172.21.15.122:0/786699127 wait complete. 2024-09-12T01:56:30.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.205+0000 7fb14266e640 1 Processor -- start 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.205+0000 7fb14266e640 1 -- start start 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb14266e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 0x7fb13c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb14266e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102570 0x7fb13c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb14266e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c102ef0 0x7fb13c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb14266e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c110f80 con 0x7fb13c100e20 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb14266e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c110e00 con 0x7fb13c102ef0 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb14266e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c111100 con 0x7fb13c102570 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 0x7fb13c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 0x7fb13c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48106/0 (socket says 172.21.15.122:48106) 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13bfff640 1 -- 172.21.15.122:0/794036044 learned_addr learned my addr 172.21.15.122:0/794036044 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13b7fe640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102570 0x7fb13c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:30.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb140c0b640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c102ef0 0x7fb13c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13bfff640 1 -- 172.21.15.122:0/794036044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102570 msgr2=0x7fb13c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13bfff640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102570 0x7fb13c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.206+0000 7fb13bfff640 1 -- 172.21.15.122:0/794036044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c102ef0 msgr2=0x7fb13c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb13bfff640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c102ef0 0x7fb13c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb13bfff640 1 -- 172.21.15.122:0/794036044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c1a69f0 con 0x7fb13c100e20 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb13b7fe640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102570 0x7fb13c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb140c0b640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c102ef0 0x7fb13c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb13bfff640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 0x7fb13c0fcc80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00d950 tx=0x7fb12c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:30.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c017030 con 0x7fb13c100e20 2024-09-12T01:56:30.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c1a6ce0 con 0x7fb13c100e20 2024-09-12T01:56:30.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12c004020 con 0x7fb13c100e20 2024-09-12T01:56:30.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0047a0 con 0x7fb13c100e20 2024-09-12T01:56:30.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.207+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c1a71e0 con 0x7fb13c100e20 2024-09-12T01:56:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.208+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13c101200 con 0x7fb13c100e20 2024-09-12T01:56:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.210+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb12c0041d0 con 0x7fb13c100e20 2024-09-12T01:56:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.211+0000 7fb1397fa640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1240a2b80 0x7fb1240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.211+0000 7fb13b7fe640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1240a2b80 0x7fb1240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.211+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb12c0ceff0 con 0x7fb13c100e20 2024-09-12T01:56:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.211+0000 7fb13b7fe640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1240a2b80 0x7fb1240a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb130009230 tx=0x7fb1300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:30.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.214+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb12c09bf50 con 0x7fb13c100e20 2024-09-12T01:56:30.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.433+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb13c0fe1b0 con 0x7fb13c100e20 2024-09-12T01:56:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.434+0000 7fb1397fa640 1 -- 172.21.15.122:0/794036044 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fb12c09b850 con 0x7fb13c100e20 2024-09-12T01:56:30.434 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:56:30.435 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:56:30.435 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1240a2b80 msgr2=0x7fb1240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1240a2b80 0x7fb1240a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb130009230 tx=0x7fb1300058d0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 msgr2=0x7fb13c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 0x7fb13c0fcc80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00d950 tx=0x7fb12c00de10 comp rx=0 tx=0).stop 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 shutdown_connections 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb1240a2b80 0x7fb1240a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb13c102ef0 0x7fb13c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb13c102570 0x7fb13c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 --2- 172.21.15.122:0/794036044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb13c100e20 0x7fb13c0fcc80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.436+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 >> 172.21.15.122:0/794036044 conn(0x7fb13c0fc7d0 msgr2=0x7fb13c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.437+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 shutdown_connections 2024-09-12T01:56:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:30.437+0000 7fb14266e640 1 -- 172.21.15.122:0/794036044 wait complete. 2024-09-12T01:56:30.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 929742 9.96 MB/sec execute 408 sec latency 23.015 ms 2024-09-12T01:56:30.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:30 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/794036044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:30 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/794036044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:31.150 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T01:56:31.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T01:56:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:30 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/794036044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:31.423 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 -- 172.21.15.122:0/2390684791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c101c30 msgr2=0x7f890c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 --2- 172.21.15.122:0/2390684791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c101c30 0x7f890c1020b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8900009900 tx=0x7f890002a410 comp rx=0 tx=0).stop 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 -- 172.21.15.122:0/2390684791 shutdown_connections 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 --2- 172.21.15.122:0/2390684791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 0x7f890c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 --2- 172.21.15.122:0/2390684791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c101c30 0x7f890c1020b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 --2- 172.21.15.122:0/2390684791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100a40 0x7f890c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 -- 172.21.15.122:0/2390684791 >> 172.21.15.122:0/2390684791 conn(0x7f890c0fc1e0 msgr2=0x7f890c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 -- 172.21.15.122:0/2390684791 shutdown_connections 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.548+0000 7f891272a640 1 -- 172.21.15.122:0/2390684791 wait complete. 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 Processor -- start 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 -- start start 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 0x7f890c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c101c30 0x7f890c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 0x7f890c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10c710 con 0x7f890c100a40 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10c590 con 0x7f890c1025f0 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f891272a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10c890 con 0x7f890c101c30 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f890bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 0x7f890c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f890bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 0x7f890c197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38670/0 (socket says 172.21.15.122:38670) 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f890bfff640 1 -- 172.21.15.122:0/411986499 learned_addr learned my addr 172.21.15.122:0/411986499 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f8910cc7640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 0x7f890c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f890b7fe640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c101c30 0x7f890c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f8910cc7640 1 -- 172.21.15.122:0/411986499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c101c30 msgr2=0x7f890c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f8910cc7640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c101c30 0x7f890c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f8910cc7640 1 -- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 msgr2=0x7f890c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.549+0000 7f8910cc7640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 0x7f890c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f8910cc7640 1 -- 172.21.15.122:0/411986499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c19f9e0 con 0x7f890c1025f0 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f890bfff640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 0x7f890c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f8910cc7640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 0x7f890c19f370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8904009fd0 tx=0x7f890400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f89097fa640 1 -- 172.21.15.122:0/411986499 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890400ce20 con 0x7f890c1025f0 2024-09-12T01:56:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c19fcd0 con 0x7f890c1025f0 2024-09-12T01:56:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f89097fa640 1 -- 172.21.15.122:0/411986499 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904005020 con 0x7f890c1025f0 2024-09-12T01:56:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f89097fa640 1 -- 172.21.15.122:0/411986499 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904010720 con 0x7f890c1025f0 2024-09-12T01:56:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.550+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a01a0 con 0x7f890c1025f0 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.553+0000 7f89097fa640 1 -- 172.21.15.122:0/411986499 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8904005540 con 0x7f890c1025f0 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89097fa640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88f40a2cb0 0x7f88f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 Processor -- start 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 -- start start 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89097fa640 1 -- 172.21.15.122:0/411986499 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89040cf6c0 con 0x7f890c1025f0 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f890bfff640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88f40a2cb0 0x7f88f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b2fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 0x55f88b302da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b305830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f88b2612f0 con 0x55f88b2fe2e0 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f88b209270 con 0x55f88b2fd9a0 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f89154c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f88b25d2b0 con 0x55f88b303370 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f890b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 0x55f88b302da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f8910cc7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b305830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f890b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 0x55f88b302da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38682/0 (socket says 172.21.15.122:38682) 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f890b7fe640 1 -- 172.21.15.122:0/2694565743 learned_addr learned my addr 172.21.15.122:0/2694565743 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.554+0000 7f890bfff640 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b2fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890b7fe640 1 -- 172.21.15.122:0/2694565743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 msgr2=0x55f88b305830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890b7fe640 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b305830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890b7fe640 1 -- 172.21.15.122:0/2694565743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 msgr2=0x55f88b2fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890b7fe640 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b2fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890b7fe640 1 -- 172.21.15.122:0/2694565743 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88b208340 con 0x55f88b2fe2e0 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f8910cc7640 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b305830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890bfff640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88f40a2cb0 0x7f88f40a5170 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005fd0 tx=0x7f88fc014da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f890b7fe640 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 0x55f88b302da0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8900000c00 tx=0x7f8900004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0284bc34a6e5705 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f88f27fc640 1 -- 172.21.15.122:0/2694565743 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900035040 con 0x55f88b2fe2e0 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f88f27fc640 1 -- 172.21.15.122:0/2694565743 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900005520 con 0x55f88b2fe2e0 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.555+0000 7f88f27fc640 1 -- 172.21.15.122:0/2694565743 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900031910 con 0x55f88b2fe2e0 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 -- 172.21.15.122:0/2694565743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 msgr2=0x55f88b302da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 0x55f88b302da0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8900000c00 tx=0x7f8900004170 comp rx=0 tx=0).stop 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 -- 172.21.15.122:0/2694565743 shutdown_connections 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b305830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fe2e0 0x55f88b302da0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 --2- 172.21.15.122:0/2694565743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b2fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 -- 172.21.15.122:0/2694565743 >> 172.21.15.122:0/2694565743 conn(0x55f88b2f9010 msgr2=0x55f88b2f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 -- 172.21.15.122:0/2694565743 shutdown_connections 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 -- 172.21.15.122:0/2694565743 wait complete. 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 -- start start 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.556+0000 7f89154c6740 1 Processor -- start 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b307bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b308130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b30f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f88b3062e0 con 0x55f88b2fd9a0 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f88b1ea290 con 0x55f88b2fe2e0 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f88b306460 con 0x55f88b303370 2024-09-12T01:56:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f89154c6740 10 client.0 _ll_register_callbacks cb 0x55f88b21b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b307bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b307bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38694/0 (socket says 172.21.15.122:38694) 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 -- 172.21.15.122:0/174234519 learned_addr learned my addr 172.21.15.122:0/174234519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890b7fe640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b308130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f8910cc7640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b30f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 -- 172.21.15.122:0/174234519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 msgr2=0x55f88b30f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b30f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 -- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 msgr2=0x55f88b308130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b308130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 -- 172.21.15.122:0/174234519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88af74210 con 0x55f88b2fd9a0 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f890bfff640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b307bf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f88fc014290 tx=0x7f88fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bd85801219cc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f88eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.557+0000 7f88eb7fe640 1 -- 172.21.15.122:0/174234519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc011e10 con 0x55f88b2fd9a0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f88eb7fe640 1 -- 172.21.15.122:0/174234519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f88fc020440 con 0x55f88b2fd9a0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8900005520 con 0x55f88b2fd9a0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f88eb7fe640 1 -- 172.21.15.122:0/174234519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88fc020640 con 0x55f88b2fd9a0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 10 client.17034 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f88b040620 con 0x55f88b2fd9a0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f88eb7fe640 1 -- 172.21.15.122:0/174234519 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900005520 con 0x55f88b2fd9a0 2024-09-12T01:56:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f88eb7fe640 1 -- 172.21.15.122:0/174234519 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f88fc032990 con 0x55f88b2fd9a0 2024-09-12T01:56:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 10 client.17034 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 20 client.17034 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b313880 0x55f88b315c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 4 client.17034 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88b26ea20 con 0x55f88b313880 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.558+0000 7f890b7fe640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b313880 0x55f88b315c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.559+0000 7f890b7fe640 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b313880 0x55f88b315c70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e2fb66ea427f6e4 server_cookie=70a4171907423403 in_seq=0 out_seq=0 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.559+0000 7f88eb7fe640 10 client.17034 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.559+0000 7f88eb7fe640 1 -- 172.21.15.122:0/174234519 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f88b26ea20 con 0x55f88b313880 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.559+0000 7f88eb7fe640 10 client.17034 handle_command_reply: tid=0 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.559+0000 7f89154c6740 1 client.17034 shutdown 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 msgr2=0x55f88b307bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b307bf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f88fc014290 tx=0x7f88fc004730 comp rx=0 tx=0).stop 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 shutdown_connections 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b313880 0x55f88b315c70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b30f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b308130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f88eb7fe640 0 client.17034 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 --2- 172.21.15.122:0/174234519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b307bf0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 >> 172.21.15.122:0/174234519 conn(0x55f88b2f9010 msgr2=0x55f88b2fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 shutdown_connections 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 1 -- 172.21.15.122:0/174234519 wait complete. 2024-09-12T01:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.560+0000 7f89154c6740 20 client.17034 trim_cache size 0 max 16384 2024-09-12T01:56:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 Processor -- start 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 -- start start 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b2f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b30f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b2fa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1ab820 con 0x55f88b2fd9a0 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a90f0 con 0x55f88b2fe2e0 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.575+0000 7f89154c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a6f20 con 0x55f88b303370 2024-09-12T01:56:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b2f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b2f9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38704/0 (socket says 172.21.15.122:38704) 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 -- 172.21.15.122:0/2814145077 learned_addr learned my addr 172.21.15.122:0/2814145077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890b7fe640 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b30f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f8910cc7640 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b2fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 -- 172.21.15.122:0/2814145077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 msgr2=0x55f88b2fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b2fa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 -- 172.21.15.122:0/2814145077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 msgr2=0x55f88b30f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b30f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 -- 172.21.15.122:0/2814145077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88b315fe0 con 0x55f88b2fd9a0 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f890bfff640 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b2f9ec0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f88fc011010 tx=0x7f88fc0205d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf723827c9c38521 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f88f1ffb640 1 -- 172.21.15.122:0/2814145077 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc028400 con 0x55f88b2fd9a0 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f88f1ffb640 1 -- 172.21.15.122:0/2814145077 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc020cc0 con 0x55f88b2fd9a0 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.576+0000 7f88f1ffb640 1 -- 172.21.15.122:0/2814145077 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc028d50 con 0x55f88b2fd9a0 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 -- 172.21.15.122:0/2814145077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 msgr2=0x55f88b2f9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b2f9ec0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f88fc011010 tx=0x7f88fc0205d0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 -- 172.21.15.122:0/2814145077 shutdown_connections 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b303370 0x55f88b2fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fe2e0 0x55f88b30f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 --2- 172.21.15.122:0/2814145077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b2fd9a0 0x55f88b2f9ec0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 -- 172.21.15.122:0/2814145077 >> 172.21.15.122:0/2814145077 conn(0x55f88b2f9010 msgr2=0x55f88b3079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 -- 172.21.15.122:0/2814145077 shutdown_connections 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 -- 172.21.15.122:0/2814145077 wait complete. 2024-09-12T01:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.577+0000 7f89154c6740 1 -- start start 2024-09-12T01:56:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 Processor -- start 2024-09-12T01:56:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b30c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b2fe2e0 0x55f88b30ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 0x55f88b30dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a75b0 con 0x55f88b303370 2024-09-12T01:56:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a8a20 con 0x55f88b2fd9a0 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a7c80 con 0x55f88b2fe2e0 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f89154c6740 10 client.0 _ll_register_callbacks cb 0x7f88fc014170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 0x55f88b30dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 0x55f88b30dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38718/0 (socket says 172.21.15.122:38718) 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f890bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b30c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 -- 172.21.15.122:0/1333534885 learned_addr learned my addr 172.21.15.122:0/1333534885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f890b7fe640 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b2fe2e0 0x55f88b30ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 -- 172.21.15.122:0/1333534885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b2fe2e0 msgr2=0x55f88b30ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b2fe2e0 0x55f88b30ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 -- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 msgr2=0x55f88b30c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b30c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.578+0000 7f8910cc7640 1 -- 172.21.15.122:0/1333534885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88b30e2f0 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f8910cc7640 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 0x55f88b30dce0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f890400c760 tx=0x7f89040cd090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1461e0ea7a6359d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f88eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f88eb7fe640 1 -- 172.21.15.122:0/1333534885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89040b7030 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f88eb7fe640 1 -- 172.21.15.122:0/1333534885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f89040051d0 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc020cc0 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f88eb7fe640 1 -- 172.21.15.122:0/1333534885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8904081050 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 10 client.17040 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f88b37e550 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f88eb7fe640 1 -- 172.21.15.122:0/1333534885 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc020cc0 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f88eb7fe640 1 -- 172.21.15.122:0/1333534885 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f890409b370 con 0x55f88b303370 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 10 client.17040 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 20 client.17040 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b37ee60 0x55f88b381280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 4 client.17040 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f88b268d70 con 0x55f88b37ee60 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.579+0000 7f890bfff640 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b37ee60 0x55f88b381280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.580+0000 7f890bfff640 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b37ee60 0x55f88b381280 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a096aedb8822d4 server_cookie=6463445650ab7213 in_seq=0 out_seq=0 2024-09-12T01:56:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.580+0000 7f88eb7fe640 10 client.17040 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.591+0000 7f88eb7fe640 1 -- 172.21.15.122:0/1333534885 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f88b268d70 con 0x55f88b37ee60 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.591+0000 7f88eb7fe640 10 client.17040 handle_command_reply: tid=0 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 client.17040 shutdown 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 msgr2=0x55f88b30dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 0x55f88b30dce0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f890400c760 tx=0x7f89040cd090 comp rx=0 tx=0).stop 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 shutdown_connections 2024-09-12T01:56:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f88b37ee60 0x55f88b381280 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f88b303370 0x55f88b30dce0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f88eb7fe640 0 client.17040 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88b2fe2e0 0x55f88b30ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 --2- 172.21.15.122:0/1333534885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f88b2fd9a0 0x55f88b30c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 >> 172.21.15.122:0/1333534885 conn(0x55f88b2f9010 msgr2=0x55f88b3079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 shutdown_connections 2024-09-12T01:56:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 1 -- 172.21.15.122:0/1333534885 wait complete. 2024-09-12T01:56:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.592+0000 7f89154c6740 20 client.17040 trim_cache size 0 max 16384 2024-09-12T01:56:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:56:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:56:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:56:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88f40a2cb0 msgr2=0x7f88f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88f40a2cb0 0x7f88f40a5170 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005fd0 tx=0x7f88fc014da0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 msgr2=0x7f890c19f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 0x7f890c19f370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8904009fd0 tx=0x7f890400ef10 comp rx=0 tx=0).stop 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 shutdown_connections 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88f40a2cb0 0x7f88f40a5170 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f890c1025f0 0x7f890c19f370 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c101c30 0x7f890c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 --2- 172.21.15.122:0/411986499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f890c100a40 0x7f890c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 >> 172.21.15.122:0/411986499 conn(0x7f890c0fc1e0 msgr2=0x7f890c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 shutdown_connections 2024-09-12T01:56:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:31.593+0000 7f891272a640 1 -- 172.21.15.122:0/411986499 wait complete. 2024-09-12T01:56:31.827 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:31 smithi122 ceph-mon[30925]: pgmap v636: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 2.0 MiB/s wr, 90 op/s 2024-09-12T01:56:31.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931674 9.97 MB/sec execute 409 sec latency 54.411 ms 2024-09-12T01:56:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 -- 172.21.15.122:0/3672083642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 msgr2=0x7f11bc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 --2- 172.21.15.122:0/3672083642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc100ed0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009930 tx=0x7f11a402a420 comp rx=0 tx=0).stop 2024-09-12T01:56:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 -- 172.21.15.122:0/3672083642 shutdown_connections 2024-09-12T01:56:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 --2- 172.21.15.122:0/3672083642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc101410 0x7f11bc10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 --2- 172.21.15.122:0/3672083642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc100ed0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 --2- 172.21.15.122:0/3672083642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc106a70 0x7f11bc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 -- 172.21.15.122:0/3672083642 >> 172.21.15.122:0/3672083642 conn(0x7f11bc0fc7d0 msgr2=0x7f11bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 -- 172.21.15.122:0/3672083642 shutdown_connections 2024-09-12T01:56:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.930+0000 7f11c2593640 1 -- 172.21.15.122:0/3672083642 wait complete. 2024-09-12T01:56:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 Processor -- start 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 -- start start 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 0x7f11bc1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc106a70 0x7f11bc19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc110e30 con 0x7f11bc100a70 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc110cb0 con 0x7f11bc101410 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c2593640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc110fb0 con 0x7f11bc106a70 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11b3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 0x7f11bc1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11b3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 0x7f11bc1978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:46788/0 (socket says 172.21.15.122:46788) 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11b3fff640 1 -- 172.21.15.122:0/1362186239 learned_addr learned my addr 172.21.15.122:0/1362186239 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc1973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38728/0 (socket says 172.21.15.122:38728) 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.931+0000 7f11c0b30640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc106a70 0x7f11bc19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b3fff640 1 -- 172.21.15.122:0/1362186239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc106a70 msgr2=0x7f11bc19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b3fff640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc106a70 0x7f11bc19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b3fff640 1 -- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 msgr2=0x7f11bc1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b3fff640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b3fff640 1 -- 172.21.15.122:0/1362186239 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19c360 con 0x7f11bc101410 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11bbfff640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b3fff640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 0x7f11bc1978f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009230 tx=0x7f11a4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b9ffb640 1 -- 172.21.15.122:0/1362186239 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a403f030 con 0x7f11bc101410 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b9ffb640 1 -- 172.21.15.122:0/1362186239 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11a4004440 con 0x7f11bc101410 2024-09-12T01:56:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19c5f0 con 0x7f11bc101410 2024-09-12T01:56:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11b9ffb640 1 -- 172.21.15.122:0/1362186239 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4035070 con 0x7f11bc101410 2024-09-12T01:56:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.932+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc06a790 con 0x7f11bc101410 2024-09-12T01:56:31.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.934+0000 7f11b9ffb640 1 -- 172.21.15.122:0/1362186239 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f11a4042030 con 0x7f11bc101410 2024-09-12T01:56:31.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 Processor -- start 2024-09-12T01:56:31.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 -- start start 2024-09-12T01:56:31.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11b9ffb640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f11a80a2b80 0x7f11a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c79462ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c7946394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c79463be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c794593160 con 0x55c79462e850 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c79453a720 con 0x55c794639a30 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c496c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c79458f120 con 0x55c79462f170 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11c0b30640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f11a80a2b80 0x7f11a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c79462ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.935+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c79462ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38738/0 (socket says 172.21.15.122:38738) 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 -- 172.21.15.122:0/1784258750 learned_addr learned my addr 172.21.15.122:0/1784258750 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11b3fff640 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c7946394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11b9ffb640 1 -- 172.21.15.122:0/1362186239 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11a40eeff0 con 0x7f11bc101410 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11c0b30640 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c79463be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 -- 172.21.15.122:0/1784258750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 msgr2=0x55c7946394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c7946394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 -- 172.21.15.122:0/1784258750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 msgr2=0x55c79463be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c79463be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 -- 172.21.15.122:0/1784258750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c794304ba0 con 0x55c79462e850 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11c0b30640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f11a80a2b80 0x7f11a80a5040 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4011bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11bbfff640 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c79462ec30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0114f0 tx=0x7f11ac0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5872394b845793a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11b27fc640 1 -- 172.21.15.122:0/1784258750 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac01e070 con 0x55c79462e850 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11b27fc640 1 -- 172.21.15.122:0/1784258750 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac011030 con 0x55c79462e850 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.936+0000 7f11b27fc640 1 -- 172.21.15.122:0/1784258750 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac010720 con 0x55c79462e850 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 -- 172.21.15.122:0/1784258750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 msgr2=0x55c79462ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c79462ec30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0114f0 tx=0x7f11ac0119b0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 -- 172.21.15.122:0/1784258750 shutdown_connections 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c79463be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c7946394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 --2- 172.21.15.122:0/1784258750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c79462ec30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 -- 172.21.15.122:0/1784258750 >> 172.21.15.122:0/1784258750 conn(0x55c79462ade0 msgr2=0x55c79462b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 -- 172.21.15.122:0/1784258750 shutdown_connections 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.937+0000 7f11c496c740 1 -- 172.21.15.122:0/1784258750 wait complete. 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 -- start start 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 Processor -- start 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 0x55c794636930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794636e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c794639a30 0x55c7946373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c79463c600 con 0x55c794639a30 2024-09-12T01:56:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c79451b900 con 0x55c79462e850 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c79463c780 con 0x55c79462f170 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11c496c740 10 client.0 _ll_register_callbacks cb 0x55c794519780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.938+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 0x55c794636930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 0x55c794636930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:46810/0 (socket says 172.21.15.122:46810) 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 -- 172.21.15.122:0/3019880155 learned_addr learned my addr 172.21.15.122:0/3019880155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11b3fff640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794636e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 -- 172.21.15.122:0/3019880155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 msgr2=0x55c794636e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794636e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11c0b30640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c794639a30 0x55c7946373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 -- 172.21.15.122:0/3019880155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c794639a30 msgr2=0x55c7946373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c794639a30 0x55c7946373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 -- 172.21.15.122:0/3019880155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c79455bc50 con 0x55c79462e850 2024-09-12T01:56:31.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11c0b30640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c794639a30 0x55c7946373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11bbfff640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 0x55c794636930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005270 tx=0x7f11ac0052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43377c9a52ce1e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f119b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f119b7fe640 1 -- 172.21.15.122:0/3019880155 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac026020 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f119b7fe640 1 -- 172.21.15.122:0/3019880155 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11ac0054c0 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.939+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac011030 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f119b7fe640 1 -- 172.21.15.122:0/3019880155 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11ac01e040 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 10 client.26255 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7946482c0 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f119b7fe640 1 -- 172.21.15.122:0/3019880155 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac011030 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f119b7fe640 1 -- 172.21.15.122:0/3019880155 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f11ac021070 con 0x55c79462e850 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 10 client.26255 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 10 client.26255 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:31.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 20 client.26255 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c794649720 0x55c79464bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 4 client.26255 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7945a0890 con 0x55c794649720 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.940+0000 7f11b3fff640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c794649720 0x55c79464bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.941+0000 7f11b3fff640 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c794649720 0x55c79464bb10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b98293297454d9 server_cookie=fcb032a4b4dce807 in_seq=0 out_seq=0 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.941+0000 7f119b7fe640 10 client.26255 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.941+0000 7f119b7fe640 1 -- 172.21.15.122:0/3019880155 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7945a0890 con 0x55c794649720 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.941+0000 7f119b7fe640 10 client.26255 handle_command_reply: tid=0 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 client.26255 shutdown 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 msgr2=0x55c794636930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 0x55c794636930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005270 tx=0x7f11ac0052a0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 shutdown_connections 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c794649720 0x55c79464bb10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c794639a30 0x55c7946373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f119b7fe640 0 client.26255 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794636e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 --2- 172.21.15.122:0/3019880155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c79462e850 0x55c794636930 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 >> 172.21.15.122:0/3019880155 conn(0x55c79462ade0 msgr2=0x55c794630ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 shutdown_connections 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 1 -- 172.21.15.122:0/3019880155 wait complete. 2024-09-12T01:56:31.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.942+0000 7f11c496c740 20 client.26255 trim_cache size 0 max 16384 2024-09-12T01:56:31.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 Processor -- start 2024-09-12T01:56:31.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 -- start start 2024-09-12T01:56:31.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c7946312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794651280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c7946517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1afed0 con 0x55c79462e850 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ad7a0 con 0x55c794639a30 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c496c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ab5d0 con 0x55c79462f170 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c7946312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c7946312b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38756/0 (socket says 172.21.15.122:38756) 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11bbfff640 1 -- 172.21.15.122:0/3096452416 learned_addr learned my addr 172.21.15.122:0/3096452416 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.963+0000 7f11c0b30640 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c7946517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11b3fff640 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794651280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11bbfff640 1 -- 172.21.15.122:0/3096452416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 msgr2=0x55c794651280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11bbfff640 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794651280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11bbfff640 1 -- 172.21.15.122:0/3096452416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 msgr2=0x55c7946517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11bbfff640 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c7946517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11bbfff640 1 -- 172.21.15.122:0/3096452416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c79462e5d0 con 0x55c79462e850 2024-09-12T01:56:31.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11b3fff640 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794651280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11bbfff640 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c7946312b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00fec0 tx=0x7f11ac004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719cad0203d3c55c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11b1ffb640 1 -- 172.21.15.122:0/3096452416 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac026030 con 0x55c79462e850 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11b1ffb640 1 -- 172.21.15.122:0/3096452416 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac004c10 con 0x55c79462e850 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.964+0000 7f11b1ffb640 1 -- 172.21.15.122:0/3096452416 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac01a450 con 0x55c79462e850 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 -- 172.21.15.122:0/3096452416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 msgr2=0x55c7946312b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c7946312b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00fec0 tx=0x7f11ac004a20 comp rx=0 tx=0).stop 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 -- 172.21.15.122:0/3096452416 shutdown_connections 2024-09-12T01:56:31.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c7946517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462f170 0x55c794651280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 --2- 172.21.15.122:0/3096452416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462e850 0x55c7946312b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 -- 172.21.15.122:0/3096452416 >> 172.21.15.122:0/3096452416 conn(0x55c79462ade0 msgr2=0x55c794635200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 -- 172.21.15.122:0/3096452416 shutdown_connections 2024-09-12T01:56:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 -- 172.21.15.122:0/3096452416 wait complete. 2024-09-12T01:56:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 -- start start 2024-09-12T01:56:31.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.965+0000 7f11c496c740 1 Processor -- start 2024-09-12T01:56:31.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 0x55c7946afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 0x55c7946b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c794641bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1abc60 con 0x55c79462f170 2024-09-12T01:56:31.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ad0d0 con 0x55c794639a30 2024-09-12T01:56:31.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ac330 con 0x55c79462e850 2024-09-12T01:56:31.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c496c740 10 client.0 _ll_register_callbacks cb 0x7f11ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:31.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 0x55c7946b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c0b30640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c794641bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 0x55c7946afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 0x55c7946afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34146/0 (socket says 172.21.15.122:34146) 2024-09-12T01:56:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11c0b30640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c794641bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:46822/0 (socket says 172.21.15.122:46822) 2024-09-12T01:56:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 -- 172.21.15.122:0/4122389340 learned_addr learned my addr 172.21.15.122:0/4122389340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 0x55c7946b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38762/0 (socket says 172.21.15.122:38762) 2024-09-12T01:56:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 -- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 msgr2=0x55c794641bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c794641bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 -- 172.21.15.122:0/4122389340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 msgr2=0x55c7946b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 0x55c7946b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 -- 172.21.15.122:0/4122389340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c794642400 con 0x55c79462e850 2024-09-12T01:56:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11b3fff640 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 0x55c7946b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:31.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f11bbfff640 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 0x55c7946afb50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0114f0 tx=0x7f11ac00f9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10afe261d3afe919 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:31.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.966+0000 7f119b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:56:31.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f119b7fe640 1 -- 172.21.15.122:0/4122389340 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac026030 con 0x55c79462e850 2024-09-12T01:56:31.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f119b7fe640 1 -- 172.21.15.122:0/4122389340 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f11ac005460 con 0x55c79462e850 2024-09-12T01:56:31.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac004c10 con 0x55c79462e850 2024-09-12T01:56:31.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f119b7fe640 1 -- 172.21.15.122:0/4122389340 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11ac005660 con 0x55c79462e850 2024-09-12T01:56:31.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 10 client.26224 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:31.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7946adc40 con 0x55c79462e850 2024-09-12T01:56:31.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f119b7fe640 1 -- 172.21.15.122:0/4122389340 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac004c10 con 0x55c79462e850 2024-09-12T01:56:31.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f119b7fe640 1 -- 172.21.15.122:0/4122389340 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f11ac034020 con 0x55c79462e850 2024-09-12T01:56:31.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 10 client.26224 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:31.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 10 client.26224 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:31.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 20 client.26224 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:31.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7946bd2a0 0x55c7946bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:31.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 4 client.26224 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T01:56:31.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.967+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55c79459abe0 con 0x55c7946bd2a0 2024-09-12T01:56:31.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.968+0000 7f11b3fff640 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7946bd2a0 0x55c7946bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:31.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.968+0000 7f119b7fe640 10 client.26224 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:31.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:31.968+0000 7f11b3fff640 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7946bd2a0 0x55c7946bf690 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6ebd60acecc77a server_cookie=fa35bfa1d49d1d9e in_seq=0 out_seq=0 2024-09-12T01:56:32.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f119b7fe640 1 -- 172.21.15.122:0/4122389340 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c79459abe0 con 0x55c7946bd2a0 2024-09-12T01:56:32.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f119b7fe640 10 client.26224 handle_command_reply: tid=0 2024-09-12T01:56:32.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 client.26224 shutdown 2024-09-12T01:56:32.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 msgr2=0x55c7946afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 0x55c7946afb50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0114f0 tx=0x7f11ac00f9b0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 shutdown_connections 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7946bd2a0 0x55c7946bf690 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c794639a30 0x55c794641bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c79462f170 0x55c7946b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f119b7fe640 0 client.26224 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 --2- 172.21.15.122:0/4122389340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c79462e850 0x55c7946afb50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 >> 172.21.15.122:0/4122389340 conn(0x55c79462ade0 msgr2=0x55c794629610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 shutdown_connections 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.037+0000 7f11c496c740 1 -- 172.21.15.122:0/4122389340 wait complete. 2024-09-12T01:56:32.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c496c740 20 client.26224 trim_cache size 0 max 16384 2024-09-12T01:56:32.040 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:56:32.040 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:56:32.040 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:56:32.040 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 42, 2024-09-12T01:56:32.040 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:56:32.040 INFO:teuthology.orchestra.run.smithi122.stdout: "cb486123": { 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 42, 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 142.6, 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:56:32.041 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:32.042 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:56:32.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f11a80a2b80 msgr2=0x7f11a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:32.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f11a80a2b80 0x7f11a80a5040 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4011bc0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 msgr2=0x7f11bc1978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 0x7f11bc1978f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009230 tx=0x7f11a4004040 comp rx=0 tx=0).stop 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 shutdown_connections 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f11a80a2b80 0x7f11a80a5040 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11bc106a70 0x7f11bc19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc101410 0x7f11bc1978f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 --2- 172.21.15.122:0/1362186239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc100a70 0x7f11bc1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 >> 172.21.15.122:0/1362186239 conn(0x7f11bc0fc7d0 msgr2=0x7f11bc0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 shutdown_connections 2024-09-12T01:56:32.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:32.038+0000 7f11c2593640 1 -- 172.21.15.122:0/1362186239 wait complete. 2024-09-12T01:56:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:31 smithi151 ceph-mon[35229]: pgmap v636: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 2.0 MiB/s wr, 90 op/s 2024-09-12T01:56:32.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:31 smithi183 ceph-mon[35127]: pgmap v636: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 2.0 MiB/s wr, 90 op/s 2024-09-12T01:56:32.819 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cb486123', quiesce took 3.4 seconds. Will release after: 19.150000000000002 2024-09-12T01:56:32.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.95 MB/sec execute 410 sec latency 817.610 ms 2024-09-12T01:56:33.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.92 MB/sec execute 411 sec latency 1817.707 ms 2024-09-12T01:56:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:33 smithi122 ceph-mon[30925]: pgmap v637: 97 pgs: 97 active+clean; 264 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 111 op/s 2024-09-12T01:56:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:33 smithi151 ceph-mon[35229]: pgmap v637: 97 pgs: 97 active+clean; 264 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 111 op/s 2024-09-12T01:56:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:33 smithi183 ceph-mon[35127]: pgmap v637: 97 pgs: 97 active+clean; 264 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 111 op/s 2024-09-12T01:56:34.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.90 MB/sec execute 412 sec latency 2817.799 ms 2024-09-12T01:56:35.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.88 MB/sec execute 413 sec latency 3817.870 ms 2024-09-12T01:56:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:35 smithi122 ceph-mon[30925]: pgmap v638: 97 pgs: 97 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 2.4 MiB/s wr, 142 op/s 2024-09-12T01:56:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:35 smithi151 ceph-mon[35229]: pgmap v638: 97 pgs: 97 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 2.4 MiB/s wr, 142 op/s 2024-09-12T01:56:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:35 smithi183 ceph-mon[35127]: pgmap v638: 97 pgs: 97 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 2.4 MiB/s wr, 142 op/s 2024-09-12T01:56:36.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.85 MB/sec execute 414 sec latency 4817.965 ms 2024-09-12T01:56:37.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.83 MB/sec execute 415 sec latency 5818.044 ms 2024-09-12T01:56:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:37 smithi122 ceph-mon[30925]: pgmap v639: 97 pgs: 97 active+clean; 267 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 3.0 MiB/s wr, 152 op/s 2024-09-12T01:56:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:37 smithi151 ceph-mon[35229]: pgmap v639: 97 pgs: 97 active+clean; 267 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 3.0 MiB/s wr, 152 op/s 2024-09-12T01:56:38.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:37 smithi183 ceph-mon[35127]: pgmap v639: 97 pgs: 97 active+clean; 267 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 3.0 MiB/s wr, 152 op/s 2024-09-12T01:56:38.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.81 MB/sec execute 416 sec latency 6818.135 ms 2024-09-12T01:56:39.853 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.78 MB/sec execute 417 sec latency 7818.238 ms 2024-09-12T01:56:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:39 smithi122 ceph-mon[30925]: pgmap v640: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 4.0 MiB/s wr, 164 op/s 2024-09-12T01:56:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:39 smithi151 ceph-mon[35229]: pgmap v640: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 4.0 MiB/s wr, 164 op/s 2024-09-12T01:56:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:39 smithi183 ceph-mon[35127]: pgmap v640: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 4.0 MiB/s wr, 164 op/s 2024-09-12T01:56:40.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.76 MB/sec execute 418 sec latency 8818.325 ms 2024-09-12T01:56:41.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.74 MB/sec execute 419 sec latency 9818.425 ms 2024-09-12T01:56:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:41 smithi122 ceph-mon[30925]: pgmap v641: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 4.2 MiB/s wr, 158 op/s 2024-09-12T01:56:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:41 smithi151 ceph-mon[35229]: pgmap v641: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 4.2 MiB/s wr, 158 op/s 2024-09-12T01:56:42.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:41 smithi183 ceph-mon[35127]: pgmap v641: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 4.2 MiB/s wr, 158 op/s 2024-09-12T01:56:42.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.71 MB/sec execute 420 sec latency 10818.519 ms 2024-09-12T01:56:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:42 smithi151 ceph-mon[35229]: pgmap v642: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 4.0 MiB/s wr, 147 op/s 2024-09-12T01:56:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:42 smithi183 ceph-mon[35127]: pgmap v642: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 4.0 MiB/s wr, 147 op/s 2024-09-12T01:56:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:42 smithi122 ceph-mon[30925]: pgmap v642: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 4.0 MiB/s wr, 147 op/s 2024-09-12T01:56:43.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.69 MB/sec execute 421 sec latency 11818.638 ms 2024-09-12T01:56:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 -- 172.21.15.122:0/1130917904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0102030 msgr2=0x7f10e0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 --2- 172.21.15.122:0/1130917904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0102030 0x7f10e0102430 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009900 tx=0x7f10d002a410 comp rx=0 tx=0).stop 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 -- 172.21.15.122:0/1130917904 shutdown_connections 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 --2- 172.21.15.122:0/1130917904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0101690 0x7f10e0108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 --2- 172.21.15.122:0/1130917904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0100cd0 0x7f10e0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 --2- 172.21.15.122:0/1130917904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0102030 0x7f10e0102430 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 -- 172.21.15.122:0/1130917904 >> 172.21.15.122:0/1130917904 conn(0x7f10e00fb970 msgr2=0x7f10e00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 -- 172.21.15.122:0/1130917904 shutdown_connections 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.689+0000 7f10e69d8640 1 -- 172.21.15.122:0/1130917904 wait complete. 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 Processor -- start 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 -- start start 2024-09-12T01:56:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0100cd0 0x7f10e0104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 0x7f10e01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0102030 0x7f10e0103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10e010c6c0 con 0x7f10e0101690 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e010c540 con 0x7f10e0100cd0 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10e69d8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10e010c840 con 0x7f10e0102030 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 0x7f10e01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 0x7f10e01030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55424/0 (socket says 172.21.15.122:55424) 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10df7fe640 1 -- 172.21.15.122:0/3638738657 learned_addr learned my addr 172.21.15.122:0/3638738657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.690+0000 7f10dffff640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0100cd0 0x7f10e0104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10e4f75640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0102030 0x7f10e0103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10df7fe640 1 -- 172.21.15.122:0/3638738657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0102030 msgr2=0x7f10e0103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10df7fe640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0102030 0x7f10e0103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10df7fe640 1 -- 172.21.15.122:0/3638738657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0100cd0 msgr2=0x7f10e0104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10df7fe640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0100cd0 0x7f10e0104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10df7fe640 1 -- 172.21.15.122:0/3638738657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e0103e90 con 0x7f10e0101690 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10e4f75640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0102030 0x7f10e0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10df7fe640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 0x7f10e01030c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f10d400d950 tx=0x7f10d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10dd7fa640 1 -- 172.21.15.122:0/3638738657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4017070 con 0x7f10e0101690 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10dd7fa640 1 -- 172.21.15.122:0/3638738657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d4009c30 con 0x7f10e0101690 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e01a1f30 con 0x7f10e0101690 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10dd7fa640 1 -- 172.21.15.122:0/3638738657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d400f6b0 con 0x7f10e0101690 2024-09-12T01:56:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.691+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e01a2390 con 0x7f10e0101690 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.693+0000 7f10dd7fa640 1 -- 172.21.15.122:0/3638738657 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f10d40109d0 con 0x7f10e0101690 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 Processor -- start 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 -- start start 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 0x55b0586a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10dd7fa640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10c80a2c30 0x7f10c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b0586ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b0586072f0 con 0x55b0586a39a0 2024-09-12T01:56:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b0585af270 con 0x55b0586a9370 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e984b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0586032b0 con 0x55b0586a42e0 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 0x55b0586a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 0x55b0586a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55438/0 (socket says 172.21.15.122:55438) 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10dffff640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10c80a2c30 0x7f10c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 -- 172.21.15.122:0/1105894645 learned_addr learned my addr 172.21.15.122:0/1105894645 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10df7fe640 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10dd7fa640 1 -- 172.21.15.122:0/3638738657 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f10d40ced00 con 0x7f10e0101690 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b0586ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 -- 172.21.15.122:0/1105894645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 msgr2=0x55b0586a8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586a8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 -- 172.21.15.122:0/1105894645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 msgr2=0x55b0586ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b0586ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10e4f75640 1 -- 172.21.15.122:0/1105894645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0585ae340 con 0x55b0586a39a0 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.694+0000 7f10df7fe640 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586a8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10e4f75640 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 0x55b0586a3da0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f10d8013960 tx=0x7f10d8009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848f1aa30c016fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10dffff640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10c80a2c30 0x7f10c80a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006010 tx=0x7f10d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10c67fc640 1 -- 172.21.15.122:0/1105894645 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8019030 con 0x55b0586a39a0 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10c67fc640 1 -- 172.21.15.122:0/1105894645 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8003e50 con 0x55b0586a39a0 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10c67fc640 1 -- 172.21.15.122:0/1105894645 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8004380 con 0x55b0586a39a0 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10e984b740 1 -- 172.21.15.122:0/1105894645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 msgr2=0x55b0586a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.695+0000 7f10e984b740 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 0x55b0586a3da0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f10d8013960 tx=0x7f10d8009a40 comp rx=0 tx=0).stop 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- 172.21.15.122:0/1105894645 shutdown_connections 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b0586ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586a8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 --2- 172.21.15.122:0/1105894645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a39a0 0x55b0586a3da0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- 172.21.15.122:0/1105894645 >> 172.21.15.122:0/1105894645 conn(0x55b05869f010 msgr2=0x55b05869f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- 172.21.15.122:0/1105894645 shutdown_connections 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- 172.21.15.122:0/1105894645 wait complete. 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- start start 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 Processor -- start 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b0586adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a42e0 0x55b0586ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b0586ac340 con 0x55b0586a9370 2024-09-12T01:56:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b058590290 con 0x55b0586a42e0 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0586ac4c0 con 0x55b0586a39a0 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.696+0000 7f10e984b740 10 client.0 _ll_register_callbacks cb 0x55b0585c1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55444/0 (socket says 172.21.15.122:55444) 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 -- 172.21.15.122:0/461509407 learned_addr learned my addr 172.21.15.122:0/461509407 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10dffff640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b0586adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10df7fe640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a42e0 0x55b0586ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 -- 172.21.15.122:0/461509407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 msgr2=0x55b0586adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b0586adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 -- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a42e0 msgr2=0x55b0586ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a42e0 0x55b0586ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 -- 172.21.15.122:0/461509407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b05831a210 con 0x55b0586a9370 2024-09-12T01:56:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e4f75640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b5140 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f10d8012010 tx=0x7f10d8005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edda23f2e177835b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10bf7fe640 1 -- 172.21.15.122:0/461509407 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8019030 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10bf7fe640 1 -- 172.21.15.122:0/461509407 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f10d8003b70 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d8003e50 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10bf7fe640 1 -- 172.21.15.122:0/461509407 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10d8004a40 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e984b740 10 client.17067 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.697+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0583e6620 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10bf7fe640 1 -- 172.21.15.122:0/461509407 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8003e50 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10bf7fe640 1 -- 172.21.15.122:0/461509407 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f10d8026020 con 0x55b0586a9370 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10e984b740 10 client.17067 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10e984b740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10e984b740 20 client.17067 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10e984b740 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0586b9510 0x55b0586bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10e984b740 4 client.17067 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b058614a20 con 0x55b0586b9510 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.698+0000 7f10dffff640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0586b9510 0x55b0586bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.699+0000 7f10bf7fe640 10 client.17067 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.699+0000 7f10dffff640 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0586b9510 0x55b0586bb900 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a6e66cd62b27fa server_cookie=21fd972e97ab66d in_seq=0 out_seq=0 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.699+0000 7f10bf7fe640 1 -- 172.21.15.122:0/461509407 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b058614a20 con 0x55b0586b9510 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.699+0000 7f10bf7fe640 10 client.17067 handle_command_reply: tid=0 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.699+0000 7f10e984b740 1 client.17067 shutdown 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 msgr2=0x55b0586b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b5140 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f10d8012010 tx=0x7f10d8005af0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 shutdown_connections 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0586b9510 0x55b0586bb900 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b5140 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a42e0 0x55b0586ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 --2- 172.21.15.122:0/461509407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b0586adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10bf7fe640 0 client.17067 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 >> 172.21.15.122:0/461509407 conn(0x55b05869f010 msgr2=0x55b0586a0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 shutdown_connections 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 1 -- 172.21.15.122:0/461509407 wait complete. 2024-09-12T01:56:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.700+0000 7f10e984b740 20 client.17067 trim_cache size 0 max 16384 2024-09-12T01:56:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 Processor -- start 2024-09-12T01:56:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 -- start start 2024-09-12T01:56:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b05869f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 0x55b0586b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b05869f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01adb40 con 0x55b0586a42e0 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01ab410 con 0x55b0586a9370 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10e984b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01a9240 con 0x55b0586a39a0 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 0x55b0586b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 0x55b0586b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55460/0 (socket says 172.21.15.122:55460) 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.715+0000 7f10df7fe640 1 -- 172.21.15.122:0/687431146 learned_addr learned my addr 172.21.15.122:0/687431146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10e4f75640 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b05869f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10dffff640 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b05869f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10df7fe640 1 -- 172.21.15.122:0/687431146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 msgr2=0x55b05869f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10df7fe640 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b05869f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10df7fe640 1 -- 172.21.15.122:0/687431146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 msgr2=0x55b05869f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10df7fe640 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b05869f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10df7fe640 1 -- 172.21.15.122:0/687431146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0586abc10 con 0x55b0586a42e0 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10dffff640 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b05869f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10df7fe640 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 0x55b0586b5060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f10d40a3c60 tx=0x7f10d4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3ace77aca7733f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10c5ffb640 1 -- 172.21.15.122:0/687431146 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4020070 con 0x55b0586a42e0 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10c5ffb640 1 -- 172.21.15.122:0/687431146 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d405ee50 con 0x55b0586a42e0 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.716+0000 7f10c5ffb640 1 -- 172.21.15.122:0/687431146 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d409f8d0 con 0x55b0586a42e0 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 -- 172.21.15.122:0/687431146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 msgr2=0x55b0586b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 0x55b0586b5060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f10d40a3c60 tx=0x7f10d4010d70 comp rx=0 tx=0).stop 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 -- 172.21.15.122:0/687431146 shutdown_connections 2024-09-12T01:56:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a9370 0x55b05869f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a42e0 0x55b0586b5060 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 --2- 172.21.15.122:0/687431146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a39a0 0x55b05869f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 -- 172.21.15.122:0/687431146 >> 172.21.15.122:0/687431146 conn(0x55b05869f010 msgr2=0x55b0586adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 -- 172.21.15.122:0/687431146 shutdown_connections 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 -- 172.21.15.122:0/687431146 wait complete. 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.717+0000 7f10e984b740 1 -- start start 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 Processor -- start 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a39a0 0x55b0586b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01a98d0 con 0x55b0586a9370 2024-09-12T01:56:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01aad40 con 0x55b0586a39a0 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10e01a9fa0 con 0x55b0586a42e0 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e984b740 10 client.0 _ll_register_callbacks cb 0x7f10d80137c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55474/0 (socket says 172.21.15.122:55474) 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 -- 172.21.15.122:0/2108305860 learned_addr learned my addr 172.21.15.122:0/2108305860 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10dffff640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a39a0 0x55b0586b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10df7fe640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 -- 172.21.15.122:0/2108305860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 msgr2=0x55b0586b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 -- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a39a0 msgr2=0x55b0586b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a39a0 0x55b0586b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 -- 172.21.15.122:0/2108305860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0586b40c0 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10df7fe640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10e4f75640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b3ab0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009ac0 tx=0x7f10d8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f366591dbd7191 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.718+0000 7f10bf7fe640 1 -- 172.21.15.122:0/2108305860 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8019070 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10bf7fe640 1 -- 172.21.15.122:0/2108305860 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f10d801e670 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d405ee50 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10bf7fe640 1 -- 172.21.15.122:0/2108305860 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10d8009e90 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 10 client.17073 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0587241e0 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10bf7fe640 1 -- 172.21.15.122:0/2108305860 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d405ee50 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10bf7fe640 1 -- 172.21.15.122:0/2108305860 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f10d8026040 con 0x55b0586a9370 2024-09-12T01:56:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 10 client.17073 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 20 client.17073 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b058724a50 0x55b058726e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 4 client.17073 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.719+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b05860ed70 con 0x55b058724a50 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.720+0000 7f10dffff640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b058724a50 0x55b058726e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.720+0000 7f10bf7fe640 10 client.17073 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.720+0000 7f10dffff640 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b058724a50 0x55b058726e70 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bd368de99e2aef server_cookie=c5e885f231c4e972 in_seq=0 out_seq=0 2024-09-12T01:56:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10bf7fe640 1 -- 172.21.15.122:0/2108305860 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b05860ed70 con 0x55b058724a50 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10bf7fe640 10 client.17073 handle_command_reply: tid=0 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 client.17073 shutdown 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 msgr2=0x55b0586b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b3ab0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009ac0 tx=0x7f10d8011ce0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 shutdown_connections 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b058724a50 0x55b058726e70 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0586a9370 0x55b0586b3ab0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0586a42e0 0x55b0586b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10bf7fe640 0 client.17073 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 --2- 172.21.15.122:0/2108305860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0586a39a0 0x55b0586b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 >> 172.21.15.122:0/2108305860 conn(0x55b05869f010 msgr2=0x55b05869d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 shutdown_connections 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 1 -- 172.21.15.122:0/2108305860 wait complete. 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.733+0000 7f10e984b740 20 client.17073 trim_cache size 0 max 16384 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:56:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:56:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10c80a2c30 msgr2=0x7f10c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10c80a2c30 0x7f10c80a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006010 tx=0x7f10d0000bc0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 msgr2=0x7f10e01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 0x7f10e01030c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f10d400d950 tx=0x7f10d400de10 comp rx=0 tx=0).stop 2024-09-12T01:56:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 shutdown_connections 2024-09-12T01:56:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f10c80a2c30 0x7f10c80a50f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10e0102030 0x7f10e0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e0101690 0x7f10e01030c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 --2- 172.21.15.122:0/3638738657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0100cd0 0x7f10e0104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 >> 172.21.15.122:0/3638738657 conn(0x7f10e00fb970 msgr2=0x7f10e00fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 shutdown_connections 2024-09-12T01:56:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:44.734+0000 7f10e69d8640 1 -- 172.21.15.122:0/3638738657 wait complete. 2024-09-12T01:56:44.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.67 MB/sec execute 422 sec latency 12818.718 ms 2024-09-12T01:56:45.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.64 MB/sec execute 423 sec latency 13818.812 ms 2024-09-12T01:56:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:45 smithi183 ceph-mon[35127]: pgmap v643: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 3.3 MiB/s wr, 114 op/s 2024-09-12T01:56:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:45 smithi122 ceph-mon[30925]: pgmap v643: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 3.3 MiB/s wr, 114 op/s 2024-09-12T01:56:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:45 smithi151 ceph-mon[35229]: pgmap v643: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 3.3 MiB/s wr, 114 op/s 2024-09-12T01:56:46.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.62 MB/sec execute 424 sec latency 14818.927 ms 2024-09-12T01:56:47.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.60 MB/sec execute 425 sec latency 15819.020 ms 2024-09-12T01:56:47.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:47 smithi183 ceph-mon[35127]: pgmap v644: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-09-12T01:56:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:47 smithi122 ceph-mon[30925]: pgmap v644: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-09-12T01:56:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:47 smithi151 ceph-mon[35229]: pgmap v644: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-09-12T01:56:48.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.58 MB/sec execute 426 sec latency 16819.139 ms 2024-09-12T01:56:49.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.55 MB/sec execute 427 sec latency 17819.237 ms 2024-09-12T01:56:49.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:49 smithi183 ceph-mon[35127]: pgmap v645: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.1 MiB/s wr, 49 op/s 2024-09-12T01:56:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:49 smithi122 ceph-mon[30925]: pgmap v645: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.1 MiB/s wr, 49 op/s 2024-09-12T01:56:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:49 smithi151 ceph-mon[35229]: pgmap v645: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 2.1 MiB/s wr, 49 op/s 2024-09-12T01:56:50.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.53 MB/sec execute 428 sec latency 18819.326 ms 2024-09-12T01:56:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:50 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:50.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:50 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:50 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:56:51.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.51 MB/sec execute 429 sec latency 19819.354 ms 2024-09-12T01:56:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: pgmap v646: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 873 KiB/s wr, 25 op/s 2024-09-12T01:56:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:51 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.970 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: pgmap v646: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 873 KiB/s wr, 25 op/s 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:51 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: pgmap v646: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 873 KiB/s wr, 25 op/s 2024-09-12T01:56:52.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:51 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.217 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 -- 172.21.15.122:0/3503584104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500102d90 msgr2=0x7fe500103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 --2- 172.21.15.122:0/3503584104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500102d90 0x7fe500103170 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009930 tx=0x7fe4f002a420 comp rx=0 tx=0).stop 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 -- 172.21.15.122:0/3503584104 shutdown_connections 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 --2- 172.21.15.122:0/3503584104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500108290 0x7fe50010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 --2- 172.21.15.122:0/3503584104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500103740 0x7fe500107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 --2- 172.21.15.122:0/3503584104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500102d90 0x7fe500103170 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 -- 172.21.15.122:0/3503584104 >> 172.21.15.122:0/3503584104 conn(0x7fe5000fc7d0 msgr2=0x7fe5000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 -- 172.21.15.122:0/3503584104 shutdown_connections 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 -- 172.21.15.122:0/3503584104 wait complete. 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 Processor -- start 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.686+0000 7fe506866640 1 -- start start 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe506866640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500102d90 0x7fe500198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe506866640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 0x7fe500198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe506866640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500108290 0x7fe50019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe506866640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001120d0 con 0x7fe500103740 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe506866640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe500111f50 con 0x7fe500108290 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe506866640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112250 con 0x7fe500102d90 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 0x7fe500198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:52.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 0x7fe500198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52164/0 (socket says 172.21.15.122:52164) 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 -- 172.21.15.122:0/3689152530 learned_addr learned my addr 172.21.15.122:0/3689152530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe504e03640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500108290 0x7fe50019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4fffff640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500102d90 0x7fe500198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 -- 172.21.15.122:0/3689152530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500102d90 msgr2=0x7fe500198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500102d90 0x7fe500198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 -- 172.21.15.122:0/3689152530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500108290 msgr2=0x7fe50019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500108290 0x7fe50019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 -- 172.21.15.122:0/3689152530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50019d520 con 0x7fe500103740 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe504e03640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500108290 0x7fe50019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.687+0000 7fe4ff7fe640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 0x7fe500198ab0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c970 tx=0x7fe4e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.688+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8014070 con 0x7fe500103740 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.688+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50019d810 con 0x7fe500103740 2024-09-12T01:56:52.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.688+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8004440 con 0x7fe500103740 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.688+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800f450 con 0x7fe500103740 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.688+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a50f0 con 0x7fe500103740 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.689+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50006a7f0 con 0x7fe500103740 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.690+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe4e8017070 con 0x7fe500103740 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.690+0000 7fe4fd7fa640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe4ec0a2b80 0x7fe4ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.691+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe4e80cef50 con 0x7fe500103740 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.691+0000 7fe4fffff640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe4ec0a2b80 0x7fe4ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.691+0000 7fe4fffff640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe4ec0a2b80 0x7fe4ec0a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0000c00 tx=0x7fe4f0031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:52.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.695+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4e801e020 con 0x7fe500103740 2024-09-12T01:56:52.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.49 MB/sec execute 430 sec latency 20819.384 ms 2024-09-12T01:56:52.874 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:52.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:52.875 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.904+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe50010a2a0 con 0x7fe500103740 2024-09-12T01:56:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.905+0000 7fe4fd7fa640 1 -- 172.21.15.122:0/3689152530 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fe4e809beb0 con 0x7fe500103740 2024-09-12T01:56:52.905 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:56:52.906 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:56:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe4ec0a2b80 msgr2=0x7fe4ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe4ec0a2b80 0x7fe4ec0a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0000c00 tx=0x7fe4f0031490 comp rx=0 tx=0).stop 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 msgr2=0x7fe500198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 0x7fe500198ab0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c970 tx=0x7fe4e800ce30 comp rx=0 tx=0).stop 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 shutdown_connections 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe4ec0a2b80 0x7fe4ec0a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe500108290 0x7fe50019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe500103740 0x7fe500198ab0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 --2- 172.21.15.122:0/3689152530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500102d90 0x7fe500198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.907+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 >> 172.21.15.122:0/3689152530 conn(0x7fe5000fc7d0 msgr2=0x7fe500106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.908+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 shutdown_connections 2024-09-12T01:56:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:52.908+0000 7fe506866640 1 -- 172.21.15.122:0/3689152530 wait complete. 2024-09-12T01:56:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:53.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:56:53.017 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:56:53.017 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:56:53.804 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id cb486123 --release --await' 2024-09-12T01:56:53.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id cb486123 --release --await 2024-09-12T01:56:53.854 INFO:tasks.workunit.client.0.smithi122.stdout: 1 931987 9.46 MB/sec execute 431 sec latency 21819.411 ms 2024-09-12T01:56:53.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:53 smithi183 ceph-mon[35127]: pgmap v647: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 12 op/s 2024-09-12T01:56:53.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:53 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3689152530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:53 smithi122 ceph-mon[30925]: pgmap v647: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 12 op/s 2024-09-12T01:56:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:53 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3689152530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:56:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:53 smithi151 ceph-mon[35229]: pgmap v647: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 12 op/s 2024-09-12T01:56:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:53 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3689152530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:56:54.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 -- 172.21.15.122:0/3101289668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a010c170 msgr2=0x7f66a010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 --2- 172.21.15.122:0/3101289668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a010c170 0x7f66a010e610 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6698009ca0 tx=0x7f669802a950 comp rx=0 tx=0).stop 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 -- 172.21.15.122:0/3101289668 shutdown_connections 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 --2- 172.21.15.122:0/3101289668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a010c170 0x7f66a010e610 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 --2- 172.21.15.122:0/3101289668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0103660 0x7f66a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 --2- 172.21.15.122:0/3101289668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a0102d40 0x7f66a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 -- 172.21.15.122:0/3101289668 >> 172.21.15.122:0/3101289668 conn(0x7f66a00fc7a0 msgr2=0x7f66a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 -- 172.21.15.122:0/3101289668 shutdown_connections 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.558+0000 7f66a74dc640 1 -- 172.21.15.122:0/3101289668 wait complete. 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 Processor -- start 2024-09-12T01:56:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 -- start start 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a0102d40 0x7f66a0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 0x7f66a0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a010c170 0x7f66a019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0112030 con 0x7f66a0103660 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0111eb0 con 0x7f66a0102d40 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a74dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01121b0 con 0x7f66a010c170 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a4a77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 0x7f66a0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a4a77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 0x7f66a0198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52174/0 (socket says 172.21.15.122:52174) 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a4a77640 1 -- 172.21.15.122:0/2739406809 learned_addr learned my addr 172.21.15.122:0/2739406809 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.559+0000 7f66a5278640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a0102d40 0x7f66a0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a4a77640 1 -- 172.21.15.122:0/2739406809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a010c170 msgr2=0x7f66a019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a4a77640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a010c170 0x7f66a019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a4a77640 1 -- 172.21.15.122:0/2739406809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a0102d40 msgr2=0x7f66a0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a4a77640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a0102d40 0x7f66a0198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a4a77640 1 -- 172.21.15.122:0/2739406809 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a019d5b0 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a4a77640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 0x7f66a0198b40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f669400b3f0 tx=0x7f669400b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66927fc640 1 -- 172.21.15.122:0/2739406809 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694015020 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66927fc640 1 -- 172.21.15.122:0/2739406809 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694011d10 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a019d8a0 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66927fc640 1 -- 172.21.15.122:0/2739406809 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694010e60 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.560+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a5180 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.562+0000 7f66927fc640 1 -- 172.21.15.122:0/2739406809 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6694018020 con 0x7f66a0103660 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 Processor -- start 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 -- start start 2024-09-12T01:56:54.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db15440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db22630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f72da7a620 con 0x55f72db15060 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f72da21e50 con 0x55f72db15980 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a98b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f72da765e0 con 0x55f72db20240 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db15440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db15440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52176/0 (socket says 172.21.15.122:52176) 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 -- 172.21.15.122:0/1386244329 learned_addr learned my addr 172.21.15.122:0/1386244329 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 -- 172.21.15.122:0/1386244329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 msgr2=0x55f72db22630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db22630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 -- 172.21.15.122:0/1386244329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 msgr2=0x55f72db1fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 -- 172.21.15.122:0/1386244329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72da12d40 con 0x55f72db15060 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66927fc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f668c0a2c00 0x7f668c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.563+0000 7f66a5a79640 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db15440 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f669802ae70 tx=0x7f6698009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d44ccbeaaa94be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f66a5278640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f668c0a2c00 0x7f668c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f66927fc640 1 -- 172.21.15.122:0/2739406809 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f66940ce7b0 con 0x7f66a0103660 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f668b7fe640 1 -- 172.21.15.122:0/1386244329 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698034040 con 0x55f72db15060 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f668b7fe640 1 -- 172.21.15.122:0/1386244329 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698003970 con 0x55f72db15060 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f668b7fe640 1 -- 172.21.15.122:0/1386244329 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004450 con 0x55f72db15060 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f66a98b5740 1 -- 172.21.15.122:0/1386244329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 msgr2=0x55f72db15440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f66a5278640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f668c0a2c00 0x7f668c0a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f669c006aa0 tx=0x7f669c006a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.564+0000 7f66a98b5740 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db15440 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f669802ae70 tx=0x7f6698009940 comp rx=0 tx=0).stop 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- 172.21.15.122:0/1386244329 shutdown_connections 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db22630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 --2- 172.21.15.122:0/1386244329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db15440 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- 172.21.15.122:0/1386244329 >> 172.21.15.122:0/1386244329 conn(0x55f72db115f0 msgr2=0x55f72db119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- 172.21.15.122:0/1386244329 shutdown_connections 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- 172.21.15.122:0/1386244329 wait complete. 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- start start 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 Processor -- start 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db1dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f72db22e10 con 0x55f72db15060 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f72da02f10 con 0x55f72db15980 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f72db22f90 con 0x55f72db20240 2024-09-12T01:56:54.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a98b5740 10 client.0 _ll_register_callbacks cb 0x55f72da26db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.565+0000 7f66a5278640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db1d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52184/0 (socket says 172.21.15.122:52184) 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 -- 172.21.15.122:0/3266328669 learned_addr learned my addr 172.21.15.122:0/3266328669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5a79640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db1dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a4a77640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 -- 172.21.15.122:0/3266328669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 msgr2=0x55f72db1dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db1dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 -- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 msgr2=0x55f72db1d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 -- 172.21.15.122:0/3266328669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72d789580 con 0x55f72db15060 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a4a77640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:54.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a5278640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db1d150 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f669c015700 tx=0x7f669c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4716ee42c88e363d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f6688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f6688ff9640 1 -- 172.21.15.122:0/3266328669 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c023020 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f6688ff9640 1 -- 172.21.15.122:0/3266328669 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f669c013650 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6698003970 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f6688ff9640 1 -- 172.21.15.122:0/3266328669 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f669c01b040 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a98b5740 10 client.17091 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f72d853960 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.566+0000 7f6688ff9640 1 -- 172.21.15.122:0/3266328669 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698003970 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f6688ff9640 1 -- 172.21.15.122:0/3266328669 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f669c01e030 con 0x55f72db15060 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a98b5740 10 client.17091 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a98b5740 10 client.17091 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:54.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a98b5740 20 client.17091 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a98b5740 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db2fe40 0x55f72db32230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a98b5740 4 client.17091 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72da87b90 con 0x55f72db2fe40 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a4a77640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db2fe40 0x55f72db32230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f6688ff9640 10 client.17091 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.567+0000 7f66a4a77640 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db2fe40 0x55f72db32230 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56f3c093de2ac7c server_cookie=3536e48778d01f8d in_seq=0 out_seq=0 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.568+0000 7f6688ff9640 1 -- 172.21.15.122:0/3266328669 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f72da87b90 con 0x55f72db2fe40 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.568+0000 7f6688ff9640 10 client.17091 handle_command_reply: tid=0 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.568+0000 7f66a98b5740 1 client.17091 shutdown 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.568+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 msgr2=0x55f72db1d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.568+0000 7f66a98b5740 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db1d150 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f669c015700 tx=0x7f669c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 shutdown_connections 2024-09-12T01:56:54.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db2fe40 0x55f72db32230 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db20240 0x55f72db1dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db15980 0x55f72db1d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f6688ff9640 0 client.17091 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 --2- 172.21.15.122:0/3266328669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15060 0x55f72db1d150 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 >> 172.21.15.122:0/3266328669 conn(0x55f72db115f0 msgr2=0x55f72db17320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 shutdown_connections 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 1 -- 172.21.15.122:0/3266328669 wait complete. 2024-09-12T01:56:54.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.569+0000 7f66a98b5740 20 client.17091 trim_cache size 0 max 16384 2024-09-12T01:56:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.585+0000 7f66a98b5740 1 Processor -- start 2024-09-12T01:56:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.585+0000 7f66a98b5740 1 -- start start 2024-09-12T01:56:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db15980 0x55f72db105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db37b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 0x55f72db281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a98b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01affa0 con 0x55f72db38040 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a98b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ad870 con 0x55f72db20240 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a98b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ab6a0 con 0x55f72db15980 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 0x55f72db281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 0x55f72db281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52200/0 (socket says 172.21.15.122:52200) 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 -- 172.21.15.122:0/2244940289 learned_addr learned my addr 172.21.15.122:0/2244940289 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 -- 172.21.15.122:0/2244940289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db15980 msgr2=0x55f72db105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db15980 0x55f72db105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 -- 172.21.15.122:0/2244940289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 msgr2=0x55f72db37b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db37b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 -- 172.21.15.122:0/2244940289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72db157b0 con 0x55f72db38040 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f66a5a79640 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 0x55f72db281d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f669802ae70 tx=0x7f669802fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41139a21d7c7c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f668affd640 1 -- 172.21.15.122:0/2244940289 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698034030 con 0x55f72db38040 2024-09-12T01:56:54.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f668affd640 1 -- 172.21.15.122:0/2244940289 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669802f5a0 con 0x55f72db38040 2024-09-12T01:56:54.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.586+0000 7f668affd640 1 -- 172.21.15.122:0/2244940289 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698048040 con 0x55f72db38040 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 -- 172.21.15.122:0/2244940289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 msgr2=0x55f72db281d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 0x55f72db281d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f669802ae70 tx=0x7f669802fbb0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 -- 172.21.15.122:0/2244940289 shutdown_connections 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db38040 0x55f72db281d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db37b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 --2- 172.21.15.122:0/2244940289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db15980 0x55f72db105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 -- 172.21.15.122:0/2244940289 >> 172.21.15.122:0/2244940289 conn(0x55f72db115f0 msgr2=0x55f72db1f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 -- 172.21.15.122:0/2244940289 shutdown_connections 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 -- 172.21.15.122:0/2244940289 wait complete. 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 -- start start 2024-09-12T01:56:54.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.587+0000 7f66a98b5740 1 Processor -- start 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 0x55f72db25000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db38040 0x55f72db25a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01abd30 con 0x55f72db15980 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ad1a0 con 0x55f72db20240 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01ac400 con 0x55f72db38040 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a98b5740 10 client.0 _ll_register_callbacks cb 0x7f669c00eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 0x55f72db25000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 0x55f72db25000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52204/0 (socket says 172.21.15.122:52204) 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 -- 172.21.15.122:0/239745172 learned_addr learned my addr 172.21.15.122:0/239745172 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5a79640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db38040 0x55f72db25a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a4a77640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 -- 172.21.15.122:0/239745172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db38040 msgr2=0x55f72db25a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db38040 0x55f72db25a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 -- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 msgr2=0x55f72db25540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db25540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.588+0000 7f66a5278640 1 -- 172.21.15.122:0/239745172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72db9ab90 con 0x55f72db15980 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a5278640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 0x55f72db25000 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f669c017010 tx=0x7f669c013880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fc4d3122722f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f6688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f6688ff9640 1 -- 172.21.15.122:0/239745172 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c038020 con 0x55f72db15980 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f6688ff9640 1 -- 172.21.15.122:0/239745172 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f669c013e00 con 0x55f72db15980 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669802f5a0 con 0x55f72db15980 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f6688ff9640 1 -- 172.21.15.122:0/239745172 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f669c023050 con 0x55f72db15980 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 10 client.17097 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f72db9beb0 con 0x55f72db15980 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f6688ff9640 1 -- 172.21.15.122:0/239745172 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669802f5a0 con 0x55f72db15980 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f6688ff9640 1 -- 172.21.15.122:0/239745172 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f669c033ce0 con 0x55f72db15980 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 10 client.17097 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 10 client.17097 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 20 client.17097 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db9c600 0x55f72db9ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 4 client.17097 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cb486123", "release": true, "await": true}] 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.589+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cb486123", "release": true, "await": true}) -- 0x55f72da820a0 con 0x55f72db9c600 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.590+0000 7f66a4a77640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db9c600 0x55f72db9ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.590+0000 7f66a4a77640 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db9c600 0x55f72db9ea20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70912d425421d86f server_cookie=b2dc756c83d33b85 in_seq=0 out_seq=0 2024-09-12T01:56:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.590+0000 7f6688ff9640 10 client.17097 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f6688ff9640 1 -- 172.21.15.122:0/239745172 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f72da820a0 con 0x55f72db9c600 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f6688ff9640 10 client.17097 handle_command_reply: tid=0 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 client.17097 shutdown 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 msgr2=0x55f72db25000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 0x55f72db25000 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f669c017010 tx=0x7f669c013880 comp rx=0 tx=0).stop 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 shutdown_connections 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f72db9c600 0x55f72db9ea20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f72db38040 0x55f72db25a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f72db20240 0x55f72db25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 --2- 172.21.15.122:0/239745172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f72db15980 0x55f72db25000 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 >> 172.21.15.122:0/239745172 conn(0x55f72db115f0 msgr2=0x55f72db16ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f6688ff9640 0 client.17097 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 shutdown_connections 2024-09-12T01:56:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 1 -- 172.21.15.122:0/239745172 wait complete. 2024-09-12T01:56:54.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.599+0000 7f66a98b5740 20 client.17097 trim_cache size 0 max 16384 2024-09-12T01:56:54.601 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:56:54.601 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 44, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "cb486123": { 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 44, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:56:54.602 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 142.6, 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:56:54.603 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:56:54.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f668c0a2c00 msgr2=0x7f668c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f668c0a2c00 0x7f668c0a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f669c006aa0 tx=0x7f669c006a30 comp rx=0 tx=0).stop 2024-09-12T01:56:54.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 msgr2=0x7f66a0198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 0x7f66a0198b40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f669400b3f0 tx=0x7f669400b7f0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 shutdown_connections 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f668c0a2c00 0x7f668c0a50c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a010c170 0x7f66a019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66a0103660 0x7f66a0198b40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 --2- 172.21.15.122:0/2739406809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66a0102d40 0x7f66a0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 >> 172.21.15.122:0/2739406809 conn(0x7f66a00fc7a0 msgr2=0x7f66a00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 shutdown_connections 2024-09-12T01:56:54.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:56:54.600+0000 7f66a74dc640 1 -- 172.21.15.122:0/2739406809 wait complete. 2024-09-12T01:56:54.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 933180 9.45 MB/sec execute 432 sec latency 22567.230 ms 2024-09-12T01:56:55.407 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cb486123', total seconds elapsed: 25.9 2024-09-12T01:56:55.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 935490 9.46 MB/sec execute 433 sec latency 15.617 ms 2024-09-12T01:56:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:55 smithi183 ceph-mon[35127]: pgmap v648: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-12T01:56:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:55 smithi122 ceph-mon[30925]: pgmap v648: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-12T01:56:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:55 smithi151 ceph-mon[35229]: pgmap v648: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-12T01:56:56.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 938449 9.48 MB/sec execute 434 sec latency 9.634 ms 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.826+0000 7fbf99731640 1 -- 172.21.15.122:0/2703164055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 msgr2=0x7fbf941049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.826+0000 7fbf99731640 1 --2- 172.21.15.122:0/2703164055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf941049b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009900 tx=0x7fbf8402a410 comp rx=0 tx=0).stop 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.826+0000 7fbf99731640 1 -- 172.21.15.122:0/2703164055 shutdown_connections 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.826+0000 7fbf99731640 1 --2- 172.21.15.122:0/2703164055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94107820 0x7fbf94109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.826+0000 7fbf99731640 1 --2- 172.21.15.122:0/2703164055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94104ef0 0x7fbf941072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.826+0000 7fbf99731640 1 --2- 172.21.15.122:0/2703164055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf941049b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- 172.21.15.122:0/2703164055 >> 172.21.15.122:0/2703164055 conn(0x7fbf940fc1e0 msgr2=0x7fbf940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- 172.21.15.122:0/2703164055 shutdown_connections 2024-09-12T01:56:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- 172.21.15.122:0/2703164055 wait complete. 2024-09-12T01:56:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 Processor -- start 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- start start 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf94197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 0x7fbf941983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94107820 0x7fbf9419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410d800 con 0x7fbf941025c0 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410d680 con 0x7fbf94104ef0 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.827+0000 7fbf99731640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410d980 con 0x7fbf94107820 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf94197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 0x7fbf941983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94107820 0x7fbf9419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 0x7fbf941983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52926/0 (socket says 172.21.15.122:52926) 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf927fc640 1 -- 172.21.15.122:0/2280247324 learned_addr learned my addr 172.21.15.122:0/2280247324 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf94197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52232/0 (socket says 172.21.15.122:52232) 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 -- 172.21.15.122:0/2280247324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94107820 msgr2=0x7fbf9419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94107820 0x7fbf9419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 -- 172.21.15.122:0/2280247324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 msgr2=0x7fbf941983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 0x7fbf941983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 -- 172.21.15.122:0/2280247324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9419fa80 con 0x7fbf941025c0 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf927fc640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 0x7fbf941983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf94197e60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbf84031440 tx=0x7fbf84031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf7bfff640 1 -- 172.21.15.122:0/2280247324 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8403f070 con 0x7fbf941025c0 2024-09-12T01:56:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf7bfff640 1 -- 172.21.15.122:0/2280247324 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf84003cd0 con 0x7fbf941025c0 2024-09-12T01:56:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9419fd10 con 0x7fbf941025c0 2024-09-12T01:56:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf7bfff640 1 -- 172.21.15.122:0/2280247324 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84035070 con 0x7fbf941025c0 2024-09-12T01:56:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.828+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf941a01b0 con 0x7fbf941025c0 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf7bfff640 1 -- 172.21.15.122:0/2280247324 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbf8403b560 con 0x7fbf941025c0 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf9c4cb740 1 Processor -- start 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf9c4cb740 1 -- start start 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a96f0 0x5647494ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 0x5647494b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.831+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647494082f0 con 0x5647494ae780 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5647493b0270 con 0x5647494a8db0 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5647494042b0 con 0x5647494a96f0 2024-09-12T01:56:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a96f0 0x5647494ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 0x5647494b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf937fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:52942/0 (socket says 172.21.15.122:52942) 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf937fe640 1 -- 172.21.15.122:0/629478364 learned_addr learned my addr 172.21.15.122:0/629478364 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 0x5647494b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52240/0 (socket says 172.21.15.122:52240) 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf7bfff640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf7c0a2c30 0x7fbf7c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf937fe640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf7c0a2c30 0x7fbf7c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 -- 172.21.15.122:0/629478364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a96f0 msgr2=0x5647494ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a96f0 0x5647494ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 -- 172.21.15.122:0/629478364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 msgr2=0x5647494a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 -- 172.21.15.122:0/629478364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647493af340 con 0x5647494ae780 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf7bfff640 1 -- 172.21.15.122:0/2280247324 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fbf840ef4d0 con 0x7fbf941025c0 2024-09-12T01:56:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf927fc640 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 0x5647494b0c40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005bc0 tx=0x7fbf8800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23bc3ff1a95ef51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf937fe640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf7c0a2c30 0x7fbf7c0a50f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005c40 tx=0x7fbf8c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf797fa640 1 -- 172.21.15.122:0/629478364 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88004230 con 0x5647494ae780 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf797fa640 1 -- 172.21.15.122:0/629478364 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf880043d0 con 0x5647494ae780 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.832+0000 7fbf797fa640 1 -- 172.21.15.122:0/629478364 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf880035f0 con 0x5647494ae780 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/629478364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 msgr2=0x5647494b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 0x5647494b0c40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005bc0 tx=0x7fbf8800bf90 comp rx=0 tx=0).stop 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/629478364 shutdown_connections 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494ae780 0x5647494b0c40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a96f0 0x5647494ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/629478364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/629478364 >> 172.21.15.122:0/629478364 conn(0x5647494a0010 msgr2=0x5647494a0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/629478364 shutdown_connections 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/629478364 wait complete. 2024-09-12T01:56:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.833+0000 7fbf9c4cb740 1 -- start start 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 Processor -- start 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 0x5647494b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a96f0 0x5647494b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x5647494ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647494b1750 con 0x5647494a8db0 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564749391290 con 0x5647494a96f0 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5647494b18d0 con 0x5647494ae780 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 10 client.0 _ll_register_callbacks cb 0x5647493c2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 0x5647494b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 0x5647494b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52256/0 (socket says 172.21.15.122:52256) 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 -- 172.21.15.122:0/3197536986 learned_addr learned my addr 172.21.15.122:0/3197536986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf927fc640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a96f0 0x5647494b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf937fe640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x5647494ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 -- 172.21.15.122:0/3197536986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 msgr2=0x5647494ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x5647494ba470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 -- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a96f0 msgr2=0x5647494b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a96f0 0x5647494b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 -- 172.21.15.122:0/3197536986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56474911b210 con 0x5647494a8db0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 0x5647494b2ef0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbf840b8770 tx=0x7fbf84003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5e1a041886f65a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3197536986 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84048040 con 0x5647494a8db0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3197536986 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fbf840b7af0 con 0x5647494a8db0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.834+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf880043d0 con 0x5647494a8db0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3197536986 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf840bb700 con 0x5647494a8db0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 10 client.17109 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647491e7620 con 0x5647494a8db0 2024-09-12T01:56:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3197536986 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf880043d0 con 0x5647494a8db0 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3197536986 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbf8404b060 con 0x5647494a8db0 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 10 client.17109 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 20 client.17109 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x5647494c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 4 client.17109 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564749415a20 con 0x5647494beb50 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.835+0000 7fbf927fc640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x5647494c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.836+0000 7fbf76ff5640 10 client.17109 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.836+0000 7fbf927fc640 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x5647494c0fc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30a8be7f6a430491 server_cookie=a658a323f11d4c3a in_seq=0 out_seq=0 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.836+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3197536986 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564749415a20 con 0x5647494beb50 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf76ff5640 10 client.17109 handle_command_reply: tid=0 2024-09-12T01:56:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 client.17109 shutdown 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 msgr2=0x5647494b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 0x5647494b2ef0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbf840b8770 tx=0x7fbf84003e80 comp rx=0 tx=0).stop 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 shutdown_connections 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x5647494c0fc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x5647494ba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf76ff5640 0 client.17109 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a96f0 0x5647494b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3197536986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a8db0 0x5647494b2ef0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 >> 172.21.15.122:0/3197536986 conn(0x5647494a0010 msgr2=0x5647494a1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 shutdown_connections 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3197536986 wait complete. 2024-09-12T01:56:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.837+0000 7fbf9c4cb740 20 client.17109 trim_cache size 0 max 16384 2024-09-12T01:56:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 Processor -- start 2024-09-12T01:56:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 -- start start 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x56474949e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941a96c0 con 0x5647494a96f0 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941a6f90 con 0x5647494a8db0 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941a4dc0 con 0x5647494ae780 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52260/0 (socket says 172.21.15.122:52260) 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 -- 172.21.15.122:0/435908302 learned_addr learned my addr 172.21.15.122:0/435908302 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf937fe640 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x56474949e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 -- 172.21.15.122:0/435908302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 msgr2=0x56474949e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x56474949e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 -- 172.21.15.122:0/435908302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 msgr2=0x5647494b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 -- 172.21.15.122:0/435908302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647494ada80 con 0x5647494a96f0 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf937fe640 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x56474949e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.853+0000 7fbf927fc640 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494ba430 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005bc0 tx=0x7fbf880051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8c62434cdf9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf78ff9640 1 -- 172.21.15.122:0/435908302 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88005510 con 0x5647494a96f0 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf78ff9640 1 -- 172.21.15.122:0/435908302 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf88003ad0 con 0x5647494a96f0 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf78ff9640 1 -- 172.21.15.122:0/435908302 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88011ba0 con 0x5647494a96f0 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/435908302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 msgr2=0x5647494ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494ba430 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005bc0 tx=0x7fbf880051e0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/435908302 shutdown_connections 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494ae780 0x56474949e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494ba430 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/435908302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494a8db0 0x5647494b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/435908302 >> 172.21.15.122:0/435908302 conn(0x5647494a0010 msgr2=0x5647494b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/435908302 shutdown_connections 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.854+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/435908302 wait complete. 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 -- start start 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 Processor -- start 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a8db0 0x5647494b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494ae780 0x5647494b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941a5450 con 0x5647494a96f0 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941a68c0 con 0x5647494ae780 2024-09-12T01:56:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941a5b20 con 0x5647494a8db0 2024-09-12T01:56:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf9c4cb740 10 client.0 _ll_register_callbacks cb 0x7fbf8403b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:56:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52270/0 (socket says 172.21.15.122:52270) 2024-09-12T01:56:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 -- 172.21.15.122:0/3058331595 learned_addr learned my addr 172.21.15.122:0/3058331595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:56:57.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 941512 9.50 MB/sec execute 435 sec latency 13.081 ms 2024-09-12T01:56:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf937fe640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494ae780 0x5647494b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a8db0 0x5647494b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 -- 172.21.15.122:0/3058331595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a8db0 msgr2=0x5647494b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a8db0 0x5647494b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 -- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494ae780 msgr2=0x5647494b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494ae780 0x5647494b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 -- 172.21.15.122:0/3058331595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647494b99c0 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.855+0000 7fbf927fc640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494b83a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbf880104a0 tx=0x7fbf880115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5319b4742cf4038 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3058331595 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88005510 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3058331595 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fbf880035f0 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf88003ad0 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3058331595 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf880037f0 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 10 client.17115 fetch_fsmap learned FSMap version 30 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564749529c00 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3058331595 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf88003ad0 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3058331595 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbf8801b860 con 0x5647494a96f0 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 10 client.17115 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:56:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 20 client.17115 populate_metadata read hostname 'smithi122' 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x56474952c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 4 client.17115 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56474940fd70 con 0x5647494beb50 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.856+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x56474952c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.857+0000 7fbf92ffd640 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x56474952c420 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e094e6ed1da326 server_cookie=ee0a5a9a1b5ec4d2 in_seq=0 out_seq=0 2024-09-12T01:56:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.857+0000 7fbf76ff5640 10 client.17115 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.868+0000 7fbf76ff5640 1 -- 172.21.15.122:0/3058331595 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56474940fd70 con 0x5647494beb50 2024-09-12T01:56:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.868+0000 7fbf76ff5640 10 client.17115 handle_command_reply: tid=0 2024-09-12T01:56:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.868+0000 7fbf9c4cb740 1 client.17115 shutdown 2024-09-12T01:56:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 msgr2=0x5647494b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494b83a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbf880104a0 tx=0x7fbf880115f0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 shutdown_connections 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5647494beb50 0x56474952c420 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647494ae780 0x5647494b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647494a96f0 0x5647494b83a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf76ff5640 0 client.17115 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 --2- 172.21.15.122:0/3058331595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647494a8db0 0x5647494b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 >> 172.21.15.122:0/3058331595 conn(0x5647494a0010 msgr2=0x5647494a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 shutdown_connections 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 1 -- 172.21.15.122:0/3058331595 wait complete. 2024-09-12T01:56:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf9c4cb740 20 client.17115 trim_cache size 0 max 16384 2024-09-12T01:56:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:56:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:56:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:56:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:56:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf7c0a2c30 msgr2=0x7fbf7c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf99731640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf7c0a2c30 0x7fbf7c0a50f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005c40 tx=0x7fbf8c004ab0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 msgr2=0x7fbf94197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:56:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.869+0000 7fbf99731640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf94197e60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbf84031440 tx=0x7fbf84031b60 comp rx=0 tx=0).stop 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 shutdown_connections 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbf7c0a2c30 0x7fbf7c0a50f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94107820 0x7fbf9419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf94104ef0 0x7fbf941983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 --2- 172.21.15.122:0/2280247324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf941025c0 0x7fbf94197e60 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 >> 172.21.15.122:0/2280247324 conn(0x7fbf940fc1e0 msgr2=0x7fbf94102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 shutdown_connections 2024-09-12T01:56:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:56:57.870+0000 7fbf99731640 1 -- 172.21.15.122:0/2280247324 wait complete. 2024-09-12T01:56:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:57 smithi183 ceph-mon[35127]: pgmap v649: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-12T01:56:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:57 smithi122 ceph-mon[30925]: pgmap v649: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-12T01:56:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:57 smithi151 ceph-mon[35229]: pgmap v649: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-09-12T01:56:58.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 944065 9.50 MB/sec execute 436 sec latency 11.112 ms 2024-09-12T01:56:59.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 946212 9.51 MB/sec execute 437 sec latency 11.103 ms 2024-09-12T01:56:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:56:59 smithi122 ceph-mon[30925]: pgmap v650: 97 pgs: 97 active+clean; 282 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 324 KiB/s wr, 60 op/s 2024-09-12T01:57:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:56:59 smithi151 ceph-mon[35229]: pgmap v650: 97 pgs: 97 active+clean; 282 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 324 KiB/s wr, 60 op/s 2024-09-12T01:57:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:56:59 smithi183 ceph-mon[35127]: pgmap v650: 97 pgs: 97 active+clean; 282 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 324 KiB/s wr, 60 op/s 2024-09-12T01:57:00.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 949078 9.53 MB/sec execute 438 sec latency 13.716 ms 2024-09-12T01:57:01.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 951567 9.53 MB/sec execute 439 sec latency 141.005 ms 2024-09-12T01:57:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:01 smithi122 ceph-mon[30925]: pgmap v651: 97 pgs: 97 active+clean; 283 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 457 KiB/s wr, 71 op/s 2024-09-12T01:57:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:01 smithi151 ceph-mon[35229]: pgmap v651: 97 pgs: 97 active+clean; 283 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 457 KiB/s wr, 71 op/s 2024-09-12T01:57:02.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:01 smithi183 ceph-mon[35127]: pgmap v651: 97 pgs: 97 active+clean; 283 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 457 KiB/s wr, 71 op/s 2024-09-12T01:57:02.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 953773 9.54 MB/sec execute 440 sec latency 22.136 ms 2024-09-12T01:57:03.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 956405 9.55 MB/sec execute 441 sec latency 18.409 ms 2024-09-12T01:57:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:03 smithi122 ceph-mon[30925]: pgmap v652: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-12T01:57:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:03 smithi151 ceph-mon[35229]: pgmap v652: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-12T01:57:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:03 smithi183 ceph-mon[35127]: pgmap v652: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-09-12T01:57:04.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 959396 9.57 MB/sec execute 442 sec latency 10.148 ms 2024-09-12T01:57:05.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 962052 9.57 MB/sec execute 443 sec latency 10.602 ms 2024-09-12T01:57:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:05 smithi122 ceph-mon[30925]: pgmap v653: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-12T01:57:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:05 smithi151 ceph-mon[35229]: pgmap v653: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-12T01:57:06.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:05 smithi183 ceph-mon[35127]: pgmap v653: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-12T01:57:06.855 INFO:tasks.workunit.client.0.smithi122.stdout: 1 964328 9.58 MB/sec execute 444 sec latency 12.752 ms 2024-09-12T01:57:07.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 967099 9.59 MB/sec execute 445 sec latency 16.703 ms 2024-09-12T01:57:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:07 smithi122 ceph-mon[30925]: pgmap v654: 97 pgs: 97 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-12T01:57:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:07 smithi151 ceph-mon[35229]: pgmap v654: 97 pgs: 97 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-12T01:57:08.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:07 smithi183 ceph-mon[35127]: pgmap v654: 97 pgs: 97 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-12T01:57:08.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 970254 9.61 MB/sec execute 446 sec latency 12.488 ms 2024-09-12T01:57:09.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 973251 9.62 MB/sec execute 447 sec latency 10.070 ms 2024-09-12T01:57:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:09 smithi122 ceph-mon[30925]: pgmap v655: 97 pgs: 97 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-12T01:57:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:09 smithi151 ceph-mon[35229]: pgmap v655: 97 pgs: 97 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-12T01:57:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:09 smithi183 ceph-mon[35127]: pgmap v655: 97 pgs: 97 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-12T01:57:10.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 976049 9.63 MB/sec execute 448 sec latency 10.136 ms 2024-09-12T01:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 -- 172.21.15.122:0/783642749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a41025c0 msgr2=0x7f96a41049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 --2- 172.21.15.122:0/783642749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a41025c0 0x7f96a41049b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9694009900 tx=0x7f969402a410 comp rx=0 tx=0).stop 2024-09-12T01:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 -- 172.21.15.122:0/783642749 shutdown_connections 2024-09-12T01:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 --2- 172.21.15.122:0/783642749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4107820 0x7f96a4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 --2- 172.21.15.122:0/783642749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a4104ef0 0x7f96a41072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 --2- 172.21.15.122:0/783642749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a41025c0 0x7f96a41049b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 -- 172.21.15.122:0/783642749 >> 172.21.15.122:0/783642749 conn(0x7f96a40fc1e0 msgr2=0x7f96a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 -- 172.21.15.122:0/783642749 shutdown_connections 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 -- 172.21.15.122:0/783642749 wait complete. 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 Processor -- start 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.959+0000 7f96aa398640 1 -- start start 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96aa398640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 0x7f96a4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96aa398640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4104ef0 0x7f96a4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96aa398640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4107820 0x7f96a419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96aa398640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d850 con 0x7f96a4107820 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96aa398640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d6d0 con 0x7f96a4104ef0 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96aa398640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d9d0 con 0x7f96a41025c0 2024-09-12T01:57:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 0x7f96a4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4107820 0x7f96a419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 0x7f96a4197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:44342/0 (socket says 172.21.15.122:44342) 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a3fff640 1 -- 172.21.15.122:0/657569190 learned_addr learned my addr 172.21.15.122:0/657569190 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a37fe640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4104ef0 0x7f96a4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 -- 172.21.15.122:0/657569190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 msgr2=0x7f96a4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 0x7f96a4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 -- 172.21.15.122:0/657569190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4104ef0 msgr2=0x7f96a4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4104ef0 0x7f96a4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 -- 172.21.15.122:0/657569190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a419fa70 con 0x7f96a4107820 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a3fff640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 0x7f96a4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a37fe640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4104ef0 0x7f96a4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.960+0000 7f96a8935640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4107820 0x7f96a419f400 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f969c005fd0 tx=0x7f969c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.961+0000 7f96a17fa640 1 -- 172.21.15.122:0/657569190 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c00cdf0 con 0x7f96a4107820 2024-09-12T01:57:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.961+0000 7f96a17fa640 1 -- 172.21.15.122:0/657569190 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c005020 con 0x7f96a4107820 2024-09-12T01:57:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.961+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a419fd60 con 0x7f96a4107820 2024-09-12T01:57:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.961+0000 7f96a17fa640 1 -- 172.21.15.122:0/657569190 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0106d0 con 0x7f96a4107820 2024-09-12T01:57:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.961+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a0200 con 0x7f96a4107820 2024-09-12T01:57:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96a17fa640 1 -- 172.21.15.122:0/657569190 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f969c0054d0 con 0x7f96a4107820 2024-09-12T01:57:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 Processor -- start 2024-09-12T01:57:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 -- start start 2024-09-12T01:57:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 0x555987b62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b63370 0x555987b65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555987ac12f0 con 0x555987b5e2e0 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555987a69270 con 0x555987b5d9a0 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.963+0000 7f96ad212740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555987abd2b0 con 0x555987b63370 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a17fa640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f968c0a2c30 0x7f968c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 0x555987b62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 0x555987b62da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35428/0 (socket says 172.21.15.122:35428) 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 -- 172.21.15.122:0/2481815105 learned_addr learned my addr 172.21.15.122:0/2481815105 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a37fe640 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b63370 0x555987b65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a8935640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f968c0a2c30 0x7f968c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a17fa640 1 -- 172.21.15.122:0/657569190 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f969c0cf710 con 0x7f96a4107820 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 -- 172.21.15.122:0/2481815105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b63370 msgr2=0x555987b65830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b63370 0x555987b65830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 -- 172.21.15.122:0/2481815105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 msgr2=0x555987b5dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b5dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 -- 172.21.15.122:0/2481815105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555987a68340 con 0x555987b5e2e0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a37fe640 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b5dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a3fff640 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 0x555987b62da0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f969402a930 tx=0x7f96940043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f16396e9bd231b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f96a8935640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f968c0a2c30 0x7f968c0a50f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f969c065130 tx=0x7f969c07c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f968a7fc640 1 -- 172.21.15.122:0/2481815105 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694040070 con 0x555987b5e2e0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f968a7fc640 1 -- 172.21.15.122:0/2481815105 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940047a0 con 0x555987b5e2e0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.964+0000 7f968a7fc640 1 -- 172.21.15.122:0/2481815105 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694035040 con 0x555987b5e2e0 2024-09-12T01:57:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 -- 172.21.15.122:0/2481815105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 msgr2=0x555987b62da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 0x555987b62da0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f969402a930 tx=0x7f96940043f0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 -- 172.21.15.122:0/2481815105 shutdown_connections 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b63370 0x555987b65830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5e2e0 0x555987b62da0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 --2- 172.21.15.122:0/2481815105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b5dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 -- 172.21.15.122:0/2481815105 >> 172.21.15.122:0/2481815105 conn(0x555987b59010 msgr2=0x555987b59410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 -- 172.21.15.122:0/2481815105 shutdown_connections 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 -- 172.21.15.122:0/2481815105 wait complete. 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.965+0000 7f96ad212740 1 -- start start 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 Processor -- start 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5d9a0 0x555987b67b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5e2e0 0x555987b68080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555987b662e0 con 0x555987b63370 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555987a4a290 con 0x555987b5e2e0 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555987b66460 con 0x555987b5d9a0 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96ad212740 10 client.0 _ll_register_callbacks cb 0x555987a7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35444/0 (socket says 172.21.15.122:35444) 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 -- 172.21.15.122:0/133228301 learned_addr learned my addr 172.21.15.122:0/133228301 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a3fff640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5e2e0 0x555987b68080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5d9a0 0x555987b67b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 -- 172.21.15.122:0/133228301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5d9a0 msgr2=0x555987b67b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5d9a0 0x555987b67b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 -- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5e2e0 msgr2=0x555987b68080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5e2e0 0x555987b68080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 -- 172.21.15.122:0/133228301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559877d4210 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96a37fe640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6f0c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9698012fd0 tx=0x7f9698004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5747b84a6e0ac529 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.966+0000 7f96837fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96837fe640 1 -- 172.21.15.122:0/133228301 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969801a040 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96837fe640 1 -- 172.21.15.122:0/133228301 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9698005020 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96940047a0 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96837fe640 1 -- 172.21.15.122:0/133228301 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9698005220 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 10 client.17127 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559878a0620 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96837fe640 1 -- 172.21.15.122:0/133228301 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940047a0 con 0x555987b63370 2024-09-12T01:57:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96837fe640 1 -- 172.21.15.122:0/133228301 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9698029030 con 0x555987b63370 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 10 client.17127 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 20 client.17127 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987b737b0 0x555987b75c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 4 client.17127 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555987acea20 con 0x555987b737b0 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.967+0000 7f96a3fff640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987b737b0 0x555987b75c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96837fe640 10 client.17127 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96a3fff640 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987b737b0 0x555987b75c20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c4327d668e2e53 server_cookie=78bd36b3b9b2752 in_seq=0 out_seq=0 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96837fe640 1 -- 172.21.15.122:0/133228301 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555987acea20 con 0x555987b737b0 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96837fe640 10 client.17127 handle_command_reply: tid=0 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96ad212740 1 client.17127 shutdown 2024-09-12T01:57:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 msgr2=0x555987b6f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.968+0000 7f96ad212740 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6f0c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9698012fd0 tx=0x7f9698004730 comp rx=0 tx=0).stop 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 shutdown_connections 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987b737b0 0x555987b75c20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6f0c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5e2e0 0x555987b68080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96837fe640 0 client.17127 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 --2- 172.21.15.122:0/133228301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5d9a0 0x555987b67b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 >> 172.21.15.122:0/133228301 conn(0x555987b59010 msgr2=0x555987b5a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 shutdown_connections 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 1 -- 172.21.15.122:0/133228301 wait complete. 2024-09-12T01:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.969+0000 7f96ad212740 20 client.17127 trim_cache size 0 max 16384 2024-09-12T01:57:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.983+0000 7f96ad212740 1 Processor -- start 2024-09-12T01:57:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.983+0000 7f96ad212740 1 -- start start 2024-09-12T01:57:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96ad212740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 0x555987b59ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96ad212740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96ad212740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b63370 0x555987b5a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96ad212740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ab8b0 con 0x555987b5d9a0 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96ad212740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a9180 con 0x555987b63370 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96ad212740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a6fb0 con 0x555987b5e2e0 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 0x555987b59ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 0x555987b59ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35460/0 (socket says 172.21.15.122:35460) 2024-09-12T01:57:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 -- 172.21.15.122:0/1078367445 learned_addr learned my addr 172.21.15.122:0/1078367445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a3fff640 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b63370 0x555987b5a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 -- 172.21.15.122:0/1078367445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 msgr2=0x555987b6f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 -- 172.21.15.122:0/1078367445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b63370 msgr2=0x555987b5a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b63370 0x555987b5a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 -- 172.21.15.122:0/1078367445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555987b5e7c0 con 0x555987b5d9a0 2024-09-12T01:57:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a3fff640 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f96a37fe640 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 0x555987b59ec0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f96980059f0 tx=0x7f9698005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e07d9b932d3651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f9689ffb640 1 -- 172.21.15.122:0/1078367445 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698000e60 con 0x555987b5d9a0 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f9689ffb640 1 -- 172.21.15.122:0/1078367445 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969800e870 con 0x555987b5d9a0 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.984+0000 7f9689ffb640 1 -- 172.21.15.122:0/1078367445 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698004290 con 0x555987b5d9a0 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 -- 172.21.15.122:0/1078367445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 msgr2=0x555987b59ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 0x555987b59ec0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f96980059f0 tx=0x7f9698005e30 comp rx=0 tx=0).stop 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 -- 172.21.15.122:0/1078367445 shutdown_connections 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b63370 0x555987b5a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 --2- 172.21.15.122:0/1078367445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b5d9a0 0x555987b59ec0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 -- 172.21.15.122:0/1078367445 >> 172.21.15.122:0/1078367445 conn(0x555987b59010 msgr2=0x555987b67b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 -- 172.21.15.122:0/1078367445 shutdown_connections 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 -- 172.21.15.122:0/1078367445 wait complete. 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 -- start start 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.985+0000 7f96ad212740 1 Processor -- start 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a7640 con 0x555987b63370 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a8ab0 con 0x555987b5d9a0 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a7d10 con 0x555987b5e2e0 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 10 client.0 _ll_register_callbacks cb 0x7f96980129a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35472/0 (socket says 172.21.15.122:35472) 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 -- 172.21.15.122:0/3032679055 learned_addr learned my addr 172.21.15.122:0/3032679055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 -- 172.21.15.122:0/3032679055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 msgr2=0x555987b6cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 -- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 msgr2=0x555987b6c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b6c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 -- 172.21.15.122:0/3032679055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555987b6e140 con 0x555987b63370 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a3fff640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96a37fe640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6db30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f969800e670 tx=0x7f9698012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83af3f9d3e380d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96837fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96837fe640 1 -- 172.21.15.122:0/3032679055 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969803e040 con 0x555987b63370 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96837fe640 1 -- 172.21.15.122:0/3032679055 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9698011a60 con 0x555987b63370 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.986+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f969800e870 con 0x555987b63370 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96837fe640 1 -- 172.21.15.122:0/3032679055 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9698012c90 con 0x555987b63370 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 10 client.17133 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555987bde500 con 0x555987b63370 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96837fe640 1 -- 172.21.15.122:0/3032679055 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969800e870 con 0x555987b63370 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96837fe640 1 -- 172.21.15.122:0/3032679055 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9698029040 con 0x555987b63370 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 10 client.17133 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 20 client.17133 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987bdec40 0x555987be1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 4 client.17133 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:57:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555987ac8d70 con 0x555987bdec40 2024-09-12T01:57:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.987+0000 7f96a3fff640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987bdec40 0x555987be1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.988+0000 7f96a3fff640 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987bdec40 0x555987be1060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b09958bf84da4eb server_cookie=92bbebbc13f5ba66 in_seq=0 out_seq=0 2024-09-12T01:57:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:10.988+0000 7f96837fe640 10 client.17133 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96837fe640 1 -- 172.21.15.122:0/3032679055 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555987ac8d70 con 0x555987bdec40 2024-09-12T01:57:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96837fe640 10 client.17133 handle_command_reply: tid=0 2024-09-12T01:57:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 client.17133 shutdown 2024-09-12T01:57:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 msgr2=0x555987b6db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6db30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f969800e670 tx=0x7f9698012be0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 shutdown_connections 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555987bdec40 0x555987be1060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555987b63370 0x555987b6db30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96837fe640 0 client.17133 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555987b5e2e0 0x555987b6cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 --2- 172.21.15.122:0/3032679055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555987b5d9a0 0x555987b6c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 >> 172.21.15.122:0/3032679055 conn(0x555987b59010 msgr2=0x555987b579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 shutdown_connections 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 1 -- 172.21.15.122:0/3032679055 wait complete. 2024-09-12T01:57:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.000+0000 7f96ad212740 20 client.17133 trim_cache size 0 max 16384 2024-09-12T01:57:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:57:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:57:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:57:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:57:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f968c0a2c30 msgr2=0x7f968c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f968c0a2c30 0x7f968c0a50f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f969c065130 tx=0x7f969c07c040 comp rx=0 tx=0).stop 2024-09-12T01:57:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4107820 msgr2=0x7f96a419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4107820 0x7f96a419f400 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f969c005fd0 tx=0x7f969c00ef10 comp rx=0 tx=0).stop 2024-09-12T01:57:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 shutdown_connections 2024-09-12T01:57:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f968c0a2c30 0x7f968c0a50f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96a4107820 0x7f96a419f400 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a4104ef0 0x7f96a4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 --2- 172.21.15.122:0/657569190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96a41025c0 0x7f96a4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 >> 172.21.15.122:0/657569190 conn(0x7f96a40fc1e0 msgr2=0x7f96a4102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 shutdown_connections 2024-09-12T01:57:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:11.001+0000 7f96aa398640 1 -- 172.21.15.122:0/657569190 wait complete. 2024-09-12T01:57:11.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 978248 9.64 MB/sec execute 449 sec latency 49.938 ms 2024-09-12T01:57:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:11 smithi122 ceph-mon[30925]: pgmap v656: 97 pgs: 97 active+clean; 294 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-09-12T01:57:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:11 smithi151 ceph-mon[35229]: pgmap v656: 97 pgs: 97 active+clean; 294 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-09-12T01:57:12.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:11 smithi183 ceph-mon[35127]: pgmap v656: 97 pgs: 97 active+clean; 294 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 229 op/s 2024-09-12T01:57:12.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 981268 9.66 MB/sec execute 450 sec latency 17.093 ms 2024-09-12T01:57:13.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 984357 9.68 MB/sec execute 451 sec latency 14.131 ms 2024-09-12T01:57:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:13 smithi122 ceph-mon[30925]: pgmap v657: 97 pgs: 97 active+clean; 299 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-09-12T01:57:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:13 smithi151 ceph-mon[35229]: pgmap v657: 97 pgs: 97 active+clean; 299 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-09-12T01:57:14.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:13 smithi183 ceph-mon[35127]: pgmap v657: 97 pgs: 97 active+clean; 299 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-09-12T01:57:14.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 987120 9.68 MB/sec execute 452 sec latency 13.281 ms 2024-09-12T01:57:15.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 990074 9.69 MB/sec execute 453 sec latency 16.696 ms 2024-09-12T01:57:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:15 smithi122 ceph-mon[30925]: pgmap v658: 97 pgs: 97 active+clean; 299 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-09-12T01:57:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:15 smithi151 ceph-mon[35229]: pgmap v658: 97 pgs: 97 active+clean; 299 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-09-12T01:57:16.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:15 smithi183 ceph-mon[35127]: pgmap v658: 97 pgs: 97 active+clean; 299 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2024-09-12T01:57:16.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 992300 9.70 MB/sec execute 454 sec latency 11.129 ms 2024-09-12T01:57:17.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 995185 9.72 MB/sec execute 455 sec latency 17.160 ms 2024-09-12T01:57:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:17 smithi122 ceph-mon[30925]: pgmap v659: 97 pgs: 97 active+clean; 303 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-12T01:57:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:17 smithi151 ceph-mon[35229]: pgmap v659: 97 pgs: 97 active+clean; 303 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-12T01:57:18.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:17 smithi183 ceph-mon[35127]: pgmap v659: 97 pgs: 97 active+clean; 303 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-09-12T01:57:18.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 998326 9.73 MB/sec execute 456 sec latency 11.582 ms 2024-09-12T01:57:19.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1000932 9.73 MB/sec execute 457 sec latency 11.712 ms 2024-09-12T01:57:19.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:19 smithi122 ceph-mon[30925]: pgmap v660: 97 pgs: 97 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-09-12T01:57:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:19 smithi151 ceph-mon[35229]: pgmap v660: 97 pgs: 97 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-09-12T01:57:20.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:19 smithi183 ceph-mon[35127]: pgmap v660: 97 pgs: 97 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-09-12T01:57:20.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1003071 9.74 MB/sec execute 458 sec latency 11.361 ms 2024-09-12T01:57:21.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1005882 9.76 MB/sec execute 459 sec latency 16.525 ms 2024-09-12T01:57:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:21 smithi122 ceph-mon[30925]: pgmap v661: 97 pgs: 97 active+clean; 283 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-09-12T01:57:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:21 smithi151 ceph-mon[35229]: pgmap v661: 97 pgs: 97 active+clean; 283 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-09-12T01:57:22.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:21 smithi183 ceph-mon[35127]: pgmap v661: 97 pgs: 97 active+clean; 283 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-09-12T01:57:22.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1009045 9.77 MB/sec execute 460 sec latency 9.192 ms 2024-09-12T01:57:23.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1011566 9.77 MB/sec execute 461 sec latency 12.594 ms 2024-09-12T01:57:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:23 smithi122 ceph-mon[30925]: pgmap v662: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2024-09-12T01:57:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.094+0000 7f3f18723640 1 -- 172.21.15.122:0/3957122386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10100a40 msgr2=0x7f3f10100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.094+0000 7f3f18723640 1 --2- 172.21.15.122:0/3957122386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10100a40 0x7f3f10100e40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009ca0 tx=0x7f3f0c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 -- 172.21.15.122:0/3957122386 shutdown_connections 2024-09-12T01:57:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 --2- 172.21.15.122:0/3957122386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 0x7f3f10108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 --2- 172.21.15.122:0/3957122386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 0x7f3f101020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 --2- 172.21.15.122:0/3957122386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10100a40 0x7f3f10100e40 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 -- 172.21.15.122:0/3957122386 >> 172.21.15.122:0/3957122386 conn(0x7f3f100fc1e0 msgr2=0x7f3f100fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 -- 172.21.15.122:0/3957122386 shutdown_connections 2024-09-12T01:57:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 -- 172.21.15.122:0/3957122386 wait complete. 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 Processor -- start 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.095+0000 7f3f18723640 1 -- start start 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f18723640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 0x7f3f10199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f18723640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 0x7f3f1019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f18723640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f101025f0 0x7f3f101a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f18723640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010b600 con 0x7f3f101025f0 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f18723640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010b480 con 0x7f3f10100a40 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f18723640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010b780 con 0x7f3f10101c30 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 0x7f3f10199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 0x7f3f10199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:60272/0 (socket says 172.21.15.122:60272) 2024-09-12T01:57:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 -- 172.21.15.122:0/2993304585 learned_addr learned my addr 172.21.15.122:0/2993304585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f101025f0 0x7f3f101a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 0x7f3f1019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 -- 172.21.15.122:0/2993304585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 msgr2=0x7f3f1019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 0x7f3f1019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 -- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f101025f0 msgr2=0x7f3f101a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f101025f0 0x7f3f101a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f164bf640 1 -- 172.21.15.122:0/2993304585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f101a1be0 con 0x7f3f10100a40 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.096+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f101025f0 0x7f3f101a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.097+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 0x7f3f10199fc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000f80 tx=0x7f3f0c0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.097+0000 7f3f077fe640 1 -- 172.21.15.122:0/2993304585 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c003b70 con 0x7f3f10100a40 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.097+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1010c6a0 con 0x7f3f10100a40 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.097+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1010cc40 con 0x7f3f10100a40 2024-09-12T01:57:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.097+0000 7f3f077fe640 1 -- 172.21.15.122:0/2993304585 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0c003d10 con 0x7f3f10100a40 2024-09-12T01:57:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.097+0000 7f3f077fe640 1 -- 172.21.15.122:0/2993304585 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c02fc50 con 0x7f3f10100a40 2024-09-12T01:57:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 Processor -- start 2024-09-12T01:57:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 -- start start 2024-09-12T01:57:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f077fe640 1 -- 172.21.15.122:0/2993304585 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3f0c004040 con 0x7f3f10100a40 2024-09-12T01:57:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c241c9a0 0x55f3c241cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2421da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c2422370 0x55f3c2424830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3c23802f0 con 0x55f3c241d2e0 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2328270 con 0x55f3c241c9a0 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f3c237c2b0 con 0x55f3c2422370 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f164bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2421da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f16cc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c241c9a0 0x55f3c241cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f164bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2421da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54082/0 (socket says 172.21.15.122:54082) 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.100+0000 7f3f164bf640 1 -- 172.21.15.122:0/2278123324 learned_addr learned my addr 172.21.15.122:0/2278123324 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c2422370 0x55f3c2424830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f164bf640 1 -- 172.21.15.122:0/2278123324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c2422370 msgr2=0x55f3c2424830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c2422370 0x55f3c2424830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f164bf640 1 -- 172.21.15.122:0/2278123324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c241c9a0 msgr2=0x55f3c241cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c241c9a0 0x55f3c241cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f164bf640 1 -- 172.21.15.122:0/2278123324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c2327340 con 0x55f3c241d2e0 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c241c9a0 0x55f3c241cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f077fe640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a2cb0 0x7f3efc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2421da0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0f8360 tx=0x7f3f0c0f8450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c68d76a5c8c3044 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a2cb0 0x7f3efc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f077fe640 1 -- 172.21.15.122:0/2993304585 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3f0c0f0330 con 0x7f3f10100a40 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f04ff9640 1 -- 172.21.15.122:0/2278123324 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0b8e30 con 0x55f3c241d2e0 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f04ff9640 1 -- 172.21.15.122:0/2278123324 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0c0bd290 con 0x55f3c241d2e0 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.101+0000 7f3f04ff9640 1 -- 172.21.15.122:0/2278123324 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0bc4f0 con 0x55f3c241d2e0 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2278123324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 msgr2=0x55f3c2421da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a2cb0 0x7f3efc0a5170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004630 tx=0x7f3f08016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2421da0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0f8360 tx=0x7f3f0c0f8450 comp rx=0 tx=0).stop 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2278123324 shutdown_connections 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c2422370 0x55f3c2424830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2421da0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2278123324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c241c9a0 0x55f3c241cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2278123324 >> 172.21.15.122:0/2278123324 conn(0x55f3c2418010 msgr2=0x55f3c2418410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2278123324 shutdown_connections 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2278123324 wait complete. 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.102+0000 7f3f1b4ba740 1 -- start start 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 Processor -- start 2024-09-12T01:57:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c24266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2426c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3c23802f0 con 0x55f3c241d2e0 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2309290 con 0x55f3c2422370 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2425100 con 0x55f3c241c9a0 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f1b4ba740 10 client.0 _ll_register_callbacks cb 0x55f3c233a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f15cbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2426c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:60294/0 (socket says 172.21.15.122:60294) 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 -- 172.21.15.122:0/3597006881 learned_addr learned my addr 172.21.15.122:0/3597006881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f164bf640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c24266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 -- 172.21.15.122:0/3597006881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 msgr2=0x55f3c24266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c24266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 -- 172.21.15.122:0/3597006881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 msgr2=0x55f3c2426c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2426c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f16cc0640 1 -- 172.21.15.122:0/3597006881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c2093210 con 0x55f3c2422370 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.103+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2426c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242dc50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3f080217b0 tx=0x7f3f08004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead579f71f24350 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:57:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/3597006881 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08025070 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/3597006881 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3f0801f810 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c0bd290 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/3597006881 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f08021e20 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3f1b4ba740 10 client.26336 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.104+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3c215f620 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/3597006881 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0c0bd290 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/3597006881 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3f0802cc90 con 0x55f3c2422370 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f1b4ba740 10 client.26336 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f1b4ba740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f1b4ba740 20 client.26336 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c2432430 0x55f3c2434890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f1b4ba740 4 client.26336 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c238da20 con 0x55f3c2432430 2024-09-12T01:57:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.105+0000 7f3f164bf640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c2432430 0x55f3c2434890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.106+0000 7f3ef9ffb640 10 client.26336 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.106+0000 7f3f164bf640 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c2432430 0x55f3c2434890 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f02f494a202fd5e5 server_cookie=68737ef14b59b80c in_seq=0 out_seq=0 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/3597006881 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3c238da20 con 0x55f3c2432430 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3ef9ffb640 10 client.26336 handle_command_reply: tid=0 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 client.26336 shutdown 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 msgr2=0x55f3c242dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242dc50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3f080217b0 tx=0x7f3f08004730 comp rx=0 tx=0).stop 2024-09-12T01:57:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 shutdown_connections 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c2432430 0x55f3c2434890 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242dc50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c2426c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/3597006881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c24266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3ef9ffb640 0 client.26336 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 >> 172.21.15.122:0/3597006881 conn(0x55f3c2418010 msgr2=0x55f3c2419820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 shutdown_connections 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/3597006881 wait complete. 2024-09-12T01:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.107+0000 7f3f1b4ba740 20 client.26336 trim_cache size 0 max 16384 2024-09-12T01:57:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:23 smithi151 ceph-mon[35229]: pgmap v662: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 Processor -- start 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 -- start start 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 0x55f3c2418ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241d2e0 0x55f3c242dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c2419400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a96a0 con 0x55f3c241c9a0 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a6f70 con 0x55f3c2422370 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.123+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a4da0 con 0x55f3c241d2e0 2024-09-12T01:57:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f164bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 0x55f3c2418ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c2419400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c2419400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:60304/0 (socket says 172.21.15.122:60304) 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f164bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 0x55f3c2418ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54102/0 (socket says 172.21.15.122:54102) 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 -- 172.21.15.122:0/2552268600 learned_addr learned my addr 172.21.15.122:0/2552268600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241d2e0 0x55f3c242dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 -- 172.21.15.122:0/2552268600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241d2e0 msgr2=0x55f3c242dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241d2e0 0x55f3c242dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 -- 172.21.15.122:0/2552268600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 msgr2=0x55f3c2418ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 0x55f3c2418ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 -- 172.21.15.122:0/2552268600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c2417e30 con 0x55f3c2422370 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 0x55f3c2418ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c2419400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004130 tx=0x7f3f08021430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e501695a6b553f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3efbfff640 1 -- 172.21.15.122:0/2552268600 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08025040 con 0x55f3c2422370 2024-09-12T01:57:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3efbfff640 1 -- 172.21.15.122:0/2552268600 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f08031070 con 0x55f3c2422370 2024-09-12T01:57:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.124+0000 7f3efbfff640 1 -- 172.21.15.122:0/2552268600 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08004a00 con 0x55f3c2422370 2024-09-12T01:57:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2552268600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 msgr2=0x55f3c2419400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c2419400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004130 tx=0x7f3f08021430 comp rx=0 tx=0).stop 2024-09-12T01:57:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2552268600 shutdown_connections 2024-09-12T01:57:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c2419400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241d2e0 0x55f3c242dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2552268600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241c9a0 0x55f3c2418ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2552268600 >> 172.21.15.122:0/2552268600 conn(0x55f3c2418010 msgr2=0x55f3c24266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2552268600 shutdown_connections 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2552268600 wait complete. 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.125+0000 7f3f1b4ba740 1 -- start start 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 Processor -- start 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c242af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c242b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a5430 con 0x55f3c241d2e0 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a68a0 con 0x55f3c2422370 2024-09-12T01:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a5b00 con 0x55f3c241c9a0 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 10 client.0 _ll_register_callbacks cb 0x7f3f08009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c242b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c242b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54106/0 (socket says 172.21.15.122:54106) 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 -- 172.21.15.122:0/2690350067 learned_addr learned my addr 172.21.15.122:0/2690350067 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f16cc0640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c242af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 -- 172.21.15.122:0/2690350067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 msgr2=0x55f3c242af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c242af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 -- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 msgr2=0x55f3c242c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 -- 172.21.15.122:0/2690350067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c242ca60 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f15cbe640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c242b440 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3f1019b4d0 tx=0x7f3f0000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7f46d64a0ca786 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/2690350067 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00023060 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/2690350067 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3f00004250 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.126+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f08031070 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/2690350067 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f00004450 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 10 client.17151 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3c242de40 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/2690350067 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f08031070 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/2690350067 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3f00028070 con 0x55f3c241d2e0 2024-09-12T01:57:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 10 client.17151 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 20 client.17151 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c249d8a0 0x55f3c249fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 4 client.17151 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3c2387d70 con 0x55f3c249d8a0 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.127+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c249d8a0 0x55f3c249fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.128+0000 7f3f164bf640 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c249d8a0 0x55f3c249fcc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d32dded3877b46 server_cookie=822f37c64b32c250 in_seq=0 out_seq=0 2024-09-12T01:57:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.128+0000 7f3ef9ffb640 10 client.17151 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.140+0000 7f3ef9ffb640 1 -- 172.21.15.122:0/2690350067 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3c2387d70 con 0x55f3c249d8a0 2024-09-12T01:57:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.140+0000 7f3ef9ffb640 10 client.17151 handle_command_reply: tid=0 2024-09-12T01:57:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.140+0000 7f3f1b4ba740 1 client.17151 shutdown 2024-09-12T01:57:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 msgr2=0x55f3c242b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c242b440 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3f1019b4d0 tx=0x7f3f0000f790 comp rx=0 tx=0).stop 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 shutdown_connections 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3c249d8a0 0x55f3c249fcc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3c2422370 0x55f3c242c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3ef9ffb640 0 client.17151 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3c241d2e0 0x55f3c242b440 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 --2- 172.21.15.122:0/2690350067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f3c241c9a0 0x55f3c242af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 >> 172.21.15.122:0/2690350067 conn(0x55f3c2418010 msgr2=0x55f3c24169a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 shutdown_connections 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 1 -- 172.21.15.122:0/2690350067 wait complete. 2024-09-12T01:57:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f1b4ba740 20 client.17151 trim_cache size 0 max 16384 2024-09-12T01:57:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:57:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:57:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:57:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a2cb0 msgr2=0x7f3efc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a2cb0 0x7f3efc0a5170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004630 tx=0x7f3f08016040 comp rx=0 tx=0).stop 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 msgr2=0x7f3f10199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 0x7f3f10199fc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000f80 tx=0x7f3f0c0319f0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 shutdown_connections 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a2cb0 0x7f3efc0a5170 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f101025f0 0x7f3f101a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 0x7f3f1019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 --2- 172.21.15.122:0/2993304585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f10100a40 0x7f3f10199fc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.141+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 >> 172.21.15.122:0/2993304585 conn(0x7f3f100fc1e0 msgr2=0x7f3f100fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.142+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 shutdown_connections 2024-09-12T01:57:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:24.142+0000 7f3f18723640 1 -- 172.21.15.122:0/2993304585 wait complete. 2024-09-12T01:57:24.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:23 smithi183 ceph-mon[35127]: pgmap v662: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2024-09-12T01:57:24.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1013914 9.79 MB/sec execute 462 sec latency 14.190 ms 2024-09-12T01:57:25.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1016521 9.80 MB/sec execute 463 sec latency 19.583 ms 2024-09-12T01:57:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:25 smithi122 ceph-mon[30925]: pgmap v663: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-09-12T01:57:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:25 smithi151 ceph-mon[35229]: pgmap v663: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-09-12T01:57:26.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:25 smithi183 ceph-mon[35127]: pgmap v663: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-09-12T01:57:26.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1019254 9.80 MB/sec execute 464 sec latency 44.654 ms 2024-09-12T01:57:27.104 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:26 smithi183 ceph-mon[35127]: pgmap v664: 97 pgs: 97 active+clean; 290 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-09-12T01:57:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:26 smithi151 ceph-mon[35229]: pgmap v664: 97 pgs: 97 active+clean; 290 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-09-12T01:57:27.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:26 smithi122 ceph-mon[30925]: pgmap v664: 97 pgs: 97 active+clean; 290 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-09-12T01:57:27.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1021492 9.81 MB/sec execute 465 sec latency 60.788 ms 2024-09-12T01:57:28.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1024041 9.82 MB/sec execute 466 sec latency 19.604 ms 2024-09-12T01:57:29.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1027150 9.84 MB/sec execute 467 sec latency 9.139 ms 2024-09-12T01:57:29.941 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:29 smithi122 ceph-mon[30925]: pgmap v665: 97 pgs: 97 active+clean; 296 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-09-12T01:57:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:29 smithi183 ceph-mon[35127]: pgmap v665: 97 pgs: 97 active+clean; 296 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-09-12T01:57:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:29 smithi151 ceph-mon[35229]: pgmap v665: 97 pgs: 97 active+clean; 296 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-09-12T01:57:30.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1030388 9.85 MB/sec execute 468 sec latency 12.112 ms 2024-09-12T01:57:31.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1033210 9.86 MB/sec execute 469 sec latency 9.755 ms 2024-09-12T01:57:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:31 smithi183 ceph-mon[35127]: pgmap v666: 97 pgs: 97 active+clean; 297 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-12T01:57:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:31 smithi122 ceph-mon[30925]: pgmap v666: 97 pgs: 97 active+clean; 297 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-12T01:57:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:31 smithi151 ceph-mon[35229]: pgmap v666: 97 pgs: 97 active+clean; 297 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-09-12T01:57:32.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1035418 9.87 MB/sec execute 470 sec latency 92.569 ms 2024-09-12T01:57:33.856 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1038112 9.88 MB/sec execute 471 sec latency 13.504 ms 2024-09-12T01:57:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:33 smithi183 ceph-mon[35127]: pgmap v667: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 307 op/s 2024-09-12T01:57:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:33 smithi122 ceph-mon[30925]: pgmap v667: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 307 op/s 2024-09-12T01:57:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:33 smithi151 ceph-mon[35229]: pgmap v667: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 307 op/s 2024-09-12T01:57:34.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1041272 9.89 MB/sec execute 472 sec latency 12.899 ms 2024-09-12T01:57:35.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1044428 9.91 MB/sec execute 473 sec latency 10.106 ms 2024-09-12T01:57:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:35 smithi183 ceph-mon[35127]: pgmap v668: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 246 op/s 2024-09-12T01:57:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:35 smithi122 ceph-mon[30925]: pgmap v668: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 246 op/s 2024-09-12T01:57:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:35 smithi151 ceph-mon[35229]: pgmap v668: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 246 op/s 2024-09-12T01:57:36.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1047294 9.92 MB/sec execute 474 sec latency 13.948 ms 2024-09-12T01:57:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 -- 172.21.15.122:0/1667184450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 msgr2=0x7f18dc10a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 --2- 172.21.15.122:0/1667184450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc10a130 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009cd0 tx=0x7f18d402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 -- 172.21.15.122:0/1667184450 shutdown_connections 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 --2- 172.21.15.122:0/1667184450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc10a130 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 --2- 172.21.15.122:0/1667184450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18dc0ffbc0 0x7f18dc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 --2- 172.21.15.122:0/1667184450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18dc103150 0x7f18dc103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 -- 172.21.15.122:0/1667184450 >> 172.21.15.122:0/1667184450 conn(0x7f18dc0fb930 msgr2=0x7f18dc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 -- 172.21.15.122:0/1667184450 shutdown_connections 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.233+0000 7f18e1a65640 1 -- 172.21.15.122:0/1667184450 wait complete. 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 Processor -- start 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 -- start start 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18dc0ffbc0 0x7f18dc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18dc103150 0x7f18dc19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10d7f0 con 0x7f18dc100580 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10d670 con 0x7f18dc103150 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18e1a65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10d970 con 0x7f18dc0ffbc0 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35272/0 (socket says 172.21.15.122:35272) 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 -- 172.21.15.122:0/406176604 learned_addr learned my addr 172.21.15.122:0/406176604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18dbfff640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18dc103150 0x7f18dc19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18db7fe640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18dc0ffbc0 0x7f18dc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 -- 172.21.15.122:0/406176604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18dc0ffbc0 msgr2=0x7f18dc197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18dc0ffbc0 0x7f18dc197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 -- 172.21.15.122:0/406176604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18dc103150 msgr2=0x7f18dc19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18dc103150 0x7f18dc19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.234+0000 7f18daffd640 1 -- 172.21.15.122:0/406176604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc19fa00 con 0x7f18dc100580 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.235+0000 7f18daffd640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc198320 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d770 tx=0x7f18cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.235+0000 7f18d8ff9640 1 -- 172.21.15.122:0/406176604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc017070 con 0x7f18dc100580 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.235+0000 7f18d8ff9640 1 -- 172.21.15.122:0/406176604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18cc005500 con 0x7f18dc100580 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.235+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc19fcf0 con 0x7f18dc100580 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.235+0000 7f18d8ff9640 1 -- 172.21.15.122:0/406176604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc004e00 con 0x7f18dc100580 2024-09-12T01:57:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.235+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a0190 con 0x7f18dc100580 2024-09-12T01:57:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.237+0000 7f18d8ff9640 1 -- 172.21.15.122:0/406176604 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f18cc004fa0 con 0x7f18dc100580 2024-09-12T01:57:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.237+0000 7f18e4805740 1 Processor -- start 2024-09-12T01:57:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.237+0000 7f18e4805740 1 -- start start 2024-09-12T01:57:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18e4805740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc50da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18e4805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18e4805740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18e4805740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a78dbb42f0 con 0x55a78dc56370 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18e4805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a78db5c270 con 0x55a78dc512e0 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18e4805740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a78dbb02b0 con 0x55a78dc509a0 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18dbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc50da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18daffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc55da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:44832/0 (socket says 172.21.15.122:44832) 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18dbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc50da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51182/0 (socket says 172.21.15.122:51182) 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 -- 172.21.15.122:0/1071594307 learned_addr learned my addr 172.21.15.122:0/1071594307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 -- 172.21.15.122:0/1071594307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 msgr2=0x55a78dc50da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc50da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 -- 172.21.15.122:0/1071594307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 msgr2=0x55a78dc58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc58830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 -- 172.21.15.122:0/1071594307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78db5b340 con 0x55a78dc512e0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18d8ff9640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18c80a2c30 0x7f18c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18dbfff640 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18dbfff640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18c80a2c30 0x7f18c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18d8ff9640 1 -- 172.21.15.122:0/406176604 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f18cc0cea10 con 0x7f18dc100580 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.238+0000 7f18db7fe640 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc55da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009ca0 tx=0x7f18c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1950d0a7adb689 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18d1ffb640 1 -- 172.21.15.122:0/1071594307 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4004220 con 0x55a78dc512e0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18d1ffb640 1 -- 172.21.15.122:0/1071594307 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c40043c0 con 0x55a78dc512e0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18d1ffb640 1 -- 172.21.15.122:0/1071594307 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c40198d0 con 0x55a78dc512e0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 -- 172.21.15.122:0/1071594307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 msgr2=0x55a78dc55da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18dbfff640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18c80a2c30 0x7f18c80a50f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005e30 tx=0x7f18d4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc55da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009ca0 tx=0x7f18c4011810 comp rx=0 tx=0).stop 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 -- 172.21.15.122:0/1071594307 shutdown_connections 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc55da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 --2- 172.21.15.122:0/1071594307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc50da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 -- 172.21.15.122:0/1071594307 >> 172.21.15.122:0/1071594307 conn(0x55a78dc4c010 msgr2=0x55a78dc4c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 -- 172.21.15.122:0/1071594307 shutdown_connections 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.239+0000 7f18e4805740 1 -- 172.21.15.122:0/1071594307 wait complete. 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 -- start start 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 Processor -- start 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc5b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a78dc59340 con 0x55a78dc509a0 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a78db3d290 con 0x55a78dc56370 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a78dc594c0 con 0x55a78dc512e0 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18e4805740 10 client.0 _ll_register_callbacks cb 0x55a78db6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18db7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18daffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc5b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18daffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc5b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51186/0 (socket says 172.21.15.122:51186) 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18dbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.240+0000 7f18daffd640 1 -- 172.21.15.122:0/197327355 learned_addr learned my addr 172.21.15.122:0/197327355 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18db7fe640 1 -- 172.21.15.122:0/197327355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 msgr2=0x55a78dc5b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18db7fe640 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc5b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18db7fe640 1 -- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 msgr2=0x55a78dc620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18db7fe640 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18db7fe640 1 -- 172.21.15.122:0/197327355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78d8c7210 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18daffd640 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18db7fe640 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5ab60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f18c401b040 tx=0x7f18c40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6027668974dd6979 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18baffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18baffd640 1 -- 172.21.15.122:0/197327355 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4028070 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18baffd640 1 -- 172.21.15.122:0/197327355 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f18c4005040 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c40043c0 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18baffd640 1 -- 172.21.15.122:0/197327355 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18c4034050 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 10 client.17163 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a78d993620 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18baffd640 1 -- 172.21.15.122:0/197327355 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c40043c0 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18baffd640 1 -- 172.21.15.122:0/197327355 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f18c40049e0 con 0x55a78dc509a0 2024-09-12T01:57:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 10 client.17163 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 20 client.17163 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dc68a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 4 client.17163 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.241+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a78dbc1a20 con 0x55a78dc66660 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.242+0000 7f18daffd640 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dc68a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.242+0000 7f18daffd640 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dc68a50 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b70df4910e540ad server_cookie=26ea64f23169af2b in_seq=0 out_seq=0 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.242+0000 7f18baffd640 10 client.17163 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18baffd640 1 -- 172.21.15.122:0/197327355 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a78dbc1a20 con 0x55a78dc66660 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18baffd640 10 client.17163 handle_command_reply: tid=0 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 client.17163 shutdown 2024-09-12T01:57:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 msgr2=0x55a78dc5ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5ab60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f18c401b040 tx=0x7f18c40038a0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 shutdown_connections 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dc68a50 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18baffd640 0 client.17163 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 --2- 172.21.15.122:0/197327355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5ab60 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 >> 172.21.15.122:0/197327355 conn(0x55a78dc4c010 msgr2=0x55a78dc4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 shutdown_connections 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 1 -- 172.21.15.122:0/197327355 wait complete. 2024-09-12T01:57:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.243+0000 7f18e4805740 20 client.17163 trim_cache size 0 max 16384 2024-09-12T01:57:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.257+0000 7f18e4805740 1 Processor -- start 2024-09-12T01:57:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.257+0000 7f18e4805740 1 -- start start 2024-09-12T01:57:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.257+0000 7f18e4805740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18e4805740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18e4805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18e4805740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1ab8b0 con 0x55a78dc509a0 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18e4805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a9180 con 0x55a78dc56370 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18e4805740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a6fb0 con 0x55a78dc512e0 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35304/0 (socket says 172.21.15.122:35304) 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 -- 172.21.15.122:0/207480175 learned_addr learned my addr 172.21.15.122:0/207480175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18dbfff640 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18daffd640 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 -- 172.21.15.122:0/207480175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 msgr2=0x55a78dc620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 -- 172.21.15.122:0/207480175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 msgr2=0x55a78dc5ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc5ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 -- 172.21.15.122:0/207480175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78dc517c0 con 0x55a78dc509a0 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18daffd640 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18db7fe640 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5a910 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f18c4004d80 tx=0x7f18c4011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6dac976308264d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18d17fa640 1 -- 172.21.15.122:0/207480175 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4028040 con 0x55a78dc509a0 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18d17fa640 1 -- 172.21.15.122:0/207480175 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c40097a0 con 0x55a78dc509a0 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.258+0000 7f18d17fa640 1 -- 172.21.15.122:0/207480175 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4005480 con 0x55a78dc509a0 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 -- 172.21.15.122:0/207480175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 msgr2=0x55a78dc5a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5a910 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f18c4004d80 tx=0x7f18c4011d10 comp rx=0 tx=0).stop 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 -- 172.21.15.122:0/207480175 shutdown_connections 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc56370 0x55a78dc5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc512e0 0x55a78dc620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 --2- 172.21.15.122:0/207480175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc509a0 0x55a78dc5a910 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 -- 172.21.15.122:0/207480175 >> 172.21.15.122:0/207480175 conn(0x55a78dc4c010 msgr2=0x55a78dc598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 -- 172.21.15.122:0/207480175 shutdown_connections 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 -- 172.21.15.122:0/207480175 wait complete. 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.259+0000 7f18e4805740 1 -- start start 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 Processor -- start 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc5f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a7640 con 0x55a78dc56370 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a8ab0 con 0x55a78dc512e0 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1a7d10 con 0x55a78dc509a0 2024-09-12T01:57:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18e4805740 10 client.0 _ll_register_callbacks cb 0x7f18cc0d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc5f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18daffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc5f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35314/0 (socket says 172.21.15.122:35314) 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 -- 172.21.15.122:0/4083402293 learned_addr learned my addr 172.21.15.122:0/4083402293 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18db7fe640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 -- 172.21.15.122:0/4083402293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 msgr2=0x55a78dc62110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc62110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 -- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 msgr2=0x55a78dc5ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc5ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 -- 172.21.15.122:0/4083402293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78dccf2e0 con 0x55a78dc56370 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18daffd640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc5ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18db7fe640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc62110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18dbfff640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc5f170 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f18d4035000 tx=0x7f18d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9ce6b1cf6886ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18baffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.260+0000 7f18baffd640 1 -- 172.21.15.122:0/4083402293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4057020 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18baffd640 1 -- 172.21.15.122:0/4083402293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f18d4004250 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c40097a0 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18baffd640 1 -- 172.21.15.122:0/4083402293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18d4004450 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 10 client.17169 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a78dcd0660 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18baffd640 1 -- 172.21.15.122:0/4083402293 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c40097a0 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18baffd640 1 -- 172.21.15.122:0/4083402293 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f18d4030070 con 0x55a78dc56370 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 10 client.17169 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 20 client.17169 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dcd3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 4 client.17169 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:57:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a78dbbbd70 con 0x55a78dc66660 2024-09-12T01:57:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.261+0000 7f18db7fe640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dcd3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.262+0000 7f18baffd640 10 client.17169 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.262+0000 7f18db7fe640 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dcd3100 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda162a9e62652d8 server_cookie=5f9f7957bf32841f in_seq=0 out_seq=0 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18baffd640 1 -- 172.21.15.122:0/4083402293 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a78dbbbd70 con 0x55a78dc66660 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18baffd640 10 client.17169 handle_command_reply: tid=0 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 client.17169 shutdown 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 msgr2=0x55a78dc5f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc5f170 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f18d4035000 tx=0x7f18d40095c0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 shutdown_connections 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a78dc66660 0x55a78dcd3100 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a78dc56370 0x55a78dc5f170 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a78dc512e0 0x55a78dc5ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18baffd640 0 client.17169 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 --2- 172.21.15.122:0/4083402293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a78dc509a0 0x55a78dc62110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 >> 172.21.15.122:0/4083402293 conn(0x55a78dc4c010 msgr2=0x55a78dc5a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 shutdown_connections 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 1 -- 172.21.15.122:0/4083402293 wait complete. 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.273+0000 7f18e4805740 20 client.17169 trim_cache size 0 max 16384 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:57:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:57:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18c80a2c30 msgr2=0x7f18c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18c80a2c30 0x7f18c80a50f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005e30 tx=0x7f18d4005d80 comp rx=0 tx=0).stop 2024-09-12T01:57:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 msgr2=0x7f18dc198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc198320 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d770 tx=0x7f18cc00dc30 comp rx=0 tx=0).stop 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 shutdown_connections 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18c80a2c30 0x7f18c80a50f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18dc103150 0x7f18dc19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18dc100580 0x7f18dc198320 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 --2- 172.21.15.122:0/406176604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18dc0ffbc0 0x7f18dc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 >> 172.21.15.122:0/406176604 conn(0x7f18dc0fb930 msgr2=0x7f18dc106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 shutdown_connections 2024-09-12T01:57:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:37.274+0000 7f18e1a65640 1 -- 172.21.15.122:0/406176604 wait complete. 2024-09-12T01:57:37.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1049216 9.92 MB/sec execute 475 sec latency 93.531 ms 2024-09-12T01:57:37.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:37 smithi183 ceph-mon[35127]: pgmap v669: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-09-12T01:57:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:37 smithi122 ceph-mon[30925]: pgmap v669: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-09-12T01:57:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:37 smithi151 ceph-mon[35229]: pgmap v669: 97 pgs: 97 active+clean; 303 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-09-12T01:57:38.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1052015 9.93 MB/sec execute 476 sec latency 19.521 ms 2024-09-12T01:57:39.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1055265 9.95 MB/sec execute 477 sec latency 10.973 ms 2024-09-12T01:57:39.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:39 smithi183 ceph-mon[35127]: pgmap v670: 97 pgs: 97 active+clean; 304 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.6 MiB/s wr, 338 op/s 2024-09-12T01:57:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:39 smithi122 ceph-mon[30925]: pgmap v670: 97 pgs: 97 active+clean; 304 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.6 MiB/s wr, 338 op/s 2024-09-12T01:57:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:39 smithi151 ceph-mon[35229]: pgmap v670: 97 pgs: 97 active+clean; 304 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.6 MiB/s wr, 338 op/s 2024-09-12T01:57:40.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1058047 9.96 MB/sec execute 478 sec latency 13.966 ms 2024-09-12T01:57:41.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1060460 9.96 MB/sec execute 479 sec latency 52.469 ms 2024-09-12T01:57:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:41 smithi183 ceph-mon[35127]: pgmap v671: 97 pgs: 97 active+clean; 307 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-09-12T01:57:41.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:41 smithi122 ceph-mon[30925]: pgmap v671: 97 pgs: 97 active+clean; 307 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-09-12T01:57:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:41 smithi151 ceph-mon[35229]: pgmap v671: 97 pgs: 97 active+clean; 307 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-09-12T01:57:42.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1062808 9.97 MB/sec execute 480 sec latency 58.470 ms 2024-09-12T01:57:43.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1065823 9.98 MB/sec execute 481 sec latency 108.109 ms 2024-09-12T01:57:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:43 smithi122 ceph-mon[30925]: pgmap v672: 97 pgs: 97 active+clean; 307 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 327 op/s 2024-09-12T01:57:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:43 smithi151 ceph-mon[35229]: pgmap v672: 97 pgs: 97 active+clean; 307 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 327 op/s 2024-09-12T01:57:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:43 smithi183 ceph-mon[35127]: pgmap v672: 97 pgs: 97 active+clean; 307 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 327 op/s 2024-09-12T01:57:44.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1068662 9.99 MB/sec execute 482 sec latency 10.178 ms 2024-09-12T01:57:45.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1071534 10.00 MB/sec execute 483 sec latency 10.687 ms 2024-09-12T01:57:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:45 smithi122 ceph-mon[30925]: pgmap v673: 97 pgs: 97 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-09-12T01:57:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:45 smithi151 ceph-mon[35229]: pgmap v673: 97 pgs: 97 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-09-12T01:57:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:45 smithi183 ceph-mon[35127]: pgmap v673: 97 pgs: 97 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-09-12T01:57:46.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1073667 10.01 MB/sec execute 484 sec latency 58.432 ms 2024-09-12T01:57:47.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1076713 10.02 MB/sec execute 485 sec latency 18.794 ms 2024-09-12T01:57:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:47 smithi122 ceph-mon[30925]: pgmap v674: 97 pgs: 97 active+clean; 311 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2024-09-12T01:57:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:47 smithi151 ceph-mon[35229]: pgmap v674: 97 pgs: 97 active+clean; 311 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2024-09-12T01:57:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:47 smithi183 ceph-mon[35127]: pgmap v674: 97 pgs: 97 active+clean; 311 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2024-09-12T01:57:48.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1079092 10.02 MB/sec execute 486 sec latency 109.581 ms 2024-09-12T01:57:49.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1081365 10.03 MB/sec execute 487 sec latency 10.978 ms 2024-09-12T01:57:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:49 smithi122 ceph-mon[30925]: pgmap v675: 97 pgs: 97 active+clean; 311 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2024-09-12T01:57:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:49 smithi151 ceph-mon[35229]: pgmap v675: 97 pgs: 97 active+clean; 311 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2024-09-12T01:57:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:49 smithi183 ceph-mon[35127]: pgmap v675: 97 pgs: 97 active+clean; 311 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 -- 172.21.15.122:0/2419198335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c102620 msgr2=0x7f383c104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 --2- 172.21.15.122:0/2419198335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c102620 0x7f383c104a10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f382c009900 tx=0x7f382c02a410 comp rx=0 tx=0).stop 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 -- 172.21.15.122:0/2419198335 shutdown_connections 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 --2- 172.21.15.122:0/2419198335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 0x7f383c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 --2- 172.21.15.122:0/2419198335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c104f50 0x7f383c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 --2- 172.21.15.122:0/2419198335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c102620 0x7f383c104a10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 -- 172.21.15.122:0/2419198335 >> 172.21.15.122:0/2419198335 conn(0x7f383c0fc1e0 msgr2=0x7f383c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 -- 172.21.15.122:0/2419198335 shutdown_connections 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 -- 172.21.15.122:0/2419198335 wait complete. 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.369+0000 7f3841d6f640 1 Processor -- start 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 -- start start 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c102620 0x7f383c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 0x7f383c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 0x7f383c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f383c10d820 con 0x7f383c104f50 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f383c10d6a0 con 0x7f383c102620 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f383c10d9a0 con 0x7f383c107880 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 0x7f383c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c102620 0x7f383c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 0x7f383c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57686/0 (socket says 172.21.15.122:57686) 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 -- 172.21.15.122:0/1750864265 learned_addr learned my addr 172.21.15.122:0/1750864265 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383bfff640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 0x7f383c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 -- 172.21.15.122:0/1750864265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 msgr2=0x7f383c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 0x7f383c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 -- 172.21.15.122:0/1750864265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c102620 msgr2=0x7f383c197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c102620 0x7f383c197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 -- 172.21.15.122:0/1750864265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c19fa70 con 0x7f383c104f50 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383b7fe640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c102620 0x7f383c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383bfff640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 0x7f383c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f383affd640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 0x7f383c198390 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f383000e9a0 tx=0x7f383000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3838ff9640 1 -- 172.21.15.122:0/1750864265 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383000cd80 con 0x7f383c104f50 2024-09-12T01:57:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.370+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c19fd60 con 0x7f383c104f50 2024-09-12T01:57:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.371+0000 7f3838ff9640 1 -- 172.21.15.122:0/1750864265 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38300035f0 con 0x7f383c104f50 2024-09-12T01:57:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.371+0000 7f3838ff9640 1 -- 172.21.15.122:0/1750864265 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830010670 con 0x7f383c104f50 2024-09-12T01:57:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.371+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c1a0220 con 0x7f383c104f50 2024-09-12T01:57:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3838ff9640 1 -- 172.21.15.122:0/1750864265 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f38300037a0 con 0x7f383c104f50 2024-09-12T01:57:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 Processor -- start 2024-09-12T01:57:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 -- start start 2024-09-12T01:57:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a4e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28a55c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ed289ad2f0 con 0x55ed28a4e6f0 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed28955270 con 0x55ed28a53780 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3844be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed289a92b0 con 0x55ed28a4ddb0 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f3838ff9640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f38240a2c30 0x7f38240a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57698/0 (socket says 172.21.15.122:57698) 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f383b7fe640 1 -- 172.21.15.122:0/58908286 learned_addr learned my addr 172.21.15.122:0/58908286 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f383bfff640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a4e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f383affd640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28a55c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.373+0000 7f383bfff640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f38240a2c30 0x7f38240a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f3838ff9640 1 -- 172.21.15.122:0/1750864265 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f38300cf340 con 0x7f383c104f50 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383b7fe640 1 -- 172.21.15.122:0/58908286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 msgr2=0x55ed28a4e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383b7fe640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a4e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383b7fe640 1 -- 172.21.15.122:0/58908286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 msgr2=0x55ed28a55c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383b7fe640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28a55c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383b7fe640 1 -- 172.21.15.122:0/58908286 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed28954340 con 0x55ed28a4e6f0 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383bfff640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a4e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383affd640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28a55c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383b7fe640 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a531b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f382c02a930 tx=0x7f382c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ca68c9b42578c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f383bfff640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f38240a2c30 0x7f38240a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3834005c40 tx=0x7f3834009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f3821ffb640 1 -- 172.21.15.122:0/58908286 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c040070 con 0x55ed28a4e6f0 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f3821ffb640 1 -- 172.21.15.122:0/58908286 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c0319b0 con 0x55ed28a4e6f0 2024-09-12T01:57:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.374+0000 7f3821ffb640 1 -- 172.21.15.122:0/58908286 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c035040 con 0x55ed28a4e6f0 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 -- 172.21.15.122:0/58908286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 msgr2=0x55ed28a531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a531b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f382c02a930 tx=0x7f382c009680 comp rx=0 tx=0).stop 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 -- 172.21.15.122:0/58908286 shutdown_connections 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28a55c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a531b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 --2- 172.21.15.122:0/58908286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a4e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 -- 172.21.15.122:0/58908286 >> 172.21.15.122:0/58908286 conn(0x55ed28a45010 msgr2=0x55ed28a45410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 -- 172.21.15.122:0/58908286 shutdown_connections 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 -- 172.21.15.122:0/58908286 wait complete. 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 -- start start 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.375+0000 7f3844be2740 1 Processor -- start 2024-09-12T01:57:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a57f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4e6f0 0x55ed28a58490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a53780 0x55ed28a5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ed28a566f0 con 0x55ed28a53780 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed28936290 con 0x55ed28a4e6f0 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed28a56870 con 0x55ed28a4ddb0 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a57f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a53780 0x55ed28a5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f3844be2740 10 client.0 _ll_register_callbacks cb 0x55ed28967140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a57f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43756/0 (socket says 172.21.15.122:43756) 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a53780 0x55ed28a5f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57712/0 (socket says 172.21.15.122:57712) 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383affd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4e6f0 0x55ed28a58490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 -- 172.21.15.122:0/4105508821 learned_addr learned my addr 172.21.15.122:0/4105508821 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 -- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4e6f0 msgr2=0x55ed28a58490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4e6f0 0x55ed28a58490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 -- 172.21.15.122:0/4105508821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a53780 msgr2=0x55ed28a5f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a53780 0x55ed28a5f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 -- 172.21.15.122:0/4105508821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed286c0210 con 0x55ed28a4ddb0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f383b7fe640 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a57f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3508120f7593e0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.376+0000 7f381affd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f381affd640 1 -- 172.21.15.122:0/4105508821 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c04b020 con 0x55ed28a4ddb0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f381affd640 1 -- 172.21.15.122:0/4105508821 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f382c003a00 con 0x55ed28a4ddb0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c0319b0 con 0x55ed28a4ddb0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f381affd640 1 -- 172.21.15.122:0/4105508821 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f382c035070 con 0x55ed28a4ddb0 2024-09-12T01:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 10 client.26329 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed2878c620 con 0x55ed28a4ddb0 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f381affd640 1 -- 172.21.15.122:0/4105508821 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c0319b0 con 0x55ed28a4ddb0 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f381affd640 1 -- 172.21.15.122:0/4105508821 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f382c03cc20 con 0x55ed28a4ddb0 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 10 client.26329 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 20 client.26329 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28a4a7c0 0x55ed28a4cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 4 client.26329 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed289baa20 con 0x55ed28a4a7c0 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.377+0000 7f383affd640 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28a4a7c0 0x55ed28a4cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.378+0000 7f381affd640 10 client.26329 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.378+0000 7f383affd640 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28a4a7c0 0x55ed28a4cc10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ccd58c4c7ca9245 server_cookie=3b4d276c1f7aaafe in_seq=0 out_seq=0 2024-09-12T01:57:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.378+0000 7f381affd640 1 -- 172.21.15.122:0/4105508821 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed289baa20 con 0x55ed28a4a7c0 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.378+0000 7f381affd640 10 client.26329 handle_command_reply: tid=0 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.378+0000 7f3844be2740 1 client.26329 shutdown 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 msgr2=0x55ed28a57f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a57f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c0042f0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 shutdown_connections 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28a4a7c0 0x55ed28a4cc10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a53780 0x55ed28a5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4e6f0 0x55ed28a58490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f381affd640 0 client.26329 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 --2- 172.21.15.122:0/4105508821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a57f50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 >> 172.21.15.122:0/4105508821 conn(0x55ed28a45010 msgr2=0x55ed28a46820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 shutdown_connections 2024-09-12T01:57:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 1 -- 172.21.15.122:0/4105508821 wait complete. 2024-09-12T01:57:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.379+0000 7f3844be2740 20 client.26329 trim_cache size 0 max 16384 2024-09-12T01:57:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 Processor -- start 2024-09-12T01:57:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 -- start start 2024-09-12T01:57:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 0x55ed28a453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a45930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a53780 0x55ed28a45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1a9700 con 0x55ed28a4e6f0 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1a6fd0 con 0x55ed28a4ddb0 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f3844be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1a4e00 con 0x55ed28a53780 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 0x55ed28a453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f383affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a45930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f383b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 0x55ed28a453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:50408/0 (socket says 172.21.15.122:50408) 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.395+0000 7f383b7fe640 1 -- 172.21.15.122:0/2002794928 learned_addr learned my addr 172.21.15.122:0/2002794928 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383bfff640 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a53780 0x55ed28a45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383b7fe640 1 -- 172.21.15.122:0/2002794928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a53780 msgr2=0x55ed28a45e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383b7fe640 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a53780 0x55ed28a45e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383b7fe640 1 -- 172.21.15.122:0/2002794928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 msgr2=0x55ed28a45930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383b7fe640 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a45930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383b7fe640 1 -- 172.21.15.122:0/2002794928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed28a572b0 con 0x55ed28a4ddb0 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383affd640 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a45930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f383b7fe640 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 0x55ed28a453f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f382c031610 tx=0x7f382c0045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3afc4d6a28c1321 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f38217fa640 1 -- 172.21.15.122:0/2002794928 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c04b040 con 0x55ed28a4ddb0 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f38217fa640 1 -- 172.21.15.122:0/2002794928 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c044e50 con 0x55ed28a4ddb0 2024-09-12T01:57:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.396+0000 7f38217fa640 1 -- 172.21.15.122:0/2002794928 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c040040 con 0x55ed28a4ddb0 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 -- 172.21.15.122:0/2002794928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 msgr2=0x55ed28a453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 0x55ed28a453f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f382c031610 tx=0x7f382c0045c0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 -- 172.21.15.122:0/2002794928 shutdown_connections 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a53780 0x55ed28a45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a45930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 --2- 172.21.15.122:0/2002794928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a4ddb0 0x55ed28a453f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 -- 172.21.15.122:0/2002794928 >> 172.21.15.122:0/2002794928 conn(0x55ed28a45010 msgr2=0x55ed28a57d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 -- 172.21.15.122:0/2002794928 shutdown_connections 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 -- 172.21.15.122:0/2002794928 wait complete. 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.397+0000 7f3844be2740 1 -- start start 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 Processor -- start 2024-09-12T01:57:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a67270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28ac88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1a5490 con 0x55ed28a4e6f0 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1a6900 con 0x55ed28a53780 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1a5b60 con 0x55ed28a4ddb0 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 10 client.0 _ll_register_callbacks cb 0x7f383000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57732/0 (socket says 172.21.15.122:57732) 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 -- 172.21.15.122:0/1278187440 learned_addr learned my addr 172.21.15.122:0/1278187440 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383b7fe640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a67270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383bfff640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28ac88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 -- 172.21.15.122:0/1278187440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 msgr2=0x55ed28a67270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a67270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 -- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 msgr2=0x55ed28ac88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28ac88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 -- 172.21.15.122:0/1278187440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed28ac8f30 con 0x55ed28a4e6f0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f383affd640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a677b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3830098000 tx=0x7f383006ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d2eddc5eb41378 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f381affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f381affd640 1 -- 172.21.15.122:0/1278187440 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383009bba0 con 0x55ed28a4e6f0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f381affd640 1 -- 172.21.15.122:0/1278187440 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3830003b10 con 0x55ed28a4e6f0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.398+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c044e50 con 0x55ed28a4e6f0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f381affd640 1 -- 172.21.15.122:0/1278187440 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f383009ff50 con 0x55ed28a4e6f0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 10 client.17187 fetch_fsmap learned FSMap version 30 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed28aca2b0 con 0x55ed28a4e6f0 2024-09-12T01:57:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f381affd640 1 -- 172.21.15.122:0/1278187440 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c044e50 con 0x55ed28a4e6f0 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f381affd640 1 -- 172.21.15.122:0/1278187440 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f383006bac0 con 0x55ed28a4e6f0 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 10 client.17187 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 20 client.17187 populate_metadata read hostname 'smithi122' 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28acabc0 0x55ed28accfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 4 client.17187 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed289b4d70 con 0x55ed28acabc0 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.399+0000 7f383b7fe640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28acabc0 0x55ed28accfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.400+0000 7f383b7fe640 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28acabc0 0x55ed28accfe0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5f6a853d198602 server_cookie=86f973c22a5a1212 in_seq=0 out_seq=0 2024-09-12T01:57:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.400+0000 7f381affd640 10 client.17187 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f381affd640 1 -- 172.21.15.122:0/1278187440 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed289b4d70 con 0x55ed28acabc0 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f381affd640 10 client.17187 handle_command_reply: tid=0 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 client.17187 shutdown 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 msgr2=0x55ed28a677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a677b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3830098000 tx=0x7f383006ab20 comp rx=0 tx=0).stop 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 shutdown_connections 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ed28acabc0 0x55ed28accfe0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed28a53780 0x55ed28ac88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f381affd640 0 client.17187 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ed28a4e6f0 0x55ed28a677b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 --2- 172.21.15.122:0/1278187440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed28a4ddb0 0x55ed28a67270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 >> 172.21.15.122:0/1278187440 conn(0x55ed28a45010 msgr2=0x55ed28a439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 shutdown_connections 2024-09-12T01:57:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 1 -- 172.21.15.122:0/1278187440 wait complete. 2024-09-12T01:57:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.411+0000 7f3844be2740 20 client.17187 trim_cache size 0 max 16384 2024-09-12T01:57:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:57:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:57:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:57:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:57:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f38240a2c30 msgr2=0x7f38240a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f38240a2c30 0x7f38240a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3834005c40 tx=0x7f3834009500 comp rx=0 tx=0).stop 2024-09-12T01:57:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 msgr2=0x7f383c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:57:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 0x7f383c198390 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f383000e9a0 tx=0x7f383000ee60 comp rx=0 tx=0).stop 2024-09-12T01:57:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 shutdown_connections 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f38240a2c30 0x7f38240a50f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c107880 0x7f383c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f383c104f50 0x7f383c198390 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 --2- 172.21.15.122:0/1750864265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f383c102620 0x7f383c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 >> 172.21.15.122:0/1750864265 conn(0x7f383c0fc1e0 msgr2=0x7f383c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 shutdown_connections 2024-09-12T01:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:57:50.412+0000 7f3841d6f640 1 -- 172.21.15.122:0/1750864265 wait complete. 2024-09-12T01:57:50.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1083968 10.04 MB/sec execute 488 sec latency 11.777 ms 2024-09-12T01:57:51.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1087108 10.06 MB/sec execute 489 sec latency 11.339 ms 2024-09-12T01:57:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:51 smithi122 ceph-mon[30925]: pgmap v676: 97 pgs: 97 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 312 op/s 2024-09-12T01:57:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:51 smithi151 ceph-mon[35229]: pgmap v676: 97 pgs: 97 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 312 op/s 2024-09-12T01:57:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:51 smithi183 ceph-mon[35127]: pgmap v676: 97 pgs: 97 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 312 op/s 2024-09-12T01:57:52.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1089520 10.05 MB/sec execute 490 sec latency 13.893 ms 2024-09-12T01:57:52.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:52.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:52.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:52 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:53.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:53.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:53.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:52 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:57:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:52 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:53.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1091234 10.06 MB/sec execute 491 sec latency 54.583 ms 2024-09-12T01:57:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:53 smithi122 ceph-mon[30925]: pgmap v677: 97 pgs: 97 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-09-12T01:57:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:53 smithi151 ceph-mon[35229]: pgmap v677: 97 pgs: 97 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-09-12T01:57:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:53 smithi183 ceph-mon[35127]: pgmap v677: 97 pgs: 97 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-09-12T01:57:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:57:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:57:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:57:54.857 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1094296 10.07 MB/sec execute 492 sec latency 11.773 ms 2024-09-12T01:57:55.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1096981 10.08 MB/sec execute 493 sec latency 13.507 ms 2024-09-12T01:57:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:55 smithi122 ceph-mon[30925]: pgmap v678: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 308 op/s 2024-09-12T01:57:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:55 smithi151 ceph-mon[35229]: pgmap v678: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 308 op/s 2024-09-12T01:57:56.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:55 smithi183 ceph-mon[35127]: pgmap v678: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 308 op/s 2024-09-12T01:57:56.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1099292 10.08 MB/sec execute 494 sec latency 10.473 ms 2024-09-12T01:57:57.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1101854 10.09 MB/sec execute 495 sec latency 11.048 ms 2024-09-12T01:57:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:57 smithi122 ceph-mon[30925]: pgmap v679: 97 pgs: 97 active+clean; 303 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:57:58.003 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:57 smithi183 ceph-mon[35127]: pgmap v679: 97 pgs: 97 active+clean; 303 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:57:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:57 smithi151 ceph-mon[35229]: pgmap v679: 97 pgs: 97 active+clean; 303 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:57:58.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1104654 10.11 MB/sec execute 496 sec latency 102.468 ms 2024-09-12T01:57:59.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1107290 10.10 MB/sec execute 497 sec latency 11.782 ms 2024-09-12T01:57:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:57:59 smithi122 ceph-mon[30925]: pgmap v680: 97 pgs: 97 active+clean; 292 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-09-12T01:58:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:57:59 smithi151 ceph-mon[35229]: pgmap v680: 97 pgs: 97 active+clean; 292 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-09-12T01:58:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:57:59 smithi183 ceph-mon[35127]: pgmap v680: 97 pgs: 97 active+clean; 292 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-09-12T01:58:00.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1109754 10.12 MB/sec execute 498 sec latency 10.702 ms 2024-09-12T01:58:01.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1112517 10.13 MB/sec execute 499 sec latency 11.084 ms 2024-09-12T01:58:01.992 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:01 smithi122 ceph-mon[30925]: pgmap v681: 97 pgs: 97 active+clean; 294 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 323 op/s 2024-09-12T01:58:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:01 smithi151 ceph-mon[35229]: pgmap v681: 97 pgs: 97 active+clean; 294 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 323 op/s 2024-09-12T01:58:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:01 smithi183 ceph-mon[35127]: pgmap v681: 97 pgs: 97 active+clean; 294 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 323 op/s 2024-09-12T01:58:02.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1115573 10.14 MB/sec execute 500 sec latency 46.690 ms 2024-09-12T01:58:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 -- 172.21.15.122:0/544116286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 msgr2=0x7fd91c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 --2- 172.21.15.122:0/544116286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c109cd0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd914009ca0 tx=0x7fd91402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 -- 172.21.15.122:0/544116286 shutdown_connections 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 --2- 172.21.15.122:0/544116286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c109cd0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 --2- 172.21.15.122:0/544116286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c104fb0 0x7fd91c1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 --2- 172.21.15.122:0/544116286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 -- 172.21.15.122:0/544116286 >> 172.21.15.122:0/544116286 conn(0x7fd91c0fc1e0 msgr2=0x7fd91c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 -- 172.21.15.122:0/544116286 shutdown_connections 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.524+0000 7fd921c42640 1 -- 172.21.15.122:0/544116286 wait complete. 2024-09-12T01:58:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 Processor -- start 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 -- start start 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c104fb0 0x7fd91c100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c100b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d7d0 con 0x7fd91c1078e0 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d650 con 0x7fd91c104fb0 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd921c42640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d950 con 0x7fd91c102680 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd91bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c100b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd91bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c100b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39898/0 (socket says 172.21.15.122:39898) 2024-09-12T01:58:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:49082/0 (socket says 172.21.15.122:49082) 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd91bfff640 1 -- 172.21.15.122:0/3130290920 learned_addr learned my addr 172.21.15.122:0/3130290920 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.525+0000 7fd91affd640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c104fb0 0x7fd91c100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91bfff640 1 -- 172.21.15.122:0/3130290920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 msgr2=0x7fd91c101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91bfff640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91bfff640 1 -- 172.21.15.122:0/3130290920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c104fb0 msgr2=0x7fd91c100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91bfff640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c104fb0 0x7fd91c100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91bfff640 1 -- 172.21.15.122:0/3130290920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c101400 con 0x7fd91c1078e0 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd91bfff640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c100b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd914031ae0 tx=0x7fd914031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd918ff9640 1 -- 172.21.15.122:0/3130290920 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914003c30 con 0x7fd91c1078e0 2024-09-12T01:58:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c1a1ef0 con 0x7fd91c1078e0 2024-09-12T01:58:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd918ff9640 1 -- 172.21.15.122:0/3130290920 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd914003dd0 con 0x7fd91c1078e0 2024-09-12T01:58:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd918ff9640 1 -- 172.21.15.122:0/3130290920 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91402fcb0 con 0x7fd91c1078e0 2024-09-12T01:58:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.526+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c1a2350 con 0x7fd91c1078e0 2024-09-12T01:58:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.528+0000 7fd918ff9640 1 -- 172.21.15.122:0/3130290920 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd914004050 con 0x7fd91c1078e0 2024-09-12T01:58:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 Processor -- start 2024-09-12T01:58:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 -- start start 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a26cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a271da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a274830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555c0a1d02f0 con 0x555c0a272370 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555c0a178270 con 0x555c0a26d2e0 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd9249d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c0a1cc2b0 con 0x555c0a26c9a0 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a271da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a271da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:60498/0 (socket says 172.21.15.122:60498) 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 -- 172.21.15.122:0/24692297 learned_addr learned my addr 172.21.15.122:0/24692297 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91affd640 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a274830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91affd640 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a26cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd918ff9640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9040a2c30 0x7fd9040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 -- 172.21.15.122:0/24692297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 msgr2=0x555c0a26cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a26cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 -- 172.21.15.122:0/24692297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 msgr2=0x555c0a274830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a274830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91b7fe640 1 -- 172.21.15.122:0/24692297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0a177340 con 0x555c0a26d2e0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd91bfff640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9040a2c30 0x7fd9040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.529+0000 7fd918ff9640 1 -- 172.21.15.122:0/3130290920 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd9140f01f0 con 0x7fd91c1078e0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a271da0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0104d0 tx=0x7fd90c010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9ca3748cec8e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd901ffb640 1 -- 172.21.15.122:0/24692297 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c004220 con 0x555c0a26d2e0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd901ffb640 1 -- 172.21.15.122:0/24692297 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0043c0 con 0x555c0a26d2e0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd901ffb640 1 -- 172.21.15.122:0/24692297 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011a70 con 0x555c0a26d2e0 2024-09-12T01:58:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd9249d2740 1 -- 172.21.15.122:0/24692297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 msgr2=0x555c0a271da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd91bfff640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9040a2c30 0x7fd9040a50f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9140bc750 tx=0x7fd914087c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.530+0000 7fd9249d2740 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a271da0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0104d0 tx=0x7fd90c010990 comp rx=0 tx=0).stop 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- 172.21.15.122:0/24692297 shutdown_connections 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a274830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a271da0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 --2- 172.21.15.122:0/24692297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a26cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- 172.21.15.122:0/24692297 >> 172.21.15.122:0/24692297 conn(0x555c0a268010 msgr2=0x555c0a268410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- 172.21.15.122:0/24692297 shutdown_connections 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- 172.21.15.122:0/24692297 wait complete. 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- start start 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 Processor -- start 2024-09-12T01:58:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26c9a0 0x555c0a276ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26d2e0 0x555c0a2770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a27e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555c0a275340 con 0x555c0a272370 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555c0a159290 con 0x555c0a26c9a0 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c0a2754c0 con 0x555c0a26d2e0 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.531+0000 7fd9249d2740 10 client.0 _ll_register_callbacks cb 0x555c0a18a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26c9a0 0x555c0a276ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a27e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a27e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39920/0 (socket says 172.21.15.122:39920) 2024-09-12T01:58:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 -- 172.21.15.122:0/2737644660 learned_addr learned my addr 172.21.15.122:0/2737644660 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26d2e0 0x555c0a2770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 -- 172.21.15.122:0/2737644660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26d2e0 msgr2=0x555c0a2770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26d2e0 0x555c0a2770e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 -- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26c9a0 msgr2=0x555c0a276ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26c9a0 0x555c0a276ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 -- 172.21.15.122:0/2737644660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c09ee3210 con 0x555c0a272370 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd91affd640 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a27e120 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd910012fc0 tx=0x7fd9100096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd918ad4c620f6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd8faffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd8faffd640 1 -- 172.21.15.122:0/2737644660 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910011050 con 0x555c0a272370 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd8faffd640 1 -- 172.21.15.122:0/2737644660 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd9100111f0 con 0x555c0a272370 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd90c0043c0 con 0x555c0a272370 2024-09-12T01:58:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd8faffd640 1 -- 172.21.15.122:0/2737644660 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd910004250 con 0x555c0a272370 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd9249d2740 10 client.17199 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.532+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c09faf620 con 0x555c0a272370 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd8faffd640 1 -- 172.21.15.122:0/2737644660 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd90c0043c0 con 0x555c0a272370 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd8faffd640 1 -- 172.21.15.122:0/2737644660 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd910021460 con 0x555c0a272370 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd9249d2740 10 client.17199 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd9249d2740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd9249d2740 20 client.17199 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd9249d2740 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a284c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd9249d2740 4 client.17199 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0a1dda20 con 0x555c0a282810 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a284c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a284c40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd23ca793cf826f2 server_cookie=3b671a246e2efc48 in_seq=0 out_seq=0 2024-09-12T01:58:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.533+0000 7fd8faffd640 10 client.17199 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd8faffd640 1 -- 172.21.15.122:0/2737644660 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c0a1dda20 con 0x555c0a282810 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd8faffd640 10 client.17199 handle_command_reply: tid=0 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 client.17199 shutdown 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 msgr2=0x555c0a27e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a27e120 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd910012fc0 tx=0x7fd9100096c0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 shutdown_connections 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a284c40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a27e120 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26d2e0 0x555c0a2770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd8faffd640 0 client.17199 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:03.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 --2- 172.21.15.122:0/2737644660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26c9a0 0x555c0a276ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 >> 172.21.15.122:0/2737644660 conn(0x555c0a268010 msgr2=0x555c0a269820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 shutdown_connections 2024-09-12T01:58:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 1 -- 172.21.15.122:0/2737644660 wait complete. 2024-09-12T01:58:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.534+0000 7fd9249d2740 20 client.17199 trim_cache size 0 max 16384 2024-09-12T01:58:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.551+0000 7fd9249d2740 1 Processor -- start 2024-09-12T01:58:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.551+0000 7fd9249d2740 1 -- start start 2024-09-12T01:58:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.551+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a271b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a27e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a268ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9249d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1adb50 con 0x555c0a272370 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9249d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1ab420 con 0x555c0a26d2e0 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9249d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1a9250 con 0x555c0a26c9a0 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a268ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a268ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39928/0 (socket says 172.21.15.122:39928) 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 -- 172.21.15.122:0/3291810525 learned_addr learned my addr 172.21.15.122:0/3291810525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a27e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a271b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 -- 172.21.15.122:0/3291810525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 msgr2=0x555c0a271b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a271b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 -- 172.21.15.122:0/3291810525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 msgr2=0x555c0a27e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a27e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 -- 172.21.15.122:0/3291810525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0a26d7c0 con 0x555c0a272370 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd91affd640 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a268ec0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd910012530 tx=0x7fd910005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e470b3f234ac9669 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9017fa640 1 -- 172.21.15.122:0/3291810525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910030040 con 0x555c0a272370 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9017fa640 1 -- 172.21.15.122:0/3291810525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9100044b0 con 0x555c0a272370 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.552+0000 7fd9017fa640 1 -- 172.21.15.122:0/3291810525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9100039b0 con 0x555c0a272370 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 -- 172.21.15.122:0/3291810525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 msgr2=0x555c0a268ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a268ec0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd910012530 tx=0x7fd910005f90 comp rx=0 tx=0).stop 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 -- 172.21.15.122:0/3291810525 shutdown_connections 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a272370 0x555c0a268ec0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a26d2e0 0x555c0a27e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 --2- 172.21.15.122:0/3291810525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a271b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 -- 172.21.15.122:0/3291810525 >> 172.21.15.122:0/3291810525 conn(0x555c0a268010 msgr2=0x555c0a276ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 -- 172.21.15.122:0/3291810525 shutdown_connections 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 -- 172.21.15.122:0/3291810525 wait complete. 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.553+0000 7fd9249d2740 1 -- start start 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 Processor -- start 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a27b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 0x555c0a27b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a272370 0x555c0a27c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1a98e0 con 0x555c0a26d2e0 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1aad50 con 0x555c0a272370 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1a9fb0 con 0x555c0a26c9a0 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd9249d2740 10 client.0 _ll_register_callbacks cb 0x7fd910012560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 0x555c0a27b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 0x555c0a27b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39938/0 (socket says 172.21.15.122:39938) 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 -- 172.21.15.122:0/1748039984 learned_addr learned my addr 172.21.15.122:0/1748039984 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91affd640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a272370 0x555c0a27c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91affd640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a27b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 -- 172.21.15.122:0/1748039984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 msgr2=0x555c0a27b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a27b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 -- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a272370 msgr2=0x555c0a27c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a272370 0x555c0a27c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 -- 172.21.15.122:0/1748039984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0a27ced0 con 0x555c0a26d2e0 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd91b7fe640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 0x555c0a27b8b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0104d0 tx=0x7fd90c00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa914616c51a110 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.554+0000 7fd8faffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd8faffd640 1 -- 172.21.15.122:0/1748039984 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c034060 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd8faffd640 1 -- 172.21.15.122:0/1748039984 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd90c004280 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9100044b0 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd8faffd640 1 -- 172.21.15.122:0/1748039984 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd90c004480 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 10 client.17205 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c0a27e250 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd8faffd640 1 -- 172.21.15.122:0/1748039984 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9100044b0 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd8faffd640 1 -- 172.21.15.122:0/1748039984 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd90c039070 con 0x555c0a26d2e0 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 10 client.17205 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 20 client.17205 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a2ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 4 client.17205 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:58:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c0a1d7d70 con 0x555c0a282810 2024-09-12T01:58:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.555+0000 7fd91affd640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a2ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.556+0000 7fd91affd640 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a2ef8c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc264787f9ae137f server_cookie=da76d041f2eded0 in_seq=0 out_seq=0 2024-09-12T01:58:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.556+0000 7fd8faffd640 10 client.17205 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.568+0000 7fd8faffd640 1 -- 172.21.15.122:0/1748039984 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c0a1d7d70 con 0x555c0a282810 2024-09-12T01:58:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.568+0000 7fd8faffd640 10 client.17205 handle_command_reply: tid=0 2024-09-12T01:58:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 client.17205 shutdown 2024-09-12T01:58:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 msgr2=0x555c0a27b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 0x555c0a27b8b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0104d0 tx=0x7fd90c00bb60 comp rx=0 tx=0).stop 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 shutdown_connections 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555c0a282810 0x555c0a2ef8c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555c0a272370 0x555c0a27c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555c0a26d2e0 0x555c0a27b8b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 --2- 172.21.15.122:0/1748039984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0a26c9a0 0x555c0a27b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd8faffd640 0 client.17205 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 >> 172.21.15.122:0/1748039984 conn(0x555c0a268010 msgr2=0x555c0a276b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 shutdown_connections 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 1 -- 172.21.15.122:0/1748039984 wait complete. 2024-09-12T01:58:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd9249d2740 20 client.17205 trim_cache size 0 max 16384 2024-09-12T01:58:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:58:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:58:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:58:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9040a2c30 msgr2=0x7fd9040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd921c42640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9040a2c30 0x7fd9040a50f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9140bc750 tx=0x7fd914087c90 comp rx=0 tx=0).stop 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 msgr2=0x7fd91c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd921c42640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c100b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd914031ae0 tx=0x7fd914031ee0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 shutdown_connections 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.569+0000 7fd921c42640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd9040a2c30 0x7fd9040a50f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.570+0000 7fd921c42640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd91c1078e0 0x7fd91c100b70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.570+0000 7fd921c42640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd91c104fb0 0x7fd91c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.570+0000 7fd921c42640 1 --2- 172.21.15.122:0/3130290920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd91c102680 0x7fd91c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.570+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 >> 172.21.15.122:0/3130290920 conn(0x7fd91c0fc1e0 msgr2=0x7fd91c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.570+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 shutdown_connections 2024-09-12T01:58:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:03.570+0000 7fd921c42640 1 -- 172.21.15.122:0/3130290920 wait complete. 2024-09-12T01:58:03.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1117795 10.14 MB/sec execute 501 sec latency 61.129 ms 2024-09-12T01:58:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:03 smithi122 ceph-mon[30925]: pgmap v682: 97 pgs: 97 active+clean; 297 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:58:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:03 smithi151 ceph-mon[35229]: pgmap v682: 97 pgs: 97 active+clean; 297 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:58:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:03 smithi183 ceph-mon[35127]: pgmap v682: 97 pgs: 97 active+clean; 297 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:58:04.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1119966 10.14 MB/sec execute 502 sec latency 11.249 ms 2024-09-12T01:58:05.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1122889 10.16 MB/sec execute 503 sec latency 10.189 ms 2024-09-12T01:58:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:05 smithi122 ceph-mon[30925]: pgmap v683: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2024-09-12T01:58:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:05 smithi151 ceph-mon[35229]: pgmap v683: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2024-09-12T01:58:06.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:05 smithi183 ceph-mon[35127]: pgmap v683: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2024-09-12T01:58:06.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1125970 10.17 MB/sec execute 504 sec latency 13.926 ms 2024-09-12T01:58:07.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1128549 10.17 MB/sec execute 505 sec latency 15.929 ms 2024-09-12T01:58:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:07 smithi122 ceph-mon[30925]: pgmap v684: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-12T01:58:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:07 smithi151 ceph-mon[35229]: pgmap v684: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-12T01:58:08.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:07 smithi183 ceph-mon[35127]: pgmap v684: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-12T01:58:08.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1130409 10.18 MB/sec execute 506 sec latency 102.971 ms 2024-09-12T01:58:09.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1133448 10.19 MB/sec execute 507 sec latency 34.460 ms 2024-09-12T01:58:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:09 smithi122 ceph-mon[30925]: pgmap v685: 97 pgs: 97 active+clean; 305 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-09-12T01:58:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:09 smithi151 ceph-mon[35229]: pgmap v685: 97 pgs: 97 active+clean; 305 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-09-12T01:58:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:09 smithi183 ceph-mon[35127]: pgmap v685: 97 pgs: 97 active+clean; 305 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-09-12T01:58:10.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1136382 10.20 MB/sec execute 508 sec latency 10.143 ms 2024-09-12T01:58:11.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1139108 10.20 MB/sec execute 509 sec latency 10.290 ms 2024-09-12T01:58:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:11 smithi122 ceph-mon[30925]: pgmap v686: 97 pgs: 97 active+clean; 306 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:58:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:11 smithi151 ceph-mon[35229]: pgmap v686: 97 pgs: 97 active+clean; 306 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:58:12.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:11 smithi183 ceph-mon[35127]: pgmap v686: 97 pgs: 97 active+clean; 306 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-12T01:58:12.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1141439 10.21 MB/sec execute 510 sec latency 16.540 ms 2024-09-12T01:58:13.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1144153 10.22 MB/sec execute 511 sec latency 75.627 ms 2024-09-12T01:58:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:13 smithi122 ceph-mon[30925]: pgmap v687: 97 pgs: 97 active+clean; 309 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-09-12T01:58:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:13 smithi151 ceph-mon[35229]: pgmap v687: 97 pgs: 97 active+clean; 309 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-09-12T01:58:14.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:13 smithi183 ceph-mon[35127]: pgmap v687: 97 pgs: 97 active+clean; 309 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-09-12T01:58:14.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1147120 10.23 MB/sec execute 512 sec latency 27.405 ms 2024-09-12T01:58:15.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1150011 10.24 MB/sec execute 513 sec latency 15.497 ms 2024-09-12T01:58:15.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:15 smithi122 ceph-mon[30925]: pgmap v688: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-09-12T01:58:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:15 smithi151 ceph-mon[35229]: pgmap v688: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-09-12T01:58:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:15 smithi183 ceph-mon[35127]: pgmap v688: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 285 op/s 2024-09-12T01:58:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 -- 172.21.15.122:0/1371282075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b01025f0 msgr2=0x7f63b0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 --2- 172.21.15.122:0/1371282075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b01025f0 0x7f63b0108e80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009ca0 tx=0x7f63a802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 -- 172.21.15.122:0/1371282075 shutdown_connections 2024-09-12T01:58:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 --2- 172.21.15.122:0/1371282075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b01025f0 0x7f63b0108e80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 --2- 172.21.15.122:0/1371282075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 0x7f63b01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 --2- 172.21.15.122:0/1371282075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b0100a40 0x7f63b0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 -- 172.21.15.122:0/1371282075 >> 172.21.15.122:0/1371282075 conn(0x7f63b00fc1e0 msgr2=0x7f63b00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 -- 172.21.15.122:0/1371282075 shutdown_connections 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.664+0000 7f63b488d640 1 -- 172.21.15.122:0/1371282075 wait complete. 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 Processor -- start 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 -- start start 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 0x7f63b010b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 0x7f63b0109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b01025f0 0x7f63b0109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d810 con 0x7f63b0100a40 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d690 con 0x7f63b0101c30 2024-09-12T01:58:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63b488d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d990 con 0x7f63b01025f0 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 0x7f63b010b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 0x7f63b010b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41766/0 (socket says 172.21.15.122:41766) 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63ae59c640 1 -- 172.21.15.122:0/1758936175 learned_addr learned my addr 172.21.15.122:0/1758936175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63add9b640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 0x7f63b0109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.665+0000 7f63aed9d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b01025f0 0x7f63b0109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63add9b640 1 -- 172.21.15.122:0/1758936175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b01025f0 msgr2=0x7f63b0109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63add9b640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b01025f0 0x7f63b0109f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63add9b640 1 -- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 msgr2=0x7f63b010b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63add9b640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 0x7f63b010b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63add9b640 1 -- 172.21.15.122:0/1758936175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b010a7d0 con 0x7f63b0101c30 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63ae59c640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 0x7f63b010b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63add9b640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 0x7f63b0109a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e9f0 tx=0x7f63a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f639f7fe640 1 -- 172.21.15.122:0/1758936175 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a400cd80 con 0x7f63b0101c30 2024-09-12T01:58:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f639f7fe640 1 -- 172.21.15.122:0/1758936175 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a40035f0 con 0x7f63b0101c30 2024-09-12T01:58:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01a1e30 con 0x7f63b0101c30 2024-09-12T01:58:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f639f7fe640 1 -- 172.21.15.122:0/1758936175 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4010690 con 0x7f63b0101c30 2024-09-12T01:58:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.666+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a2290 con 0x7f63b0101c30 2024-09-12T01:58:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.668+0000 7f63b7628740 1 Processor -- start 2024-09-12T01:58:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f639f7fe640 1 -- 172.21.15.122:0/1758936175 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f63a4003b30 con 0x7f63b0101c30 2024-09-12T01:58:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 -- start start 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc729a0 0x5571ffc72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 0x5571ffc77da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc7a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571ffbd62f0 con 0x5571ffc732e0 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571ffb7e270 con 0x5571ffc729a0 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63b7628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5571ffbd22b0 con 0x5571ffc78370 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 0x5571ffc77da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 0x5571ffc77da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41780/0 (socket says 172.21.15.122:41780) 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 -- 172.21.15.122:0/1501026244 learned_addr learned my addr 172.21.15.122:0/1501026244 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63add9b640 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc7a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63aed9d640 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc729a0 0x5571ffc72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 -- 172.21.15.122:0/1501026244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 msgr2=0x5571ffc7a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc7a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 -- 172.21.15.122:0/1501026244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc729a0 msgr2=0x5571ffc72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc729a0 0x5571ffc72da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.669+0000 7f63ae59c640 1 -- 172.21.15.122:0/1501026244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ffb7d340 con 0x5571ffc732e0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f639f7fe640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63980a2c30 0x7f63980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f63ae59c640 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 0x5571ffc77da0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010770 tx=0x7f63a0010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18cd34e62da0a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f63aed9d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63980a2c30 0x7f63980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f639f7fe640 1 -- 172.21.15.122:0/1758936175 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f63a40cf210 con 0x7f63b0101c30 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f639cff9640 1 -- 172.21.15.122:0/1501026244 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a000e930 con 0x5571ffc732e0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f639cff9640 1 -- 172.21.15.122:0/1501026244 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0011440 con 0x5571ffc732e0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.670+0000 7f639cff9640 1 -- 172.21.15.122:0/1501026244 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0019630 con 0x5571ffc732e0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 -- 172.21.15.122:0/1501026244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 msgr2=0x5571ffc77da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63aed9d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63980a2c30 0x7f63980a50f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005e30 tx=0x7f63a803d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 0x5571ffc77da0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010770 tx=0x7f63a0010c30 comp rx=0 tx=0).stop 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 -- 172.21.15.122:0/1501026244 shutdown_connections 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc7a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc732e0 0x5571ffc77da0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 --2- 172.21.15.122:0/1501026244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc729a0 0x5571ffc72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 -- 172.21.15.122:0/1501026244 >> 172.21.15.122:0/1501026244 conn(0x5571ffc6e010 msgr2=0x5571ffc6e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 -- 172.21.15.122:0/1501026244 shutdown_connections 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 -- 172.21.15.122:0/1501026244 wait complete. 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 -- start start 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 Processor -- start 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.671+0000 7f63b7628740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 0x5571ffc7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc84110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571ffc7b340 con 0x5571ffc729a0 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571ffb5f290 con 0x5571ffc732e0 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5571ffc7b4c0 con 0x5571ffc78370 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 10 client.0 _ll_register_callbacks cb 0x5571ffb90140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 0x5571ffc7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 0x5571ffc7cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41792/0 (socket says 172.21.15.122:41792) 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 -- 172.21.15.122:0/941897143 learned_addr learned my addr 172.21.15.122:0/941897143 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63add9b640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63aed9d640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc84110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 -- 172.21.15.122:0/941897143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 msgr2=0x5571ffc84110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc84110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 -- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 msgr2=0x5571ffc7d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc7d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 -- 172.21.15.122:0/941897143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ff8e9210 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63ae59c640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 0x5571ffc7cb90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f63a0000c00 tx=0x7f63a00104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad65f4e34be7e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f6395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f6395ffb640 1 -- 172.21.15.122:0/941897143 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0027030 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f6395ffb640 1 -- 172.21.15.122:0/941897143 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f63a001f070 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0011440 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f6395ffb640 1 -- 172.21.15.122:0/941897143 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f63a0010d60 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 10 client.17217 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.672+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571ff9b5620 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f6395ffb640 1 -- 172.21.15.122:0/941897143 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0011440 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f6395ffb640 1 -- 172.21.15.122:0/941897143 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f63a000ea50 con 0x5571ffc729a0 2024-09-12T01:58:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63b7628740 10 client.17217 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63b7628740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63b7628740 20 client.17217 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63b7628740 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffc885e0 0x5571ffc8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63b7628740 4 client.17217 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ffbe3a20 con 0x5571ffc885e0 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63add9b640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffc885e0 0x5571ffc8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f63add9b640 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffc885e0 0x5571ffc8aa60 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b79165f47756f11 server_cookie=20776badfe9fcee6 in_seq=0 out_seq=0 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.673+0000 7f6395ffb640 10 client.17217 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f6395ffb640 1 -- 172.21.15.122:0/941897143 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571ffbe3a20 con 0x5571ffc885e0 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f6395ffb640 10 client.17217 handle_command_reply: tid=0 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 client.17217 shutdown 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 msgr2=0x5571ffc7cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 0x5571ffc7cb90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f63a0000c00 tx=0x7f63a00104a0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 shutdown_connections 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffc885e0 0x5571ffc8aa60 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc78370 0x5571ffc84110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc7d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 --2- 172.21.15.122:0/941897143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc729a0 0x5571ffc7cb90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 >> 172.21.15.122:0/941897143 conn(0x5571ffc6e010 msgr2=0x5571ffc6f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f6395ffb640 0 client.17217 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 shutdown_connections 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 1 -- 172.21.15.122:0/941897143 wait complete. 2024-09-12T01:58:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.674+0000 7f63b7628740 20 client.17217 trim_cache size 0 max 16384 2024-09-12T01:58:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.689+0000 7f63b7628740 1 Processor -- start 2024-09-12T01:58:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 -- start start 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc7c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffc7ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01adb10 con 0x5571ffc78370 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ab3e0 con 0x5571ffc732e0 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63b7628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a9210 con 0x5571ffc729a0 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffc7ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffc7ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41804/0 (socket says 172.21.15.122:41804) 2024-09-12T01:58:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 -- 172.21.15.122:0/3602327272 learned_addr learned my addr 172.21.15.122:0/3602327272 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63ae59c640 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63add9b640 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 -- 172.21.15.122:0/3602327272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 msgr2=0x5571ffc7c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc7c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 -- 172.21.15.122:0/3602327272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 msgr2=0x5571ffc840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 -- 172.21.15.122:0/3602327272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ffc7b660 con 0x5571ffc78370 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f63aed9d640 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffc7ce80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009ca0 tx=0x7f63a8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756ede6e2de09825 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.690+0000 7f6397fff640 1 -- 172.21.15.122:0/3602327272 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8004240 con 0x5571ffc78370 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f6397fff640 1 -- 172.21.15.122:0/3602327272 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8056070 con 0x5571ffc78370 2024-09-12T01:58:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f6397fff640 1 -- 172.21.15.122:0/3602327272 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80305c0 con 0x5571ffc78370 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 -- 172.21.15.122:0/3602327272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 msgr2=0x5571ffc7ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffc7ce80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009ca0 tx=0x7f63a8041000 comp rx=0 tx=0).stop 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 -- 172.21.15.122:0/3602327272 shutdown_connections 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffc7ce80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 --2- 172.21.15.122:0/3602327272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc7c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 -- 172.21.15.122:0/3602327272 >> 172.21.15.122:0/3602327272 conn(0x5571ffc6e010 msgr2=0x5571ffc6e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 -- 172.21.15.122:0/3602327272 shutdown_connections 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.691+0000 7f63b7628740 1 -- 172.21.15.122:0/3602327272 wait complete. 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 -- start start 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 Processor -- start 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc80f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffcf12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a98a0 con 0x5571ffc78370 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aad10 con 0x5571ffc732e0 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a9f70 con 0x5571ffc729a0 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63b7628740 10 client.0 _ll_register_callbacks cb 0x7f63a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffcf12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffcf12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41820/0 (socket says 172.21.15.122:41820) 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 -- 172.21.15.122:0/2638366692 learned_addr learned my addr 172.21.15.122:0/2638366692 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63add9b640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63ae59c640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc80f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 -- 172.21.15.122:0/2638366692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 msgr2=0x5571ffc80f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc80f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 -- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 msgr2=0x5571ffc814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 -- 172.21.15.122:0/2638366692 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ffcf1830 con 0x5571ffc78370 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f63aed9d640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffcf12f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f63a802ade0 tx=0x7f63a802f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa04c13d945ef66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.692+0000 7f6395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f6395ffb640 1 -- 172.21.15.122:0/2638366692 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80309d0 con 0x5571ffc78370 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f6395ffb640 1 -- 172.21.15.122:0/2638366692 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f63a8030b70 con 0x5571ffc78370 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a8056070 con 0x5571ffc78370 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f6395ffb640 1 -- 172.21.15.122:0/2638366692 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f63a8030d70 con 0x5571ffc78370 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 10 client.17223 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571ffcf2b30 con 0x5571ffc78370 2024-09-12T01:58:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f6395ffb640 1 -- 172.21.15.122:0/2638366692 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8056070 con 0x5571ffc78370 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f6395ffb640 1 -- 172.21.15.122:0/2638366692 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f63a80464a0 con 0x5571ffc78370 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 10 client.17223 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 20 client.17223 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffcf34e0 0x5571ffcf5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 4 client.17223 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571ffbddd70 con 0x5571ffcf34e0 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.693+0000 7f63ae59c640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffcf34e0 0x5571ffcf5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.694+0000 7f6395ffb640 10 client.17223 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.694+0000 7f63ae59c640 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffcf34e0 0x5571ffcf5900 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b770ea1dce73d007 server_cookie=743502aa37b1c43b in_seq=0 out_seq=0 2024-09-12T01:58:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.705+0000 7f6395ffb640 1 -- 172.21.15.122:0/2638366692 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571ffbddd70 con 0x5571ffcf34e0 2024-09-12T01:58:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f6395ffb640 10 client.17223 handle_command_reply: tid=0 2024-09-12T01:58:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 client.17223 shutdown 2024-09-12T01:58:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 msgr2=0x5571ffcf12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffcf12f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f63a802ade0 tx=0x7f63a802f8c0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 shutdown_connections 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5571ffcf34e0 0x5571ffcf5900 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571ffc78370 0x5571ffcf12f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571ffc732e0 0x5571ffc814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 --2- 172.21.15.122:0/2638366692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571ffc729a0 0x5571ffc80f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f6395ffb640 0 client.17223 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 >> 172.21.15.122:0/2638366692 conn(0x5571ffc6e010 msgr2=0x5571ffc6c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 shutdown_connections 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 1 -- 172.21.15.122:0/2638366692 wait complete. 2024-09-12T01:58:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b7628740 20 client.17223 trim_cache size 0 max 16384 2024-09-12T01:58:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:58:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:58:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:58:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63980a2c30 msgr2=0x7f63980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63980a2c30 0x7f63980a50f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005e30 tx=0x7f63a803d040 comp rx=0 tx=0).stop 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 msgr2=0x7f63b0109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 0x7f63b0109a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e9f0 tx=0x7f63a400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 shutdown_connections 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f63980a2c30 0x7f63980a50f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.706+0000 7f63b488d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b01025f0 0x7f63b0109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.707+0000 7f63b488d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63b0101c30 0x7f63b0109a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.707+0000 7f63b488d640 1 --2- 172.21.15.122:0/1758936175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b0100a40 0x7f63b010b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.707+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 >> 172.21.15.122:0/1758936175 conn(0x7f63b00fc1e0 msgr2=0x7f63b0104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.707+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 shutdown_connections 2024-09-12T01:58:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:16.707+0000 7f63b488d640 1 -- 172.21.15.122:0/1758936175 wait complete. 2024-09-12T01:58:16.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1152408 10.25 MB/sec execute 514 sec latency 15.156 ms 2024-09-12T01:58:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:16 smithi151 ceph-mon[35229]: pgmap v689: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 292 op/s 2024-09-12T01:58:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:16 smithi183 ceph-mon[35127]: pgmap v689: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 292 op/s 2024-09-12T01:58:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:16 smithi122 ceph-mon[30925]: pgmap v689: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 292 op/s 2024-09-12T01:58:17.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1155117 10.26 MB/sec execute 515 sec latency 11.321 ms 2024-09-12T01:58:18.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1158067 10.27 MB/sec execute 516 sec latency 62.154 ms 2024-09-12T01:58:19.166 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T01:58:19.166 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:58:19.444 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:58:19.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1160427 10.27 MB/sec execute 517 sec latency 17.771 ms 2024-09-12T01:58:19.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:19 smithi122 ceph-mon[30925]: pgmap v690: 97 pgs: 97 active+clean; 312 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 371 op/s 2024-09-12T01:58:19.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 -- 172.21.15.122:0/3594921470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c103720 msgr2=0x7f961c10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:19.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 --2- 172.21.15.122:0/3594921470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c103720 0x7f961c10bc10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f960402a480 comp rx=0 tx=0).stop 2024-09-12T01:58:19.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 -- 172.21.15.122:0/3594921470 shutdown_connections 2024-09-12T01:58:19.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 --2- 172.21.15.122:0/3594921470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f961c10c1a0 0x7f961c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:19.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 --2- 172.21.15.122:0/3594921470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c103720 0x7f961c10bc10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:19.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 --2- 172.21.15.122:0/3594921470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c102e00 0x7f961c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:19.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 -- 172.21.15.122:0/3594921470 >> 172.21.15.122:0/3594921470 conn(0x7f961c0fc7a0 msgr2=0x7f961c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:19.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 -- 172.21.15.122:0/3594921470 shutdown_connections 2024-09-12T01:58:19.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.956+0000 7f96229ec640 1 -- 172.21.15.122:0/3594921470 wait complete. 2024-09-12T01:58:19.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 Processor -- start 2024-09-12T01:58:19.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 -- start start 2024-09-12T01:58:19.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 0x7f961c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f961c103720 0x7f961c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c10c1a0 0x7f961c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f961c111f70 con 0x7f961c102e00 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f961c111df0 con 0x7f961c103720 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f96229ec640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1120f0 con 0x7f961c10c1a0 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 0x7f961c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 0x7f961c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41834/0 (socket says 172.21.15.122:41834) 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 -- 172.21.15.122:0/472575041 learned_addr learned my addr 172.21.15.122:0/472575041 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f9613fff640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f961c103720 0x7f961c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f9620f89640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c10c1a0 0x7f961c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 -- 172.21.15.122:0/472575041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c10c1a0 msgr2=0x7f961c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c10c1a0 0x7f961c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:19.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 -- 172.21.15.122:0/472575041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f961c103720 msgr2=0x7f961c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f961c103720 0x7f961c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 -- 172.21.15.122:0/472575041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c19d3c0 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.957+0000 7f961bfff640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 0x7f961c198410 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f960c00d770 tx=0x7f960c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.958+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c017070 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.958+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960c005500 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.958+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c004e00 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.958+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c19d6b0 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.958+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1a4f90 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.959+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c06afb0 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.960+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f960c004fa0 con 0x7f961c102e00 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.960+0000 7f9619ffb640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96080a2b80 0x7f96080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:19.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.961+0000 7f9613fff640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96080a2b80 0x7f96080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:19.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.961+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f960c0cea10 con 0x7f961c102e00 2024-09-12T01:58:19.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.961+0000 7f9613fff640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96080a2b80 0x7f96080a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9604009230 tx=0x7f96040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:19.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:19.966+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f960c09b320 con 0x7f961c102e00 2024-09-12T01:58:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:19 smithi151 ceph-mon[35229]: pgmap v690: 97 pgs: 97 active+clean; 312 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 371 op/s 2024-09-12T01:58:20.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.185+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f961c107220 con 0x7f961c102e00 2024-09-12T01:58:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.186+0000 7f9619ffb640 1 -- 172.21.15.122:0/472575041 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f960c09f620 con 0x7f961c102e00 2024-09-12T01:58:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:58:20.187 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:58:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:58:20.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96080a2b80 msgr2=0x7f96080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:20.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96080a2b80 0x7f96080a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9604009230 tx=0x7f96040058d0 comp rx=0 tx=0).stop 2024-09-12T01:58:20.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 msgr2=0x7f961c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:20.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 0x7f961c198410 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f960c00d770 tx=0x7f960c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 shutdown_connections 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96080a2b80 0x7f96080a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f961c10c1a0 0x7f961c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f961c103720 0x7f961c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 --2- 172.21.15.122:0/472575041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f961c102e00 0x7f961c198410 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 >> 172.21.15.122:0/472575041 conn(0x7f961c0fc7a0 msgr2=0x7f961c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 shutdown_connections 2024-09-12T01:58:20.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:20.189+0000 7f96229ec640 1 -- 172.21.15.122:0/472575041 wait complete. 2024-09-12T01:58:20.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:19 smithi183 ceph-mon[35127]: pgmap v690: 97 pgs: 97 active+clean; 312 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 371 op/s 2024-09-12T01:58:20.745 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/472575041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:58:20.858 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1162398 10.27 MB/sec execute 518 sec latency 68.275 ms 2024-09-12T01:58:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/472575041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:58:21.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/472575041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:58:21.212 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T01:58:21.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T01:58:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:58:21.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165293 10.29 MB/sec execute 519 sec latency 38.476 ms 2024-09-12T01:58:21.892 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:21 smithi122 ceph-mon[30925]: pgmap v691: 97 pgs: 97 active+clean; 313 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 -- 172.21.15.122:0/3063658184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370102570 msgr2=0x7f03701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 --2- 172.21.15.122:0/3063658184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370102570 0x7f03701029b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0360009900 tx=0x7f036002a480 comp rx=0 tx=0).stop 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 -- 172.21.15.122:0/3063658184 shutdown_connections 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 --2- 172.21.15.122:0/3063658184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0370102ef0 0x7f037010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 --2- 172.21.15.122:0/3063658184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370102570 0x7f03701029b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 --2- 172.21.15.122:0/3063658184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0370100e20 0x7f0370101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 -- 172.21.15.122:0/3063658184 >> 172.21.15.122:0/3063658184 conn(0x7f03700fc7d0 msgr2=0x7f03700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 -- 172.21.15.122:0/3063658184 shutdown_connections 2024-09-12T01:58:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.992+0000 7f0377ec5640 1 -- 172.21.15.122:0/3063658184 wait complete. 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 Processor -- start 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 -- start start 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 0x7f03701993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0370102570 0x7f0370199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0370102ef0 0x7f03701a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0370110f50 con 0x7f0370100e20 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0370110dd0 con 0x7f0370102570 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0377ec5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03701110d0 con 0x7f0370102ef0 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 0x7f03701993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 0x7f03701993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45248/0 (socket says 172.21.15.122:45248) 2024-09-12T01:58:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 -- 172.21.15.122:0/1160902645 learned_addr learned my addr 172.21.15.122:0/1160902645 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375460640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0370102570 0x7f0370199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0376462640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0370102ef0 0x7f03701a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 -- 172.21.15.122:0/1160902645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0370102ef0 msgr2=0x7f03701a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0370102ef0 0x7f03701a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 -- 172.21.15.122:0/1160902645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0370102570 msgr2=0x7f0370199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0370102570 0x7f0370199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.993+0000 7f0375c61640 1 -- 172.21.15.122:0/1160902645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03701a4980 con 0x7f0370100e20 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.994+0000 7f0375c61640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 0x7f03701993c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f036c00d950 tx=0x7f036c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.994+0000 7f0366ffd640 1 -- 172.21.15.122:0/1160902645 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c017030 con 0x7f0370100e20 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.994+0000 7f0366ffd640 1 -- 172.21.15.122:0/1160902645 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c004020 con 0x7f0370100e20 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.994+0000 7f0366ffd640 1 -- 172.21.15.122:0/1160902645 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0047a0 con 0x7f0370100e20 2024-09-12T01:58:21.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.994+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03701a4b50 con 0x7f0370100e20 2024-09-12T01:58:21.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.995+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03701a5010 con 0x7f0370100e20 2024-09-12T01:58:21.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.997+0000 7f0366ffd640 1 -- 172.21.15.122:0/1160902645 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f036c0041d0 con 0x7f0370100e20 2024-09-12T01:58:21.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.997+0000 7f0366ffd640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f035c0a2c00 0x7f035c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:21.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.997+0000 7f037a29e740 1 Processor -- start 2024-09-12T01:58:21.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.997+0000 7f0366ffd640 1 -- 172.21.15.122:0/1160902645 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f036c0ce9a0 con 0x7f0370100e20 2024-09-12T01:58:21.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375460640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f035c0a2c00 0x7f035c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:21.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 -- start start 2024-09-12T01:58:21.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 0x55c83bf3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3bfa0 0x55c83bf4a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c83bea12a0 con 0x55c83bf3b680 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c83be48b10 con 0x55c83bf3bfa0 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f037a29e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c83be9d260 con 0x55c83bf4ad50 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 0x55c83bf3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 0x55c83bf3ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45264/0 (socket says 172.21.15.122:45264) 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 -- 172.21.15.122:0/3769996744 learned_addr learned my addr 172.21.15.122:0/3769996744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375460640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f035c0a2c00 0x7f035c0a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0360009230 tx=0x7f03600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375460640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3bfa0 0x55c83bf4a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0376462640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 -- 172.21.15.122:0/3769996744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 msgr2=0x55c83bf4d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 -- 172.21.15.122:0/3769996744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3bfa0 msgr2=0x55c83bf4a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3bfa0 0x55c83bf4a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375c61640 1 -- 172.21.15.122:0/3769996744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83bc12b90 con 0x55c83bf3b680 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0376462640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.998+0000 7f0375460640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3bfa0 0x55c83bf4a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.999+0000 7f0375c61640 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 0x55c83bf3ba60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f036c09af00 tx=0x7f036c09b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec0503cccacf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.999+0000 7f035bfff640 1 -- 172.21.15.122:0/3769996744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c09f5b0 con 0x55c83bf3b680 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.999+0000 7f035bfff640 1 -- 172.21.15.122:0/3769996744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c0a3e70 con 0x55c83bf3b680 2024-09-12T01:58:22.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:21.999+0000 7f035bfff640 1 -- 172.21.15.122:0/3769996744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0bb2d0 con 0x55c83bf3b680 2024-09-12T01:58:22.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 -- 172.21.15.122:0/3769996744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 msgr2=0x55c83bf3ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:22.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 0x55c83bf3ba60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f036c09af00 tx=0x7f036c09b0f0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 -- 172.21.15.122:0/3769996744 shutdown_connections 2024-09-12T01:58:22.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3bfa0 0x55c83bf4a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 --2- 172.21.15.122:0/3769996744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3b680 0x55c83bf3ba60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 -- 172.21.15.122:0/3769996744 >> 172.21.15.122:0/3769996744 conn(0x55c83bf37cf0 msgr2=0x55c83bf380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 -- 172.21.15.122:0/3769996744 shutdown_connections 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 -- 172.21.15.122:0/3769996744 wait complete. 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 -- start start 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.000+0000 7f037a29e740 1 Processor -- start 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3b680 0x55c83bf47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf48230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf4ad50 0x55c83bf48770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c83bf4d8c0 con 0x55c83bf3bfa0 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c83be296e0 con 0x55c83bf4ad50 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c83bf4da40 con 0x55c83bf3b680 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f037a29e740 10 client.0 _ll_register_callbacks cb 0x55c83be4da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf48230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375c61640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3b680 0x55c83bf47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf48230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45276/0 (socket says 172.21.15.122:45276) 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 -- 172.21.15.122:0/478210250 learned_addr learned my addr 172.21.15.122:0/478210250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0376462640 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf4ad50 0x55c83bf48770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 -- 172.21.15.122:0/478210250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3b680 msgr2=0x55c83bf47cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3b680 0x55c83bf47cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 -- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf4ad50 msgr2=0x55c83bf48770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf4ad50 0x55c83bf48770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 -- 172.21.15.122:0/478210250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83bf56f30 con 0x55c83bf3bfa0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f0375460640 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf48230 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f036002a9a0 tx=0x7f036002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1170353975af368 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.001+0000 7f03597fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f03597fa640 1 -- 172.21.15.122:0/478210250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360035050 con 0x55c83bf3bfa0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f03597fa640 1 -- 172.21.15.122:0/478210250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0360031950 con 0x55c83bf3bfa0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c0a3e70 con 0x55c83bf3bfa0 2024-09-12T01:58:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f03597fa640 1 -- 172.21.15.122:0/478210250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0360031b50 con 0x55c83bf3bfa0 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 10 client.17241 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c83bf597f0 con 0x55c83bf3bfa0 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f03597fa640 1 -- 172.21.15.122:0/478210250 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c0a3e70 con 0x55c83bf3bfa0 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f03597fa640 1 -- 172.21.15.122:0/478210250 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f036003c440 con 0x55c83bf3bfa0 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 10 client.17241 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 10 client.17241 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 20 client.17241 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf5acb0 0x55c83bf5d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 4 client.17241 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.002+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83beae9d0 con 0x55c83bf5acb0 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.003+0000 7f0375c61640 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf5acb0 0x55c83bf5d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.003+0000 7f03597fa640 10 client.17241 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.003+0000 7f0375c61640 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf5acb0 0x55c83bf5d0d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d41fa7fb86d5da server_cookie=35035d300bd1bf in_seq=0 out_seq=0 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f03597fa640 1 -- 172.21.15.122:0/478210250 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c83beae9d0 con 0x55c83bf5acb0 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f03597fa640 10 client.17241 handle_command_reply: tid=0 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 client.17241 shutdown 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 msgr2=0x55c83bf48230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf48230 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f036002a9a0 tx=0x7f036002a480 comp rx=0 tx=0).stop 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 shutdown_connections 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf5acb0 0x55c83bf5d0d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf4ad50 0x55c83bf48770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf48230 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f03597fa640 0 client.17241 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 --2- 172.21.15.122:0/478210250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3b680 0x55c83bf47cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 >> 172.21.15.122:0/478210250 conn(0x55c83bf37cf0 msgr2=0x55c83bf3d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 shutdown_connections 2024-09-12T01:58:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 1 -- 172.21.15.122:0/478210250 wait complete. 2024-09-12T01:58:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.004+0000 7f037a29e740 20 client.17241 trim_cache size 0 max 16384 2024-09-12T01:58:22.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.021+0000 7f037a29e740 1 Processor -- start 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.021+0000 7f037a29e740 1 -- start start 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f037a29e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf37660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f037a29e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3bfa0 0x55c83bf3d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f037a29e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 0x55c83bf3ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f037a29e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03701addb0 con 0x55c83bf4ad50 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f037a29e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03701ab680 con 0x55c83bf3b680 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f037a29e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03701a94b0 con 0x55c83bf3bfa0 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf37660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0376462640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 0x55c83bf3ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf37660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:51450/0 (socket says 172.21.15.122:51450) 2024-09-12T01:58:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0376462640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 0x55c83bf3ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45290/0 (socket says 172.21.15.122:45290) 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 -- 172.21.15.122:0/1062114180 learned_addr learned my addr 172.21.15.122:0/1062114180 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375460640 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3bfa0 0x55c83bf3d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 -- 172.21.15.122:0/1062114180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3bfa0 msgr2=0x55c83bf3d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3bfa0 0x55c83bf3d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 -- 172.21.15.122:0/1062114180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 msgr2=0x55c83bf3ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 0x55c83bf3ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 -- 172.21.15.122:0/1062114180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83bf33260 con 0x55c83bf3b680 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0376462640 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 0x55c83bf3ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.022+0000 7f0375c61640 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf37660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f036c09b190 tx=0x7f036c009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03b862966e00ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f035b7fe640 1 -- 172.21.15.122:0/1062114180 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0b4070 con 0x55c83bf3b680 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f035b7fe640 1 -- 172.21.15.122:0/1062114180 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c09f5b0 con 0x55c83bf3b680 2024-09-12T01:58:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f035b7fe640 1 -- 172.21.15.122:0/1062114180 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0bb0c0 con 0x55c83bf3b680 2024-09-12T01:58:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 -- 172.21.15.122:0/1062114180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 msgr2=0x55c83bf37660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf37660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f036c09b190 tx=0x7f036c009740 comp rx=0 tx=0).stop 2024-09-12T01:58:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 -- 172.21.15.122:0/1062114180 shutdown_connections 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf4ad50 0x55c83bf3ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf3bfa0 0x55c83bf3d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 --2- 172.21.15.122:0/1062114180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf37660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 -- 172.21.15.122:0/1062114180 >> 172.21.15.122:0/1062114180 conn(0x55c83bf37cf0 msgr2=0x55c83bf3d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 -- 172.21.15.122:0/1062114180 shutdown_connections 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.023+0000 7f037a29e740 1 -- 172.21.15.122:0/1062114180 wait complete. 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 -- start start 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 Processor -- start 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf52d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf53270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03701a9b40 con 0x55c83bf3bfa0 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03701aafb0 con 0x55c83bf3b680 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.024+0000 7f037a29e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03701aa210 con 0x55c83bf4ad50 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f037a29e740 10 client.0 _ll_register_callbacks cb 0x7f0360004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:44654/0 (socket says 172.21.15.122:44654) 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 -- 172.21.15.122:0/4123163344 learned_addr learned my addr 172.21.15.122:0/4123163344 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0375460640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf53270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0375c61640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf52d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 -- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 msgr2=0x55c83bf52d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf52d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 -- 172.21.15.122:0/4123163344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 msgr2=0x55c83bf53270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:22.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf53270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 -- 172.21.15.122:0/4123163344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83bf50240 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0375c61640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf52d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f0376462640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4fd00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0368012a70 tx=0x7f0368012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9eb1992ec36188 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f03597fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f03597fa640 1 -- 172.21.15.122:0/4123163344 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368004b20 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f03597fa640 1 -- 172.21.15.122:0/4123163344 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0368004cc0 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.025+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c09f5b0 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f03597fa640 1 -- 172.21.15.122:0/4123163344 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0368010b20 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 10 client.26389 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c83bf3e3a0 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f03597fa640 1 -- 172.21.15.122:0/4123163344 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c09f5b0 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f03597fa640 1 -- 172.21.15.122:0/4123163344 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f036801b410 con 0x55c83bf4ad50 2024-09-12T01:58:22.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 10 client.26389 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 10 client.26389 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 20 client.26389 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf3ec10 0x55c83bf41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 4 client.26389 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x55c83bea8d20 con 0x55c83bf3ec10 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.026+0000 7f0375c61640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf3ec10 0x55c83bf41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.027+0000 7f03597fa640 10 client.26389 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:22.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.027+0000 7f0375c61640 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf3ec10 0x55c83bf41030 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d216865e414ae9b server_cookie=83313ad59d589454 in_seq=0 out_seq=0 2024-09-12T01:58:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:21 smithi151 ceph-mon[35229]: pgmap v691: 97 pgs: 97 active+clean; 313 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-09-12T01:58:22.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f03597fa640 1 -- 172.21.15.122:0/4123163344 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c83bea8d20 con 0x55c83bf3ec10 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f03597fa640 10 client.26389 handle_command_reply: tid=0 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 client.26389 shutdown 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 msgr2=0x55c83bf4fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4fd00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0368012a70 tx=0x7f0368012f30 comp rx=0 tx=0).stop 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 shutdown_connections 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c83bf3ec10 0x55c83bf41030 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c83bf4ad50 0x55c83bf4fd00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c83bf3bfa0 0x55c83bf53270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f03597fa640 0 client.26389 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 --2- 172.21.15.122:0/4123163344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83bf3b680 0x55c83bf52d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 >> 172.21.15.122:0/4123163344 conn(0x55c83bf37cf0 msgr2=0x55c83bf3d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:22.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 shutdown_connections 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 1 -- 172.21.15.122:0/4123163344 wait complete. 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.142+0000 7f037a29e740 20 client.26389 trim_cache size 0 max 16384 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 46, 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "791b60cc": { 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 46, 2024-09-12T01:58:22.144 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 147.2, 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:58:22.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:22.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:22.146 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:58:22.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f035c0a2c00 msgr2=0x7f035c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f035c0a2c00 0x7f035c0a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0360009230 tx=0x7f03600058d0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 msgr2=0x7f03701993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 0x7f03701993c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f036c00d950 tx=0x7f036c00de10 comp rx=0 tx=0).stop 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 shutdown_connections 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f035c0a2c00 0x7f035c0a50c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0370102ef0 0x7f03701a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0370102570 0x7f0370199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 --2- 172.21.15.122:0/1160902645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0370100e20 0x7f03701993c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 >> 172.21.15.122:0/1160902645 conn(0x7f03700fc7d0 msgr2=0x7f03700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 shutdown_connections 2024-09-12T01:58:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:22.143+0000 7f0377ec5640 1 -- 172.21.15.122:0/1160902645 wait complete. 2024-09-12T01:58:22.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:21 smithi183 ceph-mon[35127]: pgmap v691: 97 pgs: 97 active+clean; 313 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-09-12T01:58:22.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.27 MB/sec execute 520 sec latency 722.947 ms 2024-09-12T01:58:22.934 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '791b60cc', quiesce took 3.8 seconds. Will release after: 23.4 2024-09-12T01:58:23.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.25 MB/sec execute 521 sec latency 1723.031 ms 2024-09-12T01:58:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:23 smithi122 ceph-mon[30925]: pgmap v692: 97 pgs: 97 active+clean; 303 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 345 op/s 2024-09-12T01:58:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:23 smithi151 ceph-mon[35229]: pgmap v692: 97 pgs: 97 active+clean; 303 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 345 op/s 2024-09-12T01:58:24.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:23 smithi183 ceph-mon[35127]: pgmap v692: 97 pgs: 97 active+clean; 303 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 345 op/s 2024-09-12T01:58:24.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.23 MB/sec execute 522 sec latency 2723.134 ms 2024-09-12T01:58:25.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.21 MB/sec execute 523 sec latency 3723.220 ms 2024-09-12T01:58:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:25 smithi122 ceph-mon[30925]: pgmap v693: 97 pgs: 97 active+clean; 293 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-09-12T01:58:26.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:25 smithi151 ceph-mon[35229]: pgmap v693: 97 pgs: 97 active+clean; 293 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-09-12T01:58:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:25 smithi183 ceph-mon[35127]: pgmap v693: 97 pgs: 97 active+clean; 293 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-09-12T01:58:26.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.19 MB/sec execute 524 sec latency 4723.321 ms 2024-09-12T01:58:27.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.17 MB/sec execute 525 sec latency 5723.420 ms 2024-09-12T01:58:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:27 smithi122 ceph-mon[30925]: pgmap v694: 97 pgs: 97 active+clean; 294 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-12T01:58:28.004 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:27 smithi183 ceph-mon[35127]: pgmap v694: 97 pgs: 97 active+clean; 294 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-12T01:58:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:27 smithi151 ceph-mon[35229]: pgmap v694: 97 pgs: 97 active+clean; 294 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-12T01:58:28.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.15 MB/sec execute 526 sec latency 6723.467 ms 2024-09-12T01:58:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1874610531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b0103170 msgr2=0x7f69b0103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1874610531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b0103170 0x7f69b0103570 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009900 tx=0x7f69a002a410 comp rx=0 tx=0).stop 2024-09-12T01:58:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1874610531 shutdown_connections 2024-09-12T01:58:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1874610531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b01005a0 0x7f69b010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1874610531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69b00ffbe0 0x7f69b0100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1874610531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b0103170 0x7f69b0103570 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1874610531 >> 172.21.15.122:0/1874610531 conn(0x7f69b00fb930 msgr2=0x7f69b00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1874610531 shutdown_connections 2024-09-12T01:58:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.802+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1874610531 wait complete. 2024-09-12T01:58:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 Processor -- start 2024-09-12T01:58:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 -- start start 2024-09-12T01:58:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69b00ffbe0 0x7f69b0102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 0x7f69b0100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b0103170 0x7f69b01013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010d810 con 0x7f69b01005a0 2024-09-12T01:58:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010d690 con 0x7f69b00ffbe0 2024-09-12T01:58:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.803+0000 7f69b4d4b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69b010d990 con 0x7f69b0103170 2024-09-12T01:58:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 0x7f69b0100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 0x7f69b0100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45314/0 (socket says 172.21.15.122:45314) 2024-09-12T01:58:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1573730553 learned_addr learned my addr 172.21.15.122:0/1573730553 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b0103170 0x7f69b01013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69ae59c640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69b00ffbe0 0x7f69b0102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1573730553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b0103170 msgr2=0x7f69b01013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b0103170 0x7f69b01013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1573730553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69b00ffbe0 msgr2=0x7f69b0102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69b00ffbe0 0x7f69b0102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1573730553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b0101c80 con 0x7f69b01005a0 2024-09-12T01:58:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 0x7f69b0100eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f699800d950 tx=0x7f699800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a7fff640 1 -- 172.21.15.122:0/1573730553 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998017070 con 0x7f69b01005a0 2024-09-12T01:58:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a7fff640 1 -- 172.21.15.122:0/1573730553 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6998009c30 con 0x7f69b01005a0 2024-09-12T01:58:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b01a1f30 con 0x7f69b01005a0 2024-09-12T01:58:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69a7fff640 1 -- 172.21.15.122:0/1573730553 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699800f6b0 con 0x7f69b01005a0 2024-09-12T01:58:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.804+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b01a2390 con 0x7f69b01005a0 2024-09-12T01:58:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.806+0000 7f69a7fff640 1 -- 172.21.15.122:0/1573730553 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f69980109d0 con 0x7f69b01005a0 2024-09-12T01:58:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 Processor -- start 2024-09-12T01:58:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 -- start start 2024-09-12T01:58:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c3824da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c3829da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c382c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f8c37882f0 con 0x55f8c382a370 2024-09-12T01:58:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3730270 con 0x55f8c38252e0 2024-09-12T01:58:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f8c37842b0 con 0x55f8c38249a0 2024-09-12T01:58:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69a7fff640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f699c0a2c30 0x7f699c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69a5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c382c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69a5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c382c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45326/0 (socket says 172.21.15.122:45326) 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69ae59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c3829da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f699c0a2c30 0x7f699c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.807+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1065495108 learned_addr learned my addr 172.21.15.122:0/1065495108 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c3824da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a7fff640 1 -- 172.21.15.122:0/1573730553 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f69980ceb60 con 0x7f69b01005a0 2024-09-12T01:58:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1065495108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 msgr2=0x55f8c3824da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c3824da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1065495108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 msgr2=0x55f8c3829da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c3829da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a5d9b640 1 -- 172.21.15.122:0/1065495108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c372f340 con 0x55f8c382a370 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f699c0a2c30 0x7f699c0a50f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005fd0 tx=0x7f69a8009de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c382c830 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6998010fd0 tx=0x7f6998010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511c85360a6a7d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a4d99640 1 -- 172.21.15.122:0/1065495108 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69980a4030 con 0x55f8c382a370 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a4d99640 1 -- 172.21.15.122:0/1065495108 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6998062040 con 0x55f8c382a370 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.808+0000 7f69a4d99640 1 -- 172.21.15.122:0/1065495108 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699809a540 con 0x55f8c382a370 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1065495108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 msgr2=0x55f8c382c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c382c830 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6998010fd0 tx=0x7f6998010d70 comp rx=0 tx=0).stop 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1065495108 shutdown_connections 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c382c830 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c3829da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1065495108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c3824da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1065495108 >> 172.21.15.122:0/1065495108 conn(0x55f8c3820010 msgr2=0x55f8c3820410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1065495108 shutdown_connections 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1065495108 wait complete. 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 -- start start 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.809+0000 7f69b7bc2740 1 Processor -- start 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c382eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c382f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c38360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f8c382d2e0 con 0x55f8c382a370 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3711290 con 0x55f8c38252e0 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f8c382d460 con 0x55f8c38249a0 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 10 client.0 _ll_register_callbacks cb 0x55f8c3742140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c38360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c38360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45340/0 (socket says 172.21.15.122:45340) 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 -- 172.21.15.122:0/1191214641 learned_addr learned my addr 172.21.15.122:0/1191214641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c382f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 -- 172.21.15.122:0/1191214641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 msgr2=0x55f8c382eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c382eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 -- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 msgr2=0x55f8c382f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c382f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 -- 172.21.15.122:0/1191214641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c349b210 con 0x55f8c382a370 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c382f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69aed9d640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c38360f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f69a8010e00 tx=0x7f69a80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290c03a40356a6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f6995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f6995ffb640 1 -- 172.21.15.122:0/1191214641 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a802a410 con 0x55f8c382a370 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f6995ffb640 1 -- 172.21.15.122:0/1191214641 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f69a802a5b0 con 0x55f8c382a370 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.810+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6998062040 con 0x55f8c382a370 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f6995ffb640 1 -- 172.21.15.122:0/1191214641 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69a802a7b0 con 0x55f8c382a370 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 10 client.17259 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8c3567620 con 0x55f8c382a370 2024-09-12T01:58:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f6995ffb640 1 -- 172.21.15.122:0/1191214641 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6998062040 con 0x55f8c382a370 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f6995ffb640 1 -- 172.21.15.122:0/1191214641 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f69a801f070 con 0x55f8c382a370 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 10 client.17259 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 20 client.17259 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c383a800 0x55f8c383ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 4 client.17259 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c3795a20 con 0x55f8c383a800 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.811+0000 7f69ae59c640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c383a800 0x55f8c383ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69ae59c640 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c383a800 0x55f8c383ccc0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60872947116eaee server_cookie=7d7f1d6052c8a85e in_seq=0 out_seq=0 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f6995ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f6995ffb640 1 -- 172.21.15.122:0/1191214641 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8c3795a20 con 0x55f8c383a800 2024-09-12T01:58:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f6995ffb640 10 client.17259 handle_command_reply: tid=0 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 client.17259 shutdown 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 msgr2=0x55f8c38360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c38360f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f69a8010e00 tx=0x7f69a80096a0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 shutdown_connections 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c383a800 0x55f8c383ccc0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c38360f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c382f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/1191214641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c382eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f6995ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 >> 172.21.15.122:0/1191214641 conn(0x55f8c3820010 msgr2=0x55f8c3821820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 shutdown_connections 2024-09-12T01:58:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.812+0000 7f69b7bc2740 1 -- 172.21.15.122:0/1191214641 wait complete. 2024-09-12T01:58:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.813+0000 7f69b7bc2740 20 client.17259 trim_cache size 0 max 16384 2024-09-12T01:58:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 Processor -- start 2024-09-12T01:58:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 -- start start 2024-09-12T01:58:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c383c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c38360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c383cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01adaa0 con 0x55f8c382a370 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ab370 con 0x55f8c38252e0 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01a91a0 con 0x55f8c38249a0 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c383cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c383cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45344/0 (socket says 172.21.15.122:45344) 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 -- 172.21.15.122:0/81113316 learned_addr learned my addr 172.21.15.122:0/81113316 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69ae59c640 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c383c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 -- 172.21.15.122:0/81113316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 msgr2=0x55f8c383c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c383c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 -- 172.21.15.122:0/81113316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 msgr2=0x55f8c38360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c38360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 -- 172.21.15.122:0/81113316 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c382e920 con 0x55f8c382a370 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69ae59c640 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c383c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f69aed9d640 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c383cb20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009b90 tx=0x7f69a8012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bd70c30d7a319f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f6997fff640 1 -- 172.21.15.122:0/81113316 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8039070 con 0x55f8c382a370 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f6997fff640 1 -- 172.21.15.122:0/81113316 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a802ae30 con 0x55f8c382a370 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.827+0000 7f6997fff640 1 -- 172.21.15.122:0/81113316 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8011950 con 0x55f8c382a370 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 -- 172.21.15.122:0/81113316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 msgr2=0x55f8c383cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c383cb20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009b90 tx=0x7f69a8012860 comp rx=0 tx=0).stop 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 -- 172.21.15.122:0/81113316 shutdown_connections 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c382a370 0x55f8c383cb20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c38252e0 0x55f8c38360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/81113316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38249a0 0x55f8c383c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 -- 172.21.15.122:0/81113316 >> 172.21.15.122:0/81113316 conn(0x55f8c3820010 msgr2=0x55f8c382eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 -- 172.21.15.122:0/81113316 shutdown_connections 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 -- 172.21.15.122:0/81113316 wait complete. 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 -- start start 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.828+0000 7f69b7bc2740 1 Processor -- start 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 0x55f8c3833000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38252e0 0x55f8c3833540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c382a370 0x55f8c38a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01a9830 con 0x55f8c38249a0 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01aaca0 con 0x55f8c382a370 2024-09-12T01:58:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01a9f00 con 0x55f8c38252e0 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 10 client.0 _ll_register_callbacks cb 0x7f69a8009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 0x55f8c3833000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 0x55f8c3833000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45346/0 (socket says 172.21.15.122:45346) 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 -- 172.21.15.122:0/3156886098 learned_addr learned my addr 172.21.15.122:0/3156886098 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69aed9d640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c382a370 0x55f8c38a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38252e0 0x55f8c3833540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 -- 172.21.15.122:0/3156886098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38252e0 msgr2=0x55f8c3833540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38252e0 0x55f8c3833540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 -- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c382a370 msgr2=0x55f8c38a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c382a370 0x55f8c38a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 -- 172.21.15.122:0/3156886098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c38a3b00 con 0x55f8c38249a0 2024-09-12T01:58:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69aed9d640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c382a370 0x55f8c38a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69ae59c640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 0x55f8c3833000 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005940 tx=0x7f69a0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65abb24b3fdf9b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f6995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f6995ffb640 1 -- 172.21.15.122:0/3156886098 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0035040 con 0x55f8c38249a0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a802ae30 con 0x55f8c38249a0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f6995ffb640 1 -- 172.21.15.122:0/3156886098 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f69a0005590 con 0x55f8c38249a0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f6995ffb640 1 -- 172.21.15.122:0/3156886098 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69a0004770 con 0x55f8c38249a0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 10 client.17265 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.829+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8c38a4dd0 con 0x55f8c38249a0 2024-09-12T01:58:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f6995ffb640 1 -- 172.21.15.122:0/3156886098 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a802ae30 con 0x55f8c38249a0 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f6995ffb640 1 -- 172.21.15.122:0/3156886098 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f69a00463a0 con 0x55f8c38249a0 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69b7bc2740 10 client.17265 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69b7bc2740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69b7bc2740 20 client.17265 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c38a5700 0x55f8c38a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69b7bc2740 4 client.17265 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8c378fd70 con 0x55f8c38a5700 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.830+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c38a5700 0x55f8c38a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.831+0000 7f6995ffb640 10 client.17265 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.831+0000 7f69a5d9b640 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c38a5700 0x55f8c38a7b20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea5566377035821 server_cookie=916d8295137c1d3 in_seq=0 out_seq=0 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f6995ffb640 1 -- 172.21.15.122:0/3156886098 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8c378fd70 con 0x55f8c38a5700 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f6995ffb640 10 client.17265 handle_command_reply: tid=0 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 client.17265 shutdown 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 msgr2=0x55f8c3833000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 0x55f8c3833000 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005940 tx=0x7f69a0004530 comp rx=0 tx=0).stop 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 shutdown_connections 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f8c38a5700 0x55f8c38a7b20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f8c382a370 0x55f8c38a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8c38252e0 0x55f8c3833540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f6995ffb640 0 client.17265 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 --2- 172.21.15.122:0/3156886098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f8c38249a0 0x55f8c3833000 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 >> 172.21.15.122:0/3156886098 conn(0x55f8c3820010 msgr2=0x55f8c383c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 shutdown_connections 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 1 -- 172.21.15.122:0/3156886098 wait complete. 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.843+0000 7f69b7bc2740 20 client.17265 trim_cache size 0 max 16384 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:58:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:58:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f699c0a2c30 msgr2=0x7f699c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f699c0a2c30 0x7f699c0a50f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005fd0 tx=0x7f69a8009de0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 msgr2=0x7f69b0100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 0x7f69b0100eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f699800d950 tx=0x7f699800de10 comp rx=0 tx=0).stop 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 shutdown_connections 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f699c0a2c30 0x7f699c0a50f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69b0103170 0x7f69b01013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b01005a0 0x7f69b0100eb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 --2- 172.21.15.122:0/1573730553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69b00ffbe0 0x7f69b0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 >> 172.21.15.122:0/1573730553 conn(0x7f69b00fb930 msgr2=0x7f69b0106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 shutdown_connections 2024-09-12T01:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:29.844+0000 7f69b4d4b640 1 -- 172.21.15.122:0/1573730553 wait complete. 2024-09-12T01:58:29.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.13 MB/sec execute 527 sec latency 7723.549 ms 2024-09-12T01:58:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:29 smithi122 ceph-mon[30925]: pgmap v695: 97 pgs: 97 active+clean; 294 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-09-12T01:58:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:29 smithi151 ceph-mon[35229]: pgmap v695: 97 pgs: 97 active+clean; 294 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-09-12T01:58:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:29 smithi183 ceph-mon[35127]: pgmap v695: 97 pgs: 97 active+clean; 294 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-09-12T01:58:30.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.11 MB/sec execute 528 sec latency 8723.621 ms 2024-09-12T01:58:31.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.09 MB/sec execute 529 sec latency 9723.679 ms 2024-09-12T01:58:31.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:31 smithi122 ceph-mon[30925]: pgmap v696: 97 pgs: 97 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-09-12T01:58:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:31 smithi151 ceph-mon[35229]: pgmap v696: 97 pgs: 97 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-09-12T01:58:32.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:31 smithi183 ceph-mon[35127]: pgmap v696: 97 pgs: 97 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-09-12T01:58:32.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.07 MB/sec execute 530 sec latency 10723.774 ms 2024-09-12T01:58:33.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.05 MB/sec execute 531 sec latency 11723.860 ms 2024-09-12T01:58:33.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:33 smithi122 ceph-mon[30925]: pgmap v697: 97 pgs: 97 active+clean; 290 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 158 op/s 2024-09-12T01:58:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:33 smithi151 ceph-mon[35229]: pgmap v697: 97 pgs: 97 active+clean; 290 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 158 op/s 2024-09-12T01:58:34.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:33 smithi183 ceph-mon[35127]: pgmap v697: 97 pgs: 97 active+clean; 290 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 158 op/s 2024-09-12T01:58:34.859 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.04 MB/sec execute 532 sec latency 12723.946 ms 2024-09-12T01:58:35.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.02 MB/sec execute 533 sec latency 13724.024 ms 2024-09-12T01:58:35.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:35 smithi122 ceph-mon[30925]: pgmap v698: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.3 MiB/s wr, 95 op/s 2024-09-12T01:58:36.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:35 smithi151 ceph-mon[35229]: pgmap v698: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.3 MiB/s wr, 95 op/s 2024-09-12T01:58:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:35 smithi183 ceph-mon[35127]: pgmap v698: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.3 MiB/s wr, 95 op/s 2024-09-12T01:58:36.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 10.00 MB/sec execute 534 sec latency 14724.123 ms 2024-09-12T01:58:37.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.98 MB/sec execute 535 sec latency 15724.223 ms 2024-09-12T01:58:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:37 smithi122 ceph-mon[30925]: pgmap v699: 97 pgs: 97 active+clean; 290 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.1 MiB/s wr, 91 op/s 2024-09-12T01:58:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:37 smithi151 ceph-mon[35229]: pgmap v699: 97 pgs: 97 active+clean; 290 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.1 MiB/s wr, 91 op/s 2024-09-12T01:58:38.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:37 smithi183 ceph-mon[35127]: pgmap v699: 97 pgs: 97 active+clean; 290 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.1 MiB/s wr, 91 op/s 2024-09-12T01:58:38.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.96 MB/sec execute 536 sec latency 16724.321 ms 2024-09-12T01:58:39.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.94 MB/sec execute 537 sec latency 17724.419 ms 2024-09-12T01:58:39.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:39 smithi122 ceph-mon[30925]: pgmap v700: 97 pgs: 97 active+clean; 289 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-09-12T01:58:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:39 smithi151 ceph-mon[35229]: pgmap v700: 97 pgs: 97 active+clean; 289 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-09-12T01:58:40.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:39 smithi183 ceph-mon[35127]: pgmap v700: 97 pgs: 97 active+clean; 289 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-09-12T01:58:40.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.92 MB/sec execute 538 sec latency 18724.507 ms 2024-09-12T01:58:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:40 smithi151 ceph-mon[35229]: pgmap v701: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.0 MiB/s wr, 89 op/s 2024-09-12T01:58:41.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:40 smithi183 ceph-mon[35127]: pgmap v701: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.0 MiB/s wr, 89 op/s 2024-09-12T01:58:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:40 smithi122 ceph-mon[30925]: pgmap v701: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.0 MiB/s wr, 89 op/s 2024-09-12T01:58:41.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.91 MB/sec execute 539 sec latency 19724.591 ms 2024-09-12T01:58:42.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.89 MB/sec execute 540 sec latency 20724.660 ms 2024-09-12T01:58:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 -- 172.21.15.122:0/3404739850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 msgr2=0x7fdaf0107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/3404739850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf0107300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009900 tx=0x7fdae002a410 comp rx=0 tx=0).stop 2024-09-12T01:58:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 -- 172.21.15.122:0/3404739850 shutdown_connections 2024-09-12T01:58:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/3404739850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0107840 0x7fdaf0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/3404739850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf0107300 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/3404739850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf01025e0 0x7fdaf01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 -- 172.21.15.122:0/3404739850 >> 172.21.15.122:0/3404739850 conn(0x7fdaf00fc1e0 msgr2=0x7fdaf00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 -- 172.21.15.122:0/3404739850 shutdown_connections 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 -- 172.21.15.122:0/3404739850 wait complete. 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.938+0000 7fdaf7c65640 1 Processor -- start 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 -- start start 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf01025e0 0x7fdaf0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf0107840 0x7fdaf019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010d820 con 0x7fdaf0104f10 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010d6a0 con 0x7fdaf0107840 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf7c65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010d9a0 con 0x7fdaf01025e0 2024-09-12T01:58:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57368/0 (socket says 172.21.15.122:57368) 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 -- 172.21.15.122:0/885371470 learned_addr learned my addr 172.21.15.122:0/885371470 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf01025e0 0x7fdaf0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf6202640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf0107840 0x7fdaf019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 -- 172.21.15.122:0/885371470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf01025e0 msgr2=0x7fdaf0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf01025e0 0x7fdaf0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 -- 172.21.15.122:0/885371470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf0107840 msgr2=0x7fdaf019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf0107840 0x7fdaf019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 -- 172.21.15.122:0/885371470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf019faa0 con 0x7fdaf0104f10 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.939+0000 7fdaf5200640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf01983c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdae0031610 tx=0x7fdae00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.940+0000 7fdae6ffd640 1 -- 172.21.15.122:0/885371470 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae003f030 con 0x7fdaf0104f10 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.940+0000 7fdae6ffd640 1 -- 172.21.15.122:0/885371470 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae0003ce0 con 0x7fdaf0104f10 2024-09-12T01:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.940+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf019fd30 con 0x7fdaf0104f10 2024-09-12T01:58:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.940+0000 7fdae6ffd640 1 -- 172.21.15.122:0/885371470 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0035070 con 0x7fdaf0104f10 2024-09-12T01:58:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.940+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01a01d0 con 0x7fdaf0104f10 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.942+0000 7fdae6ffd640 1 -- 172.21.15.122:0/885371470 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdae0042030 con 0x7fdaf0104f10 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 Processor -- start 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 -- start start 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa8c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa91780 0x56331fa93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56331f9eb2f0 con 0x56331fa8c6f0 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56331f993270 con 0x56331fa8bdb0 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdafaa02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56331f9e72b0 con 0x56331fa91780 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdae6ffd640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdadc0a2c30 0x7fdadc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdaf5a01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdaf5a01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57382/0 (socket says 172.21.15.122:57382) 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdaf5a01640 1 -- 172.21.15.122:0/4001447359 learned_addr learned my addr 172.21.15.122:0/4001447359 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdaf6202640 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa8c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdaf5200640 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa91780 0x56331fa93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.943+0000 7fdaf6202640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdadc0a2c30 0x7fdadc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdae6ffd640 1 -- 172.21.15.122:0/885371470 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdae00ef7a0 con 0x7fdaf0104f10 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf5a01640 1 -- 172.21.15.122:0/4001447359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa91780 msgr2=0x56331fa93c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa91780 0x56331fa93c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf5a01640 1 -- 172.21.15.122:0/4001447359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 msgr2=0x56331fa8c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa8c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf5a01640 1 -- 172.21.15.122:0/4001447359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331f992340 con 0x56331fa8c6f0 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa911b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00ae90 tx=0x7fdaec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef8d9c2eb5f7571 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdaf6202640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdadc0a2c30 0x7fdadc0a50f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdae800ffd0 tx=0x7fdae8005a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdadbfff640 1 -- 172.21.15.122:0/4001447359 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec011850 con 0x56331fa8c6f0 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdadbfff640 1 -- 172.21.15.122:0/4001447359 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec01a480 con 0x56331fa8c6f0 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.944+0000 7fdadbfff640 1 -- 172.21.15.122:0/4001447359 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec010630 con 0x56331fa8c6f0 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 -- 172.21.15.122:0/4001447359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 msgr2=0x56331fa911b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa911b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00ae90 tx=0x7fdaec00f730 comp rx=0 tx=0).stop 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 -- 172.21.15.122:0/4001447359 shutdown_connections 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa91780 0x56331fa93c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa911b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 --2- 172.21.15.122:0/4001447359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa8c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 -- 172.21.15.122:0/4001447359 >> 172.21.15.122:0/4001447359 conn(0x56331fa87420 msgr2=0x56331fa87820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 -- 172.21.15.122:0/4001447359 shutdown_connections 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 -- 172.21.15.122:0/4001447359 wait complete. 2024-09-12T01:58:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 -- start start 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.945+0000 7fdafaa02740 1 Processor -- start 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa95fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa91780 0x56331fa9d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56331fa94750 con 0x56331fa8c6f0 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56331f974290 con 0x56331fa91780 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56331fa948d0 con 0x56331fa8bdb0 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 10 client.0 _ll_register_callbacks cb 0x56331f9a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57394/0 (socket says 172.21.15.122:57394) 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 -- 172.21.15.122:0/524438946 learned_addr learned my addr 172.21.15.122:0/524438946 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf6202640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa91780 0x56331fa9d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa95fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 -- 172.21.15.122:0/524438946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 msgr2=0x56331fa95fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa95fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 -- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa91780 msgr2=0x56331fa9d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa91780 0x56331fa9d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 -- 172.21.15.122:0/524438946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331f6fe210 con 0x56331fa8c6f0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdaf5200640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa964e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdae00314e0 tx=0x7fdae002ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d39732638b906 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdad97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdad97fa640 1 -- 172.21.15.122:0/524438946 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0049040 con 0x56331fa8c6f0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdad97fa640 1 -- 172.21.15.122:0/524438946 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdae00b8670 con 0x56331fa8c6f0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.946+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec01a480 con 0x56331fa8c6f0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdad97fa640 1 -- 172.21.15.122:0/524438946 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdae00bb960 con 0x56331fa8c6f0 2024-09-12T01:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 10 client.17277 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56331f7ca620 con 0x56331fa8c6f0 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdad97fa640 1 -- 172.21.15.122:0/524438946 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec01a480 con 0x56331fa8c6f0 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdad97fa640 1 -- 172.21.15.122:0/524438946 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdae0049c40 con 0x56331fa8c6f0 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 10 client.17277 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 20 client.17277 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331faa1bc0 0x56331faa4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 4 client.17277 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56331f9f8a20 con 0x56331faa1bc0 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.947+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331faa1bc0 0x56331faa4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.948+0000 7fdad97fa640 10 client.17277 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.948+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331faa1bc0 0x56331faa4000 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1349ab5f45fdbd8b server_cookie=2a21ca3a784112ed in_seq=0 out_seq=0 2024-09-12T01:58:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.948+0000 7fdad97fa640 1 -- 172.21.15.122:0/524438946 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56331f9f8a20 con 0x56331faa1bc0 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.948+0000 7fdad97fa640 10 client.17277 handle_command_reply: tid=0 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.948+0000 7fdafaa02740 1 client.17277 shutdown 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 msgr2=0x56331fa964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa964e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdae00314e0 tx=0x7fdae002ffa0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 shutdown_connections 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331faa1bc0 0x56331faa4000 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa91780 0x56331fa9d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa8c6f0 0x56331fa964e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdad97fa640 0 client.17277 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 --2- 172.21.15.122:0/524438946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa95fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 >> 172.21.15.122:0/524438946 conn(0x56331fa87420 msgr2=0x56331fa88c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 shutdown_connections 2024-09-12T01:58:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 1 -- 172.21.15.122:0/524438946 wait complete. 2024-09-12T01:58:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.949+0000 7fdafaa02740 20 client.17277 trim_cache size 0 max 16384 2024-09-12T01:58:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 Processor -- start 2024-09-12T01:58:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 -- start start 2024-09-12T01:58:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8c6f0 0x56331fa9d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01a9720 con 0x56331fa91780 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01a6ff0 con 0x56331fa8bdb0 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.963+0000 7fdafaa02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01a4e20 con 0x56331fa8c6f0 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa888f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57396/0 (socket says 172.21.15.122:57396) 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 -- 172.21.15.122:0/3558089759 learned_addr learned my addr 172.21.15.122:0/3558089759 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf5200640 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8c6f0 0x56331fa9d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 -- 172.21.15.122:0/3558089759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8c6f0 msgr2=0x56331fa9d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8c6f0 0x56331fa9d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 -- 172.21.15.122:0/3558089759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 msgr2=0x56331fa883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa883b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 -- 172.21.15.122:0/3558089759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331fa8cbd0 con 0x56331fa91780 2024-09-12T01:58:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdaf6202640 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa888f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdae8007760 tx=0x7fdae8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc8757ed012a5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdadb7fe640 1 -- 172.21.15.122:0/3558089759 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800d840 con 0x56331fa91780 2024-09-12T01:58:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdadb7fe640 1 -- 172.21.15.122:0/3558089759 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae8029480 con 0x56331fa91780 2024-09-12T01:58:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.964+0000 7fdadb7fe640 1 -- 172.21.15.122:0/3558089759 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae801e660 con 0x56331fa91780 2024-09-12T01:58:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 -- 172.21.15.122:0/3558089759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 msgr2=0x56331fa888f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa888f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdae8007760 tx=0x7fdae8005ee0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 -- 172.21.15.122:0/3558089759 shutdown_connections 2024-09-12T01:58:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa888f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8c6f0 0x56331fa9d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 --2- 172.21.15.122:0/3558089759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8bdb0 0x56331fa883b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 -- 172.21.15.122:0/3558089759 >> 172.21.15.122:0/3558089759 conn(0x56331fa87420 msgr2=0x56331fa8d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 -- 172.21.15.122:0/3558089759 shutdown_connections 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 -- 172.21.15.122:0/3558089759 wait complete. 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 -- start start 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.965+0000 7fdafaa02740 1 Processor -- start 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa9af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8c6f0 0x56331fa9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa9c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01a54b0 con 0x56331fa91780 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01a6920 con 0x56331fa8c6f0 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01a5b80 con 0x56331fa8bdb0 2024-09-12T01:58:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 10 client.0 _ll_register_callbacks cb 0x7fdae0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa9c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa9c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57408/0 (socket says 172.21.15.122:57408) 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 -- 172.21.15.122:0/2207000268 learned_addr learned my addr 172.21.15.122:0/2207000268 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa9af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 -- 172.21.15.122:0/2207000268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 msgr2=0x56331fa9af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa9af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 -- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8c6f0 msgr2=0x56331fa9b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8c6f0 0x56331fa9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 -- 172.21.15.122:0/2207000268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331fa9ca60 con 0x56331fa91780 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdaf6202640 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa9c450 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdae800f590 tx=0x7fdae8029860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b11956312aa77c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdad97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdad97fa640 1 -- 172.21.15.122:0/2207000268 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800d840 con 0x56331fa91780 2024-09-12T01:58:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdad97fa640 1 -- 172.21.15.122:0/2207000268 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fdae800d9e0 con 0x56331fa91780 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae8029480 con 0x56331fa91780 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdad97fa640 1 -- 172.21.15.122:0/2207000268 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdae800dbe0 con 0x56331fa91780 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 10 client.17283 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.966+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56331fb0cbc0 con 0x56331fa91780 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdad97fa640 1 -- 172.21.15.122:0/2207000268 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae8029480 con 0x56331fa91780 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdad97fa640 1 -- 172.21.15.122:0/2207000268 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdae803c020 con 0x56331fa91780 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdafaa02740 10 client.17283 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdafaa02740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdafaa02740 20 client.17283 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdafaa02740 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331fb0d550 0x56331fb0f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdafaa02740 4 client.17283 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56331f9f2d70 con 0x56331fb0d550 2024-09-12T01:58:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331fb0d550 0x56331fb0f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdad97fa640 10 client.17283 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.967+0000 7fdaf5a01640 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331fb0d550 0x56331fb0f970 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f6c8e82225233e server_cookie=aae2b34ae749f128 in_seq=0 out_seq=0 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdad97fa640 1 -- 172.21.15.122:0/2207000268 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56331f9f2d70 con 0x56331fb0d550 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdad97fa640 10 client.17283 handle_command_reply: tid=0 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 client.17283 shutdown 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 msgr2=0x56331fa9c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa9c450 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdae800f590 tx=0x7fdae8029860 comp rx=0 tx=0).stop 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 shutdown_connections 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56331fb0d550 0x56331fb0f970 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56331fa91780 0x56331fa9c450 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56331fa8c6f0 0x56331fa9b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdad97fa640 0 client.17283 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 --2- 172.21.15.122:0/2207000268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56331fa8bdb0 0x56331fa9af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.981+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 >> 172.21.15.122:0/2207000268 conn(0x56331fa87420 msgr2=0x56331fa819a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 shutdown_connections 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdafaa02740 1 -- 172.21.15.122:0/2207000268 wait complete. 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdafaa02740 20 client.17283 trim_cache size 0 max 16384 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:58:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:58:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdadc0a2c30 msgr2=0x7fdadc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdadc0a2c30 0x7fdadc0a50f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdae800ffd0 tx=0x7fdae8005a60 comp rx=0 tx=0).stop 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 msgr2=0x7fdaf01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf01983c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fdae0031610 tx=0x7fdae00043b0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 shutdown_connections 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdadc0a2c30 0x7fdadc0a50f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf0107840 0x7fdaf019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaf0104f10 0x7fdaf01983c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 --2- 172.21.15.122:0/885371470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf01025e0 0x7fdaf0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 >> 172.21.15.122:0/885371470 conn(0x7fdaf00fc1e0 msgr2=0x7fdaf0102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 shutdown_connections 2024-09-12T01:58:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:42.982+0000 7fdaf7c65640 1 -- 172.21.15.122:0/885371470 wait complete. 2024-09-12T01:58:43.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.87 MB/sec execute 541 sec latency 21724.753 ms 2024-09-12T01:58:43.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:43 smithi122 ceph-mon[30925]: pgmap v702: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-09-12T01:58:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:43 smithi151 ceph-mon[35229]: pgmap v702: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-09-12T01:58:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:43 smithi183 ceph-mon[35127]: pgmap v702: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 2.4 MiB/s wr, 96 op/s 2024-09-12T01:58:44.860 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.85 MB/sec execute 542 sec latency 22724.818 ms 2024-09-12T01:58:45.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.83 MB/sec execute 543 sec latency 23724.901 ms 2024-09-12T01:58:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:45 smithi122 ceph-mon[30925]: pgmap v703: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-12T01:58:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:45 smithi151 ceph-mon[35229]: pgmap v703: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-12T01:58:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:45 smithi183 ceph-mon[35127]: pgmap v703: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-12T01:58:46.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:58:46.605 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:58:46.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.81 MB/sec execute 544 sec latency 24724.992 ms 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 -- 172.21.15.122:0/1224675537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218106f20 msgr2=0x7f221810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 --2- 172.21.15.122:0/1224675537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218106f20 0x7f221810eaa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2210009ca0 tx=0x7f221002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 -- 172.21.15.122:0/1224675537 shutdown_connections 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 --2- 172.21.15.122:0/1224675537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218106f20 0x7f221810eaa0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 --2- 172.21.15.122:0/1224675537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22181027a0 0x7f22181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 --2- 172.21.15.122:0/1224675537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2218101df0 0x7f22181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 -- 172.21.15.122:0/1224675537 >> 172.21.15.122:0/1224675537 conn(0x7f22180fd8f0 msgr2=0x7f22180ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 -- 172.21.15.122:0/1224675537 shutdown_connections 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.123+0000 7f221eb97640 1 -- 172.21.15.122:0/1224675537 wait complete. 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 Processor -- start 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 -- start start 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 0x7f2218198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22181027a0 0x7f2218198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2218106f20 0x7f221819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2218111f80 con 0x7f2218101df0 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2218111e00 con 0x7f22181027a0 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221eb97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2218112100 con 0x7f2218106f20 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 0x7f2218198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 0x7f2218198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57426/0 (socket says 172.21.15.122:57426) 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 -- 172.21.15.122:0/4072468994 learned_addr learned my addr 172.21.15.122:0/4072468994 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f2217fff640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22181027a0 0x7f2218198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 -- 172.21.15.122:0/4072468994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2218106f20 msgr2=0x7f221819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221d134640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2218106f20 0x7f221819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2218106f20 0x7f221819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 -- 172.21.15.122:0/4072468994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22181027a0 msgr2=0x7f2218198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22181027a0 0x7f2218198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.124+0000 7f221c933640 1 -- 172.21.15.122:0/4072468994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221819d3d0 con 0x7f2218101df0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.125+0000 7f221c933640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 0x7f2218198420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f220c00c920 tx=0x7f220c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.125+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c007c70 con 0x7f2218101df0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.125+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221819d6c0 con 0x7f2218101df0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.125+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220c007e10 con 0x7f2218101df0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.125+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c018890 con 0x7f2218101df0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.125+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22181a4fa0 con 0x7f2218101df0 2024-09-12T01:58:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.126+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22181021d0 con 0x7f2218101df0 2024-09-12T01:58:47.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.127+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f220c018a70 con 0x7f2218101df0 2024-09-12T01:58:47.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.127+0000 7f2215ffb640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22080a2b80 0x7f22080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:47.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.128+0000 7f2217fff640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22080a2b80 0x7f22080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:47.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.128+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f220c0cf190 con 0x7f2218101df0 2024-09-12T01:58:47.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.128+0000 7f2217fff640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22080a2b80 0x7f22080a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2218108890 tx=0x7f220400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:47.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.132+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f220c014030 con 0x7f2218101df0 2024-09-12T01:58:47.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.340+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f221810a100 con 0x7f2218101df0 2024-09-12T01:58:47.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.341+0000 7f2215ffb640 1 -- 172.21.15.122:0/4072468994 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f220c09c0f0 con 0x7f2218101df0 2024-09-12T01:58:47.341 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:58:47.341 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:58:47.342 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:58:47.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22080a2b80 msgr2=0x7f22080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22080a2b80 0x7f22080a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2218108890 tx=0x7f220400b390 comp rx=0 tx=0).stop 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 msgr2=0x7f2218198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 0x7f2218198420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f220c00c920 tx=0x7f220c00cde0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 shutdown_connections 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f22080a2b80 0x7f22080a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2218106f20 0x7f221819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22181027a0 0x7f2218198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 --2- 172.21.15.122:0/4072468994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2218101df0 0x7f2218198420 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 >> 172.21.15.122:0/4072468994 conn(0x7f22180fd8f0 msgr2=0x7f22180fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 shutdown_connections 2024-09-12T01:58:47.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:47.343+0000 7f221eb97640 1 -- 172.21.15.122:0/4072468994 wait complete. 2024-09-12T01:58:47.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.80 MB/sec execute 545 sec latency 25725.094 ms 2024-09-12T01:58:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:47 smithi122 ceph-mon[30925]: pgmap v704: 97 pgs: 97 active+clean; 287 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.1 MiB/s wr, 86 op/s 2024-09-12T01:58:47.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:47 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4072468994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:58:48.102 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 791b60cc --release --await' 2024-09-12T01:58:48.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 791b60cc --release --await 2024-09-12T01:58:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:47 smithi151 ceph-mon[35229]: pgmap v704: 97 pgs: 97 active+clean; 287 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.1 MiB/s wr, 86 op/s 2024-09-12T01:58:48.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:47 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4072468994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:58:48.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:47 smithi183 ceph-mon[35127]: pgmap v704: 97 pgs: 97 active+clean; 287 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.1 MiB/s wr, 86 op/s 2024-09-12T01:58:48.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:47 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4072468994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:58:48.347 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:58:48.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 -- 172.21.15.122:0/2669751925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102ef0 msgr2=0x7f9c0810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 --2- 172.21.15.122:0/2669751925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102ef0 0x7f9c0810da40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009ca0 tx=0x7f9c0002a950 comp rx=0 tx=0).stop 2024-09-12T01:58:48.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 -- 172.21.15.122:0/2669751925 shutdown_connections 2024-09-12T01:58:48.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 --2- 172.21.15.122:0/2669751925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102ef0 0x7f9c0810da40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 --2- 172.21.15.122:0/2669751925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c08102570 0x7f9c081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 --2- 172.21.15.122:0/2669751925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c08100e20 0x7f9c08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 -- 172.21.15.122:0/2669751925 >> 172.21.15.122:0/2669751925 conn(0x7f9c080fc7d0 msgr2=0x7f9c080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 -- 172.21.15.122:0/2669751925 shutdown_connections 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.833+0000 7f9c100bf640 1 -- 172.21.15.122:0/2669751925 wait complete. 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 Processor -- start 2024-09-12T01:58:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 -- start start 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c08100e20 0x7f9c081984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 0x7f9c08198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c08102ef0 0x7f9c0819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08110fa0 con 0x7f9c08102570 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08110e20 con 0x7f9c08100e20 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c100bf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08111120 con 0x7f9c08102ef0 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 0x7f9c08198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 0x7f9c08198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57456/0 (socket says 172.21.15.122:57456) 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 -- 172.21.15.122:0/1712899933 learned_addr learned my addr 172.21.15.122:0/1712899933 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c08100e20 0x7f9c081984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 -- 172.21.15.122:0/1712899933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c08102ef0 msgr2=0x7f9c0819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c08102ef0 0x7f9c0819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 -- 172.21.15.122:0/1712899933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c08100e20 msgr2=0x7f9c081984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c08100e20 0x7f9c081984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 -- 172.21.15.122:0/1712899933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0819d490 con 0x7f9c08102570 2024-09-12T01:58:48.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.834+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 0x7f9c08198a20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800b4a0 tx=0x7f9bf800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:48.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.835+0000 7f9bfeffd640 1 -- 172.21.15.122:0/1712899933 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8014070 con 0x7f9c08102570 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.835+0000 7f9bfeffd640 1 -- 172.21.15.122:0/1712899933 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf8010af0 con 0x7f9c08102570 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.835+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0819d780 con 0x7f9c08102570 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.835+0000 7f9bfeffd640 1 -- 172.21.15.122:0/1712899933 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8017d10 con 0x7f9c08102570 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.835+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081a5060 con 0x7f9c08102570 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.837+0000 7f9bfeffd640 1 -- 172.21.15.122:0/1712899933 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9bf8017070 con 0x7f9c08102570 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.837+0000 7f9c12498740 1 Processor -- start 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.837+0000 7f9c12498740 1 -- start start 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c12498740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba44440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c12498740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c12498740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba51630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c12498740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561c1b9a9620 con 0x561c1ba44060 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c12498740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561c1b950e50 con 0x561c1ba4f240 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c12498740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c1b9a55e0 con 0x561c1ba44980 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9bfeffd640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bf40a2b80 0x7f9bf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba44440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba44440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57458/0 (socket says 172.21.15.122:57458) 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 -- 172.21.15.122:0/1723965413 learned_addr learned my addr 172.21.15.122:0/1723965413 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bf40a2b80 0x7f9bf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba51630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9bfeffd640 1 -- 172.21.15.122:0/1712899933 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9bf80ce4c0 con 0x7f9c08102570 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 -- 172.21.15.122:0/1723965413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 msgr2=0x561c1ba4ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 -- 172.21.15.122:0/1723965413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 msgr2=0x561c1ba51630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba51630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0e65c640 1 -- 172.21.15.122:0/1723965413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1b941d40 con 0x561c1ba44060 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.838+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba51630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.839+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba44440 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9c00006010 tx=0x7f9c0002f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3c825f4a36a8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.839+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bf40a2b80 0x7f9bf40a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c0002ae70 tx=0x7f9c00034000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.839+0000 7f9bf3fff640 1 -- 172.21.15.122:0/1723965413 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00034070 con 0x561c1ba44060 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.839+0000 7f9bf3fff640 1 -- 172.21.15.122:0/1723965413 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0002fb60 con 0x561c1ba44060 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.839+0000 7f9bf3fff640 1 -- 172.21.15.122:0/1723965413 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000486e0 con 0x561c1ba44060 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 -- 172.21.15.122:0/1723965413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 msgr2=0x561c1ba44440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba44440 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9c00006010 tx=0x7f9c0002f5e0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 -- 172.21.15.122:0/1723965413 shutdown_connections 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba51630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 --2- 172.21.15.122:0/1723965413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba44440 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 -- 172.21.15.122:0/1723965413 >> 172.21.15.122:0/1723965413 conn(0x561c1ba405f0 msgr2=0x561c1ba409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 -- 172.21.15.122:0/1723965413 shutdown_connections 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 -- 172.21.15.122:0/1723965413 wait complete. 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 -- start start 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.840+0000 7f9c12498740 1 Processor -- start 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba4c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561c1ba51e10 con 0x561c1ba44060 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561c1b931f10 con 0x561c1ba4f240 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c1ba51f90 con 0x561c1ba44980 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c12498740 10 client.0 _ll_register_callbacks cb 0x561c1b955db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba4c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba4c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57468/0 (socket says 172.21.15.122:57468) 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1738015622 learned_addr learned my addr 172.21.15.122:0/1738015622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1738015622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 msgr2=0x561c1ba4c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 msgr2=0x561c1ba4cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba4cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1738015622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1b6b8580 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.841+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba4c170 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9c04011790 tx=0x7f9c04011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1690b78a93c3d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9bf17fa640 1 -- 172.21.15.122:0/1738015622 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400fe10 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9bf17fa640 1 -- 172.21.15.122:0/1738015622 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9c04005020 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c1b9b6b90 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9bf17fa640 1 -- 172.21.15.122:0/1738015622 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c04005220 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 10 client.17301 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561c1b782960 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9bf17fa640 1 -- 172.21.15.122:0/1738015622 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561c1b9b6b90 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9bf17fa640 1 -- 172.21.15.122:0/1738015622 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9c04018030 con 0x561c1ba44060 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 10 client.17301 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 10 client.17301 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:58:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 20 client.17301 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1ba5eec0 0x561c1ba612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 4 client.17301 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.842+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1b9b10a0 con 0x561c1ba5eec0 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.843+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1ba5eec0 0x561c1ba612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.843+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1ba5eec0 0x561c1ba612b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd600525a067628d server_cookie=f93b4f4a64bbdcd2 in_seq=0 out_seq=0 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.843+0000 7f9bf17fa640 10 client.17301 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9bf17fa640 1 -- 172.21.15.122:0/1738015622 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c1b9b10a0 con 0x561c1ba5eec0 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9bf17fa640 10 client.17301 handle_command_reply: tid=0 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 client.17301 shutdown 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 msgr2=0x561c1ba4c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba4c170 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9c04011790 tx=0x7f9c04011c50 comp rx=0 tx=0).stop 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 shutdown_connections 2024-09-12T01:58:48.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1ba5eec0 0x561c1ba612b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba4c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 --2- 172.21.15.122:0/1738015622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba4c170 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9bf17fa640 0 client.17301 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 >> 172.21.15.122:0/1738015622 conn(0x561c1ba405f0 msgr2=0x561c1ba3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 shutdown_connections 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 1 -- 172.21.15.122:0/1738015622 wait complete. 2024-09-12T01:58:48.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.844+0000 7f9c12498740 20 client.17301 trim_cache size 0 max 16384 2024-09-12T01:58:48.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1165484 9.78 MB/sec execute 546 sec latency 26725.132 ms 2024-09-12T01:58:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 Processor -- start 2024-09-12T01:58:48.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 -- start start 2024-09-12T01:58:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba47420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba66bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081aff00 con 0x561c1ba44060 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081ad7d0 con 0x561c1ba4f240 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c12498740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081ab600 con 0x561c1ba44980 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba47420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba47420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57480/0 (socket says 172.21.15.122:57480) 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1663215600 learned_addr learned my addr 172.21.15.122:0/1663215600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1663215600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 msgr2=0x561c1ba3f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba3f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1663215600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 msgr2=0x561c1ba66bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba66bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba66bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 -- 172.21.15.122:0/1663215600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1ba4e0b0 con 0x561c1ba44060 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba66bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.861+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba47420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c7b0 tx=0x7f9c04005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efb0e5b4ced5728 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9bf37fe640 1 -- 172.21.15.122:0/1663215600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04028070 con 0x561c1ba44060 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9bf37fe640 1 -- 172.21.15.122:0/1663215600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0400fc10 con 0x561c1ba44060 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9bf37fe640 1 -- 172.21.15.122:0/1663215600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040035f0 con 0x561c1ba44060 2024-09-12T01:58:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 -- 172.21.15.122:0/1663215600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 msgr2=0x561c1ba47420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba47420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c7b0 tx=0x7f9c04005d20 comp rx=0 tx=0).stop 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 -- 172.21.15.122:0/1663215600 shutdown_connections 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1ba66bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44980 0x561c1ba3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 --2- 172.21.15.122:0/1663215600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44060 0x561c1ba47420 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 -- 172.21.15.122:0/1663215600 >> 172.21.15.122:0/1663215600 conn(0x561c1ba405f0 msgr2=0x561c1ba3ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 -- 172.21.15.122:0/1663215600 shutdown_connections 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.862+0000 7f9c12498740 1 -- 172.21.15.122:0/1663215600 wait complete. 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 -- start start 2024-09-12T01:58:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 Processor -- start 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44060 0x561c1ba53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 0x561c1ba53bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1bac8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081abc90 con 0x561c1ba44980 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081ad100 con 0x561c1ba4f240 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081ac360 con 0x561c1ba44060 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c12498740 10 client.0 _ll_register_callbacks cb 0x7f9c0400ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c0d65a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 0x561c1ba53bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c0d65a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 0x561c1ba53bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57494/0 (socket says 172.21.15.122:57494) 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c0d65a640 1 -- 172.21.15.122:0/3783334232 learned_addr learned my addr 172.21.15.122:0/3783334232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1bac8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.863+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44060 0x561c1ba53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0d65a640 1 -- 172.21.15.122:0/3783334232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44060 msgr2=0x561c1ba53680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44060 0x561c1ba53680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0d65a640 1 -- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 msgr2=0x561c1bac8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1bac8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0d65a640 1 -- 172.21.15.122:0/3783334232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1bac90d0 con 0x561c1ba44980 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44060 0x561c1ba53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0e65c640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1bac8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:48.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c0d65a640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 0x561c1ba53bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80976f0 tx=0x7f9bf801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae58393f8b09846 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9bf17fa640 1 -- 172.21.15.122:0/3783334232 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf801e070 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9bf17fa640 1 -- 172.21.15.122:0/3783334232 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9bf8010ca0 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0400fc10 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9bf17fa640 1 -- 172.21.15.122:0/3783334232 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9bf809f0d0 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 10 client.17307 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561c1baca410 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9bf17fa640 1 -- 172.21.15.122:0/3783334232 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0400fc10 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9bf17fa640 1 -- 172.21.15.122:0/3783334232 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9bf8097390 con 0x561c1ba44980 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 10 client.17307 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 10 client.17307 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:58:48.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 20 client.17307 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1bacab40 0x561c1baccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 4 client.17307 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "791b60cc", "release": true, "await": true}] 2024-09-12T01:58:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.864+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "791b60cc", "release": true, "await": true}) -- 0x561c1b92d5a0 con 0x561c1bacab40 2024-09-12T01:58:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.865+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1bacab40 0x561c1baccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.866+0000 7f9bf17fa640 10 client.17307 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.866+0000 7f9c0de5b640 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1bacab40 0x561c1baccf60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=326979500b91b822 server_cookie=68e8c6c47101b628 in_seq=0 out_seq=0 2024-09-12T01:58:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9bf17fa640 1 -- 172.21.15.122:0/3783334232 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561c1b92d5a0 con 0x561c1bacab40 2024-09-12T01:58:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9bf17fa640 10 client.17307 handle_command_reply: tid=0 2024-09-12T01:58:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 client.17307 shutdown 2024-09-12T01:58:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 msgr2=0x561c1ba53bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 0x561c1ba53bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80976f0 tx=0x7f9bf801c000 comp rx=0 tx=0).stop 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 shutdown_connections 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561c1bacab40 0x561c1baccf60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c1ba4f240 0x561c1bac8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c1ba44980 0x561c1ba53bc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9bf17fa640 0 client.17307 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 --2- 172.21.15.122:0/3783334232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c1ba44060 0x561c1ba53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 >> 172.21.15.122:0/3783334232 conn(0x561c1ba405f0 msgr2=0x561c1ba3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 shutdown_connections 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 1 -- 172.21.15.122:0/3783334232 wait complete. 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.875+0000 7f9c12498740 20 client.17307 trim_cache size 0 max 16384 2024-09-12T01:58:48.882 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 48, 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "791b60cc": { 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 48, 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:58:48.883 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 147.2, 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:48.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:58:48.885 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:58:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bf40a2b80 msgr2=0x7f9bf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bf40a2b80 0x7f9bf40a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c0002ae70 tx=0x7f9c00034000 comp rx=0 tx=0).stop 2024-09-12T01:58:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 msgr2=0x7f9c08198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 0x7f9c08198a20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800b4a0 tx=0x7f9bf800b960 comp rx=0 tx=0).stop 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 shutdown_connections 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bf40a2b80 0x7f9bf40a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c08102ef0 0x7f9c0819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c08102570 0x7f9c08198a20 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 --2- 172.21.15.122:0/1712899933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c08100e20 0x7f9c081984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 >> 172.21.15.122:0/1712899933 conn(0x7f9c080fc7d0 msgr2=0x7f9c08069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 shutdown_connections 2024-09-12T01:58:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:58:48.876+0000 7f9c100bf640 1 -- 172.21.15.122:0/1712899933 wait complete. 2024-09-12T01:58:49.592 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '791b60cc', total seconds elapsed: 30.4 2024-09-12T01:58:49.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1167890 9.78 MB/sec execute 547 sec latency 26740.305 ms 2024-09-12T01:58:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:49 smithi122 ceph-mon[30925]: pgmap v705: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:58:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:49 smithi151 ceph-mon[35229]: pgmap v705: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:58:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:49 smithi183 ceph-mon[35127]: pgmap v705: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 95 op/s 2024-09-12T01:58:50.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1169992 9.79 MB/sec execute 548 sec latency 11.574 ms 2024-09-12T01:58:51.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1172519 9.79 MB/sec execute 549 sec latency 89.467 ms 2024-09-12T01:58:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:51 smithi122 ceph-mon[30925]: pgmap v706: 97 pgs: 97 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.1 MiB/s wr, 90 op/s 2024-09-12T01:58:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:51 smithi151 ceph-mon[35229]: pgmap v706: 97 pgs: 97 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.1 MiB/s wr, 90 op/s 2024-09-12T01:58:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:51 smithi183 ceph-mon[35127]: pgmap v706: 97 pgs: 97 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.1 MiB/s wr, 90 op/s 2024-09-12T01:58:52.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1175653 9.81 MB/sec execute 550 sec latency 10.849 ms 2024-09-12T01:58:53.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1178154 9.81 MB/sec execute 551 sec latency 13.563 ms 2024-09-12T01:58:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:53 smithi122 ceph-mon[30925]: pgmap v707: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 157 op/s 2024-09-12T01:58:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:53 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:53 smithi151 ceph-mon[35229]: pgmap v707: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 157 op/s 2024-09-12T01:58:54.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:54.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:53 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:53 smithi183 ceph-mon[35127]: pgmap v707: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 157 op/s 2024-09-12T01:58:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:58:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:53 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:54.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1180134 9.81 MB/sec execute 552 sec latency 32.401 ms 2024-09-12T01:58:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:58:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:58:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:58:55.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1183111 9.83 MB/sec execute 553 sec latency 12.488 ms 2024-09-12T01:58:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:55 smithi122 ceph-mon[30925]: pgmap v708: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3580784926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f01036f0 msgr2=0x7fd7f0109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3580784926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f01036f0 0x7fd7f0109f80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009ca0 tx=0x7fd7e802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3580784926 shutdown_connections 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3580784926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f01036f0 0x7fd7f0109f80 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3580784926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7f0102d30 0x7fd7f01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3580784926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f0101b40 0x7fd7f0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3580784926 >> 172.21.15.122:0/3580784926 conn(0x7fd7f00fd2e0 msgr2=0x7fd7f00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3580784926 shutdown_connections 2024-09-12T01:58:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.074+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3580784926 wait complete. 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.075+0000 7fd7f7d28640 1 Processor -- start 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.075+0000 7fd7f7d28640 1 -- start start 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.075+0000 7fd7f7d28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 0x7fd7f0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f7d28640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7f0102d30 0x7fd7f01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f7d28640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f01036f0 0x7fd7f019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f7d28640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010d7f0 con 0x7fd7f0101b40 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f7d28640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010d670 con 0x7fd7f0102d30 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f7d28640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010d970 con 0x7fd7f01036f0 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 0x7fd7f0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 0x7fd7f0197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57984/0 (socket says 172.21.15.122:57984) 2024-09-12T01:58:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/3648417799 learned_addr learned my addr 172.21.15.122:0/3648417799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f01036f0 0x7fd7f019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/3648417799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f01036f0 msgr2=0x7fd7f019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f01036f0 0x7fd7f019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/3648417799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7f0102d30 msgr2=0x7fd7f01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7f0102d30 0x7fd7f01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/3648417799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f019f980 con 0x7fd7f0101b40 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f01036f0 0x7fd7f019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.076+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 0x7fd7f0197d60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e9c0 tx=0x7fd7ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.077+0000 7fd7e6ffd640 1 -- 172.21.15.122:0/3648417799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00cd80 con 0x7fd7f0101b40 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.077+0000 7fd7e6ffd640 1 -- 172.21.15.122:0/3648417799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec0035f0 con 0x7fd7f0101b40 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.077+0000 7fd7e6ffd640 1 -- 172.21.15.122:0/3648417799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec010690 con 0x7fd7f0101b40 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.077+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f019fc70 con 0x7fd7f0101b40 2024-09-12T01:58:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.077+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a01c0 con 0x7fd7f0101b40 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.079+0000 7fd7faab7740 1 Processor -- start 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 -- start start 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7e6ffd640 1 -- 172.21.15.122:0/3648417799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd7ec004020 con 0x7fd7f0101b40 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c11d6062f0 con 0x55c11d6a32e0 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c11d5ae270 con 0x55c11d6a8370 2024-09-12T01:58:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7faab7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c11d6022b0 con 0x55c11d6a29a0 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6a7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57992/0 (socket says 172.21.15.122:57992) 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/587374643 learned_addr learned my addr 172.21.15.122:0/587374643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/587374643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 msgr2=0x55c11d6a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/587374643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 msgr2=0x55c11d6aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.080+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/587374643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11d5ad340 con 0x55c11d6a32e0 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6a7da0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0d73d0 tx=0x7fd7ec0d7890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c6a6b20ab9fbcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7e6ffd640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7dc0a2cb0 0x7fd7dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7dbfff640 1 -- 172.21.15.122:0/587374643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0e3070 con 0x55c11d6a32e0 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7dbfff640 1 -- 172.21.15.122:0/587374643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec09c380 con 0x55c11d6a32e0 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7dbfff640 1 -- 172.21.15.122:0/587374643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec068750 con 0x55c11d6a32e0 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7dc0a2cb0 0x7fd7dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.081+0000 7fd7e6ffd640 1 -- 172.21.15.122:0/3648417799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd7ec0cf420 con 0x7fd7f0101b40 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 -- 172.21.15.122:0/587374643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 msgr2=0x55c11d6a7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7dc0a2cb0 0x7fd7dc0a5170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005e30 tx=0x7fd7e8031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6a7da0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0d73d0 tx=0x7fd7ec0d7890 comp rx=0 tx=0).stop 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 -- 172.21.15.122:0/587374643 shutdown_connections 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6a7da0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 --2- 172.21.15.122:0/587374643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 -- 172.21.15.122:0/587374643 >> 172.21.15.122:0/587374643 conn(0x55c11d69e010 msgr2=0x55c11d69e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 -- 172.21.15.122:0/587374643 shutdown_connections 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 -- 172.21.15.122:0/587374643 wait complete. 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 -- start start 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.082+0000 7fd7faab7740 1 Processor -- start 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6acc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c11d6ab340 con 0x55c11d6a32e0 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c11d58f290 con 0x55c11d6a8370 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c11d6ab4c0 con 0x55c11d6a29a0 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 10 client.0 _ll_register_callbacks cb 0x55c11d5c0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f52c3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f62c5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f52c3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58008/0 (socket says 172.21.15.122:58008) 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f52c3640 1 -- 172.21.15.122:0/1706676357 learned_addr learned my addr 172.21.15.122:0/1706676357 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 msgr2=0x55c11d6b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1706676357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 msgr2=0x55c11d6ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1706676357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11d319210 con 0x55c11d6a29a0 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6acc00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0d73d0 tx=0x7fd7ec0681f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543ec8b07ad9b188 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7d97fa640 1 -- 172.21.15.122:0/1706676357 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec067790 con 0x55c11d6a29a0 2024-09-12T01:58:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7d97fa640 1 -- 172.21.15.122:0/1706676357 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd7ec066cf0 con 0x55c11d6a29a0 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.083+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7ec09c380 con 0x55c11d6a29a0 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7d97fa640 1 -- 172.21.15.122:0/1706676357 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7ec067990 con 0x55c11d6a29a0 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 10 client.26434 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c11d3e5620 con 0x55c11d6a29a0 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7d97fa640 1 -- 172.21.15.122:0/1706676357 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec09c380 con 0x55c11d6a29a0 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7d97fa640 1 -- 172.21.15.122:0/1706676357 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd7ec004310 con 0x55c11d6a29a0 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 10 client.26434 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 20 client.26434 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d6b8480 0x55c11d6ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 4 client.26434 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c11d613a20 con 0x55c11d6b8480 2024-09-12T01:58:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.084+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d6b8480 0x55c11d6ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.085+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d6b8480 0x55c11d6ba870 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b850117043812b server_cookie=da645234f71452d3 in_seq=0 out_seq=0 2024-09-12T01:58:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.085+0000 7fd7d97fa640 10 client.26434 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.085+0000 7fd7d97fa640 1 -- 172.21.15.122:0/1706676357 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c11d613a20 con 0x55c11d6b8480 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.085+0000 7fd7d97fa640 10 client.26434 handle_command_reply: tid=0 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.085+0000 7fd7faab7740 1 client.26434 shutdown 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 msgr2=0x55c11d6acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6acc00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0d73d0 tx=0x7fd7ec0681f0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 shutdown_connections 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d6b8480 0x55c11d6ba870 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d6b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a32e0 0x55c11d6ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1706676357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a29a0 0x55c11d6acc00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7d97fa640 0 client.26434 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 >> 172.21.15.122:0/1706676357 conn(0x55c11d69e010 msgr2=0x55c11d69f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 shutdown_connections 2024-09-12T01:58:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 1 -- 172.21.15.122:0/1706676357 wait complete. 2024-09-12T01:58:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.086+0000 7fd7faab7740 20 client.26434 trim_cache size 0 max 16384 2024-09-12T01:58:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.102+0000 7fd7faab7740 1 Processor -- start 2024-09-12T01:58:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.102+0000 7fd7faab7740 1 -- start start 2024-09-12T01:58:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.102+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 0x55c11d69d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.102+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.102+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 0x55c11d69d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.102+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 0x55c11d69d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58016/0 (socket says 172.21.15.122:58016) 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1794445800 learned_addr learned my addr 172.21.15.122:0/1794445800 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d69f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01ab840 con 0x55c11d6a29a0 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01a9110 con 0x55c11d6a8370 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01a6f40 con 0x55c11d6a32e0 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d69f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1794445800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 msgr2=0x55c11d6b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.103+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1794445800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 msgr2=0x55c11d69f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d69f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7f5ac4640 1 -- 172.21.15.122:0/1794445800 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11d6abbf0 con 0x55c11d6a29a0 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7f62c5640 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d69f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 0x55c11d69d0b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0058c0 tx=0x7fd7ec0d7890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4960b443b26b9da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7db7fe640 1 -- 172.21.15.122:0/1794445800 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec067bc0 con 0x55c11d6a29a0 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7db7fe640 1 -- 172.21.15.122:0/1794445800 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec0d2c70 con 0x55c11d6a29a0 2024-09-12T01:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.104+0000 7fd7db7fe640 1 -- 172.21.15.122:0/1794445800 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec067df0 con 0x55c11d6a29a0 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 msgr2=0x55c11d69d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 0x55c11d69d0b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0058c0 tx=0x7fd7ec0d7890 comp rx=0 tx=0).stop 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 shutdown_connections 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a8370 0x55c11d69f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 --2- 172.21.15.122:0/1794445800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a29a0 0x55c11d69d0b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 >> 172.21.15.122:0/1794445800 conn(0x55c11d69e010 msgr2=0x55c11d69c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 shutdown_connections 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 -- 172.21.15.122:0/1794445800 wait complete. 2024-09-12T01:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 -- start start 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.105+0000 7fd7faab7740 1 Processor -- start 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a29a0 0x55c11d6b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a8370 0x55c11d6b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01a75d0 con 0x55c11d6a8370 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01a8a40 con 0x55c11d6a29a0 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01a7ca0 con 0x55c11d6a32e0 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7faab7740 10 client.0 _ll_register_callbacks cb 0x7fd7ec0e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f5ac4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a29a0 0x55c11d6b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f62c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a8370 0x55c11d6b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34898/0 (socket says 172.21.15.122:34898) 2024-09-12T01:58:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 -- 172.21.15.122:0/771619903 learned_addr learned my addr 172.21.15.122:0/771619903 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 -- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a29a0 msgr2=0x55c11d6b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a29a0 0x55c11d6b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 -- 172.21.15.122:0/771619903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a8370 msgr2=0x55c11d6b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a8370 0x55c11d6b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 -- 172.21.15.122:0/771619903 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11d6b33c0 con 0x55c11d6a32e0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a29a0 0x55c11d6b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7f52c3640 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b1da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55c11d6baf60 tx=0x7fd7e00139d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512fdabef4c65216 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.106+0000 7fd7d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7d97fa640 1 -- 172.21.15.122:0/771619903 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0023070 con 0x55c11d6a32e0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7d97fa640 1 -- 172.21.15.122:0/771619903 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd7e0004250 con 0x55c11d6a32e0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7ec0d2c70 con 0x55c11d6a32e0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7d97fa640 1 -- 172.21.15.122:0/771619903 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd7e0004450 con 0x55c11d6a32e0 2024-09-12T01:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 10 client.26440 fetch_fsmap learned FSMap version 30 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c11d723050 con 0x55c11d6a32e0 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7d97fa640 1 -- 172.21.15.122:0/771619903 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec0d2c70 con 0x55c11d6a32e0 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7d97fa640 1 -- 172.21.15.122:0/771619903 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd7e0004bf0 con 0x55c11d6a32e0 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 10 client.26440 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 10 client.26440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 20 client.26440 populate_metadata read hostname 'smithi122' 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d7239e0 0x55c11d725e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 4 client.26440 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.107+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c11d60dd70 con 0x55c11d7239e0 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.108+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d7239e0 0x55c11d725e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.108+0000 7fd7d97fa640 10 client.26440 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.108+0000 7fd7f5ac4640 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d7239e0 0x55c11d725e00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571a9d45cc2da82e server_cookie=32c34b2c1bebf616 in_seq=0 out_seq=0 2024-09-12T01:58:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:55 smithi151 ceph-mon[35229]: pgmap v708: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-12T01:58:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7d97fa640 1 -- 172.21.15.122:0/771619903 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c11d60dd70 con 0x55c11d7239e0 2024-09-12T01:58:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7d97fa640 10 client.26440 handle_command_reply: tid=0 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 client.26440 shutdown 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 msgr2=0x55c11d6b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b1da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55c11d6baf60 tx=0x7fd7e00139d0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 shutdown_connections 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c11d7239e0 0x55c11d725e00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c11d6a8370 0x55c11d6b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c11d6a32e0 0x55c11d6b1da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 --2- 172.21.15.122:0/771619903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c11d6a29a0 0x55c11d6b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7d97fa640 0 client.26440 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 >> 172.21.15.122:0/771619903 conn(0x55c11d69e010 msgr2=0x55c11d69d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 shutdown_connections 2024-09-12T01:58:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 1 -- 172.21.15.122:0/771619903 wait complete. 2024-09-12T01:58:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.121+0000 7fd7faab7740 20 client.26440 trim_cache size 0 max 16384 2024-09-12T01:58:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:58:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:58:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:58:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7dc0a2cb0 msgr2=0x7fd7dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7dc0a2cb0 0x7fd7dc0a5170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005e30 tx=0x7fd7e8031bb0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 msgr2=0x7fd7f0197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 0x7fd7f0197d60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e9c0 tx=0x7fd7ec00ee80 comp rx=0 tx=0).stop 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 shutdown_connections 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd7dc0a2cb0 0x7fd7dc0a5170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7f01036f0 0x7fd7f019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7f0102d30 0x7fd7f01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 --2- 172.21.15.122:0/3648417799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101b40 0x7fd7f0197d60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:58:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 >> 172.21.15.122:0/3648417799 conn(0x7fd7f00fd2e0 msgr2=0x7fd7f0106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:58:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 shutdown_connections 2024-09-12T01:58:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:58:56.122+0000 7fd7f7d28640 1 -- 172.21.15.122:0/3648417799 wait complete. 2024-09-12T01:58:56.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:55 smithi183 ceph-mon[35127]: pgmap v708: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 155 op/s 2024-09-12T01:58:56.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1185935 9.83 MB/sec execute 554 sec latency 11.919 ms 2024-09-12T01:58:57.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1188475 9.84 MB/sec execute 555 sec latency 10.800 ms 2024-09-12T01:58:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:57 smithi122 ceph-mon[30925]: pgmap v709: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-09-12T01:58:58.004 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:57 smithi183 ceph-mon[35127]: pgmap v709: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-09-12T01:58:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:57 smithi151 ceph-mon[35229]: pgmap v709: 97 pgs: 97 active+clean; 286 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-09-12T01:58:58.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1190608 9.84 MB/sec execute 556 sec latency 11.323 ms 2024-09-12T01:58:59.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1193593 9.86 MB/sec execute 557 sec latency 43.407 ms 2024-09-12T01:58:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:58:59 smithi122 ceph-mon[30925]: pgmap v710: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-09-12T01:59:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:58:59 smithi151 ceph-mon[35229]: pgmap v710: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-09-12T01:59:00.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:58:59 smithi183 ceph-mon[35127]: pgmap v710: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 251 op/s 2024-09-12T01:59:00.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1196163 9.86 MB/sec execute 558 sec latency 11.809 ms 2024-09-12T01:59:01.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1198383 9.87 MB/sec execute 559 sec latency 11.313 ms 2024-09-12T01:59:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:01 smithi122 ceph-mon[30925]: pgmap v711: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-09-12T01:59:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:01 smithi151 ceph-mon[35229]: pgmap v711: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-09-12T01:59:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:01 smithi183 ceph-mon[35127]: pgmap v711: 97 pgs: 97 active+clean; 291 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-09-12T01:59:02.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1201003 9.88 MB/sec execute 560 sec latency 9.711 ms 2024-09-12T01:59:03.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1204018 9.89 MB/sec execute 561 sec latency 13.585 ms 2024-09-12T01:59:03.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:03 smithi122 ceph-mon[30925]: pgmap v712: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-09-12T01:59:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:03 smithi151 ceph-mon[35229]: pgmap v712: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-09-12T01:59:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:03 smithi183 ceph-mon[35127]: pgmap v712: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-09-12T01:59:04.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1206622 9.89 MB/sec execute 562 sec latency 11.666 ms 2024-09-12T01:59:05.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1208801 9.89 MB/sec execute 563 sec latency 12.400 ms 2024-09-12T01:59:05.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:05 smithi122 ceph-mon[30925]: pgmap v713: 97 pgs: 97 active+clean; 297 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-09-12T01:59:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:05 smithi151 ceph-mon[35229]: pgmap v713: 97 pgs: 97 active+clean; 297 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-09-12T01:59:06.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:05 smithi183 ceph-mon[35127]: pgmap v713: 97 pgs: 97 active+clean; 297 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-09-12T01:59:06.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1211510 9.91 MB/sec execute 564 sec latency 74.466 ms 2024-09-12T01:59:07.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1214117 9.91 MB/sec execute 565 sec latency 91.033 ms 2024-09-12T01:59:07.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:07 smithi122 ceph-mon[30925]: pgmap v714: 97 pgs: 97 active+clean; 299 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-09-12T01:59:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:07 smithi151 ceph-mon[35229]: pgmap v714: 97 pgs: 97 active+clean; 299 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-09-12T01:59:08.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:07 smithi183 ceph-mon[35127]: pgmap v714: 97 pgs: 97 active+clean; 299 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-09-12T01:59:08.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1216753 9.92 MB/sec execute 566 sec latency 15.898 ms 2024-09-12T01:59:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 -- 172.21.15.122:0/275018651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c100a40 msgr2=0x7f883c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 --2- 172.21.15.122:0/275018651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c100a40 0x7f883c100e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8838009900 tx=0x7f883802a410 comp rx=0 tx=0).stop 2024-09-12T01:59:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 -- 172.21.15.122:0/275018651 shutdown_connections 2024-09-12T01:59:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 --2- 172.21.15.122:0/275018651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f883c1025f0 0x7f883c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 --2- 172.21.15.122:0/275018651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c101c30 0x7f883c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 --2- 172.21.15.122:0/275018651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c100a40 0x7f883c100e40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 -- 172.21.15.122:0/275018651 >> 172.21.15.122:0/275018651 conn(0x7f883c0fc1e0 msgr2=0x7f883c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 -- 172.21.15.122:0/275018651 shutdown_connections 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 -- 172.21.15.122:0/275018651 wait complete. 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 Processor -- start 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 -- start start 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f883c100a40 0x7f883c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.214+0000 7f88433b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 0x7f883c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88433b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c1025f0 0x7f883c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88433b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10c6c0 con 0x7f883c101c30 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88433b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10c540 con 0x7f883c100a40 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88433b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10c840 con 0x7f883c1025f0 2024-09-12T01:59:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 0x7f883c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 0x7f883c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48824/0 (socket says 172.21.15.122:48824) 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8841153640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f883c100a40 0x7f883c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 -- 172.21.15.122:0/1284303669 learned_addr learned my addr 172.21.15.122:0/1284303669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8841954640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c1025f0 0x7f883c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 -- 172.21.15.122:0/1284303669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c1025f0 msgr2=0x7f883c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c1025f0 0x7f883c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 -- 172.21.15.122:0/1284303669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f883c100a40 msgr2=0x7f883c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f883c100a40 0x7f883c197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 -- 172.21.15.122:0/1284303669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c19fa40 con 0x7f883c101c30 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8841954640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c1025f0 0x7f883c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f8840952640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 0x7f883c198360 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f882c00d770 tx=0x7f882c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88327fc640 1 -- 172.21.15.122:0/1284303669 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c017070 con 0x7f883c101c30 2024-09-12T01:59:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88327fc640 1 -- 172.21.15.122:0/1284303669 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c005500 con 0x7f883c101c30 2024-09-12T01:59:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.215+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c19fd30 con 0x7f883c101c30 2024-09-12T01:59:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.216+0000 7f88327fc640 1 -- 172.21.15.122:0/1284303669 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c004e00 con 0x7f883c101c30 2024-09-12T01:59:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.216+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1a01d0 con 0x7f883c101c30 2024-09-12T01:59:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 Processor -- start 2024-09-12T01:59:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f88327fc640 1 -- 172.21.15.122:0/1284303669 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f882c004fa0 con 0x7f883c101c30 2024-09-12T01:59:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 -- start start 2024-09-12T01:59:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066c9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 0x55d066ca01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066ca2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d066bfa2f0 con 0x55d066c9b6f0 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d066ba2270 con 0x55d066c9adb0 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.218+0000 7f884622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d066bf62b0 con 0x55d066ca0780 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 0x55d066ca01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 0x55d066ca01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48834/0 (socket says 172.21.15.122:48834) 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 -- 172.21.15.122:0/3083608241 learned_addr learned my addr 172.21.15.122:0/3083608241 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841954640 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066c9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8840952640 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066ca2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f88327fc640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88280a2c30 0x7f88280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 -- 172.21.15.122:0/3083608241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 msgr2=0x55d066ca2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066ca2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 -- 172.21.15.122:0/3083608241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 msgr2=0x55d066c9b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066c9b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 -- 172.21.15.122:0/3083608241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d066ba1340 con 0x55d066c9b6f0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841954640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88280a2c30 0x7f88280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f88327fc640 1 -- 172.21.15.122:0/1284303669 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f882c0ceac0 con 0x7f883c101c30 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f8841153640 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 0x55d066ca01b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8838031610 tx=0x7f8838005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63edabd101a14c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f88277fe640 1 -- 172.21.15.122:0/3083608241 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838040070 con 0x55d066c9b6f0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f88277fe640 1 -- 172.21.15.122:0/3083608241 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88380318b0 con 0x55d066c9b6f0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.219+0000 7f88277fe640 1 -- 172.21.15.122:0/3083608241 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838035040 con 0x55d066c9b6f0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 -- 172.21.15.122:0/3083608241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 msgr2=0x55d066ca01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f8841954640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88280a2c30 0x7f88280a50f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8834005fd0 tx=0x7f8834004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 0x55d066ca01b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8838031610 tx=0x7f8838005c00 comp rx=0 tx=0).stop 2024-09-12T01:59:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 -- 172.21.15.122:0/3083608241 shutdown_connections 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066ca2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9b6f0 0x55d066ca01b0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 --2- 172.21.15.122:0/3083608241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066c9b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 -- 172.21.15.122:0/3083608241 >> 172.21.15.122:0/3083608241 conn(0x55d066c92010 msgr2=0x55d066c92410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 -- 172.21.15.122:0/3083608241 shutdown_connections 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.220+0000 7f884622b740 1 -- 172.21.15.122:0/3083608241 wait complete. 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 -- start start 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 Processor -- start 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066ca5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066ca0780 0x55d066cac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d066ca36f0 con 0x55d066c9adb0 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d066b83290 con 0x55d066ca0780 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d066ca3870 con 0x55d066c9b6f0 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8841153640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f884622b740 10 client.0 _ll_register_callbacks cb 0x55d066bb4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8841153640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48838/0 (socket says 172.21.15.122:48838) 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8841153640 1 -- 172.21.15.122:0/914316232 learned_addr learned my addr 172.21.15.122:0/914316232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8841954640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066ca0780 0x55d066cac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8840952640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066ca5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8841153640 1 -- 172.21.15.122:0/914316232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 msgr2=0x55d066ca5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.221+0000 7f8841153640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066ca5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8841153640 1 -- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066ca0780 msgr2=0x55d066cac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8841153640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066ca0780 0x55d066cac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8841153640 1 -- 172.21.15.122:0/914316232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06690d210 con 0x55d066c9adb0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8841954640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066ca0780 0x55d066cac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8841153640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca4f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8838031440 tx=0x7f8838031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d6afbc21ffcd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8824ff9640 1 -- 172.21.15.122:0/914316232 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883804b020 con 0x55d066c9adb0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8824ff9640 1 -- 172.21.15.122:0/914316232 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8838004450 con 0x55d066c9adb0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88380318b0 con 0x55d066c9adb0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8824ff9640 1 -- 172.21.15.122:0/914316232 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8838035070 con 0x55d066c9adb0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 10 client.17337 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0669d9620 con 0x55d066c9adb0 2024-09-12T01:59:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8824ff9640 1 -- 172.21.15.122:0/914316232 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88380318b0 con 0x55d066c9adb0 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f8824ff9640 1 -- 172.21.15.122:0/914316232 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f88380049f0 con 0x55d066c9adb0 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 10 client.17337 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 20 client.17337 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066cb07a0 0x55d066cb2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 4 client.17337 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.222+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d066c07a20 con 0x55d066cb07a0 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.223+0000 7f8840952640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066cb07a0 0x55d066cb2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.223+0000 7f8824ff9640 10 client.17337 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.223+0000 7f8840952640 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066cb07a0 0x55d066cb2be0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607a1697dcdbb97f server_cookie=d491cf4d0f3dbdca in_seq=0 out_seq=0 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.223+0000 7f8824ff9640 1 -- 172.21.15.122:0/914316232 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d066c07a20 con 0x55d066cb07a0 2024-09-12T01:59:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.223+0000 7f8824ff9640 10 client.17337 handle_command_reply: tid=0 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 client.17337 shutdown 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 msgr2=0x55d066ca4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca4f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8838031440 tx=0x7f8838031ad0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 shutdown_connections 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066cb07a0 0x55d066cb2be0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066ca0780 0x55d066cac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066ca5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f8824ff9640 0 client.17337 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 --2- 172.21.15.122:0/914316232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca4f20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 >> 172.21.15.122:0/914316232 conn(0x55d066c92010 msgr2=0x55d066c93820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 shutdown_connections 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 1 -- 172.21.15.122:0/914316232 wait complete. 2024-09-12T01:59:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.224+0000 7f884622b740 20 client.17337 trim_cache size 0 max 16384 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 Processor -- start 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 -- start start 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066ca54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066cac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 0x55d066c910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1a96e0 con 0x55d066ca0780 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1a6fb0 con 0x55d066c9adb0 2024-09-12T01:59:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f884622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1a4de0 con 0x55d066c9b6f0 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 0x55d066c910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 0x55d066c910b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48854/0 (socket says 172.21.15.122:48854) 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 -- 172.21.15.122:0/671954996 learned_addr learned my addr 172.21.15.122:0/671954996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841153640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066ca54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8840952640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066cac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 -- 172.21.15.122:0/671954996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 msgr2=0x55d066cac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066cac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 -- 172.21.15.122:0/671954996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 msgr2=0x55d066ca54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066ca54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841954640 1 -- 172.21.15.122:0/671954996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d066c9bbd0 con 0x55d066ca0780 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8841153640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066ca54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.239+0000 7f8840952640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066cac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.240+0000 7f8841954640 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 0x55d066c910b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f88340107e0 tx=0x7f8834018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569a9ee2a801760d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.240+0000 7f8826ffd640 1 -- 172.21.15.122:0/671954996 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883402c070 con 0x55d066ca0780 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.240+0000 7f8826ffd640 1 -- 172.21.15.122:0/671954996 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883401c960 con 0x55d066ca0780 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.240+0000 7f8826ffd640 1 -- 172.21.15.122:0/671954996 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834027aa0 con 0x55d066ca0780 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- 172.21.15.122:0/671954996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 msgr2=0x55d066c910b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 0x55d066c910b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f88340107e0 tx=0x7f8834018000 comp rx=0 tx=0).stop 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- 172.21.15.122:0/671954996 shutdown_connections 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066ca0780 0x55d066c910b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066c9b6f0 0x55d066cac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- 172.21.15.122:0/671954996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9adb0 0x55d066ca54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- 172.21.15.122:0/671954996 >> 172.21.15.122:0/671954996 conn(0x55d066c92010 msgr2=0x55d066c909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- 172.21.15.122:0/671954996 shutdown_connections 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- 172.21.15.122:0/671954996 wait complete. 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- start start 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 Processor -- start 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9b6f0 0x55d066caa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066cab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1a5470 con 0x55d066c9adb0 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1a68e0 con 0x55d066c9b6f0 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f883c1a5b40 con 0x55d066ca0780 2024-09-12T01:59:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f884622b740 10 client.0 _ll_register_callbacks cb 0x7f88340096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f8841153640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.241+0000 7f8841153640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48858/0 (socket says 172.21.15.122:48858) 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 -- 172.21.15.122:0/4265603244 learned_addr learned my addr 172.21.15.122:0/4265603244 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8840952640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9b6f0 0x55d066caa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841954640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066cab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 -- 172.21.15.122:0/4265603244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 msgr2=0x55d066cab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066cab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 -- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9b6f0 msgr2=0x55d066caa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9b6f0 0x55d066caa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 -- 172.21.15.122:0/4265603244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d066cab680 con 0x55d066c9adb0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8841153640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca9b70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8838004e10 tx=0x7f883802ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a726e30f378b302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8824ff9640 1 -- 172.21.15.122:0/4265603244 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883804b070 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8824ff9640 1 -- 172.21.15.122:0/4265603244 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f8838005280 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f883401c960 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8824ff9640 1 -- 172.21.15.122:0/4265603244 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8838040040 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f884622b740 10 client.17343 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d066d1b480 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.242+0000 7f8824ff9640 1 -- 172.21.15.122:0/4265603244 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883401c960 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f8824ff9640 1 -- 172.21.15.122:0/4265603244 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8838059070 con 0x55d066c9adb0 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f884622b740 10 client.17343 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f884622b740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f884622b740 20 client.17343 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f884622b740 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066d1bca0 0x55d066d1e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f884622b740 4 client.17343 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:59:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d066c01d70 con 0x55d066d1bca0 2024-09-12T01:59:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f8840952640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066d1bca0 0x55d066d1e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f8824ff9640 10 client.17343 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.243+0000 7f8840952640 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066d1bca0 0x55d066d1e0c0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5624a0142481e0bd server_cookie=fb20377b93b85193 in_seq=0 out_seq=0 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f8824ff9640 1 -- 172.21.15.122:0/4265603244 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d066c01d70 con 0x55d066d1bca0 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f8824ff9640 10 client.17343 handle_command_reply: tid=0 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 client.17343 shutdown 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 msgr2=0x55d066ca9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca9b70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8838004e10 tx=0x7f883802ab70 comp rx=0 tx=0).stop 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 shutdown_connections 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d066d1bca0 0x55d066d1e0c0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d066ca0780 0x55d066cab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f8824ff9640 0 client.17343 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d066c9b6f0 0x55d066caa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 --2- 172.21.15.122:0/4265603244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d066c9adb0 0x55d066ca9b70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 >> 172.21.15.122:0/4265603244 conn(0x55d066c92010 msgr2=0x55d066c911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 shutdown_connections 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 1 -- 172.21.15.122:0/4265603244 wait complete. 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.255+0000 7f884622b740 20 client.17343 trim_cache size 0 max 16384 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:59:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:59:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88280a2c30 msgr2=0x7f88280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88280a2c30 0x7f88280a50f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8834005fd0 tx=0x7f8834004a40 comp rx=0 tx=0).stop 2024-09-12T01:59:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 msgr2=0x7f883c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 0x7f883c198360 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f882c00d770 tx=0x7f882c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:59:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 shutdown_connections 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f88280a2c30 0x7f88280a50f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f883c1025f0 0x7f883c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f883c101c30 0x7f883c198360 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 --2- 172.21.15.122:0/1284303669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f883c100a40 0x7f883c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 >> 172.21.15.122:0/1284303669 conn(0x7f883c0fc1e0 msgr2=0x7f883c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 shutdown_connections 2024-09-12T01:59:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:09.256+0000 7f88433b7640 1 -- 172.21.15.122:0/1284303669 wait complete. 2024-09-12T01:59:09.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1219084 9.92 MB/sec execute 567 sec latency 10.051 ms 2024-09-12T01:59:09.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:09 smithi122 ceph-mon[30925]: pgmap v715: 97 pgs: 97 active+clean; 300 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-09-12T01:59:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:09 smithi151 ceph-mon[35229]: pgmap v715: 97 pgs: 97 active+clean; 300 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-09-12T01:59:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:09 smithi183 ceph-mon[35127]: pgmap v715: 97 pgs: 97 active+clean; 300 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-09-12T01:59:10.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1222032 9.94 MB/sec execute 568 sec latency 20.208 ms 2024-09-12T01:59:11.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1224676 9.94 MB/sec execute 569 sec latency 14.519 ms 2024-09-12T01:59:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:11 smithi151 ceph-mon[35229]: pgmap v716: 97 pgs: 97 active+clean; 303 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-12T01:59:12.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:11 smithi183 ceph-mon[35127]: pgmap v716: 97 pgs: 97 active+clean; 303 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-12T01:59:12.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:11 smithi122 ceph-mon[30925]: pgmap v716: 97 pgs: 97 active+clean; 303 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-12T01:59:12.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1226794 9.95 MB/sec execute 570 sec latency 12.273 ms 2024-09-12T01:59:13.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1229278 9.95 MB/sec execute 571 sec latency 11.073 ms 2024-09-12T01:59:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:13 smithi151 ceph-mon[35229]: pgmap v717: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-12T01:59:14.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:13 smithi183 ceph-mon[35127]: pgmap v717: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-12T01:59:14.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:13 smithi122 ceph-mon[30925]: pgmap v717: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 220 op/s 2024-09-12T01:59:14.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1231866 9.96 MB/sec execute 572 sec latency 15.374 ms 2024-09-12T01:59:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:14 smithi151 ceph-mon[35229]: pgmap v718: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-09-12T01:59:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:14 smithi183 ceph-mon[35127]: pgmap v718: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-09-12T01:59:15.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:14 smithi122 ceph-mon[30925]: pgmap v718: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2024-09-12T01:59:15.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1234107 9.96 MB/sec execute 573 sec latency 12.451 ms 2024-09-12T01:59:16.861 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1236613 9.97 MB/sec execute 574 sec latency 11.859 ms 2024-09-12T01:59:17.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1239612 9.98 MB/sec execute 575 sec latency 17.964 ms 2024-09-12T01:59:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:17 smithi122 ceph-mon[30925]: pgmap v719: 97 pgs: 97 active+clean; 309 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-09-12T01:59:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:17 smithi151 ceph-mon[35229]: pgmap v719: 97 pgs: 97 active+clean; 309 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-09-12T01:59:18.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:17 smithi183 ceph-mon[35127]: pgmap v719: 97 pgs: 97 active+clean; 309 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-09-12T01:59:18.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1242201 9.99 MB/sec execute 576 sec latency 18.593 ms 2024-09-12T01:59:19.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1244180 9.99 MB/sec execute 577 sec latency 14.898 ms 2024-09-12T01:59:19.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:19 smithi122 ceph-mon[30925]: pgmap v720: 97 pgs: 97 active+clean; 314 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-12T01:59:20.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:19 smithi151 ceph-mon[35229]: pgmap v720: 97 pgs: 97 active+clean; 314 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-12T01:59:20.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:19 smithi183 ceph-mon[35127]: pgmap v720: 97 pgs: 97 active+clean; 314 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-09-12T01:59:20.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1246854 10.00 MB/sec execute 578 sec latency 37.512 ms 2024-09-12T01:59:21.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1249596 10.00 MB/sec execute 579 sec latency 12.057 ms 2024-09-12T01:59:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:21 smithi122 ceph-mon[30925]: pgmap v721: 97 pgs: 97 active+clean; 315 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-12T01:59:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:21 smithi151 ceph-mon[35229]: pgmap v721: 97 pgs: 97 active+clean; 315 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-12T01:59:22.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:21 smithi183 ceph-mon[35127]: pgmap v721: 97 pgs: 97 active+clean; 315 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 238 op/s 2024-09-12T01:59:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 -- 172.21.15.122:0/4130025592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf41025f0 msgr2=0x7f3bf4108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/4130025592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf41025f0 0x7f3bf4108e80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009ca0 tx=0x7f3bec02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 -- 172.21.15.122:0/4130025592 shutdown_connections 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/4130025592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf41025f0 0x7f3bf4108e80 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/4130025592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bf4101c30 0x7f3bf41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/4130025592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf4100a40 0x7f3bf4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 -- 172.21.15.122:0/4130025592 >> 172.21.15.122:0/4130025592 conn(0x7f3bf40fc1e0 msgr2=0x7f3bf40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 -- 172.21.15.122:0/4130025592 shutdown_connections 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 -- 172.21.15.122:0/4130025592 wait complete. 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.347+0000 7f3bfa01e640 1 Processor -- start 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 -- start start 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bf4100a40 0x7f3bf406a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 0x7f3bf419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf41025f0 0x7f3bf419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410c6a0 con 0x7f3bf4101c30 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410c520 con 0x7f3bf4100a40 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410c820 con 0x7f3bf41025f0 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 0x7f3bf419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 0x7f3bf419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60060/0 (socket says 172.21.15.122:60060) 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/1729227497 learned_addr learned my addr 172.21.15.122:0/1729227497 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bf4100a40 0x7f3bf406a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf41025f0 0x7f3bf419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/1729227497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf41025f0 msgr2=0x7f3bf419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf41025f0 0x7f3bf419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/1729227497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bf4100a40 msgr2=0x7f3bf406a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bf4100a40 0x7f3bf406a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/1729227497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf419f950 con 0x7f3bf4101c30 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf41025f0 0x7f3bf419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 0x7f3bf419ca40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3be800e9f0 tx=0x7f3be800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf0ff9640 1 -- 172.21.15.122:0/1729227497 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be800cd80 con 0x7f3bf4101c30 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf0ff9640 1 -- 172.21.15.122:0/1729227497 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be8010910 con 0x7f3bf4101c30 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf419fc40 con 0x7f3bf4101c30 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.348+0000 7f3bf0ff9640 1 -- 172.21.15.122:0/1729227497 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8018990 con 0x7f3bf4101c30 2024-09-12T01:59:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.349+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41a0190 con 0x7f3bf4101c30 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf0ff9640 1 -- 172.21.15.122:0/1729227497 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3be8010ac0 con 0x7f3bf4101c30 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 Processor -- start 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 -- start start 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a58da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a5e370 0x55d138a60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d1389bc2f0 con 0x55d138a592e0 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d138964270 con 0x55d138a5e370 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1389b82b0 con 0x55d138a589a0 2024-09-12T01:59:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a58da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a5dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60064/0 (socket says 172.21.15.122:60064) 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf37fe640 1 -- 172.21.15.122:0/994415401 learned_addr learned my addr 172.21.15.122:0/994415401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf0ff9640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3bdc0a2c30 0x7f3bdc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.351+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a5e370 0x55d138a60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf37fe640 1 -- 172.21.15.122:0/994415401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 msgr2=0x55d138a58da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a58da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf37fe640 1 -- 172.21.15.122:0/994415401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a5e370 msgr2=0x55d138a60830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a5e370 0x55d138a60830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf37fe640 1 -- 172.21.15.122:0/994415401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d138963340 con 0x55d138a592e0 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3bdc0a2c30 0x7f3bdc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf0ff9640 1 -- 172.21.15.122:0/1729227497 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3be80cf380 con 0x7f3bf4101c30 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a5dda0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b7d0 tx=0x7f3be4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11501b847f3d1536 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bd9ffb640 1 -- 172.21.15.122:0/994415401 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4003dd0 con 0x55d138a592e0 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bd9ffb640 1 -- 172.21.15.122:0/994415401 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be4015070 con 0x55d138a592e0 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.352+0000 7f3bd9ffb640 1 -- 172.21.15.122:0/994415401 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be40117c0 con 0x55d138a592e0 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- 172.21.15.122:0/994415401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 msgr2=0x55d138a5dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3bdc0a2c30 0x7f3bdc0a50f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3bec031ae0 tx=0x7f3bec03e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a5dda0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b7d0 tx=0x7f3be4004610 comp rx=0 tx=0).stop 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- 172.21.15.122:0/994415401 shutdown_connections 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a5e370 0x55d138a60830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a5dda0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/994415401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- 172.21.15.122:0/994415401 >> 172.21.15.122:0/994415401 conn(0x55d138a54010 msgr2=0x55d138a54410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- 172.21.15.122:0/994415401 shutdown_connections 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- 172.21.15.122:0/994415401 wait complete. 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- start start 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 Processor -- start 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a62bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a5e370 0x55d138a6a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d138a61340 con 0x55d138a5e370 2024-09-12T01:59:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d138945290 con 0x55d138a592e0 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.353+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d138a614c0 con 0x55d138a589a0 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bfce8c740 10 client.0 _ll_register_callbacks cb 0x55d138976140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a5e370 0x55d138a6a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a62bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a62bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53038/0 (socket says 172.21.15.122:53038) 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf37fe640 1 -- 172.21.15.122:0/4154685585 learned_addr learned my addr 172.21.15.122:0/4154685585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/4154685585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 msgr2=0x55d138a62bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a62bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/4154685585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a5e370 msgr2=0x55d138a6a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a5e370 0x55d138a6a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/4154685585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1386cf210 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a5e370 0x55d138a6a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a63130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3be809b8e0 tx=0x7f3be809b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3de2492aaf15621 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/4154685585 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8086050 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/4154685585 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3be8018b30 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be4015070 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/4154685585 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3be80a4850 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bfce8c740 10 client.26513 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.354+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d13879b620 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/4154685585 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be4015070 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/4154685585 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3be8063030 con 0x55d138a592e0 2024-09-12T01:59:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bfce8c740 10 client.26513 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bfce8c740 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bfce8c740 20 client.26513 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138a6e8b0 0x55d138a70cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bfce8c740 4 client.26513 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1389c9a20 con 0x55d138a6e8b0 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138a6e8b0 0x55d138a70cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bd2ffd640 10 client.26513 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.355+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138a6e8b0 0x55d138a70cc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ed353e9ad42ae2 server_cookie=7eed1391018ca55a in_seq=0 out_seq=0 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/4154685585 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1389c9a20 con 0x55d138a6e8b0 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bd2ffd640 10 client.26513 handle_command_reply: tid=0 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 client.26513 shutdown 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 msgr2=0x55d138a63130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a63130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3be809b8e0 tx=0x7f3be809b600 comp rx=0 tx=0).stop 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 shutdown_connections 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138a6e8b0 0x55d138a70cc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a5e370 0x55d138a6a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a63130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/4154685585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a589a0 0x55d138a62bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bd2ffd640 0 client.26513 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.356+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 >> 172.21.15.122:0/4154685585 conn(0x55d138a54010 msgr2=0x55d138a55820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.357+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 shutdown_connections 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.357+0000 7f3bfce8c740 1 -- 172.21.15.122:0/4154685585 wait complete. 2024-09-12T01:59:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.357+0000 7f3bfce8c740 20 client.26513 trim_cache size 0 max 16384 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 Processor -- start 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 -- start start 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 0x55d138a55b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a6a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41ab850 con 0x55d138a589a0 2024-09-12T01:59:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41a9120 con 0x55d138a592e0 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41a6f50 con 0x55d138a5e370 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 0x55d138a55b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 0x55d138a55b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60088/0 (socket says 172.21.15.122:60088) 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 -- 172.21.15.122:0/1634936356 learned_addr learned my addr 172.21.15.122:0/1634936356 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a6a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 -- 172.21.15.122:0/1634936356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 msgr2=0x55d138a755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 -- 172.21.15.122:0/1634936356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 msgr2=0x55d138a6a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a6a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.372+0000 7f3bf37fe640 1 -- 172.21.15.122:0/1634936356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d138a53e30 con 0x55d138a589a0 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 0x55d138a55b10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed766b163d0afc12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bd97fa640 1 -- 172.21.15.122:0/1634936356 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4005040 con 0x55d138a589a0 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bd97fa640 1 -- 172.21.15.122:0/1634936356 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be402d070 con 0x55d138a589a0 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bd97fa640 1 -- 172.21.15.122:0/1634936356 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004680 con 0x55d138a589a0 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 -- 172.21.15.122:0/1634936356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 msgr2=0x55d138a55b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 0x55d138a55b10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be4003d60 comp rx=0 tx=0).stop 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 -- 172.21.15.122:0/1634936356 shutdown_connections 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a592e0 0x55d138a6a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/1634936356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a589a0 0x55d138a55b10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.373+0000 7f3bfce8c740 1 -- 172.21.15.122:0/1634936356 >> 172.21.15.122:0/1634936356 conn(0x55d138a54010 msgr2=0x55d138a543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 -- 172.21.15.122:0/1634936356 shutdown_connections 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 -- 172.21.15.122:0/1634936356 wait complete. 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 -- start start 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 Processor -- start 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a589a0 0x55d138a67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a68180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41a75e0 con 0x55d138a592e0 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41a8a50 con 0x55d138a589a0 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41a7cb0 con 0x55d138a5e370 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bfce8c740 10 client.0 _ll_register_callbacks cb 0x7f3be8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bf2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a68180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bf2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a68180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60098/0 (socket says 172.21.15.122:60098) 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/3719645707 learned_addr learned my addr 172.21.15.122:0/3719645707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.374+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a589a0 0x55d138a67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf3fff640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/3719645707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 msgr2=0x55d138a69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a69140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a589a0 msgr2=0x55d138a67c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a589a0 0x55d138a67c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf2ffd640 1 -- 172.21.15.122:0/3719645707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d138a69750 con 0x55d138a592e0 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bf2ffd640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a68180 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3be8098560 tx=0x7f3be809a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3611283269c255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/3719645707 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8086040 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/3719645707 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f3be809b2d0 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be402d070 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/3719645707 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3be809b9b0 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 10 client.17361 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d138ad99a0 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/3719645707 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be402d070 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/3719645707 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3be8063040 con 0x55d138a592e0 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 10 client.17361 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 20 client.17361 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.375+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138ada320 0x55d138adc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.376+0000 7f3bfce8c740 4 client.17361 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:59:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.376+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1389c3d70 con 0x55d138ada320 2024-09-12T01:59:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.376+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138ada320 0x55d138adc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.376+0000 7f3bd2ffd640 10 client.17361 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.376+0000 7f3bf37fe640 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138ada320 0x55d138adc740 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145cb757549f1c55 server_cookie=8e75230a1ecc9c8f in_seq=0 out_seq=0 2024-09-12T01:59:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.388+0000 7f3bd2ffd640 1 -- 172.21.15.122:0/3719645707 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1389c3d70 con 0x55d138ada320 2024-09-12T01:59:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.388+0000 7f3bd2ffd640 10 client.17361 handle_command_reply: tid=0 2024-09-12T01:59:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.388+0000 7f3bfce8c740 1 client.17361 shutdown 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 msgr2=0x55d138a68180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a68180 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3be8098560 tx=0x7f3be809a0f0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 shutdown_connections 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d138ada320 0x55d138adc740 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d138a5e370 0x55d138a69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bd2ffd640 0 client.17361 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d138a592e0 0x55d138a68180 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 --2- 172.21.15.122:0/3719645707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d138a589a0 0x55d138a67c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 >> 172.21.15.122:0/3719645707 conn(0x55d138a54010 msgr2=0x55d138a54c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 shutdown_connections 2024-09-12T01:59:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 1 -- 172.21.15.122:0/3719645707 wait complete. 2024-09-12T01:59:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.389+0000 7f3bfce8c740 20 client.17361 trim_cache size 0 max 16384 2024-09-12T01:59:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:59:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:59:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:59:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:59:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3bdc0a2c30 msgr2=0x7f3bdc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3bdc0a2c30 0x7f3bdc0a50f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3bec031ae0 tx=0x7f3bec03e040 comp rx=0 tx=0).stop 2024-09-12T01:59:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 msgr2=0x7f3bf419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:22.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 0x7f3bf419ca40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3be800e9f0 tx=0x7f3be800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 shutdown_connections 2024-09-12T01:59:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3bdc0a2c30 0x7f3bdc0a50f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf41025f0 0x7f3bf419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4101c30 0x7f3bf419ca40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 --2- 172.21.15.122:0/1729227497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bf4100a40 0x7f3bf406a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 >> 172.21.15.122:0/1729227497 conn(0x7f3bf40fc1e0 msgr2=0x7f3bf4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 shutdown_connections 2024-09-12T01:59:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:22.390+0000 7f3bfa01e640 1 -- 172.21.15.122:0/1729227497 wait complete. 2024-09-12T01:59:22.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1252533 10.01 MB/sec execute 580 sec latency 13.200 ms 2024-09-12T01:59:23.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1254723 10.02 MB/sec execute 581 sec latency 10.562 ms 2024-09-12T01:59:23.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:23 smithi122 ceph-mon[30925]: pgmap v722: 97 pgs: 97 active+clean; 313 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-09-12T01:59:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:23 smithi151 ceph-mon[35229]: pgmap v722: 97 pgs: 97 active+clean; 313 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-09-12T01:59:24.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:23 smithi183 ceph-mon[35127]: pgmap v722: 97 pgs: 97 active+clean; 313 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-09-12T01:59:24.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1257633 10.03 MB/sec execute 582 sec latency 9.204 ms 2024-09-12T01:59:25.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1260187 10.03 MB/sec execute 583 sec latency 99.759 ms 2024-09-12T01:59:25.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:25 smithi122 ceph-mon[30925]: pgmap v723: 97 pgs: 97 active+clean; 295 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2024-09-12T01:59:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:25 smithi151 ceph-mon[35229]: pgmap v723: 97 pgs: 97 active+clean; 295 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2024-09-12T01:59:26.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:25 smithi183 ceph-mon[35127]: pgmap v723: 97 pgs: 97 active+clean; 295 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2024-09-12T01:59:26.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1262364 10.04 MB/sec execute 584 sec latency 25.911 ms 2024-09-12T01:59:27.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1264728 10.04 MB/sec execute 585 sec latency 59.509 ms 2024-09-12T01:59:27.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:27 smithi122 ceph-mon[30925]: pgmap v724: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-09-12T01:59:28.004 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:27 smithi183 ceph-mon[35127]: pgmap v724: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-09-12T01:59:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:27 smithi151 ceph-mon[35229]: pgmap v724: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-09-12T01:59:28.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1267471 10.05 MB/sec execute 586 sec latency 36.154 ms 2024-09-12T01:59:29.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1269740 10.05 MB/sec execute 587 sec latency 11.037 ms 2024-09-12T01:59:29.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:29 smithi122 ceph-mon[30925]: pgmap v725: 97 pgs: 97 active+clean; 301 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-12T01:59:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:29 smithi151 ceph-mon[35229]: pgmap v725: 97 pgs: 97 active+clean; 301 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-12T01:59:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:29 smithi183 ceph-mon[35127]: pgmap v725: 97 pgs: 97 active+clean; 301 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-12T01:59:30.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1271807 10.06 MB/sec execute 588 sec latency 55.255 ms 2024-09-12T01:59:31.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1274575 10.06 MB/sec execute 589 sec latency 11.553 ms 2024-09-12T01:59:31.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:31 smithi122 ceph-mon[30925]: pgmap v726: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-09-12T01:59:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:31 smithi151 ceph-mon[35229]: pgmap v726: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-09-12T01:59:32.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:31 smithi183 ceph-mon[35127]: pgmap v726: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-09-12T01:59:32.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1277142 10.06 MB/sec execute 590 sec latency 11.586 ms 2024-09-12T01:59:33.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1279203 10.07 MB/sec execute 591 sec latency 37.844 ms 2024-09-12T01:59:33.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:33 smithi122 ceph-mon[30925]: pgmap v727: 97 pgs: 97 active+clean; 304 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-09-12T01:59:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:33 smithi151 ceph-mon[35229]: pgmap v727: 97 pgs: 97 active+clean; 304 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-09-12T01:59:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:33 smithi183 ceph-mon[35127]: pgmap v727: 97 pgs: 97 active+clean; 304 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-09-12T01:59:34.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1282281 10.08 MB/sec execute 592 sec latency 8.772 ms 2024-09-12T01:59:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/4085828546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb680ffe30 msgr2=0x7fbb68100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/4085828546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb680ffe30 0x7fbb68100230 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009900 tx=0x7fbb6402a410 comp rx=0 tx=0).stop 2024-09-12T01:59:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/4085828546 shutdown_connections 2024-09-12T01:59:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/4085828546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb68103860 0x7fbb6810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/4085828546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 0x7fbb68103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/4085828546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb680ffe30 0x7fbb68100230 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/4085828546 >> 172.21.15.122:0/4085828546 conn(0x7fbb680fb970 msgr2=0x7fbb680fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/4085828546 shutdown_connections 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.491+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/4085828546 wait complete. 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 Processor -- start 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 -- start start 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb680ffe30 0x7fbb68197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 0x7fbb68198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 0x7fbb6819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6810d800 con 0x7fbb68103860 2024-09-12T01:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6810d680 con 0x7fbb680ffe30 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6fd9f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6810d980 con 0x7fbb68102ea0 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6e33c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 0x7fbb6819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb680ffe30 0x7fbb68197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6e33c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 0x7fbb6819f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37912/0 (socket says 172.21.15.122:37912) 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6e33c640 1 -- 172.21.15.122:0/811621941 learned_addr learned my addr 172.21.15.122:0/811621941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 0x7fbb68198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 -- 172.21.15.122:0/811621941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb680ffe30 msgr2=0x7fbb68197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb680ffe30 0x7fbb68197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 -- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 msgr2=0x7fbb6819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 0x7fbb6819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 -- 172.21.15.122:0/811621941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6819fa50 con 0x7fbb68102ea0 2024-09-12T01:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 0x7fbb6819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb680ffe30 0x7fbb68197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.492+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 0x7fbb68198370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e9f0 tx=0x7fbb5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.493+0000 7fbb5effd640 1 -- 172.21.15.122:0/811621941 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800ccc0 con 0x7fbb68102ea0 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.493+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6819fd40 con 0x7fbb68102ea0 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.493+0000 7fbb5effd640 1 -- 172.21.15.122:0/811621941 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb58005020 con 0x7fbb68102ea0 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.493+0000 7fbb5effd640 1 -- 172.21.15.122:0/811621941 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580049e0 con 0x7fbb68102ea0 2024-09-12T01:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.493+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb681a01e0 con 0x7fbb68102ea0 2024-09-12T01:59:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb5effd640 1 -- 172.21.15.122:0/811621941 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbb58005560 con 0x7fbb68102ea0 2024-09-12T01:59:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 Processor -- start 2024-09-12T01:59:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 -- start start 2024-09-12T01:59:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd419a0 0x55778fd41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 0x55778fd46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd47370 0x55778fd49830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55778fca52f0 con 0x55778fd422e0 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55778fc4d270 con 0x55778fd47370 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb72c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55778fca12b0 con 0x55778fd419a0 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 0x55778fd46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 0x55778fd46da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37924/0 (socket says 172.21.15.122:37924) 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb6db3b640 1 -- 172.21.15.122:0/358486293 learned_addr learned my addr 172.21.15.122:0/358486293 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd47370 0x55778fd49830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.496+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd419a0 0x55778fd41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6db3b640 1 -- 172.21.15.122:0/358486293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd419a0 msgr2=0x55778fd41da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd419a0 0x55778fd41da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6db3b640 1 -- 172.21.15.122:0/358486293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd47370 msgr2=0x55778fd49830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd47370 0x55778fd49830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6db3b640 1 -- 172.21.15.122:0/358486293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778fc4c340 con 0x55778fd422e0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb5effd640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb540a2c30 0x7fbb540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb540a2c30 0x7fbb540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 0x55778fd46da0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009900 tx=0x7fbb64031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb61dd593ec41a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb5effd640 1 -- 172.21.15.122:0/811621941 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fbb580cef90 con 0x7fbb68102ea0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb53fff640 1 -- 172.21.15.122:0/358486293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64040070 con 0x55778fd422e0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb53fff640 1 -- 172.21.15.122:0/358486293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64005040 con 0x55778fd422e0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.497+0000 7fbb53fff640 1 -- 172.21.15.122:0/358486293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64035040 con 0x55778fd422e0 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 -- 172.21.15.122:0/358486293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 msgr2=0x55778fd46da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 0x55778fd46da0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009900 tx=0x7fbb64031730 comp rx=0 tx=0).stop 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb540a2c30 0x7fbb540a50f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbb60010860 tx=0x7fbb600075e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 -- 172.21.15.122:0/358486293 shutdown_connections 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd47370 0x55778fd49830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd422e0 0x55778fd46da0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 --2- 172.21.15.122:0/358486293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd419a0 0x55778fd41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 -- 172.21.15.122:0/358486293 >> 172.21.15.122:0/358486293 conn(0x55778fd3d010 msgr2=0x55778fd3d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 -- 172.21.15.122:0/358486293 shutdown_connections 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 -- 172.21.15.122:0/358486293 wait complete. 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 -- start start 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.498+0000 7fbb72c19740 1 Processor -- start 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd4c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55778fd4a2e0 con 0x55778fd419a0 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55778fc2e290 con 0x55778fd422e0 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55778fd4a460 con 0x55778fd47370 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 10 client.0 _ll_register_callbacks cb 0x55778fc5f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd4bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37928/0 (socket says 172.21.15.122:37928) 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 -- 172.21.15.122:0/266975575 learned_addr learned my addr 172.21.15.122:0/266975575 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd4c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 -- 172.21.15.122:0/266975575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 msgr2=0x55778fd530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 -- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 msgr2=0x55778fd4c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd4c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 -- 172.21.15.122:0/266975575 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778f9b8210 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd4bb60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbb6402a930 tx=0x7fbb64005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416d22f3c26ffe98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb517fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb517fa640 1 -- 172.21.15.122:0/266975575 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64040070 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb517fa640 1 -- 172.21.15.122:0/266975575 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fbb64004ae0 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.499+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb64005040 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb517fa640 1 -- 172.21.15.122:0/266975575 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbb64035040 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 10 client.17373 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55778fa84620 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb517fa640 1 -- 172.21.15.122:0/266975575 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64005040 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb517fa640 1 -- 172.21.15.122:0/266975575 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbb64051400 con 0x55778fd419a0 2024-09-12T01:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 10 client.17373 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 20 client.17373 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fd573e0 0x55778fd597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 4 client.17373 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55778fcb2a20 con 0x55778fd573e0 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.500+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fd573e0 0x55778fd597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fd573e0 0x55778fd597d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682933e53178e248 server_cookie=bf228e8deef2b04b in_seq=0 out_seq=0 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb517fa640 10 client.17373 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb517fa640 1 -- 172.21.15.122:0/266975575 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55778fcb2a20 con 0x55778fd573e0 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb517fa640 10 client.17373 handle_command_reply: tid=0 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 client.17373 shutdown 2024-09-12T01:59:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 msgr2=0x55778fd4bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd4bb60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbb6402a930 tx=0x7fbb64005270 comp rx=0 tx=0).stop 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 shutdown_connections 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fd573e0 0x55778fd597d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd4c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb517fa640 0 client.17373 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 --2- 172.21.15.122:0/266975575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd4bb60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.501+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 >> 172.21.15.122:0/266975575 conn(0x55778fd3d010 msgr2=0x55778fd3e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.502+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 shutdown_connections 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.502+0000 7fbb72c19740 1 -- 172.21.15.122:0/266975575 wait complete. 2024-09-12T01:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.502+0000 7fbb72c19740 20 client.17373 trim_cache size 0 max 16384 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 Processor -- start 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 -- start start 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd419a0 0x55778fd3d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd422e0 0x55778fd530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 0x55778fd3d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681ab890 con 0x55778fd47370 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681a9160 con 0x55778fd419a0 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb72c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681a6f90 con 0x55778fd422e0 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 0x55778fd3d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 0x55778fd3d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37942/0 (socket says 172.21.15.122:37942) 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 -- 172.21.15.122:0/2264351250 learned_addr learned my addr 172.21.15.122:0/2264351250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd422e0 0x55778fd530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd419a0 0x55778fd3d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 -- 172.21.15.122:0/2264351250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd422e0 msgr2=0x55778fd530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd422e0 0x55778fd530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 -- 172.21.15.122:0/2264351250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd419a0 msgr2=0x55778fd3d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd419a0 0x55778fd3d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6e33c640 1 -- 172.21.15.122:0/2264351250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778fd59b40 con 0x55778fd47370 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.516+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd422e0 0x55778fd530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.517+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 0x55778fd3d930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000fd70 tx=0x7fbb60018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543c04878fac02ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.517+0000 7fbb537fe640 1 -- 172.21.15.122:0/2264351250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000fda0 con 0x55778fd47370 2024-09-12T01:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.517+0000 7fbb537fe640 1 -- 172.21.15.122:0/2264351250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60003e50 con 0x55778fd47370 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.517+0000 7fbb537fe640 1 -- 172.21.15.122:0/2264351250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60027680 con 0x55778fd47370 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.517+0000 7fbb72c19740 1 -- 172.21.15.122:0/2264351250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 msgr2=0x55778fd3d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.517+0000 7fbb72c19740 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 0x55778fd3d930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000fd70 tx=0x7fbb60018000 comp rx=0 tx=0).stop 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- 172.21.15.122:0/2264351250 shutdown_connections 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd47370 0x55778fd3d930 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd422e0 0x55778fd530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 --2- 172.21.15.122:0/2264351250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd419a0 0x55778fd3d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- 172.21.15.122:0/2264351250 >> 172.21.15.122:0/2264351250 conn(0x55778fd3d010 msgr2=0x55778fd4b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- 172.21.15.122:0/2264351250 shutdown_connections 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- 172.21.15.122:0/2264351250 wait complete. 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- start start 2024-09-12T01:59:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 Processor -- start 2024-09-12T01:59:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd50750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd51ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681a7620 con 0x55778fd419a0 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681a8a90 con 0x55778fd422e0 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681a7cf0 con 0x55778fd47370 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb72c19740 10 client.0 _ll_register_callbacks cb 0x7fbb580d28e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.518+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd50750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd50750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37950/0 (socket says 172.21.15.122:37950) 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 -- 172.21.15.122:0/1429407573 learned_addr learned my addr 172.21.15.122:0/1429407573 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd50c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6e33c640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd51ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 -- 172.21.15.122:0/1429407573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 msgr2=0x55778fd51ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd51ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 -- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 msgr2=0x55778fd50c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd50c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 -- 172.21.15.122:0/1429407573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778fd522b0 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb6db3b640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd50750 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbb64051a10 tx=0x7fbb6402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd75598ed0c3e675 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb517fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb517fa640 1 -- 172.21.15.122:0/1429407573 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64040040 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb517fa640 1 -- 172.21.15.122:0/1429407573 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fbb64004030 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb60003e50 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb517fa640 1 -- 172.21.15.122:0/1429407573 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbb640052e0 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb72c19740 10 client.17379 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.519+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55778fdc1fd0 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb517fa640 1 -- 172.21.15.122:0/1429407573 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60003e50 con 0x55778fd419a0 2024-09-12T01:59:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb517fa640 1 -- 172.21.15.122:0/1429407573 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbb6405b020 con 0x55778fd419a0 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb72c19740 10 client.17379 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb72c19740 10 client.17379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb72c19740 20 client.17379 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb72c19740 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fdc2960 0x55778fdc4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb72c19740 4 client.17379 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55778fcacd70 con 0x55778fdc2960 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fdc2960 0x55778fdc4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb6d33a640 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fdc2960 0x55778fdc4d80 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d6059cd2bd61ec server_cookie=f6df222b8afff270 in_seq=0 out_seq=0 2024-09-12T01:59:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.520+0000 7fbb517fa640 10 client.17379 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.534+0000 7fbb517fa640 1 -- 172.21.15.122:0/1429407573 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55778fcacd70 con 0x55778fdc2960 2024-09-12T01:59:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.534+0000 7fbb517fa640 10 client.17379 handle_command_reply: tid=0 2024-09-12T01:59:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.534+0000 7fbb72c19740 1 client.17379 shutdown 2024-09-12T01:59:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 msgr2=0x55778fd50750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd50750 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbb64051a10 tx=0x7fbb6402a410 comp rx=0 tx=0).stop 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 shutdown_connections 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55778fdc2960 0x55778fdc4d80 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55778fd47370 0x55778fd51ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55778fd422e0 0x55778fd50c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 --2- 172.21.15.122:0/1429407573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55778fd419a0 0x55778fd50750 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 >> 172.21.15.122:0/1429407573 conn(0x55778fd3d010 msgr2=0x55778fd3b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb517fa640 0 client.17379 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 shutdown_connections 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 1 -- 172.21.15.122:0/1429407573 wait complete. 2024-09-12T01:59:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.535+0000 7fbb72c19740 20 client.17379 trim_cache size 0 max 16384 2024-09-12T01:59:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:59:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:59:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:59:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb540a2c30 msgr2=0x7fbb540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb540a2c30 0x7fbb540a50f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbb60010860 tx=0x7fbb600075e0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 msgr2=0x7fbb68198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 0x7fbb68198370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e9f0 tx=0x7fbb5800eeb0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 shutdown_connections 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbb540a2c30 0x7fbb540a50f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb68103860 0x7fbb6819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb68102ea0 0x7fbb68198370 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 --2- 172.21.15.122:0/811621941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb680ffe30 0x7fbb68197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 >> 172.21.15.122:0/811621941 conn(0x7fbb680fb970 msgr2=0x7fbb68106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 shutdown_connections 2024-09-12T01:59:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:35.536+0000 7fbb6fd9f640 1 -- 172.21.15.122:0/811621941 wait complete. 2024-09-12T01:59:35.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1284091 10.08 MB/sec execute 593 sec latency 154.417 ms 2024-09-12T01:59:35.988 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:35 smithi122 ceph-mon[30925]: pgmap v728: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-09-12T01:59:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:35 smithi151 ceph-mon[35229]: pgmap v728: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-09-12T01:59:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:35 smithi183 ceph-mon[35127]: pgmap v728: 97 pgs: 97 active+clean; 303 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-09-12T01:59:36.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1286362 10.08 MB/sec execute 594 sec latency 11.473 ms 2024-09-12T01:59:37.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1289379 10.10 MB/sec execute 595 sec latency 15.703 ms 2024-09-12T01:59:37.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:37 smithi122 ceph-mon[30925]: pgmap v729: 97 pgs: 97 active+clean; 305 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-12T01:59:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:37 smithi151 ceph-mon[35229]: pgmap v729: 97 pgs: 97 active+clean; 305 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-12T01:59:38.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:37 smithi183 ceph-mon[35127]: pgmap v729: 97 pgs: 97 active+clean; 305 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-12T01:59:38.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1292099 10.10 MB/sec execute 596 sec latency 19.065 ms 2024-09-12T01:59:39.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1294681 10.11 MB/sec execute 597 sec latency 13.313 ms 2024-09-12T01:59:40.108 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:59:40.108 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T01:59:40.109 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:59:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:39 smithi151 ceph-mon[35229]: pgmap v730: 97 pgs: 97 active+clean; 310 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 345 op/s 2024-09-12T01:59:40.132 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:39 smithi122 ceph-mon[30925]: pgmap v730: 97 pgs: 97 active+clean; 310 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 345 op/s 2024-09-12T01:59:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:39 smithi183 ceph-mon[35127]: pgmap v730: 97 pgs: 97 active+clean; 310 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 345 op/s 2024-09-12T01:59:40.370 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:40.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1296796 10.11 MB/sec execute 598 sec latency 78.062 ms 2024-09-12T01:59:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 -- 172.21.15.122:0/908161307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418102780 msgr2=0x7fb4181067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/908161307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418102780 0x7fb4181067d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb408009900 tx=0x7fb40802a410 comp rx=0 tx=0).stop 2024-09-12T01:59:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 -- 172.21.15.122:0/908161307 shutdown_connections 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/908161307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb418106f00 0x7fb41810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/908161307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418102780 0x7fb4181067d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/908161307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418101dd0 0x7fb4181021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.873+0000 7fb41f5e9640 1 -- 172.21.15.122:0/908161307 >> 172.21.15.122:0/908161307 conn(0x7fb4180fd8b0 msgr2=0x7fb4180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 -- 172.21.15.122:0/908161307 shutdown_connections 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 -- 172.21.15.122:0/908161307 wait complete. 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 Processor -- start 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 -- start start 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 0x7fb418198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418102780 0x7fb4181989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb418106f00 0x7fb41819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb418111f60 con 0x7fb418101dd0 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb418111de0 con 0x7fb418106f00 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41f5e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181120e0 con 0x7fb418102780 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41d385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 0x7fb418198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41d385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 0x7fb418198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57474/0 (socket says 172.21.15.122:57474) 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41d385640 1 -- 172.21.15.122:0/3011465661 learned_addr learned my addr 172.21.15.122:0/3011465661 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.874+0000 7fb41db86640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb418106f00 0x7fb41819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41cb84640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418102780 0x7fb4181989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41d385640 1 -- 172.21.15.122:0/3011465661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418102780 msgr2=0x7fb4181989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41d385640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418102780 0x7fb4181989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41d385640 1 -- 172.21.15.122:0/3011465661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb418106f00 msgr2=0x7fb41819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41d385640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb418106f00 0x7fb41819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41d385640 1 -- 172.21.15.122:0/3011465661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819d410 con 0x7fb418101dd0 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41d385640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 0x7fb418198460 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb41400e9f0 tx=0x7fb41400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400ccc0 con 0x7fb418101dd0 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819d700 con 0x7fb418101dd0 2024-09-12T01:59:40.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41400ce60 con 0x7fb418101dd0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414010690 con 0x7fb418101dd0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.875+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a4fe0 con 0x7fb418101dd0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.876+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4181021b0 con 0x7fb418101dd0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.877+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb4140046c0 con 0x7fb418101dd0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.878+0000 7fb40e7fc640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4040a2b80 0x7fb4040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.878+0000 7fb41cb84640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4040a2b80 0x7fb4040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.878+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fb4140cf5f0 con 0x7fb418101dd0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.878+0000 7fb41cb84640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4040a2b80 0x7fb4040a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb408009230 tx=0x7fb4080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:40.881+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4140541b0 con 0x7fb418101dd0 2024-09-12T01:59:41.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.106+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4181997c0 con 0x7fb418101dd0 2024-09-12T01:59:41.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.107+0000 7fb40e7fc640 1 -- 172.21.15.122:0/3011465661 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7fb41409c550 con 0x7fb418101dd0 2024-09-12T01:59:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T01:59:41.107 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T01:59:41.107 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T01:59:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4040a2b80 msgr2=0x7fb4040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4040a2b80 0x7fb4040a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb408009230 tx=0x7fb4080058d0 comp rx=0 tx=0).stop 2024-09-12T01:59:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 msgr2=0x7fb418198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 0x7fb418198460 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb41400e9f0 tx=0x7fb41400edf0 comp rx=0 tx=0).stop 2024-09-12T01:59:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 shutdown_connections 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb4040a2b80 0x7fb4040a5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb418106f00 0x7fb41819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb418102780 0x7fb4181989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 --2- 172.21.15.122:0/3011465661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418101dd0 0x7fb418198460 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 >> 172.21.15.122:0/3011465661 conn(0x7fb4180fd8b0 msgr2=0x7fb4180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 shutdown_connections 2024-09-12T01:59:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:41.109+0000 7fb41f5e9640 1 -- 172.21.15.122:0/3011465661 wait complete. 2024-09-12T01:59:41.862 INFO:tasks.workunit.client.0.smithi122.stdout: 1 1299758 10.12 MB/sec execute 599 sec latency 37.581 ms 2024-09-12T01:59:41.871 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:41 smithi122 ceph-mon[30925]: pgmap v731: 97 pgs: 97 active+clean; 311 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-09-12T01:59:41.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:41 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3011465661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:41.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T01:59:41.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T01:59:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:41 smithi151 ceph-mon[35229]: pgmap v731: 97 pgs: 97 active+clean; 311 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-09-12T01:59:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:41 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3011465661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:42.201 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:59:42.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:41 smithi183 ceph-mon[35127]: pgmap v731: 97 pgs: 97 active+clean; 311 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-09-12T01:59:42.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:41 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3011465661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:59:42.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 -- 172.21.15.122:0/3117606067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 msgr2=0x7f98601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 --2- 172.21.15.122:0/3117606067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f98601029b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9850009900 tx=0x7f985002a480 comp rx=0 tx=0).stop 2024-09-12T01:59:42.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 -- 172.21.15.122:0/3117606067 shutdown_connections 2024-09-12T01:59:42.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 --2- 172.21.15.122:0/3117606067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860102ef0 0x7f986010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 --2- 172.21.15.122:0/3117606067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f98601029b0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 --2- 172.21.15.122:0/3117606067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860100e20 0x7f9860101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.703+0000 7f98673a2640 1 -- 172.21.15.122:0/3117606067 >> 172.21.15.122:0/3117606067 conn(0x7f98600fc7d0 msgr2=0x7f98600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:42.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 -- 172.21.15.122:0/3117606067 shutdown_connections 2024-09-12T01:59:42.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 -- 172.21.15.122:0/3117606067 wait complete. 2024-09-12T01:59:42.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 Processor -- start 2024-09-12T01:59:42.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 -- start start 2024-09-12T01:59:42.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 0x7f9860197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f9860197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860102ef0 0x7f986019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9860110f50 con 0x7f9860102570 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9860110dd0 con 0x7f9860100e20 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.704+0000 7f98673a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98601110d0 con 0x7f9860102ef0 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 0x7f9860197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986493d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f9860197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 0x7f9860197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:59486/0 (socket says 172.21.15.122:59486) 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 -- 172.21.15.122:0/2478734800 learned_addr learned my addr 172.21.15.122:0/2478734800 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986593f640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860102ef0 0x7f986019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 -- 172.21.15.122:0/2478734800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860102ef0 msgr2=0x7f986019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860102ef0 0x7f986019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 -- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 msgr2=0x7f9860197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f9860197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 -- 172.21.15.122:0/2478734800 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986019c3e0 con 0x7f9860100e20 2024-09-12T01:59:42.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986493d640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f9860197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:42.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f986513e640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 0x7f9860197430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f985c00d950 tx=0x7f985c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:42.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f98567fc640 1 -- 172.21.15.122:0/2478734800 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c017030 con 0x7f9860100e20 2024-09-12T01:59:42.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986019c6d0 con 0x7f9860100e20 2024-09-12T01:59:42.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f98567fc640 1 -- 172.21.15.122:0/2478734800 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c004020 con 0x7f9860100e20 2024-09-12T01:59:42.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.705+0000 7f98567fc640 1 -- 172.21.15.122:0/2478734800 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0047a0 con 0x7f9860100e20 2024-09-12T01:59:42.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.706+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98601a3fb0 con 0x7f9860100e20 2024-09-12T01:59:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.708+0000 7f98567fc640 1 -- 172.21.15.122:0/2478734800 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f985c0041d0 con 0x7f9860100e20 2024-09-12T01:59:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.708+0000 7f986977b740 1 Processor -- start 2024-09-12T01:59:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.708+0000 7f986977b740 1 -- start start 2024-09-12T01:59:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.708+0000 7f986977b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce02b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.708+0000 7f986977b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 0x55f4dce0d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986977b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce0d940 0x55f4dce0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986977b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f4dcd682a0 con 0x55f4dce02760 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986977b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f4dcd0fb10 con 0x55f4dce03080 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986977b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f4dcd64260 con 0x55f4dce0d940 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f98567fc640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f984c0a2b80 0x7f984c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 0x55f4dce0d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 0x55f4dce0d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:59500/0 (socket says 172.21.15.122:59500) 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986593f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce02b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 -- 172.21.15.122:0/3451008235 learned_addr learned my addr 172.21.15.122:0/3451008235 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986493d640 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce0d940 0x55f4dce0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986593f640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f984c0a2b80 0x7f984c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f98567fc640 1 -- 172.21.15.122:0/2478734800 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f985c0cea20 con 0x7f9860100e20 2024-09-12T01:59:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 -- 172.21.15.122:0/3451008235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce0d940 msgr2=0x55f4dce0fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce0d940 0x55f4dce0fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 -- 172.21.15.122:0/3451008235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 msgr2=0x55f4dce02b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce02b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 -- 172.21.15.122:0/3451008235 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dcad9b90 con 0x55f4dce03080 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986593f640 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce02b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986493d640 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce0d940 0x55f4dce0fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f986513e640 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 0x55f4dce0d400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f985c009610 tx=0x7f985c097650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9feca99c56ccc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.709+0000 7f984b7fe640 1 -- 172.21.15.122:0/3451008235 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c09b980 con 0x55f4dce03080 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986593f640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f984c0a2b80 0x7f984c0a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f9858009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f984b7fe640 1 -- 172.21.15.122:0/3451008235 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c0cee50 con 0x55f4dce03080 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f984b7fe640 1 -- 172.21.15.122:0/3451008235 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0a51b0 con 0x55f4dce03080 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 -- 172.21.15.122:0/3451008235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 msgr2=0x55f4dce0d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 0x55f4dce0d400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f985c009610 tx=0x7f985c097650 comp rx=0 tx=0).stop 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 -- 172.21.15.122:0/3451008235 shutdown_connections 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce0d940 0x55f4dce0fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce03080 0x55f4dce0d400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 --2- 172.21.15.122:0/3451008235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce02b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 -- 172.21.15.122:0/3451008235 >> 172.21.15.122:0/3451008235 conn(0x55f4dcdfecf0 msgr2=0x55f4dcdff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 -- 172.21.15.122:0/3451008235 shutdown_connections 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.710+0000 7f986977b740 1 -- 172.21.15.122:0/3451008235 wait complete. 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 -- start start 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 Processor -- start 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce0ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce0b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f4dce126c0 con 0x55f4dce0d940 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f4dccf06e0 con 0x55f4dce02760 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f4dce12840 con 0x55f4dce03080 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986977b740 10 client.0 _ll_register_callbacks cb 0x55f4dcd14a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986593f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce0b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986593f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce0b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57500/0 (socket says 172.21.15.122:57500) 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.711+0000 7f986593f640 1 -- 172.21.15.122:0/3803743324 learned_addr learned my addr 172.21.15.122:0/3803743324 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986493d640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce0ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986513e640 1 -- 172.21.15.122:0/3803743324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 msgr2=0x55f4dce0ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986513e640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce0ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986513e640 1 -- 172.21.15.122:0/3803743324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 msgr2=0x55f4dce0b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986513e640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce0b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986513e640 1 -- 172.21.15.122:0/3803743324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dce1bbe0 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986593f640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce0b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986513e640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce0a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f985c09b1c0 tx=0x7f985c0cecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccb4ff41caffbd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f9848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f9848ff9640 1 -- 172.21.15.122:0/3803743324 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c09b980 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c0cee50 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f9848ff9640 1 -- 172.21.15.122:0/3803743324 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f985c0a5a40 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f9848ff9640 1 -- 172.21.15.122:0/3803743324 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f985c09b280 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986977b740 10 client.26552 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.712+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4dce11d50 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f9848ff9640 1 -- 172.21.15.122:0/3803743324 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c0cee50 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f9848ff9640 1 -- 172.21.15.122:0/3803743324 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f985c09b4e0 con 0x55f4dce02760 2024-09-12T01:59:42.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986977b740 10 client.26552 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986977b740 10 client.26552 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986977b740 20 client.26552 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986977b740 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce11fe0 0x55f4dce24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986977b740 4 client.26552 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4dcd759d0 con 0x55f4dce11fe0 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986493d640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce11fe0 0x55f4dce24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f986493d640 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce11fe0 0x55f4dce24010 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969da03d32923051 server_cookie=f1a7d15e3aa9aad5 in_seq=0 out_seq=0 2024-09-12T01:59:42.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.713+0000 7f9848ff9640 10 client.26552 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f9848ff9640 1 -- 172.21.15.122:0/3803743324 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4dcd759d0 con 0x55f4dce11fe0 2024-09-12T01:59:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f9848ff9640 10 client.26552 handle_command_reply: tid=0 2024-09-12T01:59:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 client.26552 shutdown 2024-09-12T01:59:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 msgr2=0x55f4dce0a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce0a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f985c09b1c0 tx=0x7f985c0cecf0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 shutdown_connections 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce11fe0 0x55f4dce24010 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce0b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce0ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 --2- 172.21.15.122:0/3803743324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce0a840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f9848ff9640 0 client.26552 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 >> 172.21.15.122:0/3803743324 conn(0x55f4dcdfecf0 msgr2=0x55f4dce04a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 shutdown_connections 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 1 -- 172.21.15.122:0/3803743324 wait complete. 2024-09-12T01:59:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.714+0000 7f986977b740 20 client.26552 trim_cache size 0 max 16384 2024-09-12T01:59:42.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.731+0000 7f986977b740 1 Processor -- start 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.731+0000 7f986977b740 1 -- start start 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986977b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986977b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce27570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986977b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce0d940 0x55f4dce07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986977b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98601adce0 con 0x55f4dce02760 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986977b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98601ab5b0 con 0x55f4dce0d940 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986977b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98601a93e0 con 0x55f4dce03080 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce27030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57504/0 (socket says 172.21.15.122:57504) 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 -- 172.21.15.122:0/1789678699 learned_addr learned my addr 172.21.15.122:0/1789678699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986593f640 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce0d940 0x55f4dce07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986493d640 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce27570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 -- 172.21.15.122:0/1789678699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 msgr2=0x55f4dce27570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce27570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 -- 172.21.15.122:0/1789678699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce0d940 msgr2=0x55f4dce07ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce0d940 0x55f4dce07ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 -- 172.21.15.122:0/1789678699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dcdfeb20 con 0x55f4dce02760 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986493d640 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce27570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f986513e640 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce27030 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f985c009610 tx=0x7f985c099790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fe41b5ab219879 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f984affd640 1 -- 172.21.15.122:0/1789678699 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c07c070 con 0x55f4dce02760 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f984affd640 1 -- 172.21.15.122:0/1789678699 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c0a8060 con 0x55f4dce02760 2024-09-12T01:59:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.732+0000 7f984affd640 1 -- 172.21.15.122:0/1789678699 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c09b4e0 con 0x55f4dce02760 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 -- 172.21.15.122:0/1789678699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 msgr2=0x55f4dce27030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce27030 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f985c009610 tx=0x7f985c099790 comp rx=0 tx=0).stop 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 -- 172.21.15.122:0/1789678699 shutdown_connections 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce0d940 0x55f4dce07ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce27570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 --2- 172.21.15.122:0/1789678699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce02760 0x55f4dce27030 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 -- 172.21.15.122:0/1789678699 >> 172.21.15.122:0/1789678699 conn(0x55f4dcdfecf0 msgr2=0x55f4dce1b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 -- 172.21.15.122:0/1789678699 shutdown_connections 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.733+0000 7f986977b740 1 -- 172.21.15.122:0/1789678699 wait complete. 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 -- start start 2024-09-12T01:59:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 Processor -- start 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce25a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce8b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98601a9a70 con 0x55f4dce0d940 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98601aaee0 con 0x55f4dce02760 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98601aa140 con 0x55f4dce03080 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986977b740 10 client.0 _ll_register_callbacks cb 0x7f985c0a3ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986593f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce8b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986513e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce254c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:59542/0 (socket says 172.21.15.122:59542) 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986513e640 1 -- 172.21.15.122:0/1014196927 learned_addr learned my addr 172.21.15.122:0/1014196927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.734+0000 7f986493d640 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce25a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986513e640 1 -- 172.21.15.122:0/1014196927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 msgr2=0x55f4dce25a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986513e640 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce25a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986513e640 1 -- 172.21.15.122:0/1014196927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 msgr2=0x55f4dce8b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986513e640 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce8b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986513e640 1 -- 172.21.15.122:0/1014196927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dcdfa260 con 0x55f4dce02760 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986513e640 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce254c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f985c09b210 tx=0x7f985c0b41f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbced59f2126a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f9848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f9848ff9640 1 -- 172.21.15.122:0/1014196927 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c07c070 con 0x55f4dce02760 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f9848ff9640 1 -- 172.21.15.122:0/1014196927 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f985c0cc5a0 con 0x55f4dce02760 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.735+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c0a8060 con 0x55f4dce02760 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f9848ff9640 1 -- 172.21.15.122:0/1014196927 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f985c09abe0 con 0x55f4dce02760 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 10 client.26558 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4dcdfa260 con 0x55f4dce02760 2024-09-12T01:59:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f9848ff9640 1 -- 172.21.15.122:0/1014196927 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c0a8060 con 0x55f4dce02760 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f9848ff9640 1 -- 172.21.15.122:0/1014196927 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f985c0a3a60 con 0x55f4dce02760 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 10 client.26558 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 10 client.26558 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 20 client.26558 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce8ce00 0x55f4dce8f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 4 client.26558 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.736+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55f4dcd6fd20 con 0x55f4dce8ce00 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.737+0000 7f986493d640 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce8ce00 0x55f4dce8f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.737+0000 7f986493d640 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce8ce00 0x55f4dce8f1f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41d18a2d4080298 server_cookie=fd044f0957b48aea in_seq=0 out_seq=0 2024-09-12T01:59:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.737+0000 7f9848ff9640 10 client.26558 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.801+0000 7f9848ff9640 1 -- 172.21.15.122:0/1014196927 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4dcd6fd20 con 0x55f4dce8ce00 2024-09-12T01:59:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.801+0000 7f9848ff9640 10 client.26558 handle_command_reply: tid=0 2024-09-12T01:59:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.801+0000 7f986977b740 1 client.26558 shutdown 2024-09-12T01:59:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 msgr2=0x55f4dce254c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce254c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f985c09b210 tx=0x7f985c0b41f0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 shutdown_connections 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f4dce8ce00 0x55f4dce8f1f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4dce0d940 0x55f4dce8b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4dce03080 0x55f4dce25a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 --2- 172.21.15.122:0/1014196927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4dce02760 0x55f4dce254c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f9848ff9640 0 client.26558 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 >> 172.21.15.122:0/1014196927 conn(0x55f4dcdfecf0 msgr2=0x55f4dce041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 shutdown_connections 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 1 -- 172.21.15.122:0/1014196927 wait complete. 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f986977b740 20 client.26558 trim_cache size 0 max 16384 2024-09-12T01:59:42.803 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 50, 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "327dfbb5": { 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 50, 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:59:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:59:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T01:59:42.806 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T01:59:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f984c0a2b80 msgr2=0x7f984c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f984c0a2b80 0x7f984c0a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f9858009500 comp rx=0 tx=0).stop 2024-09-12T01:59:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 msgr2=0x7f9860197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 0x7f9860197430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f985c00d950 tx=0x7f985c00de10 comp rx=0 tx=0).stop 2024-09-12T01:59:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 shutdown_connections 2024-09-12T01:59:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f984c0a2b80 0x7f984c0a5040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860102ef0 0x7f986019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9860102570 0x7f9860197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 --2- 172.21.15.122:0/2478734800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9860100e20 0x7f9860197430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.802+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 >> 172.21.15.122:0/2478734800 conn(0x7f98600fc7d0 msgr2=0x7f98600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.803+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 shutdown_connections 2024-09-12T01:59:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T01:59:42.803+0000 7f98673a2640 1 -- 172.21.15.122:0/2478734800 wait complete. 2024-09-12T01:59:42.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 600 sec 2024-09-12T01:59:43.519 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '327dfbb5', quiesce took 3.4 seconds. Will release after: 26.6 2024-09-12T01:59:43.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 601 sec 2024-09-12T01:59:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:43 smithi151 ceph-mon[35229]: pgmap v732: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-12T01:59:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:43 smithi183 ceph-mon[35127]: pgmap v732: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-12T01:59:44.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:43 smithi122 ceph-mon[30925]: pgmap v732: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-09-12T01:59:44.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 602 sec 2024-09-12T01:59:45.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 603 sec 2024-09-12T01:59:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:45 smithi151 ceph-mon[35229]: pgmap v733: 97 pgs: 97 active+clean; 311 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-09-12T01:59:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:45 smithi183 ceph-mon[35127]: pgmap v733: 97 pgs: 97 active+clean; 311 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-09-12T01:59:46.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:45 smithi122 ceph-mon[30925]: pgmap v733: 97 pgs: 97 active+clean; 311 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-09-12T01:59:46.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 604 sec 2024-09-12T01:59:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:46 smithi151 ceph-mon[35229]: pgmap v734: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-09-12T01:59:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:46 smithi183 ceph-mon[35127]: pgmap v734: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-09-12T01:59:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:46 smithi122 ceph-mon[30925]: pgmap v734: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-09-12T01:59:47.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 605 sec 2024-09-12T01:59:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 -- 172.21.15.122:0/1349510759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd8104ef0 msgr2=0x7fccd81072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/1349510759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd8104ef0 0x7fccd81072e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009900 tx=0x7fcccc02a410 comp rx=0 tx=0).stop 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 -- 172.21.15.122:0/1349510759 shutdown_connections 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/1349510759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccd8107820 0x7fccd8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/1349510759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd8104ef0 0x7fccd81072e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/1349510759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd81025c0 0x7fccd81049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 -- 172.21.15.122:0/1349510759 >> 172.21.15.122:0/1349510759 conn(0x7fccd80fc1e0 msgr2=0x7fccd80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 -- 172.21.15.122:0/1349510759 shutdown_connections 2024-09-12T01:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.632+0000 7fccdcf2c640 1 -- 172.21.15.122:0/1349510759 wait complete. 2024-09-12T01:59:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 Processor -- start 2024-09-12T01:59:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 -- start start 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 0x7fccd8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccd8104ef0 0x7fccd81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd8107820 0x7fccd819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fccd810d7f0 con 0x7fccd81025c0 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccd810d670 con 0x7fccd8104ef0 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccdcf2c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccd810d970 con 0x7fccd8107820 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.633+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 0x7fccd8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 0x7fccd8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57538/0 (socket says 172.21.15.122:57538) 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 -- 172.21.15.122:0/3441895116 learned_addr learned my addr 172.21.15.122:0/3441895116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccd8104ef0 0x7fccd81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd8107820 0x7fccd819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 -- 172.21.15.122:0/3441895116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd8107820 msgr2=0x7fccd819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd8107820 0x7fccd819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 -- 172.21.15.122:0/3441895116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccd8104ef0 msgr2=0x7fccd81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccd8104ef0 0x7fccd81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 -- 172.21.15.122:0/3441895116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd819faa0 con 0x7fccd81025c0 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd8107820 0x7fccd819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccd659c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 0x7fccd8197e80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fccc800c970 tx=0x7fccc800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccc77fe640 1 -- 172.21.15.122:0/3441895116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8007bb0 con 0x7fccd81025c0 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccc77fe640 1 -- 172.21.15.122:0/3441895116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc8007d50 con 0x7fccd81025c0 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd819fd90 con 0x7fccd81025c0 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccc77fe640 1 -- 172.21.15.122:0/3441895116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800f450 con 0x7fccd81025c0 2024-09-12T01:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.634+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a01a0 con 0x7fccd81025c0 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.636+0000 7fccc77fe640 1 -- 172.21.15.122:0/3441895116 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fccc8005020 con 0x7fccd81025c0 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 Processor -- start 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 -- start start 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a25490ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25490fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a254912830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a25486e2f0 con 0x55a25490b2e0 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a254816270 con 0x55a254910370 2024-09-12T01:59:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a25486a2b0 con 0x55a25490a9a0 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25490fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25490fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57554/0 (socket says 172.21.15.122:57554) 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccd659c640 1 -- 172.21.15.122:0/911078970 learned_addr learned my addr 172.21.15.122:0/911078970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.637+0000 7fccc77fe640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fccc00a2c30 0x7fccc00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a254912830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fccc00a2c30 0x7fccc00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccc77fe640 1 -- 172.21.15.122:0/3441895116 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fccc80cee30 con 0x7fccd81025c0 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd659c640 1 -- 172.21.15.122:0/911078970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 msgr2=0x55a25490ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a25490ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd659c640 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a25490ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd659c640 1 -- 172.21.15.122:0/911078970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 msgr2=0x55a254912830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd659c640 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a254912830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd659c640 1 -- 172.21.15.122:0/911078970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a254815340 con 0x55a25490b2e0 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a254912830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd659c640 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25490fda0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fccc80c6e80 tx=0x7fccc8004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a1580070ad6223 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fccc00a2c30 0x7fccc00a50f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fccd806a980 tx=0x7fccd0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccc4ff9640 1 -- 172.21.15.122:0/911078970 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc801d050 con 0x55a25490b2e0 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccc4ff9640 1 -- 172.21.15.122:0/911078970 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc809bd90 con 0x55a25490b2e0 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.638+0000 7fccc4ff9640 1 -- 172.21.15.122:0/911078970 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80997c0 con 0x55a25490b2e0 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 -- 172.21.15.122:0/911078970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 msgr2=0x55a25490fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25490fda0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fccc80c6e80 tx=0x7fccc8004a20 comp rx=0 tx=0).stop 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 -- 172.21.15.122:0/911078970 shutdown_connections 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a254912830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25490fda0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/911078970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a25490ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 -- 172.21.15.122:0/911078970 >> 172.21.15.122:0/911078970 conn(0x55a254906010 msgr2=0x55a254906410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 -- 172.21.15.122:0/911078970 shutdown_connections 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.639+0000 7fccdfcbf740 1 -- 172.21.15.122:0/911078970 wait complete. 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 -- start start 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 Processor -- start 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254914b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490b2e0 0x55a2549150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a25491c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a2549132e0 con 0x55a25490a9a0 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a2547f7290 con 0x55a254910370 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a254913460 con 0x55a25490b2e0 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccdfcbf740 10 client.0 _ll_register_callbacks cb 0x55a254828140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254914b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254914b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57558/0 (socket says 172.21.15.122:57558) 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 -- 172.21.15.122:0/2178177973 learned_addr learned my addr 172.21.15.122:0/2178177973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a25491c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490b2e0 0x55a2549150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 -- 172.21.15.122:0/2178177973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490b2e0 msgr2=0x55a2549150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490b2e0 0x55a2549150b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 -- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 msgr2=0x55a25491c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a25491c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd659c640 1 -- 172.21.15.122:0/2178177973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a254581210 con 0x55a25490a9a0 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.640+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a25491c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccd659c640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254914b70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fccc80058c0 tx=0x7fccc809fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520cde064d9948 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccbdffb640 1 -- 172.21.15.122:0/2178177973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80b4060 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccbdffb640 1 -- 172.21.15.122:0/2178177973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fccc8005310 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc809bd90 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccbdffb640 1 -- 172.21.15.122:0/2178177973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccc801d020 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 10 client.17415 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a25464d620 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccbdffb640 1 -- 172.21.15.122:0/2178177973 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc809bd90 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccbdffb640 1 -- 172.21.15.122:0/2178177973 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fccc809aae0 con 0x55a25490a9a0 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 10 client.17415 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 10 client.17415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 20 client.17415 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a2549203f0 0x55a2549227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 4 client.17415 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.641+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a25487ba20 con 0x55a2549203f0 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.642+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a2549203f0 0x55a2549227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.642+0000 7fccbdffb640 10 client.17415 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.642+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a2549203f0 0x55a2549227e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f520c1311e57b8bd server_cookie=40ac64d5696f6d81 in_seq=0 out_seq=0 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccbdffb640 1 -- 172.21.15.122:0/2178177973 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a25487ba20 con 0x55a2549203f0 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccbdffb640 10 client.17415 handle_command_reply: tid=0 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 client.17415 shutdown 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 msgr2=0x55a254914b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254914b70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fccc80058c0 tx=0x7fccc809fa40 comp rx=0 tx=0).stop 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 shutdown_connections 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a2549203f0 0x55a2549227e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a25491c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490b2e0 0x55a2549150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccbdffb640 0 client.17415 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/2178177973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254914b70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 >> 172.21.15.122:0/2178177973 conn(0x55a254906010 msgr2=0x55a254907820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 shutdown_connections 2024-09-12T01:59:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 1 -- 172.21.15.122:0/2178177973 wait complete. 2024-09-12T01:59:48.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.643+0000 7fccdfcbf740 20 client.17415 trim_cache size 0 max 16384 2024-09-12T01:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 Processor -- start 2024-09-12T01:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 -- start start 2024-09-12T01:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a254915140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25491c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a2549059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81ab900 con 0x55a25490b2e0 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81a91d0 con 0x55a254910370 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81a7000 con 0x55a25490a9a0 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25491c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25491c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57570/0 (socket says 172.21.15.122:57570) 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 -- 172.21.15.122:0/3335380237 learned_addr learned my addr 172.21.15.122:0/3335380237 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd659c640 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a254915140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 -- 172.21.15.122:0/3335380237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 msgr2=0x55a254915140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a254915140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd6d9d640 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a2549059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 -- 172.21.15.122:0/3335380237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 msgr2=0x55a2549059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a2549059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 -- 172.21.15.122:0/3335380237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2549146b0 con 0x55a25490b2e0 2024-09-12T01:59:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd659c640 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a254915140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.658+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25491c0b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0314e0 tx=0x7fcccc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6873c01decc6960f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccbffff640 1 -- 172.21.15.122:0/3335380237 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc048070 con 0x55a25490b2e0 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccbffff640 1 -- 172.21.15.122:0/3335380237 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcccc005040 con 0x55a25490b2e0 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccbffff640 1 -- 172.21.15.122:0/3335380237 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc035040 con 0x55a25490b2e0 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 -- 172.21.15.122:0/3335380237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 msgr2=0x55a25491c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25491c0b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0314e0 tx=0x7fcccc000bc0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 -- 172.21.15.122:0/3335380237 shutdown_connections 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a254910370 0x55a2549059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490b2e0 0x55a25491c0b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/3335380237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a25490a9a0 0x55a254915140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 -- 172.21.15.122:0/3335380237 >> 172.21.15.122:0/3335380237 conn(0x55a254906010 msgr2=0x55a254906f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 -- 172.21.15.122:0/3335380237 shutdown_connections 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 -- 172.21.15.122:0/3335380237 wait complete. 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.659+0000 7fccdfcbf740 1 -- start start 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 Processor -- start 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254919f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a25490b2e0 0x55a25491a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a254910370 0x55a25491b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81a7690 con 0x55a25490a9a0 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81a8b00 con 0x55a25490b2e0 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccd81a7d60 con 0x55a254910370 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 10 client.0 _ll_register_callbacks cb 0x7fccc8005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254919f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254919f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57572/0 (socket says 172.21.15.122:57572) 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 -- 172.21.15.122:0/713166508 learned_addr learned my addr 172.21.15.122:0/713166508 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a25490b2e0 0x55a25491a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 -- 172.21.15.122:0/713166508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a254910370 msgr2=0x55a25491b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a254910370 0x55a25491b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 -- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a25490b2e0 msgr2=0x55a25491a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a25490b2e0 0x55a25491a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 -- 172.21.15.122:0/713166508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25491baa0 con 0x55a25490a9a0 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a25490b2e0 0x55a25491a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccd659c640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254919f40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fccc809b620 tx=0x7fccc8099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb70c28e422aaaaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccbdffb640 1 -- 172.21.15.122:0/713166508 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80b4040 con 0x55a25490a9a0 2024-09-12T01:59:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccbdffb640 1 -- 172.21.15.122:0/713166508 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fccc80afdb0 con 0x55a25490a9a0 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.660+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcccc005040 con 0x55a25490a9a0 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccbdffb640 1 -- 172.21.15.122:0/713166508 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccc8097930 con 0x55a25490a9a0 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 10 client.17421 fetch_fsmap learned FSMap version 30 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a25498b680 con 0x55a25490a9a0 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccbdffb640 1 -- 172.21.15.122:0/713166508 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcccc005040 con 0x55a25490a9a0 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccbdffb640 1 -- 172.21.15.122:0/713166508 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fccc80a0bd0 con 0x55a25490a9a0 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 10 client.17421 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 20 client.17421 populate_metadata read hostname 'smithi122' 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a25498be10 0x55a25498e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 4 client.17421 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a254875d70 con 0x55a25498be10 2024-09-12T01:59:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.661+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a25498be10 0x55a25498e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:59:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.662+0000 7fccbdffb640 10 client.17421 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.662+0000 7fccd5d9b640 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a25498be10 0x55a25498e250 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec898808c07801d server_cookie=5f3c9a740a8fab77 in_seq=0 out_seq=0 2024-09-12T01:59:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccbdffb640 1 -- 172.21.15.122:0/713166508 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a254875d70 con 0x55a25498be10 2024-09-12T01:59:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccbdffb640 10 client.17421 handle_command_reply: tid=0 2024-09-12T01:59:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 client.17421 shutdown 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 msgr2=0x55a254919f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254919f40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fccc809b620 tx=0x7fccc8099730 comp rx=0 tx=0).stop 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 shutdown_connections 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a25498be10 0x55a25498e250 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a254910370 0x55a25491b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccbdffb640 0 client.17421 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a25490b2e0 0x55a25491a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 --2- 172.21.15.122:0/713166508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a25490a9a0 0x55a254919f40 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 >> 172.21.15.122:0/713166508 conn(0x55a254906010 msgr2=0x55a254907740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 shutdown_connections 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 1 -- 172.21.15.122:0/713166508 wait complete. 2024-09-12T01:59:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.677+0000 7fccdfcbf740 20 client.17421 trim_cache size 0 max 16384 2024-09-12T01:59:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T01:59:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T01:59:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T01:59:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fccc00a2c30 msgr2=0x7fccc00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fccc00a2c30 0x7fccc00a50f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fccd806a980 tx=0x7fccd0009550 comp rx=0 tx=0).stop 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 msgr2=0x7fccd8197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 0x7fccd8197e80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fccc800c970 tx=0x7fccc800ce30 comp rx=0 tx=0).stop 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 shutdown_connections 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fccc00a2c30 0x7fccc00a50f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd8107820 0x7fccd819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccd8104ef0 0x7fccd81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 --2- 172.21.15.122:0/3441895116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccd81025c0 0x7fccd8197e80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:59:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 >> 172.21.15.122:0/3441895116 conn(0x7fccd80fc1e0 msgr2=0x7fccd8102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:59:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 shutdown_connections 2024-09-12T01:59:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T01:59:48.678+0000 7fccdcf2c640 1 -- 172.21.15.122:0/3441895116 wait complete. 2024-09-12T01:59:48.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 606 sec 2024-09-12T01:59:49.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 607 sec 2024-09-12T01:59:49.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:49 smithi122 ceph-mon[30925]: pgmap v735: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-12T01:59:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:49 smithi151 ceph-mon[35229]: pgmap v735: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-12T01:59:50.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:49 smithi183 ceph-mon[35127]: pgmap v735: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-12T01:59:50.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 608 sec 2024-09-12T01:59:51.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 609 sec 2024-09-12T01:59:51.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:51 smithi122 ceph-mon[30925]: pgmap v736: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 171 op/s 2024-09-12T01:59:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:51 smithi151 ceph-mon[35229]: pgmap v736: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 171 op/s 2024-09-12T01:59:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:51 smithi183 ceph-mon[35127]: pgmap v736: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 171 op/s 2024-09-12T01:59:52.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 610 sec 2024-09-12T01:59:53.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 611 sec 2024-09-12T01:59:53.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:53 smithi122 ceph-mon[30925]: pgmap v737: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-09-12T01:59:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:53 smithi151 ceph-mon[35229]: pgmap v737: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-09-12T01:59:54.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:53 smithi183 ceph-mon[35127]: pgmap v737: 97 pgs: 97 active+clean; 313 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 137 op/s 2024-09-12T01:59:54.863 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 612 sec 2024-09-12T01:59:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:59:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:59:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:59:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:59:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:59:55.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T01:59:55.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 613 sec 2024-09-12T01:59:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:55 smithi122 ceph-mon[30925]: pgmap v738: 97 pgs: 97 active+clean; 314 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-12T01:59:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:55 smithi151 ceph-mon[35229]: pgmap v738: 97 pgs: 97 active+clean; 314 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-12T01:59:56.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:55 smithi183 ceph-mon[35127]: pgmap v738: 97 pgs: 97 active+clean; 314 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.1 MiB/s wr, 106 op/s 2024-09-12T01:59:56.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 614 sec 2024-09-12T01:59:57.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 615 sec 2024-09-12T01:59:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:57 smithi122 ceph-mon[30925]: pgmap v739: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-09-12T01:59:58.004 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:57 smithi183 ceph-mon[35127]: pgmap v739: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-09-12T01:59:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:57 smithi151 ceph-mon[35229]: pgmap v739: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-09-12T01:59:58.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 616 sec 2024-09-12T01:59:59.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 617 sec 2024-09-12T02:00:00.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 01:59:59 smithi122 ceph-mon[30925]: pgmap v740: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 83 op/s 2024-09-12T02:00:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 01:59:59 smithi151 ceph-mon[35229]: pgmap v740: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 83 op/s 2024-09-12T02:00:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 01:59:59 smithi183 ceph-mon[35127]: pgmap v740: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 83 op/s 2024-09-12T02:00:00.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T01:59:59.999+0000 7f42afbd7640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:00:00.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 618 sec 2024-09-12T02:00:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:00 smithi151 ceph-mon[35229]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:00:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:00 smithi183 ceph-mon[35127]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:00:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:00 smithi122 ceph-mon[30925]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:00:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 -- 172.21.15.122:0/553303722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8107830 msgr2=0x7fc9a8109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/553303722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8107830 0x7fc9a8109ca0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009be0 tx=0x7fc9a002a6c0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 -- 172.21.15.122:0/553303722 shutdown_connections 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/553303722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8107830 0x7fc9a8109ca0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/553303722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a8104f00 0x7fc9a81072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/553303722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9a81025d0 0x7fc9a81049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 -- 172.21.15.122:0/553303722 >> 172.21.15.122:0/553303722 conn(0x7fc9a80fc1b0 msgr2=0x7fc9a80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 -- 172.21.15.122:0/553303722 shutdown_connections 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 -- 172.21.15.122:0/553303722 wait complete. 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.774+0000 7fc9adbb7640 1 Processor -- start 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 -- start start 2024-09-12T02:00:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a81025d0 0x7fc9a8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 0x7fc9a81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9a8107830 0x7fc9a819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810d7a0 con 0x7fc9a8104f00 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810d620 con 0x7fc9a8107830 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9adbb7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a810d920 con 0x7fc9a81025d0 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 0x7fc9a81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 0x7fc9a81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48318/0 (socket says 172.21.15.122:48318) 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/1191953468 learned_addr learned my addr 172.21.15.122:0/1191953468 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9a8107830 0x7fc9a819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a81025d0 0x7fc9a8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/1191953468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a81025d0 msgr2=0x7fc9a8197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a81025d0 0x7fc9a8197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/1191953468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9a8107830 msgr2=0x7fc9a819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9a8107830 0x7fc9a819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/1191953468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a819f9d0 con 0x7fc9a8104f00 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.775+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 0x7fc9a81982f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc99000d770 tx=0x7fc99000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.776+0000 7fc9a4ff9640 1 -- 172.21.15.122:0/1191953468 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990017070 con 0x7fc9a8104f00 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.776+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a819fcc0 con 0x7fc9a8104f00 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.776+0000 7fc9a4ff9640 1 -- 172.21.15.122:0/1191953468 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc990010460 con 0x7fc9a8104f00 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.776+0000 7fc9a4ff9640 1 -- 172.21.15.122:0/1191953468 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000f620 con 0x7fc9a8104f00 2024-09-12T02:00:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.776+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a81a0200 con 0x7fc9a8104f00 2024-09-12T02:00:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 Processor -- start 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9a4ff9640 1 -- 172.21.15.122:0/1191953468 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc99001a070 con 0x7fc9a8104f00 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 -- start start 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 0x5592d9b3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592d9a9e2f0 con 0x5592d9b3a9a0 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592d9a46270 con 0x5592d9b40370 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5592d9a9a2b0 con 0x5592d9b3b2e0 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 0x5592d9b3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 0x5592d9b3ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48332/0 (socket says 172.21.15.122:48332) 2024-09-12T02:00:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.778+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2997390659 learned_addr learned my addr 172.21.15.122:0/2997390659 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2997390659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 msgr2=0x5592d9b3fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b3fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2997390659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 msgr2=0x5592d9b42830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b42830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2997390659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d9a45340 con 0x5592d9b3a9a0 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a4ff9640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9940a2c30 0x7fc9940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 0x5592d9b3ada0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc9a806b3e0 tx=0x7fc9a0005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e622d4b659117eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9940a2c30 0x7fc9940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc9a4ff9640 1 -- 172.21.15.122:0/1191953468 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc9900cea50 con 0x7fc9a8104f00 2024-09-12T02:00:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc99dffb640 1 -- 172.21.15.122:0/2997390659 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0042030 con 0x5592d9b3a9a0 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc99dffb640 1 -- 172.21.15.122:0/2997390659 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0037040 con 0x5592d9b3a9a0 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.779+0000 7fc99dffb640 1 -- 172.21.15.122:0/2997390659 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0036070 con 0x5592d9b3a9a0 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2997390659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 msgr2=0x5592d9b3ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9940a2c30 0x7fc9940a50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5592d9b3bd10 tx=0x7fc998009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 0x5592d9b3ada0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc9a806b3e0 tx=0x7fc9a0005ec0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2997390659 shutdown_connections 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b3fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2997390659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3a9a0 0x5592d9b3ada0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2997390659 >> 172.21.15.122:0/2997390659 conn(0x5592d9b36010 msgr2=0x5592d9b36410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2997390659 shutdown_connections 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2997390659 wait complete. 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.780+0000 7fc9b0a2c740 1 -- start start 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 Processor -- start 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b44b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3b2e0 0x5592d9b45050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592d9b432e0 con 0x5592d9b40370 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592d9a27290 con 0x5592d9b3b2e0 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5592d9b43460 con 0x5592d9b3a9a0 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9b0a2c740 10 client.0 _ll_register_callbacks cb 0x5592d9a58140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b4c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48344/0 (socket says 172.21.15.122:48344) 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2800314681 learned_addr learned my addr 172.21.15.122:0/2800314681 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3b2e0 0x5592d9b45050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b44b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2800314681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 msgr2=0x5592d9b44b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b44b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3b2e0 msgr2=0x5592d9b45050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3b2e0 0x5592d9b45050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 -- 172.21.15.122:0/2800314681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d97b1210 con 0x5592d9b40370 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3b2e0 0x5592d9b45050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b4c090 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc9a002abe0 tx=0x7fc9a0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4fc5920d2c94af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.781+0000 7fc986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc986ffd640 1 -- 172.21.15.122:0/2800314681 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0042040 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc986ffd640 1 -- 172.21.15.122:0/2800314681 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc9a00303a0 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0037040 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc986ffd640 1 -- 172.21.15.122:0/2800314681 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc9a0036040 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 10 client.17433 fetch_fsmap learned FSMap version 30 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592d987d620 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc986ffd640 1 -- 172.21.15.122:0/2800314681 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0037040 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc986ffd640 1 -- 172.21.15.122:0/2800314681 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a003c040 con 0x5592d9b40370 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 10 client.17433 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 20 client.17433 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9b50390 0x5592d9b52780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 4 client.17433 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592d9aaba20 con 0x5592d9b50390 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.782+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9b50390 0x5592d9b52780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.783+0000 7fc986ffd640 10 client.17433 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.783+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9b50390 0x5592d9b52780 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf54fc032bbae2c server_cookie=ec9cd571a6928c26 in_seq=0 out_seq=0 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.783+0000 7fc986ffd640 1 -- 172.21.15.122:0/2800314681 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592d9aaba20 con 0x5592d9b50390 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.783+0000 7fc986ffd640 10 client.17433 handle_command_reply: tid=0 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.783+0000 7fc9b0a2c740 1 client.17433 shutdown 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 msgr2=0x5592d9b4c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b4c090 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc9a002abe0 tx=0x7fc9a0009800 comp rx=0 tx=0).stop 2024-09-12T02:00:01.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 shutdown_connections 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9b50390 0x5592d9b52780 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b4c090 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3b2e0 0x5592d9b45050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2800314681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b44b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc986ffd640 0 client.17433 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 >> 172.21.15.122:0/2800314681 conn(0x5592d9b36010 msgr2=0x5592d9b37820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 shutdown_connections 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2800314681 wait complete. 2024-09-12T02:00:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.784+0000 7fc9b0a2c740 20 client.17433 trim_cache size 0 max 16384 2024-09-12T02:00:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 Processor -- start 2024-09-12T02:00:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 -- start start 2024-09-12T02:00:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3a9a0 0x5592d9b349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b4c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81ab740 con 0x5592d9b40370 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81a9010 con 0x5592d9b3a9a0 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81a6e40 con 0x5592d9b3b2e0 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b34ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48360/0 (socket says 172.21.15.122:48360) 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 -- 172.21.15.122:0/3259173077 learned_addr learned my addr 172.21.15.122:0/3259173077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 -- 172.21.15.122:0/3259173077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 msgr2=0x5592d9b4c050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b4c050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 -- 172.21.15.122:0/3259173077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3a9a0 msgr2=0x5592d9b349a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3a9a0 0x5592d9b349a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 -- 172.21.15.122:0/3259173077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d9b52de0 con 0x5592d9b40370 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.798+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b34ee0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0002230 tx=0x7fc9a0005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637060cdc006d225 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc99d7fa640 1 -- 172.21.15.122:0/3259173077 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0042040 con 0x5592d9b40370 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc99d7fa640 1 -- 172.21.15.122:0/3259173077 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0030cf0 con 0x5592d9b40370 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc99d7fa640 1 -- 172.21.15.122:0/3259173077 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0003fd0 con 0x5592d9b40370 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/3259173077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 msgr2=0x5592d9b34ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b34ee0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0002230 tx=0x7fc9a0005ec0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/3259173077 shutdown_connections 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b40370 0x5592d9b34ee0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3b2e0 0x5592d9b4c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/3259173077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b3a9a0 0x5592d9b349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/3259173077 >> 172.21.15.122:0/3259173077 conn(0x5592d9b36010 msgr2=0x5592d9b44b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/3259173077 shutdown_connections 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/3259173077 wait complete. 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.799+0000 7fc9b0a2c740 1 -- start start 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 Processor -- start 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b47430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 0x5592d9b47970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b4a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81a74d0 con 0x5592d9b3b2e0 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81a8940 con 0x5592d9b40370 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81a7ba0 con 0x5592d9b3a9a0 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9b0a2c740 10 client.0 _ll_register_callbacks cb 0x7fc9900d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 0x5592d9b47970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 0x5592d9b47970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48366/0 (socket says 172.21.15.122:48366) 2024-09-12T02:00:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/2915183993 learned_addr learned my addr 172.21.15.122:0/2915183993 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b47430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b4a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/2915183993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 msgr2=0x5592d9b47430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b47430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 msgr2=0x5592d9b4a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b4a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 -- 172.21.15.122:0/2915183993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d9b3a5e0 con 0x5592d9b3b2e0 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a7fff640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b4a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc9a6ffd640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 0x5592d9b47970 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc9900cc550 tx=0x7fc99009aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580630139c654106 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.800+0000 7fc986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc986ffd640 1 -- 172.21.15.122:0/2915183993 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990021070 con 0x5592d9b3b2e0 2024-09-12T02:00:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc986ffd640 1 -- 172.21.15.122:0/2915183993 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc990010610 con 0x5592d9b3b2e0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0030cf0 con 0x5592d9b3b2e0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc986ffd640 1 -- 172.21.15.122:0/2915183993 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc99009f660 con 0x5592d9b3b2e0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 10 client.17439 fetch_fsmap learned FSMap version 30 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592d9b4c150 con 0x5592d9b3b2e0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc986ffd640 1 -- 172.21.15.122:0/2915183993 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0030cf0 con 0x5592d9b3b2e0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc986ffd640 1 -- 172.21.15.122:0/2915183993 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc9900bb0b0 con 0x5592d9b3b2e0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 10 client.17439 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 20 client.17439 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9bbb5c0 0x5592d9bbd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 4 client.17439 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592d9aa5d70 con 0x5592d9bbb5c0 2024-09-12T02:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.801+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9bbb5c0 0x5592d9bbd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.802+0000 7fc986ffd640 10 client.17439 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.802+0000 7fc9a77fe640 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9bbb5c0 0x5592d9bbd9e0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16e7c45538b4660 server_cookie=1b370db609ee762d in_seq=0 out_seq=0 2024-09-12T02:00:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc986ffd640 1 -- 172.21.15.122:0/2915183993 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592d9aa5d70 con 0x5592d9bbb5c0 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc986ffd640 10 client.17439 handle_command_reply: tid=0 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 client.17439 shutdown 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 msgr2=0x5592d9b47970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 0x5592d9b47970 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc9900cc550 tx=0x7fc99009aeb0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 shutdown_connections 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5592d9bbb5c0 0x5592d9bbd9e0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592d9b40370 0x5592d9b4a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592d9b3b2e0 0x5592d9b47970 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc986ffd640 0 client.17439 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 --2- 172.21.15.122:0/2915183993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5592d9b3a9a0 0x5592d9b47430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 >> 172.21.15.122:0/2915183993 conn(0x5592d9b36010 msgr2=0x5592d9b36ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 shutdown_connections 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 1 -- 172.21.15.122:0/2915183993 wait complete. 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.816+0000 7fc9b0a2c740 20 client.17439 trim_cache size 0 max 16384 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:00:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9940a2c30 msgr2=0x7fc9940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9940a2c30 0x7fc9940a50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5592d9b3bd10 tx=0x7fc998009260 comp rx=0 tx=0).stop 2024-09-12T02:00:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 msgr2=0x7fc9a81982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 0x7fc9a81982f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc99000d770 tx=0x7fc99000db70 comp rx=0 tx=0).stop 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 shutdown_connections 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9940a2c30 0x7fc9940a50f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9a8107830 0x7fc9a819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9a8104f00 0x7fc9a81982f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 --2- 172.21.15.122:0/1191953468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9a81025d0 0x7fc9a8197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 >> 172.21.15.122:0/1191953468 conn(0x7fc9a80fc1b0 msgr2=0x7fc9a8102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 shutdown_connections 2024-09-12T02:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:01.817+0000 7fc9adbb7640 1 -- 172.21.15.122:0/1191953468 wait complete. 2024-09-12T02:00:01.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 619 sec 2024-09-12T02:00:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:01 smithi151 ceph-mon[35229]: pgmap v741: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-12T02:00:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:01 smithi183 ceph-mon[35127]: pgmap v741: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-12T02:00:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:01 smithi122 ceph-mon[30925]: pgmap v741: 97 pgs: 97 active+clean; 315 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-12T02:00:02.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 620 sec 2024-09-12T02:00:03.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 621 sec 2024-09-12T02:00:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:03 smithi151 ceph-mon[35229]: pgmap v742: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-09-12T02:00:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:03 smithi183 ceph-mon[35127]: pgmap v742: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-09-12T02:00:04.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:03 smithi122 ceph-mon[30925]: pgmap v742: 97 pgs: 97 active+clean; 312 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.4 MiB/s wr, 89 op/s 2024-09-12T02:00:04.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 622 sec 2024-09-12T02:00:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:04 smithi151 ceph-mon[35229]: pgmap v743: 97 pgs: 97 active+clean; 309 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.1 MiB/s wr, 86 op/s 2024-09-12T02:00:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:04 smithi183 ceph-mon[35127]: pgmap v743: 97 pgs: 97 active+clean; 309 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.1 MiB/s wr, 86 op/s 2024-09-12T02:00:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:04 smithi122 ceph-mon[30925]: pgmap v743: 97 pgs: 97 active+clean; 309 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.1 MiB/s wr, 86 op/s 2024-09-12T02:00:05.864 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 623 sec 2024-09-12T02:00:06.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 624 sec 2024-09-12T02:00:07.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 625 sec 2024-09-12T02:00:07.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:07 smithi122 ceph-mon[30925]: pgmap v744: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-12T02:00:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:07 smithi151 ceph-mon[35229]: pgmap v744: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-12T02:00:08.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:07 smithi183 ceph-mon[35127]: pgmap v744: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.0 MiB/s wr, 92 op/s 2024-09-12T02:00:08.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 626 sec 2024-09-12T02:00:09.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 627 sec 2024-09-12T02:00:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:09 smithi122 ceph-mon[30925]: pgmap v745: 97 pgs: 97 active+clean; 308 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.2 MiB/s wr, 99 op/s 2024-09-12T02:00:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:09 smithi151 ceph-mon[35229]: pgmap v745: 97 pgs: 97 active+clean; 308 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.2 MiB/s wr, 99 op/s 2024-09-12T02:00:10.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:00:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:09 smithi183 ceph-mon[35127]: pgmap v745: 97 pgs: 97 active+clean; 308 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.2 MiB/s wr, 99 op/s 2024-09-12T02:00:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:10.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 628 sec 2024-09-12T02:00:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 -- 172.21.15.122:0/3790415906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 msgr2=0x7f1524106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 --2- 172.21.15.122:0/3790415906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f1524106e60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1514009930 tx=0x7f151402a420 comp rx=0 tx=0).stop 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 -- 172.21.15.122:0/3790415906 shutdown_connections 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 --2- 172.21.15.122:0/3790415906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524101420 0x7f152410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 --2- 172.21.15.122:0/3790415906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524100a80 0x7f1524100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 --2- 172.21.15.122:0/3790415906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f1524106e60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 -- 172.21.15.122:0/3790415906 >> 172.21.15.122:0/3790415906 conn(0x7f15240fc7a0 msgr2=0x7f15240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 -- 172.21.15.122:0/3790415906 shutdown_connections 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.016+0000 7f152bee8640 1 -- 172.21.15.122:0/3790415906 wait complete. 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 Processor -- start 2024-09-12T02:00:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 -- start start 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524100a80 0x7f15240fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524101420 0x7f15240fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f15240fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1524110e40 con 0x7f1524106a80 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1524110cc0 con 0x7f1524101420 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152bee8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1524110fc0 con 0x7f1524100a80 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152a485640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f15240fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152a485640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f15240fd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56870/0 (socket says 172.21.15.122:56870) 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f152a485640 1 -- 172.21.15.122:0/769848612 learned_addr learned my addr 172.21.15.122:0/769848612 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.017+0000 7f1529483640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524101420 0x7f15240fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152a485640 1 -- 172.21.15.122:0/769848612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524100a80 msgr2=0x7f15240fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f1529c84640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524100a80 0x7f15240fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152a485640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524100a80 0x7f15240fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152a485640 1 -- 172.21.15.122:0/769848612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524101420 msgr2=0x7f15240fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152a485640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524101420 0x7f15240fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152a485640 1 -- 172.21.15.122:0/769848612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15241a68c0 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f1529c84640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524100a80 0x7f15240fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f1529483640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524101420 0x7f15240fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152a485640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f15240fd880 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f151c00eaa0 tx=0x7f151c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c00cdb0 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f151c005020 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0116b0 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15241a6bb0 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.018+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241a70f0 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.019+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1524100ee0 con 0x7f1524106a80 2024-09-12T02:00:11.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.020+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f151c005560 con 0x7f1524106a80 2024-09-12T02:00:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.021+0000 7f151affd640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15100a2c00 0x7f15100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.021+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f151c0cf280 con 0x7f1524106a80 2024-09-12T02:00:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.021+0000 7f1529c84640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15100a2c00 0x7f15100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.021+0000 7f1529c84640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15100a2c00 0x7f15100a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1514000c00 tx=0x7f1514032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.025+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f151c016030 con 0x7f1524106a80 2024-09-12T02:00:11.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.236+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1524109a70 con 0x7f1524106a80 2024-09-12T02:00:11.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.241+0000 7f151affd640 1 -- 172.21.15.122:0/769848612 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f151c09c1e0 con 0x7f1524106a80 2024-09-12T02:00:11.242 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:00:11.242 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T02:00:11.242 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15100a2c00 msgr2=0x7f15100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15100a2c00 0x7f15100a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1514000c00 tx=0x7f1514032c70 comp rx=0 tx=0).stop 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 msgr2=0x7f15240fd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f15240fd880 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f151c00eaa0 tx=0x7f151c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 shutdown_connections 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15100a2c00 0x7f15100a50c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1524106a80 0x7f15240fd880 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1524101420 0x7f15240fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 --2- 172.21.15.122:0/769848612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524100a80 0x7f15240fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 >> 172.21.15.122:0/769848612 conn(0x7f15240fc7a0 msgr2=0x7f152406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 shutdown_connections 2024-09-12T02:00:11.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:11.246+0000 7f152bee8640 1 -- 172.21.15.122:0/769848612 wait complete. 2024-09-12T02:00:11.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 629 sec 2024-09-12T02:00:11.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:11 smithi122 ceph-mon[30925]: pgmap v746: 97 pgs: 97 active+clean; 309 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.9 MiB/s wr, 83 op/s 2024-09-12T02:00:11.959 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:11 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/769848612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:11.988 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 327dfbb5 --release --await' 2024-09-12T02:00:11.988 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 327dfbb5 --release --await 2024-09-12T02:00:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:11 smithi151 ceph-mon[35229]: pgmap v746: 97 pgs: 97 active+clean; 309 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.9 MiB/s wr, 83 op/s 2024-09-12T02:00:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:11 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/769848612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:12.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:11 smithi183 ceph-mon[35127]: pgmap v746: 97 pgs: 97 active+clean; 309 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.9 MiB/s wr, 83 op/s 2024-09-12T02:00:12.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:11 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/769848612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 -- 172.21.15.122:0/1406444887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 msgr2=0x7f86f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 --2- 172.21.15.122:0/1406444887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f810d950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009cd0 tx=0x7f86f002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 -- 172.21.15.122:0/1406444887 shutdown_connections 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 --2- 172.21.15.122:0/1406444887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f810d950 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 --2- 172.21.15.122:0/1406444887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8100a80 0x7f86f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 --2- 172.21.15.122:0/1406444887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8106a80 0x7f86f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 -- 172.21.15.122:0/1406444887 >> 172.21.15.122:0/1406444887 conn(0x7f86f80fc7a0 msgr2=0x7f86f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 -- 172.21.15.122:0/1406444887 shutdown_connections 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.716+0000 7f86fcff7640 1 -- 172.21.15.122:0/1406444887 wait complete. 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 Processor -- start 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 -- start start 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8100a80 0x7f86f8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8106a80 0x7f86f81a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8110ea0 con 0x7f86f8101420 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8110d20 con 0x7f86f8100a80 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86fcff7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8111020 con 0x7f86f8106a80 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86effff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86effff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f8199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56894/0 (socket says 172.21.15.122:56894) 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86effff640 1 -- 172.21.15.122:0/457769401 learned_addr learned my addr 172.21.15.122:0/457769401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86f659c640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8100a80 0x7f86f8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.717+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8106a80 0x7f86f81a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86effff640 1 -- 172.21.15.122:0/457769401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8106a80 msgr2=0x7f86f81a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86effff640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8106a80 0x7f86f81a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86effff640 1 -- 172.21.15.122:0/457769401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8100a80 msgr2=0x7f86f8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86effff640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8100a80 0x7f86f8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86effff640 1 -- 172.21.15.122:0/457769401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f81a48d0 con 0x7f86f8101420 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8106a80 0x7f86f81a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86effff640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f8199850 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b4a0 tx=0x7f86e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86ef7fe640 1 -- 172.21.15.122:0/457769401 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0014070 con 0x7f86f8101420 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81a4aa0 con 0x7f86f8101420 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86ef7fe640 1 -- 172.21.15.122:0/457769401 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0010af0 con 0x7f86f8101420 2024-09-12T02:00:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86ef7fe640 1 -- 172.21.15.122:0/457769401 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0017d10 con 0x7f86f8101420 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.718+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a4f60 con 0x7f86f8101420 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.720+0000 7f86ef7fe640 1 -- 172.21.15.122:0/457769401 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f86e0017070 con 0x7f86f8101420 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 Processor -- start 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 -- start start 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d66b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ef7fe640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86e40a2b80 0x7f86e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d67a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d67ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55908d5d0620 con 0x55908d67a650 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55908d577e50 con 0x55908d66b860 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55908d5cc5e0 con 0x55908d66af40 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d67ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86f659c640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86e40a2b80 0x7f86e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d67ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56898/0 (socket says 172.21.15.122:56898) 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3294045296 learned_addr learned my addr 172.21.15.122:0/3294045296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86ef7fe640 1 -- 172.21.15.122:0/457769401 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86e00ce450 con 0x7f86f8101420 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d66b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.721+0000 7f86effff640 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d67a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3294045296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 msgr2=0x55908d66b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d66b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3294045296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 msgr2=0x55908d67a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d67a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3294045296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908d568d40 con 0x55908d67a650 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f659c640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86e40a2b80 0x7f86e40a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f86e800bbe0 tx=0x7f86e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d67ca40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f86f0035000 tx=0x7f86f0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd08a2b3afc44530 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86ecff9640 1 -- 172.21.15.122:0/3294045296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00319a0 con 0x55908d67a650 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86ecff9640 1 -- 172.21.15.122:0/3294045296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0036040 con 0x55908d67a650 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.722+0000 7f86ecff9640 1 -- 172.21.15.122:0/3294045296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00045c0 con 0x55908d67a650 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3294045296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 msgr2=0x55908d67ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d67ca40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f86f0035000 tx=0x7f86f0005ef0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3294045296 shutdown_connections 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d67ca40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d67a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3294045296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d66b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3294045296 >> 172.21.15.122:0/3294045296 conn(0x55908d6675f0 msgr2=0x55908d6679d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3294045296 shutdown_connections 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3294045296 wait complete. 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- start start 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 Processor -- start 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d677520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d677a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d677fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55908d67d220 con 0x55908d67a650 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55908d558f10 con 0x55908d66b860 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.723+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55908d67d3a0 con 0x55908d66af40 2024-09-12T02:00:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ff3d0740 10 client.0 _ll_register_callbacks cb 0x55908d57cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d677fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d677fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56914/0 (socket says 172.21.15.122:56914) 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3598069476 learned_addr learned my addr 172.21.15.122:0/3598069476 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f659c640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d677520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86effff640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d677a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3598069476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 msgr2=0x55908d677520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d677520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 msgr2=0x55908d677a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d677a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 -- 172.21.15.122:0/3598069476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908d2df580 con 0x55908d67a650 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86effff640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d677a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d677fa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f86f002ae00 tx=0x7f86f0004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4881b91d5767493 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ddffb640 1 -- 172.21.15.122:0/3598069476 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0003970 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ddffb640 1 -- 172.21.15.122:0/3598069476 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86f0003b10 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0036040 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ddffb640 1 -- 172.21.15.122:0/3598069476 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86f0003d10 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ff3d0740 10 client.17457 fetch_fsmap learned FSMap version 30 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.724+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55908d3a9960 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ddffb640 1 -- 172.21.15.122:0/3598069476 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0036040 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ddffb640 1 -- 172.21.15.122:0/3598069476 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f86f0030070 con 0x55908d67a650 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ff3d0740 10 client.17457 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ff3d0740 10 client.17457 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ff3d0740 20 client.17457 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d68a090 0x55908d68c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ff3d0740 4 client.17457 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55908d5ddb90 con 0x55908d68a090 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.725+0000 7f86f659c640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d68a090 0x55908d68c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ddffb640 10 client.17457 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86f659c640 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d68a090 0x55908d68c480 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6006966ba0e22851 server_cookie=9530bd891116492d in_seq=0 out_seq=0 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ddffb640 1 -- 172.21.15.122:0/3598069476 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55908d5ddb90 con 0x55908d68a090 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ddffb640 10 client.17457 handle_command_reply: tid=0 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 client.17457 shutdown 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 msgr2=0x55908d677fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d677fa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f86f002ae00 tx=0x7f86f0004590 comp rx=0 tx=0).stop 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 shutdown_connections 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d68a090 0x55908d68c480 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d677fa0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d677a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ddffb640 0 client.17457 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:12.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/3598069476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d677520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.726+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 >> 172.21.15.122:0/3598069476 conn(0x55908d6675f0 msgr2=0x55908d66d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.727+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 shutdown_connections 2024-09-12T02:00:12.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.727+0000 7f86ff3d0740 1 -- 172.21.15.122:0/3598069476 wait complete. 2024-09-12T02:00:12.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.727+0000 7f86ff3d0740 20 client.17457 trim_cache size 0 max 16384 2024-09-12T02:00:12.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.743+0000 7f86ff3d0740 1 Processor -- start 2024-09-12T02:00:12.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.743+0000 7f86ff3d0740 1 -- start start 2024-09-12T02:00:12.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d685d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d674780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d674cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81add00 con 0x55908d67a650 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81ab5d0 con 0x55908d66b860 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81a9400 con 0x55908d66af40 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d674cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d674cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56926/0 (socket says 172.21.15.122:56926) 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 -- 172.21.15.122:0/915425919 learned_addr learned my addr 172.21.15.122:0/915425919 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86effff640 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d674780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f659c640 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d685d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 -- 172.21.15.122:0/915425919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 msgr2=0x55908d685d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d685d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 -- 172.21.15.122:0/915425919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 msgr2=0x55908d674780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d674780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 -- 172.21.15.122:0/915425919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908d6764d0 con 0x55908d67a650 2024-09-12T02:00:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d674cc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f86f002afd0 tx=0x7f86f0004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c020ef4667dfe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86dffff640 1 -- 172.21.15.122:0/915425919 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0003eb0 con 0x55908d67a650 2024-09-12T02:00:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86dffff640 1 -- 172.21.15.122:0/915425919 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0041070 con 0x55908d67a650 2024-09-12T02:00:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.744+0000 7f86dffff640 1 -- 172.21.15.122:0/915425919 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0053450 con 0x55908d67a650 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 -- 172.21.15.122:0/915425919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 msgr2=0x55908d674cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d674cc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f86f002afd0 tx=0x7f86f0004990 comp rx=0 tx=0).stop 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 -- 172.21.15.122:0/915425919 shutdown_connections 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d67a650 0x55908d674cc0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d66b860 0x55908d674780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/915425919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d685d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 -- 172.21.15.122:0/915425919 >> 172.21.15.122:0/915425919 conn(0x55908d6675f0 msgr2=0x55908d685660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 -- 172.21.15.122:0/915425919 shutdown_connections 2024-09-12T02:00:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.745+0000 7f86ff3d0740 1 -- 172.21.15.122:0/915425919 wait complete. 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 -- start start 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 Processor -- start 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d68d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 0x55908d68ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d67a650 0x55908d6f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81a9a90 con 0x55908d66b860 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81aaf00 con 0x55908d67a650 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81aa160 con 0x55908d66af40 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ff3d0740 10 client.0 _ll_register_callbacks cb 0x7f86f00318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 0x55908d68ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 0x55908d68ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56936/0 (socket says 172.21.15.122:56936) 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 -- 172.21.15.122:0/2491125798 learned_addr learned my addr 172.21.15.122:0/2491125798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86f659c640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d68d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 -- 172.21.15.122:0/2491125798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 msgr2=0x55908d68d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d68d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86f6d9d640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d67a650 0x55908d6f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 -- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d67a650 msgr2=0x55908d6f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d67a650 0x55908d6f39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 -- 172.21.15.122:0/2491125798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908d66b690 con 0x55908d66b860 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86effff640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 0x55908d68ddf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b470 tx=0x7f86e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd029c0cf89e91d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.746+0000 7f86ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ddffb640 1 -- 172.21.15.122:0/2491125798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e001e040 con 0x55908d66b860 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ddffb640 1 -- 172.21.15.122:0/2491125798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f86e0017360 con 0x55908d66b860 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0041070 con 0x55908d66b860 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ddffb640 1 -- 172.21.15.122:0/2491125798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86e009f020 con 0x55908d66b860 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 10 client.17463 fetch_fsmap learned FSMap version 30 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55908d6f5020 con 0x55908d66b860 2024-09-12T02:00:12.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ddffb640 1 -- 172.21.15.122:0/2491125798 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0041070 con 0x55908d66b860 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ddffb640 1 -- 172.21.15.122:0/2491125798 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f86e0010ca0 con 0x55908d66b860 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 10 client.17463 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 10 client.17463 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 20 client.17463 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d6f56f0 0x55908d6f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 4 client.17463 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "327dfbb5", "release": true, "await": true}] 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "327dfbb5", "release": true, "await": true}) -- 0x55908d5d80a0 con 0x55908d6f56f0 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.747+0000 7f86f659c640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d6f56f0 0x55908d6f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.748+0000 7f86ddffb640 10 client.17463 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:12.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.748+0000 7f86f659c640 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d6f56f0 0x55908d6f7b10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d9c74257edb97c server_cookie=55d5356569f5be7b in_seq=0 out_seq=0 2024-09-12T02:00:12.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ddffb640 1 -- 172.21.15.122:0/2491125798 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55908d5d80a0 con 0x55908d6f56f0 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ddffb640 10 client.17463 handle_command_reply: tid=0 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 client.17463 shutdown 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 msgr2=0x55908d68ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 0x55908d68ddf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b470 tx=0x7f86e00046c0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 shutdown_connections 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55908d6f56f0 0x55908d6f7b10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55908d67a650 0x55908d6f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55908d66b860 0x55908d68ddf0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ddffb640 0 client.17463 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 --2- 172.21.15.122:0/2491125798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55908d66af40 0x55908d68d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 >> 172.21.15.122:0/2491125798 conn(0x55908d6675f0 msgr2=0x55908d685e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 shutdown_connections 2024-09-12T02:00:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.755+0000 7f86ff3d0740 1 -- 172.21.15.122:0/2491125798 wait complete. 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86ff3d0740 20 client.17463 trim_cache size 0 max 16384 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 30, 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 52, 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "327dfbb5": { 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 52, 2024-09-12T02:00:12.757 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:00:12.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86e40a2b80 msgr2=0x7f86e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86e40a2b80 0x7f86e40a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f86e800bbe0 tx=0x7f86e800b390 comp rx=0 tx=0).stop 2024-09-12T02:00:12.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 msgr2=0x7f86f8199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f8199850 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b4a0 tx=0x7f86e000b960 comp rx=0 tx=0).stop 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 shutdown_connections 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f86e40a2b80 0x7f86e40a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8106a80 0x7f86f81a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86f8101420 0x7f86f8199850 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 --2- 172.21.15.122:0/457769401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86f8100a80 0x7f86f8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 >> 172.21.15.122:0/457769401 conn(0x7f86f80fc7a0 msgr2=0x7f86f80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 shutdown_connections 2024-09-12T02:00:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:12.756+0000 7f86fcff7640 1 -- 172.21.15.122:0/457769401 wait complete. 2024-09-12T02:00:12.865 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 630 sec 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: 0 cleanup 630 sec 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: Operation Count AvgLat MaxLat 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: ---------------------------------------- 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: NTCreateX 193641 1.404 26740.300 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: Close 142234 0.002 0.108 2024-09-12T02:00:12.945 INFO:tasks.workunit.client.0.smithi122.stdout: Rename 8202 6.506 19254.544 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Unlink 39118 2.902 22567.224 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Deltree 4 76.396 117.393 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Mkdir 2 0.008 0.011 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Qpathinfo 175531 0.614 140.994 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Qfileinfo 30728 0.001 0.041 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Qfsinfo 32186 0.126 5.368 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Sfileinfo 15792 0.020 16.796 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Find 67860 0.068 49.651 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: WriteX 96385 0.325 29954.656 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: ReadX 303527 0.015 89.461 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: LockX 630 0.418 1.311 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: UnlockX 630 0.407 3.144 2024-09-12T02:00:12.946 INFO:tasks.workunit.client.0.smithi122.stdout: Flush 13584 2.565 154.403 2024-09-12T02:00:12.947 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-09-12T02:00:12.947 INFO:tasks.workunit.client.0.smithi122.stdout:Throughput 10.1235 MB/sec 1 clients 1 procs max_latency=26740.305 ms 2024-09-12T02:00:12.947 INFO:tasks.workunit.client.0.smithi122.stderr:+ dbench 10 2024-09-12T02:00:12.947 INFO:tasks.workunit.client.0.smithi122.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-12T02:00:12.948 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-09-12T02:00:12.948 INFO:tasks.workunit.client.0.smithi122.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-12T02:00:12.948 INFO:tasks.workunit.client.0.smithi122.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-12T02:00:13.046 INFO:tasks.workunit.client.0.smithi122.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-12T02:00:13.047 INFO:tasks.workunit.client.0.smithi122.stdout:releasing clients 2024-09-12T02:00:13.483 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '327dfbb5', total seconds elapsed: 33.4 2024-09-12T02:00:13.483 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T02:00:13.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:00:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:13 smithi122 ceph-mon[30925]: pgmap v747: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.0 MiB/s wr, 90 op/s 2024-09-12T02:00:14.046 INFO:tasks.workunit.client.0.smithi122.stdout: 10 420 148.40 MB/sec warmup 1 sec latency 60.917 ms 2024-09-12T02:00:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:13 smithi151 ceph-mon[35229]: pgmap v747: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.0 MiB/s wr, 90 op/s 2024-09-12T02:00:14.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:13 smithi183 ceph-mon[35127]: pgmap v747: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.0 MiB/s wr, 90 op/s 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 -- 172.21.15.122:0/2469175717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c103740 msgr2=0x7f0d4c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 --2- 172.21.15.122:0/2469175717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c103740 0x7f0d4c107b80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009930 tx=0x7f0d4002a420 comp rx=0 tx=0).stop 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 -- 172.21.15.122:0/2469175717 shutdown_connections 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 --2- 172.21.15.122:0/2469175717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4c108290 0x7f0d4c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 --2- 172.21.15.122:0/2469175717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c103740 0x7f0d4c107b80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 --2- 172.21.15.122:0/2469175717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4c102d90 0x7f0d4c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 -- 172.21.15.122:0/2469175717 >> 172.21.15.122:0/2469175717 conn(0x7f0d4c0fc7d0 msgr2=0x7f0d4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 -- 172.21.15.122:0/2469175717 shutdown_connections 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.251+0000 7f0d52854640 1 -- 172.21.15.122:0/2469175717 wait complete. 2024-09-12T02:00:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.252+0000 7f0d52854640 1 Processor -- start 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.252+0000 7f0d52854640 1 -- start start 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4c102d90 0x7f0d4c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4c103740 0x7f0d4c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 0x7f0d4c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c112090 con 0x7f0d4c108290 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111f10 con 0x7f0d4c102d90 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c112210 con 0x7f0d4c103740 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 0x7f0d4c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 0x7f0d4c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56944/0 (socket says 172.21.15.122:56944) 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 -- 172.21.15.122:0/3530213935 learned_addr learned my addr 172.21.15.122:0/3530213935 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d4bfff640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4c102d90 0x7f0d4c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d4b7fe640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4c103740 0x7f0d4c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 -- 172.21.15.122:0/3530213935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4c103740 msgr2=0x7f0d4c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4c103740 0x7f0d4c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 -- 172.21.15.122:0/3530213935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4c102d90 msgr2=0x7f0d4c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4c102d90 0x7f0d4c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 -- 172.21.15.122:0/3530213935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c19ad30 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d50df1640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 0x7f0d4c19a600 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009fd0 tx=0x7f0d4400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44017070 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c19b020 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d44003e50 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440106c0 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.253+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a7310 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.254+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c103170 con 0x7f0d4c108290 2024-09-12T02:00:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.256+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0d44003970 con 0x7f0d4c108290 2024-09-12T02:00:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.256+0000 7f0d497fa640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0d340a2b80 0x7f0d340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.256+0000 7f0d4bfff640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0d340a2b80 0x7f0d340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.256+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0d440ced40 con 0x7f0d4c108290 2024-09-12T02:00:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.257+0000 7f0d4bfff640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0d340a2b80 0x7f0d340a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005d00 tx=0x7f0d3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.261+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d4409b610 con 0x7f0d4c108290 2024-09-12T02:00:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.494+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4c10c2b0 con 0x7f0d4c108290 2024-09-12T02:00:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.495+0000 7f0d497fa640 1 -- 172.21.15.122:0/3530213935 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11931 (secure 0 0 0) 0x7f0d4409f910 con 0x7f0d4c108290 2024-09-12T02:00:14.495 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:00:14.495 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"btime":"2024-09-12T01:50:11:604468+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T01:50:11.183697+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T02:00:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0d340a2b80 msgr2=0x7f0d340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0d340a2b80 0x7f0d340a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005d00 tx=0x7f0d3c00b410 comp rx=0 tx=0).stop 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 msgr2=0x7f0d4c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 0x7f0d4c19a600 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009fd0 tx=0x7f0d4400b260 comp rx=0 tx=0).stop 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 shutdown_connections 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0d340a2b80 0x7f0d340a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c108290 0x7f0d4c19a600 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4c103740 0x7f0d4c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 --2- 172.21.15.122:0/3530213935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4c102d90 0x7f0d4c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 >> 172.21.15.122:0/3530213935 conn(0x7f0d4c0fc7d0 msgr2=0x7f0d4c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 shutdown_connections 2024-09-12T02:00:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:14.497+0000 7f0d52854640 1 -- 172.21.15.122:0/3530213935 wait complete. 2024-09-12T02:00:14.899 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:14 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3530213935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 -- 172.21.15.122:0/2222236932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c100f80 msgr2=0x7f954c101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 --2- 172.21.15.122:0/2222236932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c100f80 0x7f954c101380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9540009860 tx=0x7f954002a3d0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 -- 172.21.15.122:0/2222236932 shutdown_connections 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 --2- 172.21.15.122:0/2222236932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c102650 0x7f954c108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 --2- 172.21.15.122:0/2222236932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f954c101c90 0x7f954c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 --2- 172.21.15.122:0/2222236932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c100f80 0x7f954c101380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 -- 172.21.15.122:0/2222236932 >> 172.21.15.122:0/2222236932 conn(0x7f954c0fb940 msgr2=0x7f954c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 -- 172.21.15.122:0/2222236932 shutdown_connections 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.927+0000 7f9552d11640 1 -- 172.21.15.122:0/2222236932 wait complete. 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 Processor -- start 2024-09-12T02:00:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 -- start start 2024-09-12T02:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c100f80 0x7f954c195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c101c90 0x7f954c196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f954c102650 0x7f954c19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10b5e0 con 0x7f954c100f80 2024-09-12T02:00:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10b460 con 0x7f954c102650 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9552d11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10b760 con 0x7f954c101c90 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f954bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c101c90 0x7f954c196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f9550aad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c100f80 0x7f954c195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f954bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c101c90 0x7f954c196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:40896/0 (socket says 172.21.15.122:40896) 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f954bfff640 1 -- 172.21.15.122:0/3095616844 learned_addr learned my addr 172.21.15.122:0/3095616844 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.928+0000 7f95512ae640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f954c102650 0x7f954c19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9550aad640 1 -- 172.21.15.122:0/3095616844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c101c90 msgr2=0x7f954c196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9550aad640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c101c90 0x7f954c196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9550aad640 1 -- 172.21.15.122:0/3095616844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f954c102650 msgr2=0x7f954c19d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9550aad640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f954c102650 0x7f954c19d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9550aad640 1 -- 172.21.15.122:0/3095616844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c19d830 con 0x7f954c100f80 2024-09-12T02:00:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9550aad640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c100f80 0x7f954c195c20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f954002a8f0 tx=0x7f95400306c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9549ffb640 1 -- 172.21.15.122:0/3095616844 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540005460 con 0x7f954c100f80 2024-09-12T02:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9549ffb640 1 -- 172.21.15.122:0/3095616844 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9540005600 con 0x7f954c100f80 2024-09-12T02:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c19dac0 con 0x7f954c100f80 2024-09-12T02:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9549ffb640 1 -- 172.21.15.122:0/3095616844 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954002f570 con 0x7f954c100f80 2024-09-12T02:00:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.929+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c19dfb0 con 0x7f954c100f80 2024-09-12T02:00:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.932+0000 7f9549ffb640 1 -- 172.21.15.122:0/3095616844 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9540039040 con 0x7f954c100f80 2024-09-12T02:00:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.932+0000 7f9555aa5740 1 Processor -- start 2024-09-12T02:00:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.932+0000 7f9555aa5740 1 -- start start 2024-09-12T02:00:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 0x55d9c751bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751c2e0 0x55d9c7520da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c7521370 0x55d9c7523830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9555aa5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d9c747f2f0 con 0x55d9c751b9a0 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9555aa5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d9c7427270 con 0x55d9c751c2e0 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9555aa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9c747b2b0 con 0x55d9c7521370 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 0x55d9c751bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 0x55d9c751bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56976/0 (socket says 172.21.15.122:56976) 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 -- 172.21.15.122:0/3004266236 learned_addr learned my addr 172.21.15.122:0/3004266236 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9550aad640 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751c2e0 0x55d9c7520da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f954bfff640 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c7521370 0x55d9c7523830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9549ffb640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f953c0a2c30 0x7f953c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 -- 172.21.15.122:0/3004266236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c7521370 msgr2=0x55d9c7523830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c7521370 0x55d9c7523830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 -- 172.21.15.122:0/3004266236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751c2e0 msgr2=0x55d9c7520da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751c2e0 0x55d9c7520da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 -- 172.21.15.122:0/3004266236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c7426340 con 0x55d9c751b9a0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9549ffb640 1 -- 172.21.15.122:0/3095616844 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f95400f03c0 con 0x7f954c100f80 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f953c0a2c30 0x7f953c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f95512ae640 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 0x55d9c751bda0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f954400e7d0 tx=0x7f954400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1db8b9401ec20d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.933+0000 7f9536ffd640 1 -- 172.21.15.122:0/3004266236 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954401ccc0 con 0x55d9c751b9a0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9536ffd640 1 -- 172.21.15.122:0/3004266236 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544027900 con 0x55d9c751b9a0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9536ffd640 1 -- 172.21.15.122:0/3004266236 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544025640 con 0x55d9c751b9a0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f95512ae640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f953c0a2c30 0x7f953c0a50f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9544027000 tx=0x7f9544024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 -- 172.21.15.122:0/3004266236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 msgr2=0x55d9c751bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 0x55d9c751bda0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f954400e7d0 tx=0x7f954400ec90 comp rx=0 tx=0).stop 2024-09-12T02:00:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 -- 172.21.15.122:0/3004266236 shutdown_connections 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c7521370 0x55d9c7523830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751c2e0 0x55d9c7520da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 --2- 172.21.15.122:0/3004266236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751b9a0 0x55d9c751bda0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 -- 172.21.15.122:0/3004266236 >> 172.21.15.122:0/3004266236 conn(0x55d9c7517010 msgr2=0x55d9c7517410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 -- 172.21.15.122:0/3004266236 shutdown_connections 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.934+0000 7f9555aa5740 1 -- 172.21.15.122:0/3004266236 wait complete. 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 -- start start 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 Processor -- start 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c7525ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c75260e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d9c7524340 con 0x55d9c751c2e0 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d9c7408290 con 0x55d9c7521370 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9c75244c0 con 0x55d9c751b9a0 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f954bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c75260e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f954bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c75260e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56978/0 (socket says 172.21.15.122:56978) 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f954bfff640 1 -- 172.21.15.122:0/4178195229 learned_addr learned my addr 172.21.15.122:0/4178195229 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9550aad640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c7525ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f95512ae640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.935+0000 7f9555aa5740 10 client.0 _ll_register_callbacks cb 0x55d9c7439140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f954bfff640 1 -- 172.21.15.122:0/4178195229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 msgr2=0x55d9c7525ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f954bfff640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c7525ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f954bfff640 1 -- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 msgr2=0x55d9c752d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f954bfff640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f954bfff640 1 -- 172.21.15.122:0/4178195229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c7192210 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f95512ae640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f954bfff640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c75260e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9538013800 tx=0x7f9538013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561eb9ea9e07e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f952bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f952bfff640 1 -- 172.21.15.122:0/4178195229 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953801b070 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f952bfff640 1 -- 172.21.15.122:0/4178195229 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f9538004460 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9544027900 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f952bfff640 1 -- 172.21.15.122:0/4178195229 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9538027050 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f9555aa5740 10 client.17481 fetch_fsmap learned FSMap version 31 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.936+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d9c725e620 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f952bfff640 1 -- 172.21.15.122:0/4178195229 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544027900 con 0x55d9c751c2e0 2024-09-12T02:00:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f952bfff640 1 -- 172.21.15.122:0/4178195229 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9538003b50 con 0x55d9c751c2e0 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9555aa5740 10 client.17481 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9555aa5740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9555aa5740 20 client.17481 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9555aa5740 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c7531c20 0x55d9c7534010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9555aa5740 4 client.17481 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c748ca20 con 0x55d9c7531c20 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9550aad640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c7531c20 0x55d9c7534010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f952bfff640 10 client.17481 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.937+0000 7f9550aad640 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c7531c20 0x55d9c7534010 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e5ea84f3794476 server_cookie=3f7361826240bb9b in_seq=0 out_seq=0 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f952bfff640 1 -- 172.21.15.122:0/4178195229 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9c748ca20 con 0x55d9c7531c20 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f952bfff640 10 client.17481 handle_command_reply: tid=0 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 client.17481 shutdown 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 msgr2=0x55d9c75260e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c75260e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9538013800 tx=0x7f9538013cc0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 shutdown_connections 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c7531c20 0x55d9c7534010 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f952bfff640 0 client.17481 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c75260e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 --2- 172.21.15.122:0/4178195229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c7525ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.938+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 >> 172.21.15.122:0/4178195229 conn(0x55d9c7517010 msgr2=0x55d9c7518820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.939+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 shutdown_connections 2024-09-12T02:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.939+0000 7f9555aa5740 1 -- 172.21.15.122:0/4178195229 wait complete. 2024-09-12T02:00:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.939+0000 7f9555aa5740 20 client.17481 trim_cache size 0 max 16384 2024-09-12T02:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.955+0000 7f9555aa5740 1 Processor -- start 2024-09-12T02:00:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.955+0000 7f9555aa5740 1 -- start start 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751b9a0 0x55d9c75160b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751c2e0 0x55d9c752d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 0x55d9c7518400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9555aa5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1a9610 con 0x55d9c7521370 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9555aa5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1a6e80 con 0x55d9c751b9a0 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9555aa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1a4cb0 con 0x55d9c751c2e0 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 0x55d9c7518400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 0x55d9c7518400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56982/0 (socket says 172.21.15.122:56982) 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 -- 172.21.15.122:0/1912628596 learned_addr learned my addr 172.21.15.122:0/1912628596 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f954bfff640 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751c2e0 0x55d9c752d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f9550aad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751b9a0 0x55d9c75160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 -- 172.21.15.122:0/1912628596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751c2e0 msgr2=0x55d9c752d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751c2e0 0x55d9c752d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 -- 172.21.15.122:0/1912628596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751b9a0 msgr2=0x55d9c75160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751b9a0 0x55d9c75160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 -- 172.21.15.122:0/1912628596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c75301a0 con 0x55d9c7521370 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95512ae640 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 0x55d9c7518400 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9544015fd0 tx=0x7f954400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e52dd8a4b91b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.956+0000 7f95367fc640 1 -- 172.21.15.122:0/1912628596 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544034030 con 0x55d9c7521370 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.957+0000 7f95367fc640 1 -- 172.21.15.122:0/1912628596 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544009d10 con 0x55d9c7521370 2024-09-12T02:00:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.957+0000 7f95367fc640 1 -- 172.21.15.122:0/1912628596 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544027680 con 0x55d9c7521370 2024-09-12T02:00:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.957+0000 7f9555aa5740 1 -- 172.21.15.122:0/1912628596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 msgr2=0x55d9c7518400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.957+0000 7f9555aa5740 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 0x55d9c7518400 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9544015fd0 tx=0x7f954400eef0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.957+0000 7f9555aa5740 1 -- 172.21.15.122:0/1912628596 shutdown_connections 2024-09-12T02:00:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c7521370 0x55d9c7518400 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751c2e0 0x55d9c752d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 --2- 172.21.15.122:0/1912628596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c751b9a0 0x55d9c75160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- 172.21.15.122:0/1912628596 >> 172.21.15.122:0/1912628596 conn(0x55d9c7517010 msgr2=0x55d9c75159a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- 172.21.15.122:0/1912628596 shutdown_connections 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- 172.21.15.122:0/1912628596 wait complete. 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- start start 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 Processor -- start 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c752aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c752afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1a5340 con 0x55d9c751c2e0 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1a67b0 con 0x55d9c7521370 2024-09-12T02:00:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.958+0000 7f9555aa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f954c1a5a10 con 0x55d9c751b9a0 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9555aa5740 10 client.0 _ll_register_callbacks cb 0x7f953800bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f954bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c752afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c752aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c752aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:40938/0 (socket says 172.21.15.122:40938) 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 -- 172.21.15.122:0/543738734 learned_addr learned my addr 172.21.15.122:0/543738734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f95512ae640 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 -- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 msgr2=0x55d9c752bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 -- 172.21.15.122:0/543738734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 msgr2=0x55d9c752afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c752afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 -- 172.21.15.122:0/543738734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c752c600 con 0x55d9c751b9a0 2024-09-12T02:00:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9550aad640 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c752aaa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95400b8ff0 tx=0x7f9540031b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe0a2848a1b26ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f952bfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f952bfff640 1 -- 172.21.15.122:0/543738734 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95400bc760 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f952bfff640 1 -- 172.21.15.122:0/543738734 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f95400b9200 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.959+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9544009d10 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f952bfff640 1 -- 172.21.15.122:0/543738734 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95400bad60 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 10 client.26572 fetch_fsmap learned FSMap version 31 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d9c759c940 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f952bfff640 1 -- 172.21.15.122:0/543738734 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544009d10 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f952bfff640 1 -- 172.21.15.122:0/543738734 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f95400f3c70 con 0x55d9c751b9a0 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 10 client.26572 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 10 client.26572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 20 client.26572 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c759d250 0x55d9c759f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 4 client.26572 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:00:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9c7486d70 con 0x55d9c759d250 2024-09-12T02:00:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.960+0000 7f954bfff640 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c759d250 0x55d9c759f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.961+0000 7f952bfff640 10 client.26572 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:14.961+0000 7f954bfff640 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c759d250 0x55d9c759f670 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4bdd9a20575f7 server_cookie=8344183a4a1700f4 in_seq=0 out_seq=0 2024-09-12T02:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.028+0000 7f952bfff640 1 -- 172.21.15.122:0/543738734 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9c7486d70 con 0x55d9c759d250 2024-09-12T02:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.028+0000 7f952bfff640 10 client.26572 handle_command_reply: tid=0 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 client.26572 shutdown 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 msgr2=0x55d9c752aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c752aaa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95400b8ff0 tx=0x7f9540031b90 comp rx=0 tx=0).stop 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 shutdown_connections 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d9c759d250 0x55d9c759f670 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9c7521370 0x55d9c752bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9c751c2e0 0x55d9c752afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 --2- 172.21.15.122:0/543738734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9c751b9a0 0x55d9c752aaa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f952bfff640 0 client.26572 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 >> 172.21.15.122:0/543738734 conn(0x55d9c7517010 msgr2=0x55d9c75161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 shutdown_connections 2024-09-12T02:00:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 1 -- 172.21.15.122:0/543738734 wait complete. 2024-09-12T02:00:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9555aa5740 20 client.26572 trim_cache size 0 max 16384 2024-09-12T02:00:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:00:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:00:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:00:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f953c0a2c30 msgr2=0x7f953c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9552d11640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f953c0a2c30 0x7f953c0a50f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9544027000 tx=0x7f9544024000 comp rx=0 tx=0).stop 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c100f80 msgr2=0x7f954c195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.029+0000 7f9552d11640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c100f80 0x7f954c195c20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f954002a8f0 tx=0x7f95400306c0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 shutdown_connections 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f953c0a2c30 0x7f953c0a50f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f954c102650 0x7f954c19d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f954c101c90 0x7f954c196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 --2- 172.21.15.122:0/3095616844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f954c100f80 0x7f954c195c20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 >> 172.21.15.122:0/3095616844 conn(0x7f954c0fb940 msgr2=0x7f954c104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 shutdown_connections 2024-09-12T02:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:15.030+0000 7f9552d11640 1 -- 172.21.15.122:0/3095616844 wait complete. 2024-09-12T02:00:15.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 909 108.60 MB/sec warmup 2 sec latency 90.795 ms 2024-09-12T02:00:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:14 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3530213935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:15.136 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T02:00:15.137 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T02:00:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:14 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3530213935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:15.446 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:15.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:15 smithi122 ceph-mon[30925]: pgmap v748: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-12T02:00:15.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:15 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:15.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 -- 172.21.15.122:0/1641184535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 msgr2=0x7f435010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 --2- 172.21.15.122:0/1641184535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f435010ea20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4348009ca0 tx=0x7f434802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 -- 172.21.15.122:0/1641184535 shutdown_connections 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 --2- 172.21.15.122:0/1641184535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f435010ea20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 --2- 172.21.15.122:0/1641184535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 --2- 172.21.15.122:0/1641184535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350106b80 0x7f4350106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 -- 172.21.15.122:0/1641184535 >> 172.21.15.122:0/1641184535 conn(0x7f4350077230 msgr2=0x7f43500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 -- 172.21.15.122:0/1641184535 shutdown_connections 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.951+0000 7f4354b5e640 1 -- 172.21.15.122:0/1641184535 wait complete. 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 Processor -- start 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 -- start start 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f43501989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350106b80 0x7f435019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4350112050 con 0x7f4350100aa0 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4350111ed0 con 0x7f4350106b80 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f4354b5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43501121d0 con 0x7f4350100100 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f434e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f43501989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f434e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:40966/0 (socket says 172.21.15.122:40966) 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f43501989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56998/0 (socket says 172.21.15.122:56998) 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f434e59c640 1 -- 172.21.15.122:0/3901008971 learned_addr learned my addr 172.21.15.122:0/3901008971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.952+0000 7f434ed9d640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350106b80 0x7f435019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434e59c640 1 -- 172.21.15.122:0/3901008971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350106b80 msgr2=0x7f435019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434e59c640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350106b80 0x7f435019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434e59c640 1 -- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 msgr2=0x7f43501989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434e59c640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f43501989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434e59c640 1 -- 172.21.15.122:0/3901008971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435019d440 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434dd9b640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f43501989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f434e59c640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350198490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9f0 tx=0x7f434000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f433f7fe640 1 -- 172.21.15.122:0/3901008971 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434000cd80 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435019d730 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f433f7fe640 1 -- 172.21.15.122:0/3901008971 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4340010910 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f433f7fe640 1 -- 172.21.15.122:0/3901008971 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340018990 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.953+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501a5010 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f433f7fe640 1 -- 172.21.15.122:0/3901008971 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4340010ac0 con 0x7f4350100100 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 Processor -- start 2024-09-12T02:00:15.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 -- start start 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea4f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea506810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea509140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea45d2a0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea404b10 con 0x55b5ea506d50 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f4356f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea459260 con 0x55b5ea4f7660 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.956+0000 7f433f7fe640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f43380a2b80 0x7f43380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea506810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea4f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea4f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:40980/0 (socket says 172.21.15.122:40980) 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea506810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57004/0 (socket says 172.21.15.122:57004) 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434ed9d640 1 -- 172.21.15.122:0/556993485 learned_addr learned my addr 172.21.15.122:0/556993485 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:15.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434dd9b640 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea509140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434ed9d640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f43380a2b80 0x7f43380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f433f7fe640 1 -- 172.21.15.122:0/3901008971 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f43400cf390 con 0x7f4350100100 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 -- 172.21.15.122:0/556993485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 msgr2=0x55b5ea4f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea4f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 -- 172.21.15.122:0/556993485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 msgr2=0x55b5ea509140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea509140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 -- 172.21.15.122:0/556993485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ea1ceb90 con 0x55b5ea4f7f80 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434e59c640 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea506810 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f434009b960 tx=0x7f434009b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bbc7bcd37f88c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f434ed9d640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f43380a2b80 0x7f43380a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4348009a60 tx=0x7f4348004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f433cff9640 1 -- 172.21.15.122:0/556993485 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340067510 con 0x55b5ea4f7f80 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f433cff9640 1 -- 172.21.15.122:0/556993485 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434009bc60 con 0x55b5ea4f7f80 2024-09-12T02:00:15.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.957+0000 7f433cff9640 1 -- 172.21.15.122:0/556993485 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340014ea0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 -- 172.21.15.122:0/556993485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 msgr2=0x55b5ea506810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea506810 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f434009b960 tx=0x7f434009b5c0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 -- 172.21.15.122:0/556993485 shutdown_connections 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea509140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea506810 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 --2- 172.21.15.122:0/556993485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea4f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 -- 172.21.15.122:0/556993485 >> 172.21.15.122:0/556993485 conn(0x55b5ea4f3cf0 msgr2=0x55b5ea4f40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 -- 172.21.15.122:0/556993485 shutdown_connections 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.958+0000 7f4356f37740 1 -- 172.21.15.122:0/556993485 wait complete. 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 -- start start 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 Processor -- start 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 0x55b5ea4ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7f80 0x55b5ea5003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea504740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea4fa580 con 0x55b5ea4f7660 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea3e56e0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b5ea4fa700 con 0x55b5ea506d50 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f4356f37740 10 client.0 _ll_register_callbacks cb 0x55b5ea409a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 0x55b5ea4ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 0x55b5ea4ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57006/0 (socket says 172.21.15.122:57006) 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434e59c640 1 -- 172.21.15.122:0/1720746853 learned_addr learned my addr 172.21.15.122:0/1720746853 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434dd9b640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7f80 0x55b5ea5003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434ed9d640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea504740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434e59c640 1 -- 172.21.15.122:0/1720746853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 msgr2=0x55b5ea504740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434e59c640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea504740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.959+0000 7f434e59c640 1 -- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7f80 msgr2=0x55b5ea5003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f434e59c640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7f80 0x55b5ea5003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f434e59c640 1 -- 172.21.15.122:0/1720746853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ea500cf0 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f434dd9b640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7f80 0x55b5ea5003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f434ed9d640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea504740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f434e59c640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 0x55b5ea4ffeb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f434000cfd0 tx=0x7f4340098490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff05e677bca7de6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4335ffb640 1 -- 172.21.15.122:0/1720746853 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400b2020 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4335ffb640 1 -- 172.21.15.122:0/1720746853 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f43400a45a0 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434009bc60 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4335ffb640 1 -- 172.21.15.122:0/1720746853 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f434006f070 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4356f37740 10 client.17499 fetch_fsmap learned FSMap version 31 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b5ea4fa580 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4335ffb640 1 -- 172.21.15.122:0/1720746853 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434009bc60 con 0x55b5ea4f7660 2024-09-12T02:00:15.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4335ffb640 1 -- 172.21.15.122:0/1720746853 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4340018b30 con 0x55b5ea4f7660 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4356f37740 10 client.17499 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.960+0000 7f4356f37740 10 client.17499 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f4356f37740 20 client.17499 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f4356f37740 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea512600 0x55b5ea5149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f4356f37740 4 client.17499 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5ea46a9d0 con 0x55b5ea512600 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f434dd9b640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea512600 0x55b5ea5149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f434dd9b640 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea512600 0x55b5ea5149f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcb450dad1953a server_cookie=cc27cce6615e9008 in_seq=0 out_seq=0 2024-09-12T02:00:15.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.961+0000 7f4335ffb640 10 client.17499 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:15.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4335ffb640 1 -- 172.21.15.122:0/1720746853 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5ea46a9d0 con 0x55b5ea512600 2024-09-12T02:00:15.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4335ffb640 10 client.17499 handle_command_reply: tid=0 2024-09-12T02:00:15.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 client.17499 shutdown 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 msgr2=0x55b5ea4ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 0x55b5ea4ffeb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f434000cfd0 tx=0x7f4340098490 comp rx=0 tx=0).stop 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 shutdown_connections 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea512600 0x55b5ea5149f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea504740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7f80 0x55b5ea5003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 --2- 172.21.15.122:0/1720746853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7660 0x55b5ea4ffeb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 >> 172.21.15.122:0/1720746853 conn(0x55b5ea4f3cf0 msgr2=0x55b5ea4f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.963+0000 7f4335ffb640 0 client.17499 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.964+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 shutdown_connections 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.964+0000 7f4356f37740 1 -- 172.21.15.122:0/1720746853 wait complete. 2024-09-12T02:00:15.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.964+0000 7f4356f37740 20 client.17499 trim_cache size 0 max 16384 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 Processor -- start 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 -- start start 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7660 0x55b5ea504730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea4f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea51a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43501adcb0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43501ab520 con 0x55b5ea4f7660 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f4356f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43501a9350 con 0x55b5ea506d50 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea4f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea4f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57014/0 (socket says 172.21.15.122:57014) 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f434dd9b640 1 -- 172.21.15.122:0/4269019738 learned_addr learned my addr 172.21.15.122:0/4269019738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f434e59c640 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7660 0x55b5ea504730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.983+0000 7f434ed9d640 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea51a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f434dd9b640 1 -- 172.21.15.122:0/4269019738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 msgr2=0x55b5ea51a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f434dd9b640 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea51a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f434dd9b640 1 -- 172.21.15.122:0/4269019738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7660 msgr2=0x55b5ea504730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f434dd9b640 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7660 0x55b5ea504730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f434dd9b640 1 -- 172.21.15.122:0/4269019738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ea4f3b20 con 0x55b5ea4f7f80 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f434dd9b640 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea4f2d60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x55b5ea4f9200 tx=0x7f434400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79463ca000f979 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f4337fff640 1 -- 172.21.15.122:0/4269019738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434402b040 con 0x55b5ea4f7f80 2024-09-12T02:00:15.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f4337fff640 1 -- 172.21.15.122:0/4269019738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434400ed10 con 0x55b5ea4f7f80 2024-09-12T02:00:15.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.984+0000 7f4337fff640 1 -- 172.21.15.122:0/4269019738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344025ad0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 -- 172.21.15.122:0/4269019738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 msgr2=0x55b5ea4f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:15.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea4f2d60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x55b5ea4f9200 tx=0x7f434400ebe0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 -- 172.21.15.122:0/4269019738 shutdown_connections 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea506d50 0x55b5ea51a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea4f2d60 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 --2- 172.21.15.122:0/4269019738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea4f7660 0x55b5ea504730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 -- 172.21.15.122:0/4269019738 >> 172.21.15.122:0/4269019738 conn(0x55b5ea4f3cf0 msgr2=0x55b5ea4f2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 -- 172.21.15.122:0/4269019738 shutdown_connections 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 -- 172.21.15.122:0/4269019738 wait complete. 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.985+0000 7f4356f37740 1 -- start start 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 Processor -- start 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea50aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea50af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea50b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43501a99e0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43501aae50 con 0x55b5ea506d50 2024-09-12T02:00:15.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43501aa0b0 con 0x55b5ea4f7660 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f4356f37740 10 client.0 _ll_register_callbacks cb 0x7f4344005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea50af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea50af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57018/0 (socket says 172.21.15.122:57018) 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 -- 172.21.15.122:0/2741880899 learned_addr learned my addr 172.21.15.122:0/2741880899 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434ed9d640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea50b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434e59c640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea50aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 -- 172.21.15.122:0/2741880899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 msgr2=0x55b5ea50aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea50aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 -- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 msgr2=0x55b5ea50b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea50b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.986+0000 7f434dd9b640 1 -- 172.21.15.122:0/2741880899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ea4ef260 con 0x55b5ea4f7f80 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f434dd9b640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea50af90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f434401b010 tx=0x7f434400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd92cff0acdbe26f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:15.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4335ffb640 1 -- 172.21.15.122:0/2741880899 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344035020 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4335ffb640 1 -- 172.21.15.122:0/2741880899 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f4344004030 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434400ed10 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4335ffb640 1 -- 172.21.15.122:0/2741880899 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f434402b070 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 10 client.17505 fetch_fsmap learned FSMap version 31 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b5ea57d4a0 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4335ffb640 1 -- 172.21.15.122:0/2741880899 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434400ed10 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4335ffb640 1 -- 172.21.15.122:0/2741880899 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f4344004750 con 0x55b5ea4f7f80 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 10 client.17505 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 10 client.17505 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 20 client.17505 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:15.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea57dbd0 0x55b5ea57fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:15.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.987+0000 7f4356f37740 4 client.17505 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T02:00:15.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.988+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55b5ea464d20 con 0x55b5ea57dbd0 2024-09-12T02:00:15.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.988+0000 7f434e59c640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea57dbd0 0x55b5ea57fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:15.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.988+0000 7f4335ffb640 10 client.17505 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:15.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:15.988+0000 7f434e59c640 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea57dbd0 0x55b5ea57fff0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f0b1bdb6890c23 server_cookie=72063be4a674fac1 in_seq=0 out_seq=0 2024-09-12T02:00:16.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1776 91.30 MB/sec warmup 3 sec latency 71.150 ms 2024-09-12T02:00:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:15 smithi151 ceph-mon[35229]: pgmap v748: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-12T02:00:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:15 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:16.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:15 smithi183 ceph-mon[35127]: pgmap v748: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.7 MiB/s wr, 81 op/s 2024-09-12T02:00:16.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:15 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.867+0000 7f4335ffb640 1 -- 172.21.15.122:0/2741880899 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b5ea464d20 con 0x55b5ea57dbd0 2024-09-12T02:00:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.867+0000 7f4335ffb640 10 client.17505 handle_command_reply: tid=0 2024-09-12T02:00:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.867+0000 7f4356f37740 1 client.17505 shutdown 2024-09-12T02:00:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 msgr2=0x55b5ea50af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea50af90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f434401b010 tx=0x7f434400ef40 comp rx=0 tx=0).stop 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 shutdown_connections 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b5ea57dbd0 0x55b5ea57fff0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b5ea506d50 0x55b5ea50b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5ea4f7f80 0x55b5ea50af90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 --2- 172.21.15.122:0/2741880899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5ea4f7660 0x55b5ea50aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 >> 172.21.15.122:0/2741880899 conn(0x55b5ea4f3cf0 msgr2=0x55b5ea4f2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4335ffb640 0 client.17505 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 shutdown_connections 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 1 -- 172.21.15.122:0/2741880899 wait complete. 2024-09-12T02:00:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4356f37740 20 client.17505 trim_cache size 0 max 16384 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 31, 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 54, 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "d4d363ac": { 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 54, 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:00:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:16.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:16.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:16.872 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:00:16.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f43380a2b80 msgr2=0x7f43380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:16.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4354b5e640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f43380a2b80 0x7f43380a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4348009a60 tx=0x7f4348004a80 comp rx=0 tx=0).stop 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 msgr2=0x7f4350198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4354b5e640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350198490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f434000e9f0 tx=0x7f434000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 shutdown_connections 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.868+0000 7f4354b5e640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f43380a2b80 0x7f43380a5040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.869+0000 7f4354b5e640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350106b80 0x7f435019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.869+0000 7f4354b5e640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4350100aa0 0x7f43501989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.869+0000 7f4354b5e640 1 --2- 172.21.15.122:0/3901008971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4350100100 0x7f4350198490 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.869+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 >> 172.21.15.122:0/3901008971 conn(0x7f4350077230 msgr2=0x7f43500fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.869+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 shutdown_connections 2024-09-12T02:00:16.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:16.869+0000 7f4354b5e640 1 -- 172.21.15.122:0/3901008971 wait complete. 2024-09-12T02:00:17.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 68.49 MB/sec warmup 4 sec latency 1071.229 ms 2024-09-12T02:00:17.578 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd4d363ac', quiesce took 4.1 seconds. Will release after: 25.9 2024-09-12T02:00:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:17 smithi122 ceph-mon[30925]: pgmap v749: 97 pgs: 97 active+clean; 299 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-12T02:00:18.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 54.79 MB/sec warmup 5 sec latency 2071.327 ms 2024-09-12T02:00:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:17 smithi151 ceph-mon[35229]: pgmap v749: 97 pgs: 97 active+clean; 299 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-12T02:00:18.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:17 smithi183 ceph-mon[35127]: pgmap v749: 97 pgs: 97 active+clean; 299 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-09-12T02:00:19.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 45.66 MB/sec warmup 6 sec latency 3071.412 ms 2024-09-12T02:00:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:18 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:18 smithi151 ceph-mon[35229]: pgmap v750: 97 pgs: 97 active+clean; 295 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-09-12T02:00:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:18 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:18 smithi183 ceph-mon[35127]: pgmap v750: 97 pgs: 97 active+clean; 295 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-09-12T02:00:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:18 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:00:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:18 smithi122 ceph-mon[30925]: pgmap v750: 97 pgs: 97 active+clean; 295 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-09-12T02:00:20.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 39.14 MB/sec warmup 7 sec latency 4071.511 ms 2024-09-12T02:00:21.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 34.24 MB/sec warmup 8 sec latency 5071.599 ms 2024-09-12T02:00:21.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:21 smithi122 ceph-mon[30925]: pgmap v751: 97 pgs: 97 active+clean; 292 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-09-12T02:00:22.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 30.44 MB/sec warmup 9 sec latency 6071.700 ms 2024-09-12T02:00:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:21 smithi151 ceph-mon[35229]: pgmap v751: 97 pgs: 97 active+clean; 292 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-09-12T02:00:22.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:21 smithi183 ceph-mon[35127]: pgmap v751: 97 pgs: 97 active+clean; 292 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-09-12T02:00:23.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 27.40 MB/sec warmup 10 sec latency 7071.810 ms 2024-09-12T02:00:23.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:23 smithi122 ceph-mon[30925]: pgmap v752: 97 pgs: 97 active+clean; 364 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 8.2 MiB/s wr, 254 op/s 2024-09-12T02:00:24.047 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 24.91 MB/sec warmup 11 sec latency 8071.875 ms 2024-09-12T02:00:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:23 smithi151 ceph-mon[35229]: pgmap v752: 97 pgs: 97 active+clean; 364 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 8.2 MiB/s wr, 254 op/s 2024-09-12T02:00:24.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:23 smithi183 ceph-mon[35127]: pgmap v752: 97 pgs: 97 active+clean; 364 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 8.2 MiB/s wr, 254 op/s 2024-09-12T02:00:25.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 22.83 MB/sec warmup 12 sec latency 9071.996 ms 2024-09-12T02:00:26.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 21.07 MB/sec warmup 13 sec latency 10072.096 ms 2024-09-12T02:00:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:25 smithi151 ceph-mon[35229]: pgmap v753: 97 pgs: 97 active+clean; 414 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 12 MiB/s wr, 254 op/s 2024-09-12T02:00:26.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:25 smithi183 ceph-mon[35127]: pgmap v753: 97 pgs: 97 active+clean; 414 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 12 MiB/s wr, 254 op/s 2024-09-12T02:00:26.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:25 smithi122 ceph-mon[30925]: pgmap v753: 97 pgs: 97 active+clean; 414 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 12 MiB/s wr, 254 op/s 2024-09-12T02:00:27.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 19.57 MB/sec warmup 14 sec latency 11072.217 ms 2024-09-12T02:00:28.005 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:27 smithi183 ceph-mon[35127]: pgmap v754: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 17 MiB/s wr, 256 op/s 2024-09-12T02:00:28.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 18.26 MB/sec warmup 15 sec latency 12072.292 ms 2024-09-12T02:00:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:27 smithi151 ceph-mon[35229]: pgmap v754: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 17 MiB/s wr, 256 op/s 2024-09-12T02:00:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 -- 172.21.15.122:0/1798227591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 msgr2=0x7f680c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 --2- 172.21.15.122:0/1798227591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c100e50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009900 tx=0x7f67fc02a410 comp rx=0 tx=0).stop 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 -- 172.21.15.122:0/1798227591 shutdown_connections 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 --2- 172.21.15.122:0/1798227591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c102600 0x7f680c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 --2- 172.21.15.122:0/1798227591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c101c40 0x7f680c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 --2- 172.21.15.122:0/1798227591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c100e50 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 -- 172.21.15.122:0/1798227591 >> 172.21.15.122:0/1798227591 conn(0x7f680c0fc1b0 msgr2=0x7f680c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 -- 172.21.15.122:0/1798227591 shutdown_connections 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.127+0000 7f6813e65640 1 -- 172.21.15.122:0/1798227591 wait complete. 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 Processor -- start 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 -- start start 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c101c40 0x7f680c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c102600 0x7f680c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f680c10d880 con 0x7f680c100a50 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f680c10d700 con 0x7f680c101c40 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6813e65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f680c10da00 con 0x7f680c102600 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33138/0 (socket says 172.21.15.122:33138) 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 -- 172.21.15.122:0/4173577785 learned_addr learned my addr 172.21.15.122:0/4173577785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811400640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c101c40 0x7f680c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6812402640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c102600 0x7f680c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 -- 172.21.15.122:0/4173577785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c102600 msgr2=0x7f680c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c102600 0x7f680c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 -- 172.21.15.122:0/4173577785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c101c40 msgr2=0x7f680c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c101c40 0x7f680c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.128+0000 7f6811c01640 1 -- 172.21.15.122:0/4173577785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c19fa40 con 0x7f680c100a50 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f6812402640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c102600 0x7f680c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f6811c01640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c197e20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f67fc031440 tx=0x7f67fc032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f680affd640 1 -- 172.21.15.122:0/4173577785 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc03f030 con 0x7f680c100a50 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f680affd640 1 -- 172.21.15.122:0/4173577785 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc003ce0 con 0x7f680c100a50 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c19fcd0 con 0x7f680c100a50 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f680affd640 1 -- 172.21.15.122:0/4173577785 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc035070 con 0x7f680c100a50 2024-09-12T02:00:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.129+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c1a01c0 con 0x7f680c100a50 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.131+0000 7f680affd640 1 -- 172.21.15.122:0/4173577785 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f67fc042030 con 0x7f680c100a50 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 Processor -- start 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 -- start start 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f359fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 0x55c0f35a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c0f35032f0 con 0x55c0f35a5370 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0f34ab270 con 0x55c0f35a02e0 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6816c04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0f34ff2b0 con 0x55c0f359f9a0 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f680affd640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f67f40a2cb0 0x7f67f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6811400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 0x55c0f35a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.132+0000 7f6811400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 0x55c0f35a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33140/0 (socket says 172.21.15.122:33140) 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 -- 172.21.15.122:0/3091107003 learned_addr learned my addr 172.21.15.122:0/3091107003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811c01640 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6812402640 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f359fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6812402640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f67f40a2cb0 0x7f67f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f680affd640 1 -- 172.21.15.122:0/4173577785 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f67fc0ef720 con 0x7f680c100a50 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 -- 172.21.15.122:0/3091107003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 msgr2=0x55c0f359fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f359fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 -- 172.21.15.122:0/3091107003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 msgr2=0x55c0f35a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 -- 172.21.15.122:0/3091107003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f34aa340 con 0x55c0f35a5370 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6811400640 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 0x55c0f35a7830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f680000f390 tx=0x7f680000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8dcae05bcf3bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f6812402640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f67f40a2cb0 0x7f67f40a5170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f680400efd0 tx=0x7f6804005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f67f3fff640 1 -- 172.21.15.122:0/3091107003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800004d20 con 0x55c0f35a5370 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f67f3fff640 1 -- 172.21.15.122:0/3091107003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68000119e0 con 0x55c0f35a5370 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.133+0000 7f67f3fff640 1 -- 172.21.15.122:0/3091107003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800005580 con 0x55c0f35a5370 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 -- 172.21.15.122:0/3091107003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 msgr2=0x55c0f35a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 0x55c0f35a7830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f680000f390 tx=0x7f680000f850 comp rx=0 tx=0).stop 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 -- 172.21.15.122:0/3091107003 shutdown_connections 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a5370 0x55c0f35a7830 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 --2- 172.21.15.122:0/3091107003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f359fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 -- 172.21.15.122:0/3091107003 >> 172.21.15.122:0/3091107003 conn(0x55c0f359b010 msgr2=0x55c0f359b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 -- 172.21.15.122:0/3091107003 shutdown_connections 2024-09-12T02:00:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 -- 172.21.15.122:0/3091107003 wait complete. 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.134+0000 7f6816c04740 1 -- start start 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 Processor -- start 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a02e0 0x55c0f35aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c0f35a8340 con 0x55c0f359f9a0 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0f348c290 con 0x55c0f35a5370 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0f35a84c0 con 0x55c0f35a02e0 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6816c04740 10 client.0 _ll_register_callbacks cb 0x55c0f34bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35a9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33150/0 (socket says 172.21.15.122:33150) 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 -- 172.21.15.122:0/1972930289 learned_addr learned my addr 172.21.15.122:0/1972930289 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6812402640 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 -- 172.21.15.122:0/1972930289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a02e0 msgr2=0x55c0f35aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a02e0 0x55c0f35aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 -- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 msgr2=0x55c0f35b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.135+0000 7f6811c01640 1 -- 172.21.15.122:0/1972930289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f3216210 con 0x55c0f359f9a0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6811c01640 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35a9b60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0bbc80 tx=0x7f67fc08be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49f99b821cf01c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f67f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f67f17fa640 1 -- 172.21.15.122:0/1972930289 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc049040 con 0x55c0f359f9a0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f67f17fa640 1 -- 172.21.15.122:0/1972930289 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f67fc0b85f0 con 0x55c0f359f9a0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68000119e0 con 0x55c0f359f9a0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f67f17fa640 1 -- 172.21.15.122:0/1972930289 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f67fc0bb8e0 con 0x55c0f359f9a0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 10 client.17517 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0f32e2620 con 0x55c0f359f9a0 2024-09-12T02:00:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f67f17fa640 1 -- 172.21.15.122:0/1972930289 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68000119e0 con 0x55c0f359f9a0 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f67f17fa640 1 -- 172.21.15.122:0/1972930289 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f67fc088070 con 0x55c0f359f9a0 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 10 client.17517 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 20 client.17517 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f35b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 4 client.17517 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.136+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f3510a20 con 0x55c0f35b5750 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.137+0000 7f6811400640 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f35b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.137+0000 7f67f17fa640 10 client.17517 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.137+0000 7f6811400640 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f35b7bc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49bd30a720d2da server_cookie=5fd1469d49a26ae9 in_seq=0 out_seq=0 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f67f17fa640 1 -- 172.21.15.122:0/1972930289 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0f3510a20 con 0x55c0f35b5750 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f67f17fa640 10 client.17517 handle_command_reply: tid=0 2024-09-12T02:00:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 client.17517 shutdown 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 msgr2=0x55c0f35a9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35a9b60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0bbc80 tx=0x7f67fc08be30 comp rx=0 tx=0).stop 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 shutdown_connections 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f35b7bc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a02e0 0x55c0f35aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f67f17fa640 0 client.17517 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 --2- 172.21.15.122:0/1972930289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35a9b60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 >> 172.21.15.122:0/1972930289 conn(0x55c0f359b010 msgr2=0x55c0f359c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 shutdown_connections 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 1 -- 172.21.15.122:0/1972930289 wait complete. 2024-09-12T02:00:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.138+0000 7f6816c04740 20 client.17517 trim_cache size 0 max 16384 2024-09-12T02:00:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.152+0000 7f6816c04740 1 Processor -- start 2024-09-12T02:00:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.152+0000 7f6816c04740 1 -- start start 2024-09-12T02:00:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6816c04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6816c04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6816c04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a5370 0x55c0f359bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6816c04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1ab970 con 0x55c0f359f9a0 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6816c04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1a91b0 con 0x55c0f35a02e0 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6816c04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1a6fe0 con 0x55c0f35a5370 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33156/0 (socket says 172.21.15.122:33156) 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 -- 172.21.15.122:0/2907918752 learned_addr learned my addr 172.21.15.122:0/2907918752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811400640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 -- 172.21.15.122:0/2907918752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a5370 msgr2=0x55c0f359bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6812402640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a5370 0x55c0f359bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a5370 0x55c0f359bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 -- 172.21.15.122:0/2907918752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 msgr2=0x55c0f35b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 -- 172.21.15.122:0/2907918752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f35a96a0 con 0x55c0f359f9a0 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6812402640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a5370 0x55c0f359bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811400640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f6811c01640 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35aa020 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f67fc03bfd0 tx=0x7f67fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba975a2b62e71fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f67f37fe640 1 -- 172.21.15.122:0/2907918752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc049070 con 0x55c0f359f9a0 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f67f37fe640 1 -- 172.21.15.122:0/2907918752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc0bc680 con 0x55c0f359f9a0 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.153+0000 7f67f37fe640 1 -- 172.21.15.122:0/2907918752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0c49a0 con 0x55c0f359f9a0 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 -- 172.21.15.122:0/2907918752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 msgr2=0x55c0f35aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35aa020 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f67fc03bfd0 tx=0x7f67fc0045e0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 -- 172.21.15.122:0/2907918752 shutdown_connections 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f35a5370 0x55c0f359bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a02e0 0x55c0f35b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 --2- 172.21.15.122:0/2907918752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f359f9a0 0x55c0f35aa020 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 -- 172.21.15.122:0/2907918752 >> 172.21.15.122:0/2907918752 conn(0x55c0f359b010 msgr2=0x55c0f35a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 -- 172.21.15.122:0/2907918752 shutdown_connections 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 -- 172.21.15.122:0/2907918752 wait complete. 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 -- start start 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.154+0000 7f6816c04740 1 Processor -- start 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f35ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 0x55c0f35aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1a7670 con 0x55c0f35a02e0 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1a8ae0 con 0x55c0f35a5370 2024-09-12T02:00:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f680c1a7d40 con 0x55c0f359f9a0 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 10 client.0 _ll_register_callbacks cb 0x7f67fc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 0x55c0f35aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 0x55c0f35aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33166/0 (socket says 172.21.15.122:33166) 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 -- 172.21.15.122:0/3172040258 learned_addr learned my addr 172.21.15.122:0/3172040258 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811c01640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f35ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6812402640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 -- 172.21.15.122:0/3172040258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 msgr2=0x55c0f35ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f35ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 -- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 msgr2=0x55c0f35afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 -- 172.21.15.122:0/3172040258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f35a8580 con 0x55c0f35a02e0 2024-09-12T02:00:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6811400640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 0x55c0f35aee00 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f680000f390 tx=0x7f6800009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800655d4bfc2dbca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f67f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f67f17fa640 1 -- 172.21.15.122:0/3172040258 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800011e60 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f67f17fa640 1 -- 172.21.15.122:0/3172040258 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f6800004be0 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.155+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67fc0bc680 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f67f17fa640 1 -- 172.21.15.122:0/3172040258 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6800004de0 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 10 client.17523 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0f36204a0 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f67f17fa640 1 -- 172.21.15.122:0/3172040258 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc0bc680 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f67f17fa640 1 -- 172.21.15.122:0/3172040258 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f6800026040 con 0x55c0f35a02e0 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 10 client.17523 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 20 client.17523 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f3622d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 4 client.17523 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:00:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0f350ad70 con 0x55c0f35b5750 2024-09-12T02:00:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.156+0000 7f6811c01640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f3622d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.157+0000 7f67f17fa640 10 client.17523 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.157+0000 7f6811c01640 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f3622d30 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148038918a03253d server_cookie=e842fabae9adeb6f in_seq=0 out_seq=0 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f67f17fa640 1 -- 172.21.15.122:0/3172040258 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0f350ad70 con 0x55c0f35b5750 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f67f17fa640 10 client.17523 handle_command_reply: tid=0 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 client.17523 shutdown 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 msgr2=0x55c0f35aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 0x55c0f35aee00 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f680000f390 tx=0x7f6800009880 comp rx=0 tx=0).stop 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 shutdown_connections 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c0f35b5750 0x55c0f3622d30 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0f35a5370 0x55c0f35afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0f35a02e0 0x55c0f35aee00 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f67f17fa640 0 client.17523 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 --2- 172.21.15.122:0/3172040258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0f359f9a0 0x55c0f35ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 >> 172.21.15.122:0/3172040258 conn(0x55c0f359b010 msgr2=0x55c0f35999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 shutdown_connections 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 1 -- 172.21.15.122:0/3172040258 wait complete. 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.203+0000 7f6816c04740 20 client.17523 trim_cache size 0 max 16384 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:00:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:00:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f67f40a2cb0 msgr2=0x7f67f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f67f40a2cb0 0x7f67f40a5170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f680400efd0 tx=0x7f6804005ed0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 msgr2=0x7f680c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c197e20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f67fc031440 tx=0x7f67fc032e20 comp rx=0 tx=0).stop 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 shutdown_connections 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f67f40a2cb0 0x7f67f40a5170 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680c102600 0x7f680c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f680c101c40 0x7f680c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 --2- 172.21.15.122:0/4173577785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f680c100a50 0x7f680c197e20 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 >> 172.21.15.122:0/4173577785 conn(0x7f680c0fc1b0 msgr2=0x7f680c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 shutdown_connections 2024-09-12T02:00:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:28.204+0000 7f6813e65640 1 -- 172.21.15.122:0/4173577785 wait complete. 2024-09-12T02:00:28.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:27 smithi122 ceph-mon[30925]: pgmap v754: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 17 MiB/s wr, 256 op/s 2024-09-12T02:00:29.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 17.12 MB/sec warmup 16 sec latency 13072.419 ms 2024-09-12T02:00:30.041 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:29 smithi122 ceph-mon[30925]: pgmap v755: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 16 MiB/s wr, 237 op/s 2024-09-12T02:00:30.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 16.11 MB/sec warmup 17 sec latency 14072.468 ms 2024-09-12T02:00:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:29 smithi151 ceph-mon[35229]: pgmap v755: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 16 MiB/s wr, 237 op/s 2024-09-12T02:00:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:29 smithi183 ceph-mon[35127]: pgmap v755: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 16 MiB/s wr, 237 op/s 2024-09-12T02:00:31.048 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 15.22 MB/sec warmup 18 sec latency 15072.536 ms 2024-09-12T02:00:32.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 14.42 MB/sec warmup 19 sec latency 16072.650 ms 2024-09-12T02:00:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:31 smithi151 ceph-mon[35229]: pgmap v756: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 83 op/s 2024-09-12T02:00:32.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:31 smithi183 ceph-mon[35127]: pgmap v756: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 83 op/s 2024-09-12T02:00:32.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:31 smithi122 ceph-mon[30925]: pgmap v756: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 83 op/s 2024-09-12T02:00:33.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 13.70 MB/sec warmup 20 sec latency 17073.083 ms 2024-09-12T02:00:34.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 13.05 MB/sec warmup 21 sec latency 18073.153 ms 2024-09-12T02:00:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:33 smithi151 ceph-mon[35229]: pgmap v757: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 80 op/s 2024-09-12T02:00:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:33 smithi183 ceph-mon[35127]: pgmap v757: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 80 op/s 2024-09-12T02:00:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:33 smithi122 ceph-mon[30925]: pgmap v757: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 80 op/s 2024-09-12T02:00:35.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 12.45 MB/sec warmup 22 sec latency 19073.253 ms 2024-09-12T02:00:36.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 11.91 MB/sec warmup 23 sec latency 20073.360 ms 2024-09-12T02:00:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:35 smithi151 ceph-mon[35229]: pgmap v758: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.9 MiB/s wr, 59 op/s 2024-09-12T02:00:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:35 smithi183 ceph-mon[35127]: pgmap v758: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.9 MiB/s wr, 59 op/s 2024-09-12T02:00:36.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:35 smithi122 ceph-mon[30925]: pgmap v758: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.9 MiB/s wr, 59 op/s 2024-09-12T02:00:37.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 11.41 MB/sec warmup 24 sec latency 21073.464 ms 2024-09-12T02:00:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:36 smithi151 ceph-mon[35229]: pgmap v759: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 41 op/s 2024-09-12T02:00:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:36 smithi183 ceph-mon[35127]: pgmap v759: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 41 op/s 2024-09-12T02:00:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:36 smithi122 ceph-mon[30925]: pgmap v759: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 41 op/s 2024-09-12T02:00:38.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 10.96 MB/sec warmup 25 sec latency 22073.570 ms 2024-09-12T02:00:39.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 10.54 MB/sec warmup 26 sec latency 23073.668 ms 2024-09-12T02:00:40.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 10.15 MB/sec warmup 27 sec latency 24073.768 ms 2024-09-12T02:00:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:39 smithi151 ceph-mon[35229]: pgmap v760: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-09-12T02:00:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:39 smithi183 ceph-mon[35127]: pgmap v760: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-09-12T02:00:40.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:39 smithi122 ceph-mon[30925]: pgmap v760: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-09-12T02:00:41.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 9.78 MB/sec warmup 28 sec latency 25073.872 ms 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 -- 172.21.15.122:0/453832072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc0ffba0 msgr2=0x7fc6bc100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/453832072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc0ffba0 0x7fc6bc100020 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009930 tx=0x7fc6a402a420 comp rx=0 tx=0).stop 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 -- 172.21.15.122:0/453832072 shutdown_connections 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/453832072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6bc100560 0x7fc6bc10a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/453832072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc0ffba0 0x7fc6bc100020 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/453832072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6bc103130 0x7fc6bc103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 -- 172.21.15.122:0/453832072 >> 172.21.15.122:0/453832072 conn(0x7fc6bc0fb970 msgr2=0x7fc6bc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 -- 172.21.15.122:0/453832072 shutdown_connections 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 -- 172.21.15.122:0/453832072 wait complete. 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 Processor -- start 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.305+0000 7fc6c28f1640 1 -- start start 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c28f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6bc0ffba0 0x7fc6bc102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c28f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 0x7fc6bc100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c28f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6bc103130 0x7fc6bc1013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c28f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10d7d0 con 0x7fc6bc100560 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c28f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10d650 con 0x7fc6bc103130 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c28f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10d950 con 0x7fc6bc0ffba0 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 0x7fc6bc100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 0x7fc6bc100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50672/0 (socket says 172.21.15.122:50672) 2024-09-12T02:00:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 -- 172.21.15.122:0/494767780 learned_addr learned my addr 172.21.15.122:0/494767780 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6bc0ffba0 0x7fc6bc102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6bc103130 0x7fc6bc1013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 -- 172.21.15.122:0/494767780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6bc0ffba0 msgr2=0x7fc6bc102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6bc0ffba0 0x7fc6bc102830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 -- 172.21.15.122:0/494767780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6bc103130 msgr2=0x7fc6bc1013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6bc103130 0x7fc6bc1013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.306+0000 7fc6b3fff640 1 -- 172.21.15.122:0/494767780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc101c40 con 0x7fc6bc100560 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.307+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 0x7fc6bc100e70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005ab0 tx=0x7fc6a40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.307+0000 7fc6b9ffb640 1 -- 172.21.15.122:0/494767780 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a403f030 con 0x7fc6bc100560 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.307+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc1a1ee0 con 0x7fc6bc100560 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.307+0000 7fc6b9ffb640 1 -- 172.21.15.122:0/494767780 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a40044d0 con 0x7fc6bc100560 2024-09-12T02:00:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.307+0000 7fc6b9ffb640 1 -- 172.21.15.122:0/494767780 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4035070 con 0x7fc6bc100560 2024-09-12T02:00:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.307+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc1a2350 con 0x7fc6bc100560 2024-09-12T02:00:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 Processor -- start 2024-09-12T02:00:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6b9ffb640 1 -- 172.21.15.122:0/494767780 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc6a4042030 con 0x7fc6bc100560 2024-09-12T02:00:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 -- start start 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 0x55a8891c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a88912b2f0 con 0x55a8891c79a0 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a8890d3270 con 0x55a8891cd370 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c568b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a8891272b0 con 0x55a8891c82e0 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c0e8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 0x55a8891c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c0e8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 0x55a8891c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50684/0 (socket says 172.21.15.122:50684) 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6c0e8e640 1 -- 172.21.15.122:0/2891818820 learned_addr learned my addr 172.21.15.122:0/2891818820 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.310+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6b9ffb640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6a80a2c30 0x7fc6a80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 -- 172.21.15.122:0/2891818820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 msgr2=0x55a8891ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 -- 172.21.15.122:0/2891818820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 msgr2=0x55a8891cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 -- 172.21.15.122:0/2891818820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8890d2340 con 0x55a8891c79a0 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6b9ffb640 1 -- 172.21.15.122:0/494767780 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc6a40eefe0 con 0x7fc6bc100560 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 0x55a8891c7da0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4003a40 tx=0x7fc6b4003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebd09ea3eba5c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6a80a2c30 0x7fc6a80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6b27fc640 1 -- 172.21.15.122:0/2891818820 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4022070 con 0x55a8891c79a0 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6b27fc640 1 -- 172.21.15.122:0/2891818820 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b4015070 con 0x55a8891c79a0 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.311+0000 7fc6b27fc640 1 -- 172.21.15.122:0/2891818820 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40109d0 con 0x55a8891c79a0 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 -- 172.21.15.122:0/2891818820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 msgr2=0x55a8891c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6a80a2c30 0x7fc6a80a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc6b401f000 tx=0x7fc6b4020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 0x55a8891c7da0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4003a40 tx=0x7fc6b4003f00 comp rx=0 tx=0).stop 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 -- 172.21.15.122:0/2891818820 shutdown_connections 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2891818820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c79a0 0x55a8891c7da0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 -- 172.21.15.122:0/2891818820 >> 172.21.15.122:0/2891818820 conn(0x55a8891c3010 msgr2=0x55a8891c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 -- 172.21.15.122:0/2891818820 shutdown_connections 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 -- 172.21.15.122:0/2891818820 wait complete. 2024-09-12T02:00:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.312+0000 7fc6c568b740 1 -- start start 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 Processor -- start 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 0x55a8891d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a8891d0340 con 0x55a8891cd370 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a8890b4290 con 0x55a8891c79a0 2024-09-12T02:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a8891d04c0 con 0x55a8891c82e0 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c568b740 10 client.0 _ll_register_callbacks cb 0x55a8890e5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c0e8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 0x55a8891d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c0e8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 0x55a8891d9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50698/0 (socket says 172.21.15.122:50698) 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6c0e8e640 1 -- 172.21.15.122:0/2562677090 learned_addr learned my addr 172.21.15.122:0/2562677090 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.313+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6b3fff640 1 -- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 msgr2=0x55a8891d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6b3fff640 1 -- 172.21.15.122:0/2562677090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 msgr2=0x55a8891d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 0x55a8891d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6b3fff640 1 -- 172.21.15.122:0/2562677090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a888e3e210 con 0x55a8891c82e0 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 0x55a8891d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891d2140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ecbd0 tx=0x7fc6a403b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b7d0c31cf642ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc69b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc69b7fe640 1 -- 172.21.15.122:0/2562677090 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4049070 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc69b7fe640 1 -- 172.21.15.122:0/2562677090 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc6a40f2ce0 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b4015070 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc69b7fe640 1 -- 172.21.15.122:0/2562677090 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc6a40bb4d0 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6c568b740 10 client.26611 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a888f0a620 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.314+0000 7fc69b7fe640 1 -- 172.21.15.122:0/2562677090 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b4015070 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc69b7fe640 1 -- 172.21.15.122:0/2562677090 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fc6a404c060 con 0x55a8891c82e0 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6c568b740 10 client.26611 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6c568b740 10 client.26611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6c568b740 20 client.26611 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a8891dd810 0x55a8891dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6c568b740 4 client.26611 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a889138a20 con 0x55a8891dd810 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a8891dd810 0x55a8891dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc69b7fe640 10 client.26611 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.315+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a8891dd810 0x55a8891dfc40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4241e3d2e18634 server_cookie=405ae8a21f536d70 in_seq=0 out_seq=0 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc69b7fe640 1 -- 172.21.15.122:0/2562677090 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a889138a20 con 0x55a8891dd810 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc69b7fe640 10 client.26611 handle_command_reply: tid=0 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 client.26611 shutdown 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 msgr2=0x55a8891d2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891d2140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ecbd0 tx=0x7fc6a403b890 comp rx=0 tx=0).stop 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 shutdown_connections 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a8891dd810 0x55a8891dfc40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891cd370 0x55a8891d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc69b7fe640 0 client.26611 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c82e0 0x55a8891d2140 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 --2- 172.21.15.122:0/2562677090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 >> 172.21.15.122:0/2562677090 conn(0x55a8891c3010 msgr2=0x55a8891c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 shutdown_connections 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.316+0000 7fc6c568b740 1 -- 172.21.15.122:0/2562677090 wait complete. 2024-09-12T02:00:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.317+0000 7fc6c568b740 20 client.26611 trim_cache size 0 max 16384 2024-09-12T02:00:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.331+0000 7fc6c568b740 1 Processor -- start 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.331+0000 7fc6c568b740 1 -- start start 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.331+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c79a0 0x55a8891c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6c568b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc1adb90 con 0x55a8891c82e0 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6c568b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc1ab3d0 con 0x55a8891cd370 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6c568b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc1a9200 con 0x55a8891c79a0 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50708/0 (socket says 172.21.15.122:50708) 2024-09-12T02:00:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 -- 172.21.15.122:0/4208255393 learned_addr learned my addr 172.21.15.122:0/4208255393 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c79a0 0x55a8891c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 -- 172.21.15.122:0/4208255393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c79a0 msgr2=0x55a8891c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c79a0 0x55a8891c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 -- 172.21.15.122:0/4208255393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 msgr2=0x55a8891c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 -- 172.21.15.122:0/4208255393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8891d1710 con 0x55a8891c82e0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d9140 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40bb690 tx=0x7fc6a403b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d1c8c54063dd2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.332+0000 7fc6b1ffb640 1 -- 172.21.15.122:0/4208255393 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4049070 con 0x55a8891c82e0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6b1ffb640 1 -- 172.21.15.122:0/4208255393 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a40bfc90 con 0x55a8891c82e0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6b1ffb640 1 -- 172.21.15.122:0/4208255393 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40bb2f0 con 0x55a8891c82e0 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 -- 172.21.15.122:0/4208255393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 msgr2=0x55a8891d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d9140 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40bb690 tx=0x7fc6a403b600 comp rx=0 tx=0).stop 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 -- 172.21.15.122:0/4208255393 shutdown_connections 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891cd370 0x55a8891c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d9140 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 --2- 172.21.15.122:0/4208255393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891c79a0 0x55a8891c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 -- 172.21.15.122:0/4208255393 >> 172.21.15.122:0/4208255393 conn(0x55a8891c3010 msgr2=0x55a8891d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 -- 172.21.15.122:0/4208255393 shutdown_connections 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.333+0000 7fc6c568b740 1 -- 172.21.15.122:0/4208255393 wait complete. 2024-09-12T02:00:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 -- start start 2024-09-12T02:00:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 Processor -- start 2024-09-12T02:00:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891cd370 0x55a8891d8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc1a9890 con 0x55a8891c82e0 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc1aad00 con 0x55a8891c79a0 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc1a9f60 con 0x55a8891cd370 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c568b740 10 client.0 _ll_register_callbacks cb 0x7fc6a4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50724/0 (socket says 172.21.15.122:50724) 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6b3fff640 1 -- 172.21.15.122:0/84798362 learned_addr learned my addr 172.21.15.122:0/84798362 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.334+0000 7fc6c0e8e640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891cd370 0x55a8891d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6b3fff640 1 -- 172.21.15.122:0/84798362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891cd370 msgr2=0x55a8891d8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891cd370 0x55a8891d8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6b3fff640 1 -- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 msgr2=0x55a8891d6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6b3fff640 1 -- 172.21.15.122:0/84798362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8891d8870 con 0x55a8891c82e0 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6b3fff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d7340 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc6a403d040 tx=0x7fc6a403b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542aef854cfbea06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc69b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc69b7fe640 1 -- 172.21.15.122:0/84798362 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40d3020 con 0x55a8891c82e0 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc69b7fe640 1 -- 172.21.15.122:0/84798362 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fc6a40c4260 con 0x55a8891c82e0 2024-09-12T02:00:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a40bfc90 con 0x55a8891c82e0 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc69b7fe640 1 -- 172.21.15.122:0/84798362 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc6a4049070 con 0x55a8891c82e0 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6c568b740 10 client.17541 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a8891d8870 con 0x55a8891c82e0 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.335+0000 7fc69b7fe640 1 -- 172.21.15.122:0/84798362 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a40bfc90 con 0x55a8891c82e0 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc69b7fe640 1 -- 172.21.15.122:0/84798362 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fc6a40f2ce0 con 0x55a8891c82e0 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6c568b740 10 client.17541 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6c568b740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6c568b740 20 client.17541 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6c568b740 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a889248fb0 0x55a88924b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6c568b740 4 client.17541 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:00:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a889132d70 con 0x55a889248fb0 2024-09-12T02:00:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a889248fb0 0x55a88924b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc69b7fe640 10 client.17541 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.336+0000 7fc6bbfff640 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a889248fb0 0x55a88924b3d0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38994036c85aaed server_cookie=df39ea5829ac7c8c in_seq=0 out_seq=0 2024-09-12T02:00:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.355+0000 7fc69b7fe640 1 -- 172.21.15.122:0/84798362 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a889132d70 con 0x55a889248fb0 2024-09-12T02:00:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.355+0000 7fc69b7fe640 10 client.17541 handle_command_reply: tid=0 2024-09-12T02:00:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.355+0000 7fc6c568b740 1 client.17541 shutdown 2024-09-12T02:00:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 msgr2=0x55a8891d7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d7340 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc6a403d040 tx=0x7fc6a403b790 comp rx=0 tx=0).stop 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 shutdown_connections 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a889248fb0 0x55a88924b3d0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8891cd370 0x55a8891d8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8891c82e0 0x55a8891d7340 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 --2- 172.21.15.122:0/84798362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8891c79a0 0x55a8891d6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc69b7fe640 0 client.17541 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 >> 172.21.15.122:0/84798362 conn(0x55a8891c3010 msgr2=0x55a8891c19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 shutdown_connections 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 1 -- 172.21.15.122:0/84798362 wait complete. 2024-09-12T02:00:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c568b740 20 client.17541 trim_cache size 0 max 16384 2024-09-12T02:00:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:00:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:00:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:00:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6a80a2c30 msgr2=0x7fc6a80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6a80a2c30 0x7fc6a80a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc6b401f000 tx=0x7fc6b4020000 comp rx=0 tx=0).stop 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 msgr2=0x7fc6bc100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.356+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 0x7fc6bc100e70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005ab0 tx=0x7fc6a40042e0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 shutdown_connections 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6a80a2c30 0x7fc6a80a50f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6bc103130 0x7fc6bc1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6bc100560 0x7fc6bc100e70 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 --2- 172.21.15.122:0/494767780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6bc0ffba0 0x7fc6bc102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 >> 172.21.15.122:0/494767780 conn(0x7fc6bc0fb970 msgr2=0x7fc6bc106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 shutdown_connections 2024-09-12T02:00:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:41.357+0000 7fc6c28f1640 1 -- 172.21.15.122:0/494767780 wait complete. 2024-09-12T02:00:42.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 9.45 MB/sec warmup 29 sec latency 26073.956 ms 2024-09-12T02:00:42.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:41 smithi151 ceph-mon[35229]: pgmap v761: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 24 op/s 2024-09-12T02:00:42.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:41 smithi183 ceph-mon[35127]: pgmap v761: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 24 op/s 2024-09-12T02:00:42.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:41 smithi122 ceph-mon[30925]: pgmap v761: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 24 op/s 2024-09-12T02:00:43.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 9.13 MB/sec warmup 30 sec latency 27074.083 ms 2024-09-12T02:00:43.479 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:00:43.737 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:43.858 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:43 smithi122 ceph-mon[30925]: pgmap v762: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-09-12T02:00:44.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 8.84 MB/sec warmup 31 sec latency 28074.128 ms 2024-09-12T02:00:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:43 smithi151 ceph-mon[35229]: pgmap v762: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-09-12T02:00:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 -- 172.21.15.122:0/3857152641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c102d30 msgr2=0x7fe32c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3857152641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c102d30 0x7fe32c103110 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009cd0 tx=0x7fe31c02a910 comp rx=0 tx=0).stop 2024-09-12T02:00:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 -- 172.21.15.122:0/3857152641 shutdown_connections 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3857152641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1082a0 0x7fe32c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3857152641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe32c1036e0 0x7fe32c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3857152641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c102d30 0x7fe32c103110 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 -- 172.21.15.122:0/3857152641 >> 172.21.15.122:0/3857152641 conn(0x7fe32c0fc7d0 msgr2=0x7fe32c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 -- 172.21.15.122:0/3857152641 shutdown_connections 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.158+0000 7fe3344ff640 1 -- 172.21.15.122:0/3857152641 wait complete. 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 Processor -- start 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 -- start start 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe32c102d30 0x7fe32c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 0x7fe32c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1082a0 0x7fe32c10de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c112100 con 0x7fe32c1036e0 2024-09-12T02:00:44.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c111f80 con 0x7fe32c102d30 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe3344ff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c112280 con 0x7fe32c1082a0 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe331a9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 0x7fe32c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe331a9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 0x7fe32c1a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50742/0 (socket says 172.21.15.122:50742) 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.159+0000 7fe331a9a640 1 -- 172.21.15.122:0/3798879415 learned_addr learned my addr 172.21.15.122:0/3798879415 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe33229b640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe32c102d30 0x7fe32c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe332a9c640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1082a0 0x7fe32c10de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe331a9a640 1 -- 172.21.15.122:0/3798879415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1082a0 msgr2=0x7fe32c10de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe331a9a640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1082a0 0x7fe32c10de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe331a9a640 1 -- 172.21.15.122:0/3798879415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe32c102d30 msgr2=0x7fe32c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe331a9a640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe32c102d30 0x7fe32c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe331a9a640 1 -- 172.21.15.122:0/3798879415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c10e6c0 con 0x7fe32c1036e0 2024-09-12T02:00:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe331a9a640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 0x7fe32c1a4e80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe320006fd0 tx=0x7fe32000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320011830 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c10e9b0 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe320016070 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000f2e0 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.160+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1ab8e0 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.161+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32c06a850 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.162+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe32001b460 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.163+0000 7fe32b7fe640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe3140a2b80 0x7fe3140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.163+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fe3200ce620 con 0x7fe32c1036e0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.163+0000 7fe33229b640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe3140a2b80 0x7fe3140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.163+0000 7fe33229b640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe3140a2b80 0x7fe3140a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe31c000f80 tx=0x7fe31c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:44.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.167+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe32009b580 con 0x7fe32c1036e0 2024-09-12T02:00:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:43 smithi183 ceph-mon[35127]: pgmap v762: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-09-12T02:00:44.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.367+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe32c19ff90 con 0x7fe32c1036e0 2024-09-12T02:00:44.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.367+0000 7fe32b7fe640 1 -- 172.21.15.122:0/3798879415 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11935 (secure 0 0 0) 0x7fe32009ae80 con 0x7fe32c1036e0 2024-09-12T02:00:44.369 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:00:44.369 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":32,"btime":"2024-09-12T02:00:17:747426+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:00:16.926630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1190566748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1190566748},{"type":"v1","addr":"172.21.15.183:6835","nonce":1190566748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T02:00:44.370 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 32 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.369+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe3140a2b80 msgr2=0x7fe3140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.369+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe3140a2b80 0x7fe3140a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe31c000f80 tx=0x7fe31c009360 comp rx=0 tx=0).stop 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.369+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 msgr2=0x7fe32c1a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.369+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 0x7fe32c1a4e80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe320006fd0 tx=0x7fe32000b8e0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 shutdown_connections 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe3140a2b80 0x7fe3140a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1082a0 0x7fe32c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe32c1036e0 0x7fe32c1a4e80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 --2- 172.21.15.122:0/3798879415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe32c102d30 0x7fe32c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 >> 172.21.15.122:0/3798879415 conn(0x7fe32c0fc7d0 msgr2=0x7fe32c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:44.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 shutdown_connections 2024-09-12T02:00:44.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:44.370+0000 7fe3344ff640 1 -- 172.21.15.122:0/3798879415 wait complete. 2024-09-12T02:00:45.035 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:44 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3798879415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:45.049 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1788 8.56 MB/sec warmup 32 sec latency 29074.159 ms 2024-09-12T02:00:45.064 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id d4d363ac --release --await' 2024-09-12T02:00:45.065 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id d4d363ac --release --await 2024-09-12T02:00:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:44 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3798879415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:44 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3798879415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:00:45.319 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:00:45.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 -- 172.21.15.122:0/214009217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c102d30 msgr2=0x7f830c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 --2- 172.21.15.122:0/214009217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c102d30 0x7f830c103110 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009930 tx=0x7f82fc02a440 comp rx=0 tx=0).stop 2024-09-12T02:00:45.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 -- 172.21.15.122:0/214009217 shutdown_connections 2024-09-12T02:00:45.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 --2- 172.21.15.122:0/214009217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c1082a0 0x7f830c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 --2- 172.21.15.122:0/214009217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c1036e0 0x7f830c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 --2- 172.21.15.122:0/214009217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c102d30 0x7f830c103110 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 -- 172.21.15.122:0/214009217 >> 172.21.15.122:0/214009217 conn(0x7f830c0fc7d0 msgr2=0x7f830c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 -- 172.21.15.122:0/214009217 shutdown_connections 2024-09-12T02:00:45.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 -- 172.21.15.122:0/214009217 wait complete. 2024-09-12T02:00:45.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 Processor -- start 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.794+0000 7f8311c96640 1 -- start start 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f8311c96640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c102d30 0x7f830c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f8311c96640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c1036e0 0x7f830c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f8311c96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 0x7f830c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f8311c96640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1120e0 con 0x7f830c1082a0 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f8311c96640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f830c111f60 con 0x7f830c102d30 2024-09-12T02:00:45.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f8311c96640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f830c112260 con 0x7f830c1036e0 2024-09-12T02:00:45.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 0x7f830c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 0x7f830c19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50750/0 (socket says 172.21.15.122:50750) 2024-09-12T02:00:45.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 -- 172.21.15.122:0/1783184204 learned_addr learned my addr 172.21.15.122:0/1783184204 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:45.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 -- 172.21.15.122:0/1783184204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c1036e0 msgr2=0x7f830c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830b7fe640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c102d30 0x7f830c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c1036e0 0x7f830c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 -- 172.21.15.122:0/1783184204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c102d30 msgr2=0x7f830c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c102d30 0x7f830c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 -- 172.21.15.122:0/1783184204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c19d5d0 con 0x7f830c1082a0 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830b7fe640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c102d30 0x7f830c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f830bfff640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 0x7f830c19cef0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f830400cad0 tx=0x7f830400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.795+0000 7f83097fa640 1 -- 172.21.15.122:0/1783184204 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830400ae20 con 0x7f830c1082a0 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.796+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c19d8c0 con 0x7f830c1082a0 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.796+0000 7f83097fa640 1 -- 172.21.15.122:0/1783184204 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8304005020 con 0x7f830c1082a0 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.796+0000 7f83097fa640 1 -- 172.21.15.122:0/1783184204 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830400f6f0 con 0x7f830c1082a0 2024-09-12T02:00:45.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.796+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a51a0 con 0x7f830c1082a0 2024-09-12T02:00:45.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f83097fa640 1 -- 172.21.15.122:0/1783184204 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f83040051d0 con 0x7f830c1082a0 2024-09-12T02:00:45.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 Processor -- start 2024-09-12T02:00:45.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 -- start start 2024-09-12T02:00:45.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79dc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6d7934620 con 0x55b6d79cf060 2024-09-12T02:00:45.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6d78dbe50 con 0x55b6d79da240 2024-09-12T02:00:45.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f831406f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6d79305e0 con 0x55b6d79cf980 2024-09-12T02:00:45.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.798+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50752/0 (socket says 172.21.15.122:50752) 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 -- 172.21.15.122:0/1427752764 learned_addr learned my addr 172.21.15.122:0/1427752764 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f830b7fe640 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 -- 172.21.15.122:0/1427752764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 msgr2=0x55b6d79d9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 -- 172.21.15.122:0/1427752764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 msgr2=0x55b6d79dc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79dc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 -- 172.21.15.122:0/1427752764 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d78ccd40 con 0x55b6d79cf060 2024-09-12T02:00:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f83097fa640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f82f80a2b80 0x7f82f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f830bfff640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f82f80a2b80 0x7f82f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8302ffd640 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79cf440 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098e0 tx=0x7f82f4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76d8ecbf5c063e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f83097fa640 1 -- 172.21.15.122:0/1783184204 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f83040cf190 con 0x7f830c1082a0 2024-09-12T02:00:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8301ffb640 1 -- 172.21.15.122:0/1427752764 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4015020 con 0x55b6d79cf060 2024-09-12T02:00:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8301ffb640 1 -- 172.21.15.122:0/1427752764 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f4011420 con 0x55b6d79cf060 2024-09-12T02:00:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.799+0000 7f8301ffb640 1 -- 172.21.15.122:0/1427752764 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f40105e0 con 0x55b6d79cf060 2024-09-12T02:00:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 -- 172.21.15.122:0/1427752764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 msgr2=0x55b6d79cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f830bfff640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f82f80a2b80 0x7f82f80a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f83040983c0 tx=0x7f830406da00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79cf440 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098e0 tx=0x7f82f4009da0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 -- 172.21.15.122:0/1427752764 shutdown_connections 2024-09-12T02:00:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79dc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 --2- 172.21.15.122:0/1427752764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79cf440 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 -- 172.21.15.122:0/1427752764 >> 172.21.15.122:0/1427752764 conn(0x55b6d79cb5f0 msgr2=0x55b6d79cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 -- 172.21.15.122:0/1427752764 shutdown_connections 2024-09-12T02:00:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.800+0000 7f831406f740 1 -- 172.21.15.122:0/1427752764 wait complete. 2024-09-12T02:00:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 -- start start 2024-09-12T02:00:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 Processor -- start 2024-09-12T02:00:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6d79dce10 con 0x55b6d79cf060 2024-09-12T02:00:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6d78bcf10 con 0x55b6d79da240 2024-09-12T02:00:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6d79dcf90 con 0x55b6d79cf980 2024-09-12T02:00:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f831406f740 10 client.0 _ll_register_callbacks cb 0x55b6d78e0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79d71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50766/0 (socket says 172.21.15.122:50766) 2024-09-12T02:00:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f8302ffd640 1 -- 172.21.15.122:0/1505420436 learned_addr learned my addr 172.21.15.122:0/1505420436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.801+0000 7f830b7fe640 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f8302ffd640 1 -- 172.21.15.122:0/1505420436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 msgr2=0x55b6d79d7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f8302ffd640 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f8302ffd640 1 -- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 msgr2=0x55b6d79d7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f8302ffd640 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79d7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f8302ffd640 1 -- 172.21.15.122:0/1505420436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d7643580 con 0x55b6d79cf060 2024-09-12T02:00:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f8302ffd640 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79d71d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f82f400a010 tx=0x7f82f4011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bda10c1c3a0a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f82eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:45.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f82eaffd640 1 -- 172.21.15.122:0/1505420436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4015050 con 0x55b6d79cf060 2024-09-12T02:00:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f82eaffd640 1 -- 172.21.15.122:0/1505420436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f82f400bce0 con 0x55b6d79cf060 2024-09-12T02:00:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f4011420 con 0x55b6d79cf060 2024-09-12T02:00:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f82eaffd640 1 -- 172.21.15.122:0/1505420436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82f4009470 con 0x55b6d79cf060 2024-09-12T02:00:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f831406f740 10 client.17559 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6d770d960 con 0x55b6d79cf060 2024-09-12T02:00:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f82eaffd640 1 -- 172.21.15.122:0/1505420436 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f4011420 con 0x55b6d79cf060 2024-09-12T02:00:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.802+0000 7f82eaffd640 1 -- 172.21.15.122:0/1505420436 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f82f4021070 con 0x55b6d79cf060 2024-09-12T02:00:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f831406f740 10 client.17559 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f831406f740 10 client.17559 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:00:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f831406f740 20 client.17559 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f831406f740 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d79e9ce0 0x55b6d79ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f831406f740 4 client.17559 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6d7941b90 con 0x55b6d79e9ce0 2024-09-12T02:00:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f830b7fe640 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d79e9ce0 0x55b6d79ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f82eaffd640 10 client.17559 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.803+0000 7f830b7fe640 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d79e9ce0 0x55b6d79ec0d0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc195d9bfa6ffe3 server_cookie=a310bd4133b1ac3e in_seq=0 out_seq=0 2024-09-12T02:00:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f82eaffd640 1 -- 172.21.15.122:0/1505420436 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6d7941b90 con 0x55b6d79e9ce0 2024-09-12T02:00:45.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f82eaffd640 10 client.17559 handle_command_reply: tid=0 2024-09-12T02:00:45.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 client.17559 shutdown 2024-09-12T02:00:45.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 msgr2=0x55b6d79d71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79d71d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f82f400a010 tx=0x7f82f4011800 comp rx=0 tx=0).stop 2024-09-12T02:00:45.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 shutdown_connections 2024-09-12T02:00:45.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d79e9ce0 0x55b6d79ec0d0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79d7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 --2- 172.21.15.122:0/1505420436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf060 0x55b6d79d71d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 >> 172.21.15.122:0/1505420436 conn(0x55b6d79cb5f0 msgr2=0x55b6d79d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f82eaffd640 0 client.17559 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:45.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 shutdown_connections 2024-09-12T02:00:45.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 1 -- 172.21.15.122:0/1505420436 wait complete. 2024-09-12T02:00:45.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.804+0000 7f831406f740 20 client.17559 trim_cache size 0 max 16384 2024-09-12T02:00:45.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 Processor -- start 2024-09-12T02:00:45.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 -- start start 2024-09-12T02:00:45.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf060 0x55b6d79d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 0x55b6d79ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79f1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1b0090 con 0x55b6d79cf980 2024-09-12T02:00:45.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1ad8d0 con 0x55b6d79da240 2024-09-12T02:00:45.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f831406f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1ab700 con 0x55b6d79cf060 2024-09-12T02:00:45.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 0x55b6d79ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 0x55b6d79ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50778/0 (socket says 172.21.15.122:50778) 2024-09-12T02:00:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 -- 172.21.15.122:0/630351186 learned_addr learned my addr 172.21.15.122:0/630351186 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 -- 172.21.15.122:0/630351186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf060 msgr2=0x55b6d79d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf060 0x55b6d79d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 -- 172.21.15.122:0/630351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 msgr2=0x55b6d79f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.821+0000 7f830b7fe640 1 -- 172.21.15.122:0/630351186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d79d2df0 con 0x55b6d79cf980 2024-09-12T02:00:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.822+0000 7f830b7fe640 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 0x55b6d79ca640 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009900 tx=0x7f82fc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e8dea6c9413105 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.822+0000 7f83017fa640 1 -- 172.21.15.122:0/630351186 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc048070 con 0x55b6d79cf980 2024-09-12T02:00:45.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.822+0000 7f83017fa640 1 -- 172.21.15.122:0/630351186 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc005520 con 0x55b6d79cf980 2024-09-12T02:00:45.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.822+0000 7f83017fa640 1 -- 172.21.15.122:0/630351186 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc035040 con 0x55b6d79cf980 2024-09-12T02:00:45.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- 172.21.15.122:0/630351186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 msgr2=0x55b6d79ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 0x55b6d79ca640 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009900 tx=0x7f82fc005a20 comp rx=0 tx=0).stop 2024-09-12T02:00:45.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- 172.21.15.122:0/630351186 shutdown_connections 2024-09-12T02:00:45.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79da240 0x55b6d79f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79cf980 0x55b6d79ca640 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- 172.21.15.122:0/630351186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf060 0x55b6d79d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- 172.21.15.122:0/630351186 >> 172.21.15.122:0/630351186 conn(0x55b6d79cb5f0 msgr2=0x55b6d79c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- 172.21.15.122:0/630351186 shutdown_connections 2024-09-12T02:00:45.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- 172.21.15.122:0/630351186 wait complete. 2024-09-12T02:00:45.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- start start 2024-09-12T02:00:45.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 Processor -- start 2024-09-12T02:00:45.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79cf060 0x55b6d79d4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 0x55b6d79d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1abd90 con 0x55b6d79da240 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1ad200 con 0x55b6d79cf060 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.823+0000 7f831406f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1ac460 con 0x55b6d79cf980 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f831406f740 10 client.0 _ll_register_callbacks cb 0x7f82f4009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 0x55b6d79d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 0x55b6d79d4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50792/0 (socket says 172.21.15.122:50792) 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 -- 172.21.15.122:0/2610319441 learned_addr learned my addr 172.21.15.122:0/2610319441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f830b7fe640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79cf060 0x55b6d79d4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 -- 172.21.15.122:0/2610319441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 msgr2=0x55b6d79d46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 -- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79cf060 msgr2=0x55b6d79d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:45.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79cf060 0x55b6d79d4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 -- 172.21.15.122:0/2610319441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d7a53810 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f8302ffd640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 0x55b6d79d4c10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009630 tx=0x7f82f400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65828e5c2697eaf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f82eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f82eaffd640 1 -- 172.21.15.122:0/2610319441 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4015070 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f82eaffd640 1 -- 172.21.15.122:0/2610319441 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f82f401a820 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82fc005520 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f82eaffd640 1 -- 172.21.15.122:0/2610319441 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82f400bae0 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f831406f740 10 client.17565 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.824+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6d7a54af0 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f82eaffd640 1 -- 172.21.15.122:0/2610319441 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc005520 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f82eaffd640 1 -- 172.21.15.122:0/2610319441 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f82f4037020 con 0x55b6d79da240 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f831406f740 10 client.17565 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:45.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f831406f740 10 client.17565 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f831406f740 20 client.17565 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f831406f740 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d7a55150 0x55b6d7a57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f831406f740 4 client.17565 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d4d363ac", "release": true, "await": true}] 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d4d363ac", "release": true, "await": true}) -- 0x55b6d793c0a0 con 0x55b6d7a55150 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f830b7fe640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d7a55150 0x55b6d7a57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f830b7fe640 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d7a55150 0x55b6d7a57570 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8776323ca4c16f server_cookie=dfe4578c10d1aa43 in_seq=0 out_seq=0 2024-09-12T02:00:45.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.825+0000 7f82eaffd640 10 client.17565 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:45.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f82eaffd640 1 -- 172.21.15.122:0/2610319441 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b6d793c0a0 con 0x55b6d7a55150 2024-09-12T02:00:45.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f82eaffd640 10 client.17565 handle_command_reply: tid=0 2024-09-12T02:00:45.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 client.17565 shutdown 2024-09-12T02:00:45.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 msgr2=0x55b6d79d4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 0x55b6d79d4c10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009630 tx=0x7f82f400b050 comp rx=0 tx=0).stop 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 shutdown_connections 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b6d7a55150 0x55b6d7a57570 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6d79da240 0x55b6d79d4c10 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6d79cf980 0x55b6d79d46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 --2- 172.21.15.122:0/2610319441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6d79cf060 0x55b6d79d4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 >> 172.21.15.122:0/2610319441 conn(0x55b6d79cb5f0 msgr2=0x55b6d79ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f82eaffd640 0 client.17565 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 shutdown_connections 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 1 -- 172.21.15.122:0/2610319441 wait complete. 2024-09-12T02:00:45.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.897+0000 7f831406f740 20 client.17565 trim_cache size 0 max 16384 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 32, 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 56, 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "d4d363ac": { 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 56, 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:00:45.900 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:45.901 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:45.902 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:45.902 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:00:45.902 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:00:45.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f82f80a2b80 msgr2=0x7f82f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f82f80a2b80 0x7f82f80a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f83040983c0 tx=0x7f830406da00 comp rx=0 tx=0).stop 2024-09-12T02:00:45.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 msgr2=0x7f830c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 0x7f830c19cef0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f830400cad0 tx=0x7f830400cf90 comp rx=0 tx=0).stop 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 shutdown_connections 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f82f80a2b80 0x7f82f80a5040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f830c1082a0 0x7f830c19cef0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c1036e0 0x7f830c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 --2- 172.21.15.122:0/1783184204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f830c102d30 0x7f830c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 >> 172.21.15.122:0/1783184204 conn(0x7f830c0fc7d0 msgr2=0x7f830c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 shutdown_connections 2024-09-12T02:00:45.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:00:45.898+0000 7f8311c96640 1 -- 172.21.15.122:0/1783184204 wait complete. 2024-09-12T02:00:45.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:45 smithi122 ceph-mon[30925]: pgmap v763: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-09-12T02:00:46.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1905 8.36 MB/sec warmup 33 sec latency 29916.186 ms 2024-09-12T02:00:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:45 smithi151 ceph-mon[35229]: pgmap v763: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-09-12T02:00:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:45 smithi183 ceph-mon[35127]: pgmap v763: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-09-12T02:00:46.723 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd4d363ac', total seconds elapsed: 33.2 2024-09-12T02:00:47.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 2762 8.87 MB/sec warmup 34 sec latency 70.571 ms 2024-09-12T02:00:48.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 3513 10.41 MB/sec warmup 35 sec latency 99.392 ms 2024-09-12T02:00:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:47 smithi151 ceph-mon[35229]: pgmap v764: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T02:00:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:47 smithi183 ceph-mon[35127]: pgmap v764: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T02:00:48.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:47 smithi122 ceph-mon[30925]: pgmap v764: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T02:00:49.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 4242 11.54 MB/sec warmup 36 sec latency 77.563 ms 2024-09-12T02:00:50.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 4663 11.85 MB/sec warmup 37 sec latency 98.231 ms 2024-09-12T02:00:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:49 smithi151 ceph-mon[35229]: pgmap v765: 97 pgs: 97 active+clean; 481 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.8 MiB/s wr, 64 op/s 2024-09-12T02:00:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:49 smithi183 ceph-mon[35127]: pgmap v765: 97 pgs: 97 active+clean; 481 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.8 MiB/s wr, 64 op/s 2024-09-12T02:00:50.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:49 smithi122 ceph-mon[30925]: pgmap v765: 97 pgs: 97 active+clean; 481 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 1.8 MiB/s wr, 64 op/s 2024-09-12T02:00:51.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 5144 12.03 MB/sec warmup 38 sec latency 84.939 ms 2024-09-12T02:00:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:50 smithi151 ceph-mon[35229]: pgmap v766: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-12T02:00:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:50 smithi183 ceph-mon[35127]: pgmap v766: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-12T02:00:51.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:50 smithi122 ceph-mon[30925]: pgmap v766: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-09-12T02:00:52.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 5575 12.08 MB/sec warmup 39 sec latency 92.892 ms 2024-09-12T02:00:53.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 6392 12.92 MB/sec warmup 40 sec latency 53.760 ms 2024-09-12T02:00:54.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7284 13.37 MB/sec warmup 41 sec latency 54.990 ms 2024-09-12T02:00:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:53 smithi151 ceph-mon[35229]: pgmap v767: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 195 op/s 2024-09-12T02:00:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:53 smithi183 ceph-mon[35127]: pgmap v767: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 195 op/s 2024-09-12T02:00:54.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:53 smithi122 ceph-mon[30925]: pgmap v767: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 195 op/s 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3613478644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d0100410 msgr2=0x7fd0d0100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3613478644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d0100410 0x7fd0d0100810 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009900 tx=0x7fd0cc02a410 comp rx=0 tx=0).stop 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3613478644 shutdown_connections 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3613478644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0d01016f0 0x7fd0d0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3613478644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0d0100d50 0x7fd0d01011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3613478644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d0100410 0x7fd0d0100810 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3613478644 >> 172.21.15.122:0/3613478644 conn(0x7fd0d00fc1e0 msgr2=0x7fd0d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3613478644 shutdown_connections 2024-09-12T02:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.454+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3613478644 wait complete. 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 Processor -- start 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 -- start start 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0d0100410 0x7fd0d019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0d0100d50 0x7fd0d019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 0x7fd0d01a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d010f980 con 0x7fd0d01016f0 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d010f800 con 0x7fd0d0100410 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d82ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d010fb00 con 0x7fd0d0100d50 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 0x7fd0d01a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d6056640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0d0100410 0x7fd0d019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 0x7fd0d01a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47884/0 (socket says 172.21.15.122:47884) 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.455+0000 7fd0d6857640 1 -- 172.21.15.122:0/3673081666 learned_addr learned my addr 172.21.15.122:0/3673081666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d5855640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0d0100d50 0x7fd0d019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d6857640 1 -- 172.21.15.122:0/3673081666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0d0100d50 msgr2=0x7fd0d019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d6857640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0d0100d50 0x7fd0d019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d6857640 1 -- 172.21.15.122:0/3673081666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0d0100410 msgr2=0x7fd0d019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d6857640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0d0100410 0x7fd0d019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d6857640 1 -- 172.21.15.122:0/3673081666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d01a1c20 con 0x7fd0d01016f0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d6857640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 0x7fd0d01a15b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800eb10 tx=0x7fd0c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0c77fe640 1 -- 172.21.15.122:0/3673081666 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8004b20 con 0x7fd0d01016f0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0c77fe640 1 -- 172.21.15.122:0/3673081666 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c8004cc0 con 0x7fd0d01016f0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d01a1f10 con 0x7fd0d01016f0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0c77fe640 1 -- 172.21.15.122:0/3673081666 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8010760 con 0x7fd0d01016f0 2024-09-12T02:00:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.456+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d01a2380 con 0x7fd0d01016f0 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.459+0000 7fd0c77fe640 1 -- 172.21.15.122:0/3673081666 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd0c8005020 con 0x7fd0d01016f0 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 Processor -- start 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 -- start start 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0c77fe640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0bc0a2c30 0x7fd0bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5338da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e533dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e5340830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7e529c2f0 con 0x55c7e533e370 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7e5244270 con 0x55c7e53389a0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.460+0000 7fd0db057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c7e52982b0 con 0x55c7e53392e0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d5855640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0bc0a2c30 0x7fd0bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0c77fe640 1 -- 172.21.15.122:0/3673081666 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd0c80cf680 con 0x7fd0d01016f0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e5340830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d5855640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e533dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d5855640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e533dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33520/0 (socket says 172.21.15.122:33520) 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e5340830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47896/0 (socket says 172.21.15.122:47896) 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d5855640 1 -- 172.21.15.122:0/99491661 learned_addr learned my addr 172.21.15.122:0/99491661 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6056640 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5338da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 -- 172.21.15.122:0/99491661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 msgr2=0x55c7e533dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e533dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 -- 172.21.15.122:0/99491661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 msgr2=0x55c7e5338da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5338da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 -- 172.21.15.122:0/99491661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e5243340 con 0x55c7e533e370 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d5855640 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e533dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d5855640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0bc0a2c30 0x7fd0bc0a50f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0012940 tx=0x7fd0c0009ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0d6857640 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e5340830 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80049d0 tx=0x7fd0c809bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6217dcc110de3fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0c4bfe640 1 -- 172.21.15.122:0/99491661 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c809c5e0 con 0x55c7e533e370 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0c4bfe640 1 -- 172.21.15.122:0/99491661 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c809bee0 con 0x55c7e533e370 2024-09-12T02:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.461+0000 7fd0c4bfe640 1 -- 172.21.15.122:0/99491661 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80a46c0 con 0x55c7e533e370 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 -- 172.21.15.122:0/99491661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 msgr2=0x55c7e5340830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e5340830 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80049d0 tx=0x7fd0c809bcb0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 -- 172.21.15.122:0/99491661 shutdown_connections 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e5340830 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e533dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 --2- 172.21.15.122:0/99491661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5338da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 -- 172.21.15.122:0/99491661 >> 172.21.15.122:0/99491661 conn(0x55c7e5334010 msgr2=0x55c7e5334410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 -- 172.21.15.122:0/99491661 shutdown_connections 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 -- 172.21.15.122:0/99491661 wait complete. 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.462+0000 7fd0db057740 1 -- start start 2024-09-12T02:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 Processor -- start 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5342b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5343040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e534a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7e5341390 con 0x55c7e533e370 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7e5225290 con 0x55c7e53389a0 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c7e5341510 con 0x55c7e53392e0 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 10 client.0 _ll_register_callbacks cb 0x55c7e5256140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e534a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6056640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5342b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e534a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47902/0 (socket says 172.21.15.122:47902) 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 -- 172.21.15.122:0/1739501922 learned_addr learned my addr 172.21.15.122:0/1739501922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d5855640 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5343040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 -- 172.21.15.122:0/1739501922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 msgr2=0x55c7e5343040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5343040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 -- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 msgr2=0x55c7e5342b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5342b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 -- 172.21.15.122:0/1739501922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e4faf210 con 0x55c7e533e370 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0d6857640 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e534a080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800cfb0 tx=0x7fd0c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b26cce71417a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/1739501922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80b4020 con 0x55c7e533e370 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/1739501922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd0c8005520 con 0x55c7e533e370 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.463+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c809bee0 con 0x55c7e533e370 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/1739501922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0c80a4b50 con 0x55c7e533e370 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 10 client.17577 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c7e507b620 con 0x55c7e533e370 2024-09-12T02:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/1739501922 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c809bee0 con 0x55c7e533e370 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/1739501922 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fd0c8097cc0 con 0x55c7e533e370 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 10 client.17577 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 10 client.17577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 20 client.17577 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e534e380 0x55c7e5350770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 4 client.17577 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7e52a9a20 con 0x55c7e534e380 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0d6056640 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e534e380 0x55c7e5350770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.465+0000 7fd0b9ffb640 10 client.17577 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.464+0000 7fd0d6056640 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e534e380 0x55c7e5350770 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6ba40d2a11bce1 server_cookie=c59304c957f59840 in_seq=0 out_seq=0 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.465+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/1739501922 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7e52a9a20 con 0x55c7e534e380 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.465+0000 7fd0b9ffb640 10 client.17577 handle_command_reply: tid=0 2024-09-12T02:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.465+0000 7fd0db057740 1 client.17577 shutdown 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 msgr2=0x55c7e534a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e534a080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800cfb0 tx=0x7fd0c8004ab0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 shutdown_connections 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e534e380 0x55c7e5350770 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e534a080 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5343040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0b9ffb640 0 client.17577 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 --2- 172.21.15.122:0/1739501922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5342b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 >> 172.21.15.122:0/1739501922 conn(0x55c7e5334010 msgr2=0x55c7e5335820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 shutdown_connections 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 1 -- 172.21.15.122:0/1739501922 wait complete. 2024-09-12T02:00:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.466+0000 7fd0db057740 20 client.17577 trim_cache size 0 max 16384 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 Processor -- start 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 -- start start 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53389a0 0x55c7e5334f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 0x55c7e534a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e533e370 0x55c7e5335460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01adb60 con 0x55c7e53392e0 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01ab3a0 con 0x55c7e533e370 2024-09-12T02:00:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0db057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01a91d0 con 0x55c7e53389a0 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0d5855640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 0x55c7e534a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0d6056640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53389a0 0x55c7e5334f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0d5855640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 0x55c7e534a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47912/0 (socket says 172.21.15.122:47912) 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0d5855640 1 -- 172.21.15.122:0/4123235210 learned_addr learned my addr 172.21.15.122:0/4123235210 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.482+0000 7fd0d6857640 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e533e370 0x55c7e5335460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0d5855640 1 -- 172.21.15.122:0/4123235210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53389a0 msgr2=0x55c7e5334f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0d5855640 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53389a0 0x55c7e5334f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0d5855640 1 -- 172.21.15.122:0/4123235210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e533e370 msgr2=0x55c7e5335460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0d5855640 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e533e370 0x55c7e5335460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0d5855640 1 -- 172.21.15.122:0/4123235210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e5350da0 con 0x55c7e53392e0 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0d5855640 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 0x55c7e534a040 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0010b30 tx=0x7fd0c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73a038d0699ed13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0bbfff640 1 -- 172.21.15.122:0/4123235210 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c002d070 con 0x55c7e53392e0 2024-09-12T02:00:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0bbfff640 1 -- 172.21.15.122:0/4123235210 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c00119e0 con 0x55c7e53392e0 2024-09-12T02:00:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.483+0000 7fd0bbfff640 1 -- 172.21.15.122:0/4123235210 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0020b80 con 0x55c7e53392e0 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 -- 172.21.15.122:0/4123235210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 msgr2=0x55c7e534a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 0x55c7e534a040 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0010b30 tx=0x7fd0c0004730 comp rx=0 tx=0).stop 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 -- 172.21.15.122:0/4123235210 shutdown_connections 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e533e370 0x55c7e5335460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e53392e0 0x55c7e534a040 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 --2- 172.21.15.122:0/4123235210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53389a0 0x55c7e5334f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 -- 172.21.15.122:0/4123235210 >> 172.21.15.122:0/4123235210 conn(0x55c7e5334010 msgr2=0x55c7e5342b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 -- 172.21.15.122:0/4123235210 shutdown_connections 2024-09-12T02:00:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 -- 172.21.15.122:0/4123235210 wait complete. 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 -- start start 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.484+0000 7fd0db057740 1 Processor -- start 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5345df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5346330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e53492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01a9860 con 0x55c7e533e370 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01aacd0 con 0x55c7e53389a0 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d01a9f30 con 0x55c7e53392e0 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0db057740 10 client.0 _ll_register_callbacks cb 0x7fd0c8004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e53492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5346330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d6056640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5345df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5346330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33544/0 (socket says 172.21.15.122:33544) 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d6857640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e53492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47924/0 (socket says 172.21.15.122:47924) 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 -- 172.21.15.122:0/2797988133 learned_addr learned my addr 172.21.15.122:0/2797988133 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 -- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 msgr2=0x55c7e5345df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5345df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 -- 172.21.15.122:0/2797988133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 msgr2=0x55c7e53492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e53492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 -- 172.21.15.122:0/2797988133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e532f2d0 con 0x55c7e53392e0 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d6857640 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e53492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0d5855640 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5346330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0012f60 tx=0x7fd0c0011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb4ec16571d8924 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.485+0000 7fd0b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/2797988133 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0037020 con 0x55c7e53392e0 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/2797988133 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7fd0c0011500 con 0x55c7e53392e0 2024-09-12T02:00:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c00119e0 con 0x55c7e53392e0 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/2797988133 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0c002d040 con 0x55c7e53392e0 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 10 client.26650 fetch_fsmap learned FSMap version 32 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c7e53b93f0 con 0x55c7e53392e0 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/2797988133 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c00119e0 con 0x55c7e53392e0 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/2797988133 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fd0c0030030 con 0x55c7e53392e0 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 10 client.26650 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 10 client.26650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 20 client.26650 populate_metadata read hostname 'smithi122' 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e53b9d10 0x55c7e53bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 4 client.26650 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7e52a3d70 con 0x55c7e53b9d10 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.486+0000 7fd0d6056640 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e53b9d10 0x55c7e53bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:00:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.487+0000 7fd0b9ffb640 10 client.26650 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.487+0000 7fd0d6056640 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e53b9d10 0x55c7e53bc130 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd2e456dfd24b35 server_cookie=5654136dfa40470 in_seq=0 out_seq=0 2024-09-12T02:00:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0b9ffb640 1 -- 172.21.15.122:0/2797988133 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7e52a3d70 con 0x55c7e53b9d10 2024-09-12T02:00:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0b9ffb640 10 client.26650 handle_command_reply: tid=0 2024-09-12T02:00:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 client.26650 shutdown 2024-09-12T02:00:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 msgr2=0x55c7e5346330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5346330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0012f60 tx=0x7fd0c0011dc0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 shutdown_connections 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7e53b9d10 0x55c7e53bc130 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7e533e370 0x55c7e53492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0b9ffb640 0 client.26650 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7e53392e0 0x55c7e5346330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 --2- 172.21.15.122:0/2797988133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e53389a0 0x55c7e5345df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.612+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 >> 172.21.15.122:0/2797988133 conn(0x55c7e5334010 msgr2=0x55c7e53329a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 shutdown_connections 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0db057740 1 -- 172.21.15.122:0/2797988133 wait complete. 2024-09-12T02:00:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0db057740 20 client.26650 trim_cache size 0 max 16384 2024-09-12T02:00:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:00:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:00:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:00:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0bc0a2c30 msgr2=0x7fd0bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0bc0a2c30 0x7fd0bc0a50f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0012940 tx=0x7fd0c0009ae0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 msgr2=0x7fd0d01a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 0x7fd0d01a15b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800eb10 tx=0x7fd0c800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 shutdown_connections 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd0bc0a2c30 0x7fd0bc0a50f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0d01016f0 0x7fd0d01a15b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0d0100d50 0x7fd0d019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 --2- 172.21.15.122:0/3673081666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0d0100410 0x7fd0d019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 >> 172.21.15.122:0/3673081666 conn(0x7fd0d00fc1e0 msgr2=0x7fd0d00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:00:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 shutdown_connections 2024-09-12T02:00:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:00:54.613+0000 7fd0d82ba640 1 -- 172.21.15.122:0/3673081666 wait complete. 2024-09-12T02:00:55.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 8056 14.34 MB/sec warmup 42 sec latency 93.870 ms 2024-09-12T02:00:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:54 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:00:55.147 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:54 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:00:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:54 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:00:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:55 smithi122 ceph-mon[30925]: pgmap v768: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 328 op/s 2024-09-12T02:00:56.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 8633 15.07 MB/sec warmup 43 sec latency 246.281 ms 2024-09-12T02:00:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:55 smithi151 ceph-mon[35229]: pgmap v768: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 328 op/s 2024-09-12T02:00:56.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:55 smithi183 ceph-mon[35127]: pgmap v768: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 328 op/s 2024-09-12T02:00:57.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 9185 15.57 MB/sec warmup 44 sec latency 62.932 ms 2024-09-12T02:00:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:00:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:00:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:00:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:00:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:00:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:00:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:00:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:00:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:00:58.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 9508 15.40 MB/sec warmup 45 sec latency 133.256 ms 2024-09-12T02:00:58.103 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:57 smithi183 ceph-mon[35127]: pgmap v769: 97 pgs: 97 active+clean; 491 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 370 op/s 2024-09-12T02:00:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:57 smithi151 ceph-mon[35229]: pgmap v769: 97 pgs: 97 active+clean; 491 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 370 op/s 2024-09-12T02:00:58.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:57 smithi122 ceph-mon[30925]: pgmap v769: 97 pgs: 97 active+clean; 491 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 370 op/s 2024-09-12T02:00:59.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 10063 15.54 MB/sec warmup 46 sec latency 95.187 ms 2024-09-12T02:01:00.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:00:59 smithi122 ceph-mon[30925]: pgmap v770: 97 pgs: 97 active+clean; 491 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-09-12T02:01:00.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 10595 15.73 MB/sec warmup 47 sec latency 120.391 ms 2024-09-12T02:01:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:00:59 smithi151 ceph-mon[35229]: pgmap v770: 97 pgs: 97 active+clean; 491 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-09-12T02:01:00.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:00:59 smithi183 ceph-mon[35127]: pgmap v770: 97 pgs: 97 active+clean; 491 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-09-12T02:01:01.050 INFO:tasks.workunit.client.0.smithi122.stdout: 10 11506 16.39 MB/sec warmup 48 sec latency 50.659 ms 2024-09-12T02:01:02.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 12349 16.94 MB/sec warmup 49 sec latency 64.594 ms 2024-09-12T02:01:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:01 smithi151 ceph-mon[35229]: pgmap v771: 97 pgs: 97 active+clean; 500 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.3 MiB/s wr, 530 op/s 2024-09-12T02:01:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:01 smithi183 ceph-mon[35127]: pgmap v771: 97 pgs: 97 active+clean; 500 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.3 MiB/s wr, 530 op/s 2024-09-12T02:01:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:01 smithi122 ceph-mon[30925]: pgmap v771: 97 pgs: 97 active+clean; 500 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.3 MiB/s wr, 530 op/s 2024-09-12T02:01:03.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 13066 17.81 MB/sec warmup 50 sec latency 66.329 ms 2024-09-12T02:01:04.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 13490 17.90 MB/sec warmup 51 sec latency 68.934 ms 2024-09-12T02:01:04.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:03 smithi151 ceph-mon[35229]: pgmap v772: 97 pgs: 97 active+clean; 504 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.8 MiB/s wr, 473 op/s 2024-09-12T02:01:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:03 smithi183 ceph-mon[35127]: pgmap v772: 97 pgs: 97 active+clean; 504 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.8 MiB/s wr, 473 op/s 2024-09-12T02:01:04.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:03 smithi122 ceph-mon[30925]: pgmap v772: 97 pgs: 97 active+clean; 504 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.8 MiB/s wr, 473 op/s 2024-09-12T02:01:05.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 13933 17.92 MB/sec warmup 52 sec latency 104.207 ms 2024-09-12T02:01:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:04 smithi151 ceph-mon[35229]: pgmap v773: 97 pgs: 97 active+clean; 513 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 529 op/s 2024-09-12T02:01:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:04 smithi183 ceph-mon[35127]: pgmap v773: 97 pgs: 97 active+clean; 513 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 529 op/s 2024-09-12T02:01:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:04 smithi122 ceph-mon[30925]: pgmap v773: 97 pgs: 97 active+clean; 513 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 529 op/s 2024-09-12T02:01:06.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 14353 17.87 MB/sec warmup 53 sec latency 91.481 ms 2024-09-12T02:01:07.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 15037 18.06 MB/sec warmup 54 sec latency 50.465 ms 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 -- 172.21.15.122:0/341439769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 msgr2=0x7f0574105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 --2- 172.21.15.122:0/341439769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f0574105170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0568009900 tx=0x7f056802a410 comp rx=0 tx=0).stop 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 -- 172.21.15.122:0/341439769 shutdown_connections 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 --2- 172.21.15.122:0/341439769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05741056b0 0x7f0574105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 --2- 172.21.15.122:0/341439769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f0574105170 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 --2- 172.21.15.122:0/341439769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05741003b0 0x7f05741027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 -- 172.21.15.122:0/341439769 >> 172.21.15.122:0/341439769 conn(0x7f05740fc1e0 msgr2=0x7f05740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 -- 172.21.15.122:0/341439769 shutdown_connections 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.708+0000 7f0579b76640 1 -- 172.21.15.122:0/341439769 wait complete. 2024-09-12T02:01:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 Processor -- start 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 -- start start 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05741003b0 0x7f057419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f057419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05741056b0 0x7f05741a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0574113df0 con 0x7f0574102d00 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0574113c70 con 0x7f05741003b0 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.709+0000 7f0579b76640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0574113f70 con 0x7f05741056b0 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f057419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f057419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53888/0 (socket says 172.21.15.122:53888) 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f05737fe640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05741003b0 0x7f057419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 -- 172.21.15.122:0/234912167 learned_addr learned my addr 172.21.15.122:0/234912167 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0573fff640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05741056b0 0x7f05741a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 -- 172.21.15.122:0/234912167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05741056b0 msgr2=0x7f05741a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05741056b0 0x7f05741a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 -- 172.21.15.122:0/234912167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05741003b0 msgr2=0x7f057419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05741003b0 0x7f057419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 -- 172.21.15.122:0/234912167 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05741a5fa0 con 0x7f0574102d00 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0572ffd640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f057419e8c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0568009230 tx=0x7f05680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0570ff9640 1 -- 172.21.15.122:0/234912167 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056803f030 con 0x7f0574102d00 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05741a6230 con 0x7f0574102d00 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0570ff9640 1 -- 172.21.15.122:0/234912167 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0568003ce0 con 0x7f0574102d00 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0570ff9640 1 -- 172.21.15.122:0/234912167 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568035070 con 0x7f0574102d00 2024-09-12T02:01:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.710+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05741a6720 con 0x7f0574102d00 2024-09-12T02:01:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 Processor -- start 2024-09-12T02:01:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f0570ff9640 1 -- 172.21.15.122:0/234912167 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0568042030 con 0x7f0574102d00 2024-09-12T02:01:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 -- start start 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c04830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7b602f0 con 0x55b8e7bfd2e0 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7b08270 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.713+0000 7f057c9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7b5c2b0 con 0x55b8e7c02370 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f05737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:47410/0 (socket says 172.21.15.122:47410) 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 -- 172.21.15.122:0/1899938962 learned_addr learned my addr 172.21.15.122:0/1899938962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0572ffd640 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c04830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 -- 172.21.15.122:0/1899938962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 msgr2=0x55b8e7c04830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c04830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 -- 172.21.15.122:0/1899938962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfd2e0 msgr2=0x55b8e7c01da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c01da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 -- 172.21.15.122:0/1899938962 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e7b07340 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f05737fe640 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c01da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0570ff9640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f055c0a2c30 0x7f055c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0573fff640 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcda0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f056c009fd0 tx=0x7f056c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e896007dfda05f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f05737fe640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f055c0a2c30 0x7f055c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0570ff9640 1 -- 172.21.15.122:0/234912167 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f05680ef460 con 0x7f0574102d00 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.714+0000 7f0559ffb640 1 -- 172.21.15.122:0/1899938962 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c004d20 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f0559ffb640 1 -- 172.21.15.122:0/1899938962 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056c004050 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f0559ffb640 1 -- 172.21.15.122:0/1899938962 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c005280 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 -- 172.21.15.122:0/1899938962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 msgr2=0x55b8e7bfcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f05737fe640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f055c0a2c30 0x7f055c0a50f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55b8e7bfdd10 tx=0x7f0564005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcda0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f056c009fd0 tx=0x7f056c0118d0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 -- 172.21.15.122:0/1899938962 shutdown_connections 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c04830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c01da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1899938962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcda0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 -- 172.21.15.122:0/1899938962 >> 172.21.15.122:0/1899938962 conn(0x55b8e7bf8010 msgr2=0x55b8e7bf8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 -- 172.21.15.122:0/1899938962 shutdown_connections 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.715+0000 7f057c9ed740 1 -- 172.21.15.122:0/1899938962 wait complete. 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 -- start start 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 Processor -- start 2024-09-12T02:01:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c06bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c0e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7c05340 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7ae9290 con 0x55b8e7bfd2e0 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8e7c054c0 con 0x55b8e7c02370 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f057c9ed740 10 client.0 _ll_register_callbacks cb 0x55b8e7b1a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f05737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c06bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f05737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c06bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53914/0 (socket says 172.21.15.122:53914) 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f05737fe640 1 -- 172.21.15.122:0/3816726286 learned_addr learned my addr 172.21.15.122:0/3816726286 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f0572ffd640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.716+0000 7f0573fff640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c0e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f05737fe640 1 -- 172.21.15.122:0/3816726286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 msgr2=0x55b8e7c0e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f05737fe640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c0e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f05737fe640 1 -- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfd2e0 msgr2=0x55b8e7c07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f05737fe640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f05737fe640 1 -- 172.21.15.122:0/3816726286 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e7873210 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0573fff640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c0e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0572ffd640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f05737fe640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c06bc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0564009fd0 tx=0x7f0564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b20333871866df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0552ffd640 1 -- 172.21.15.122:0/3816726286 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056402e060 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056c004050 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0552ffd640 1 -- 172.21.15.122:0/3816726286 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f0564018030 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0552ffd640 1 -- 172.21.15.122:0/3816726286 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0564009be0 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f057c9ed740 10 client.17595 fetch_fsmap learned FSMap version 32 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b8e793f620 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.717+0000 7f0552ffd640 1 -- 172.21.15.122:0/3816726286 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056c004050 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f0552ffd640 1 -- 172.21.15.122:0/3816726286 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f0564033070 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f057c9ed740 10 client.17595 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f057c9ed740 10 client.17595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f057c9ed740 20 client.17595 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f057c9ed740 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c12cd0 0x55b8e7c15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f057c9ed740 4 client.17595 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8e7b6da20 con 0x55b8e7c12cd0 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f0572ffd640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c12cd0 0x55b8e7c15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f0572ffd640 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c12cd0 0x55b8e7c15170 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b81ea69d68218c14 server_cookie=76a3177140259d3f in_seq=0 out_seq=0 2024-09-12T02:01:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.718+0000 7f0552ffd640 10 client.17595 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.719+0000 7f0552ffd640 1 -- 172.21.15.122:0/3816726286 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8e7b6da20 con 0x55b8e7c12cd0 2024-09-12T02:01:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.719+0000 7f0552ffd640 10 client.17595 handle_command_reply: tid=0 2024-09-12T02:01:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.719+0000 7f057c9ed740 1 client.17595 shutdown 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 msgr2=0x55b8e7c06bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c06bc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0564009fd0 tx=0x7f0564004730 comp rx=0 tx=0).stop 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 shutdown_connections 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c12cd0 0x55b8e7c15170 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7c02370 0x55b8e7c0e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 --2- 172.21.15.122:0/3816726286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c06bc0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f0552ffd640 0 client.17595 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 >> 172.21.15.122:0/3816726286 conn(0x55b8e7bf8010 msgr2=0x55b8e7bf9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 shutdown_connections 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 1 -- 172.21.15.122:0/3816726286 wait complete. 2024-09-12T02:01:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.720+0000 7f057c9ed740 20 client.17595 trim_cache size 0 max 16384 2024-09-12T02:01:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 Processor -- start 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 -- start start 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7bf83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05741b1f50 con 0x55b8e7c02370 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05741af790 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f057c9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05741ad5c0 con 0x55b8e7bfd2e0 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.735+0000 7f0573fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7bf83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7bf83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53928/0 (socket says 172.21.15.122:53928) 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 -- 172.21.15.122:0/4238268118 learned_addr learned my addr 172.21.15.122:0/4238268118 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0572ffd640 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f05737fe640 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 -- 172.21.15.122:0/4238268118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 msgr2=0x55b8e7c0e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 -- 172.21.15.122:0/4238268118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 msgr2=0x55b8e7bfcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 -- 172.21.15.122:0/4238268118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e7c06700 con 0x55b8e7c02370 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0572ffd640 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f0573fff640 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7bf83f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f056c005c40 tx=0x7f056c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95ddd4f9d64b475 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f05597fa640 1 -- 172.21.15.122:0/4238268118 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c003c30 con 0x55b8e7c02370 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f05597fa640 1 -- 172.21.15.122:0/4238268118 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056c004530 con 0x55b8e7c02370 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.736+0000 7f05597fa640 1 -- 172.21.15.122:0/4238268118 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c00fcb0 con 0x55b8e7c02370 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 -- 172.21.15.122:0/4238268118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 msgr2=0x55b8e7bf83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7bf83f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f056c005c40 tx=0x7f056c0047e0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 -- 172.21.15.122:0/4238268118 shutdown_connections 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7bf83f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 --2- 172.21.15.122:0/4238268118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7bfcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 -- 172.21.15.122:0/4238268118 >> 172.21.15.122:0/4238268118 conn(0x55b8e7bf8010 msgr2=0x55b8e7c01b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 -- 172.21.15.122:0/4238268118 shutdown_connections 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 -- 172.21.15.122:0/4238268118 wait complete. 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.737+0000 7f057c9ed740 1 -- start start 2024-09-12T02:01:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 Processor -- start 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c0e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7c0b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05741adc50 con 0x55b8e7c02370 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05741af0c0 con 0x55b8e7bfc9a0 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05741ae320 con 0x55b8e7bfd2e0 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f057c9ed740 10 client.0 _ll_register_callbacks cb 0x7f0564009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7c0b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7c0b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53934/0 (socket says 172.21.15.122:53934) 2024-09-12T02:01:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 -- 172.21.15.122:0/1108632585 learned_addr learned my addr 172.21.15.122:0/1108632585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f05737fe640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c0e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0572ffd640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 -- 172.21.15.122:0/1108632585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 msgr2=0x55b8e7c0ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 -- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 msgr2=0x55b8e7c0e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c0e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.738+0000 7f0573fff640 1 -- 172.21.15.122:0/1108632585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e7c7ba70 con 0x55b8e7c02370 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0573fff640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7c0b100 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f056c01b040 tx=0x7f056c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ce745f533f677 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0552ffd640 1 -- 172.21.15.122:0/1108632585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c033020 con 0x55b8e7c02370 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0552ffd640 1 -- 172.21.15.122:0/1108632585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f056c003e40 con 0x55b8e7c02370 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056c004530 con 0x55b8e7c02370 2024-09-12T02:01:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0552ffd640 1 -- 172.21.15.122:0/1108632585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f056c00fe60 con 0x55b8e7c02370 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 10 client.17601 fetch_fsmap learned FSMap version 32 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b8e7c7cd80 con 0x55b8e7c02370 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0552ffd640 1 -- 172.21.15.122:0/1108632585 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056c004530 con 0x55b8e7c02370 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f0552ffd640 1 -- 172.21.15.122:0/1108632585 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f056c024410 con 0x55b8e7c02370 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 10 client.17601 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 20 client.17601 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c7d4e0 0x55b8e7c7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 4 client.17601 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.739+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8e7b67d70 con 0x55b8e7c7d4e0 2024-09-12T02:01:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.740+0000 7f05737fe640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c7d4e0 0x55b8e7c7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.740+0000 7f05737fe640 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c7d4e0 0x55b8e7c7f8d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b0afde651415e1 server_cookie=41916049584c79f4 in_seq=0 out_seq=0 2024-09-12T02:01:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.740+0000 7f0552ffd640 10 client.17601 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f0552ffd640 1 -- 172.21.15.122:0/1108632585 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8e7b67d70 con 0x55b8e7c7d4e0 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f0552ffd640 10 client.17601 handle_command_reply: tid=0 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 client.17601 shutdown 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 msgr2=0x55b8e7c0b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7c0b100 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f056c01b040 tx=0x7f056c00f710 comp rx=0 tx=0).stop 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 shutdown_connections 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b8e7c7d4e0 0x55b8e7c7f8d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b8e7c02370 0x55b8e7c0b100 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8e7bfd2e0 0x55b8e7c0ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f0552ffd640 0 client.17601 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 --2- 172.21.15.122:0/1108632585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b8e7bfc9a0 0x55b8e7c0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.905+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 >> 172.21.15.122:0/1108632585 conn(0x55b8e7bf8010 msgr2=0x55b8e7bf69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 shutdown_connections 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f057c9ed740 1 -- 172.21.15.122:0/1108632585 wait complete. 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f057c9ed740 20 client.17601 trim_cache size 0 max 16384 2024-09-12T02:01:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:01:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:01:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:01:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:01:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f055c0a2c30 msgr2=0x7f055c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f055c0a2c30 0x7f055c0a50f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55b8e7bfdd10 tx=0x7f0564005f90 comp rx=0 tx=0).stop 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 msgr2=0x7f057419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f057419e8c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0568009230 tx=0x7f05680042f0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 shutdown_connections 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f055c0a2c30 0x7f055c0a50f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05741056b0 0x7f05741a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0574102d00 0x7f057419e8c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 --2- 172.21.15.122:0/234912167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05741003b0 0x7f057419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.906+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 >> 172.21.15.122:0/234912167 conn(0x7f05740fc1e0 msgr2=0x7f0574100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.907+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 shutdown_connections 2024-09-12T02:01:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:07.907+0000 7f0579b76640 1 -- 172.21.15.122:0/234912167 wait complete. 2024-09-12T02:01:08.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 15794 18.48 MB/sec warmup 55 sec latency 91.425 ms 2024-09-12T02:01:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:07 smithi151 ceph-mon[35229]: pgmap v774: 97 pgs: 97 active+clean; 516 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 445 op/s 2024-09-12T02:01:08.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:07 smithi183 ceph-mon[35127]: pgmap v774: 97 pgs: 97 active+clean; 516 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 445 op/s 2024-09-12T02:01:08.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:07 smithi122 ceph-mon[30925]: pgmap v774: 97 pgs: 97 active+clean; 516 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 445 op/s 2024-09-12T02:01:09.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 16723 19.05 MB/sec warmup 56 sec latency 65.432 ms 2024-09-12T02:01:10.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 17336 19.57 MB/sec warmup 57 sec latency 97.795 ms 2024-09-12T02:01:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:09 smithi151 ceph-mon[35229]: pgmap v775: 97 pgs: 97 active+clean; 523 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.8 MiB/s wr, 493 op/s 2024-09-12T02:01:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:09 smithi183 ceph-mon[35127]: pgmap v775: 97 pgs: 97 active+clean; 523 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.8 MiB/s wr, 493 op/s 2024-09-12T02:01:10.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:09 smithi122 ceph-mon[30925]: pgmap v775: 97 pgs: 97 active+clean; 523 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.8 MiB/s wr, 493 op/s 2024-09-12T02:01:11.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 17896 19.80 MB/sec warmup 58 sec latency 81.856 ms 2024-09-12T02:01:12.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 18324 19.80 MB/sec warmup 59 sec latency 85.750 ms 2024-09-12T02:01:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:11 smithi151 ceph-mon[35229]: pgmap v776: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.9 MiB/s wr, 490 op/s 2024-09-12T02:01:12.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:11 smithi183 ceph-mon[35127]: pgmap v776: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.9 MiB/s wr, 490 op/s 2024-09-12T02:01:12.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:11 smithi122 ceph-mon[30925]: pgmap v776: 97 pgs: 97 active+clean; 516 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.9 MiB/s wr, 490 op/s 2024-09-12T02:01:13.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 18615 19.67 MB/sec warmup 60 sec latency 396.274 ms 2024-09-12T02:01:14.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 18638 19.36 MB/sec warmup 61 sec latency 1275.020 ms 2024-09-12T02:01:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:13 smithi151 ceph-mon[35229]: pgmap v777: 97 pgs: 97 active+clean; 506 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.6 MiB/s wr, 485 op/s 2024-09-12T02:01:14.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:13 smithi183 ceph-mon[35127]: pgmap v777: 97 pgs: 97 active+clean; 506 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.6 MiB/s wr, 485 op/s 2024-09-12T02:01:14.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:13 smithi122 ceph-mon[30925]: pgmap v777: 97 pgs: 97 active+clean; 506 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.6 MiB/s wr, 485 op/s 2024-09-12T02:01:15.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 19003 19.28 MB/sec warmup 62 sec latency 316.180 ms 2024-09-12T02:01:16.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 19500 19.33 MB/sec warmup 63 sec latency 278.893 ms 2024-09-12T02:01:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:15 smithi151 ceph-mon[35229]: pgmap v778: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.1 MiB/s wr, 588 op/s 2024-09-12T02:01:16.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:15 smithi183 ceph-mon[35127]: pgmap v778: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.1 MiB/s wr, 588 op/s 2024-09-12T02:01:16.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:15 smithi122 ceph-mon[30925]: pgmap v778: 97 pgs: 97 active+clean; 529 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.1 MiB/s wr, 588 op/s 2024-09-12T02:01:17.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 20244 19.66 MB/sec warmup 64 sec latency 147.653 ms 2024-09-12T02:01:18.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 21038 20.00 MB/sec warmup 65 sec latency 78.347 ms 2024-09-12T02:01:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:17 smithi151 ceph-mon[35229]: pgmap v779: 97 pgs: 97 active+clean; 544 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 602 op/s 2024-09-12T02:01:18.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:17 smithi183 ceph-mon[35127]: pgmap v779: 97 pgs: 97 active+clean; 544 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 602 op/s 2024-09-12T02:01:18.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:17 smithi122 ceph-mon[30925]: pgmap v779: 97 pgs: 97 active+clean; 544 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 602 op/s 2024-09-12T02:01:19.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 21650 20.32 MB/sec warmup 66 sec latency 124.219 ms 2024-09-12T02:01:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:18 smithi151 ceph-mon[35229]: pgmap v780: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-09-12T02:01:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:18 smithi183 ceph-mon[35127]: pgmap v780: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-09-12T02:01:19.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:18 smithi122 ceph-mon[30925]: pgmap v780: 97 pgs: 97 active+clean; 519 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 850 op/s 2024-09-12T02:01:20.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 22205 20.58 MB/sec warmup 67 sec latency 73.928 ms 2024-09-12T02:01:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 -- 172.21.15.122:0/3185165978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc100a60 msgr2=0x7f96fc100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 --2- 172.21.15.122:0/3185165978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc100a60 0x7f96fc100e60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009900 tx=0x7f96f002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 -- 172.21.15.122:0/3185165978 shutdown_connections 2024-09-12T02:01:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 --2- 172.21.15.122:0/3185165978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96fc102610 0x7f96fc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 --2- 172.21.15.122:0/3185165978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 0x7f96fc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 --2- 172.21.15.122:0/3185165978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc100a60 0x7f96fc100e60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 -- 172.21.15.122:0/3185165978 >> 172.21.15.122:0/3185165978 conn(0x7f96fc0fc1e0 msgr2=0x7f96fc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.003+0000 7f9702b66640 1 -- 172.21.15.122:0/3185165978 shutdown_connections 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 -- 172.21.15.122:0/3185165978 wait complete. 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 Processor -- start 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 -- start start 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96fc100a60 0x7f96fc196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 0x7f96fc197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 0x7f96fc19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10c730 con 0x7f96fc102610 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10c5b0 con 0x7f96fc100a60 2024-09-12T02:01:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9702b66640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10c8b0 con 0x7f96fc101c50 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9701103640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 0x7f96fc19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9701103640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 0x7f96fc19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43248/0 (socket says 172.21.15.122:43248) 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9701103640 1 -- 172.21.15.122:0/3735954909 learned_addr learned my addr 172.21.15.122:0/3735954909 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f9700902640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96fc100a60 0x7f96fc196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.004+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 0x7f96fc197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9701103640 1 -- 172.21.15.122:0/3735954909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 msgr2=0x7f96fc197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9701103640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 0x7f96fc197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9701103640 1 -- 172.21.15.122:0/3735954909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96fc100a60 msgr2=0x7f96fc196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9701103640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96fc100a60 0x7f96fc196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9701103640 1 -- 172.21.15.122:0/3735954909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc19e920 con 0x7f96fc102610 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f96fbfff640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 0x7f96fc197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9701103640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 0x7f96fc19e280 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f96f400eb10 tx=0x7f96f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f96f9ffb640 1 -- 172.21.15.122:0/3735954909 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4004b20 con 0x7f96fc102610 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f96f9ffb640 1 -- 172.21.15.122:0/3735954909 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4004cc0 con 0x7f96fc102610 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc06a360 con 0x7f96fc102610 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f96f9ffb640 1 -- 172.21.15.122:0/3735954909 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4005440 con 0x7f96fc102610 2024-09-12T02:01:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.005+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc06a880 con 0x7f96fc102610 2024-09-12T02:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.007+0000 7f96f9ffb640 1 -- 172.21.15.122:0/3735954909 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f96f4005600 con 0x7f96fc102610 2024-09-12T02:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 Processor -- start 2024-09-12T02:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 -- start start 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab0e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab13780 0x55676ab15c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55676aa6d2f0 con 0x55676ab0e6f0 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55676aa15270 con 0x55676ab13780 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f97059de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55676aa692b0 con 0x55676ab0ddb0 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f96f9ffb640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96ec0a2cb0 0x7f96ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f9700902640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab0e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f9700902640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43262/0 (socket says 172.21.15.122:43262) 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f9700902640 1 -- 172.21.15.122:0/1394829650 learned_addr learned my addr 172.21.15.122:0/1394829650 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f96fbfff640 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab0e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33794/0 (socket says 172.21.15.122:33794) 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f96fbfff640 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab13780 0x55676ab15c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.008+0000 7f9701103640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96ec0a2cb0 0x7f96ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96fbfff640 1 -- 172.21.15.122:0/1394829650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab13780 msgr2=0x55676ab15c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96fbfff640 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab13780 0x55676ab15c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96fbfff640 1 -- 172.21.15.122:0/1394829650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 msgr2=0x55676ab131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96fbfff640 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96fbfff640 1 -- 172.21.15.122:0/1394829650 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55676aa14340 con 0x55676ab0ddb0 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96f9ffb640 1 -- 172.21.15.122:0/3735954909 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f96f40d13a0 con 0x7f96fc102610 2024-09-12T02:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96fbfff640 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab0e1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96e8011b20 tx=0x7f96e8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e5c9efbec2ec03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f9701103640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96ec0a2cb0 0x7f96ec0a5170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f96f40986c0 tx=0x7f96f40c8790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96e6ffd640 1 -- 172.21.15.122:0/1394829650 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8011050 con 0x55676ab0ddb0 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96e6ffd640 1 -- 172.21.15.122:0/1394829650 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e800fd60 con 0x55676ab0ddb0 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.009+0000 7f96e6ffd640 1 -- 172.21.15.122:0/1394829650 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e80047a0 con 0x55676ab0ddb0 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 -- 172.21.15.122:0/1394829650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 msgr2=0x55676ab0e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab0e1b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96e8011b20 tx=0x7f96e8005850 comp rx=0 tx=0).stop 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 -- 172.21.15.122:0/1394829650 shutdown_connections 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab13780 0x55676ab15c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 --2- 172.21.15.122:0/1394829650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab0e1b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 -- 172.21.15.122:0/1394829650 >> 172.21.15.122:0/1394829650 conn(0x55676ab05010 msgr2=0x55676ab05410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 -- 172.21.15.122:0/1394829650 shutdown_connections 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 -- 172.21.15.122:0/1394829650 wait complete. 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 -- start start 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.010+0000 7f97059de740 1 Processor -- start 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab18060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55676ab166f0 con 0x55676ab13780 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55676a9f6290 con 0x55676ab0e6f0 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55676ab16870 con 0x55676ab0ddb0 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f97059de740 10 client.0 _ll_register_callbacks cb 0x55676aa27140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43272/0 (socket says 172.21.15.122:43272) 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96fbfff640 1 -- 172.21.15.122:0/3934923574 learned_addr learned my addr 172.21.15.122:0/3934923574 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96fbfff640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab18060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 -- 172.21.15.122:0/3934923574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 msgr2=0x55676ab18060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab18060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 -- 172.21.15.122:0/3934923574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 msgr2=0x55676ab1f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 -- 172.21.15.122:0/3934923574 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55676a780210 con 0x55676ab0e6f0 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96fbfff640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f9700902640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab185a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f96f0031440 tx=0x7f96f00043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ba1014280430ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.011+0000 7f96dbfff640 1 -- 172.21.15.122:0/3934923574 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0035040 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e800fd60 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f96dbfff640 1 -- 172.21.15.122:0/3934923574 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f96f0005450 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f96dbfff640 1 -- 172.21.15.122:0/3934923574 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96f0005650 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 10 client.26726 fetch_fsmap learned FSMap version 32 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55676a84c620 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f96dbfff640 1 -- 172.21.15.122:0/3934923574 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e800fd60 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f96dbfff640 1 -- 172.21.15.122:0/3934923574 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f96f0031b40 con 0x55676ab0e6f0 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 10 client.26726 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 20 client.26726 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab23cb0 0x55676ab260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 4 client.26726 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55676aa7aa20 con 0x55676ab23cb0 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.012+0000 7f96fbfff640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab23cb0 0x55676ab260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.013+0000 7f96fbfff640 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab23cb0 0x55676ab260a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf2afa10e06c888 server_cookie=a8152228669952cf in_seq=0 out_seq=0 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.013+0000 7f96dbfff640 10 client.26726 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.013+0000 7f96dbfff640 1 -- 172.21.15.122:0/3934923574 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55676aa7aa20 con 0x55676ab23cb0 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.013+0000 7f96dbfff640 10 client.26726 handle_command_reply: tid=0 2024-09-12T02:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.013+0000 7f97059de740 1 client.26726 shutdown 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 msgr2=0x55676ab185a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab185a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f96f0031440 tx=0x7f96f00043b0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 shutdown_connections 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab23cb0 0x55676ab260a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f96dbfff640 0 client.26726 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab185a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 --2- 172.21.15.122:0/3934923574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab18060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 >> 172.21.15.122:0/3934923574 conn(0x55676ab05010 msgr2=0x55676ab06820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 shutdown_connections 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 1 -- 172.21.15.122:0/3934923574 wait complete. 2024-09-12T02:01:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.014+0000 7f97059de740 20 client.26726 trim_cache size 0 max 16384 2024-09-12T02:01:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.029+0000 7f97059de740 1 Processor -- start 2024-09-12T02:01:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.029+0000 7f97059de740 1 -- start start 2024-09-12T02:01:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.029+0000 7f97059de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0ddb0 0x55676ab06b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.029+0000 7f97059de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab1f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f97059de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab13780 0x55676ab2a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f97059de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a9670 con 0x55676ab0e6f0 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f97059de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a6eb0 con 0x55676ab0ddb0 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f97059de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a4ce0 con 0x55676ab13780 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab1f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0ddb0 0x55676ab06b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab1f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43276/0 (socket says 172.21.15.122:43276) 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0ddb0 0x55676ab06b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:58528/0 (socket says 172.21.15.122:58528) 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 -- 172.21.15.122:0/4128924683 learned_addr learned my addr 172.21.15.122:0/4128924683 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f96fbfff640 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab13780 0x55676ab2a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 -- 172.21.15.122:0/4128924683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab13780 msgr2=0x55676ab2a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab13780 0x55676ab2a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 -- 172.21.15.122:0/4128924683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0ddb0 msgr2=0x55676ab06b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0ddb0 0x55676ab06b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 -- 172.21.15.122:0/4128924683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55676ab26700 con 0x55676ab0e6f0 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f9700902640 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab1f530 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f0031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386aca5674697dac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f96e67fc640 1 -- 172.21.15.122:0/4128924683 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0035070 con 0x55676ab0e6f0 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f96e67fc640 1 -- 172.21.15.122:0/4128924683 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0055020 con 0x55676ab0e6f0 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.030+0000 7f96e67fc640 1 -- 172.21.15.122:0/4128924683 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0031910 con 0x55676ab0e6f0 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 -- 172.21.15.122:0/4128924683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 msgr2=0x55676ab1f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab1f530 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f0031f90 comp rx=0 tx=0).stop 2024-09-12T02:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 -- 172.21.15.122:0/4128924683 shutdown_connections 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab13780 0x55676ab2a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab0e6f0 0x55676ab1f530 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 --2- 172.21.15.122:0/4128924683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0ddb0 0x55676ab06b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 -- 172.21.15.122:0/4128924683 >> 172.21.15.122:0/4128924683 conn(0x55676ab05010 msgr2=0x55676ab053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 -- 172.21.15.122:0/4128924683 shutdown_connections 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 -- 172.21.15.122:0/4128924683 wait complete. 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 -- start start 2024-09-12T02:01:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.031+0000 7f97059de740 1 Processor -- start 2024-09-12T02:01:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab1cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab1d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a5370 con 0x55676ab13780 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a67e0 con 0x55676ab0e6f0 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a5a40 con 0x55676ab0ddb0 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 10 client.0 _ll_register_callbacks cb 0x7f96f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43280/0 (socket says 172.21.15.122:43280) 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 -- 172.21.15.122:0/323034089 learned_addr learned my addr 172.21.15.122:0/323034089 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f9700902640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab1d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab1cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 -- 172.21.15.122:0/323034089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 msgr2=0x55676ab1cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab1cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 -- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 msgr2=0x55676ab1d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab1d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 -- 172.21.15.122:0/323034089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55676ab1ea90 con 0x55676ab13780 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96fbfff640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1e480 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x55676ab17590 tx=0x7f96e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd263e733f511b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96dbfff640 1 -- 172.21.15.122:0/323034089 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8033070 con 0x55676ab13780 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f0055020 con 0x55676ab13780 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.032+0000 7f96dbfff640 1 -- 172.21.15.122:0/323034089 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f96e80035f0 con 0x55676ab13780 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f96dbfff640 1 -- 172.21.15.122:0/323034089 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96e80037f0 con 0x55676ab13780 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 10 client.17619 fetch_fsmap learned FSMap version 32 2024-09-12T02:01:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55676ab8ec50 con 0x55676ab13780 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f96dbfff640 1 -- 172.21.15.122:0/323034089 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f0055020 con 0x55676ab13780 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f96dbfff640 1 -- 172.21.15.122:0/323034089 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f96e8039040 con 0x55676ab13780 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 10 client.17619 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 10 client.17619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 20 client.17619 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab8f410 0x55676ab91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 4 client.17619 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55676aa74d70 con 0x55676ab8f410 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.033+0000 7f9700902640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab8f410 0x55676ab91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.034+0000 7f96dbfff640 10 client.17619 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.034+0000 7f9700902640 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab8f410 0x55676ab91800 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6023f90bb2d88aaf server_cookie=ac3a2d4c0e01b9d2 in_seq=0 out_seq=0 2024-09-12T02:01:21.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 22666 20.64 MB/sec warmup 68 sec latency 80.999 ms 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f96dbfff640 1 -- 172.21.15.122:0/323034089 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55676aa74d70 con 0x55676ab8f410 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f96dbfff640 10 client.17619 handle_command_reply: tid=0 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 client.17619 shutdown 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 msgr2=0x55676ab1e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1e480 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x55676ab17590 tx=0x7f96e8005850 comp rx=0 tx=0).stop 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 shutdown_connections 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55676ab8f410 0x55676ab91800 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55676ab13780 0x55676ab1e480 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55676ab0e6f0 0x55676ab1d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f96dbfff640 0 client.17619 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 --2- 172.21.15.122:0/323034089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55676ab0ddb0 0x55676ab1cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.189+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 >> 172.21.15.122:0/323034089 conn(0x55676ab05010 msgr2=0x55676ab05c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.190+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 shutdown_connections 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.190+0000 7f97059de740 1 -- 172.21.15.122:0/323034089 wait complete. 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.190+0000 7f97059de740 20 client.17619 trim_cache size 0 max 16384 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:01:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:01:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:01:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96ec0a2cb0 msgr2=0x7f96ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96ec0a2cb0 0x7f96ec0a5170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f96f40986c0 tx=0x7f96f40c8790 comp rx=0 tx=0).stop 2024-09-12T02:01:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 msgr2=0x7f96fc19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 0x7f96fc19e280 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f96f400eb10 tx=0x7f96f400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 shutdown_connections 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f96ec0a2cb0 0x7f96ec0a5170 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96fc102610 0x7f96fc19e280 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96fc101c50 0x7f96fc197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 --2- 172.21.15.122:0/3735954909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96fc100a60 0x7f96fc196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 >> 172.21.15.122:0/3735954909 conn(0x7f96fc0fc1e0 msgr2=0x7f96fc104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 shutdown_connections 2024-09-12T02:01:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:21.191+0000 7f9702b66640 1 -- 172.21.15.122:0/3735954909 wait complete. 2024-09-12T02:01:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:21 smithi183 ceph-mon[35127]: pgmap v781: 97 pgs: 97 active+clean; 498 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 847 op/s 2024-09-12T02:01:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:21 smithi183 ceph-mon[35127]: daemon mds.f restarted 2024-09-12T02:01:21.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setuser ceph since I am not root 2024-09-12T02:01:21.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setgroup ceph since I am not root 2024-09-12T02:01:21.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:21 smithi183 ceph-mds[64439]: -- 172.21.15.183:0/2902254917 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634acd2dc20 con 0x5634acd0a800 2024-09-12T02:01:21.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: starting mds.f at 2024-09-12T02:01:22.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 23106 20.68 MB/sec warmup 69 sec latency 116.289 ms 2024-09-12T02:01:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:21 smithi151 ceph-mon[35229]: pgmap v781: 97 pgs: 97 active+clean; 498 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 847 op/s 2024-09-12T02:01:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:21 smithi151 ceph-mon[35229]: daemon mds.f restarted 2024-09-12T02:01:22.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:21 smithi122 ceph-mon[30925]: pgmap v781: 97 pgs: 97 active+clean; 498 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 847 op/s 2024-09-12T02:01:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:21 smithi122 ceph-mon[30925]: daemon mds.f restarted 2024-09-12T02:01:23.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:22 smithi151 ceph-mon[35229]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:01:23.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:22 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6834/682523101,v1:172.21.15.183:6835/682523101] up:boot 2024-09-12T02:01:23.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:22 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:23.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:22 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:23.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:22 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:23.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 23698 20.62 MB/sec warmup 70 sec latency 57.737 ms 2024-09-12T02:01:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:22 smithi183 ceph-mon[35127]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:01:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:22 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6834/682523101,v1:172.21.15.183:6835/682523101] up:boot 2024-09-12T02:01:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:22 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:22 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:22 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:22 smithi122 ceph-mon[30925]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:01:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:22 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6834/682523101,v1:172.21.15.183:6835/682523101] up:boot 2024-09-12T02:01:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:22 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:22 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:22 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:24.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 24457 20.91 MB/sec warmup 71 sec latency 66.265 ms 2024-09-12T02:01:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:23 smithi151 ceph-mon[35229]: pgmap v783: 97 pgs: 97 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 848 op/s 2024-09-12T02:01:24.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:23 smithi183 ceph-mon[35127]: pgmap v783: 97 pgs: 97 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 848 op/s 2024-09-12T02:01:24.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:23 smithi122 ceph-mon[30925]: pgmap v783: 97 pgs: 97 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 848 op/s 2024-09-12T02:01:25.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 25107 21.19 MB/sec warmup 72 sec latency 224.190 ms 2024-09-12T02:01:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:24 smithi151 ceph-mon[35229]: pgmap v784: 97 pgs: 97 active+clean; 499 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 872 op/s 2024-09-12T02:01:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:24 smithi183 ceph-mon[35127]: pgmap v784: 97 pgs: 97 active+clean; 499 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 872 op/s 2024-09-12T02:01:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:24 smithi122 ceph-mon[30925]: pgmap v784: 97 pgs: 97 active+clean; 499 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 872 op/s 2024-09-12T02:01:26.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 25909 21.48 MB/sec warmup 73 sec latency 90.767 ms 2024-09-12T02:01:27.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 26489 21.67 MB/sec warmup 74 sec latency 63.776 ms 2024-09-12T02:01:28.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 27069 21.88 MB/sec warmup 75 sec latency 136.067 ms 2024-09-12T02:01:28.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:27 smithi151 ceph-mon[35229]: pgmap v785: 97 pgs: 97 active+clean; 497 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.9 MiB/s wr, 769 op/s 2024-09-12T02:01:28.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:27 smithi183 ceph-mon[35127]: pgmap v785: 97 pgs: 97 active+clean; 497 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.9 MiB/s wr, 769 op/s 2024-09-12T02:01:28.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:27 smithi122 ceph-mon[30925]: pgmap v785: 97 pgs: 97 active+clean; 497 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.9 MiB/s wr, 769 op/s 2024-09-12T02:01:29.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 27602 21.90 MB/sec warmup 76 sec latency 83.609 ms 2024-09-12T02:01:30.004 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:29 smithi183 ceph-mon[35127]: pgmap v786: 97 pgs: 97 active+clean; 502 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.0 MiB/s wr, 483 op/s 2024-09-12T02:01:30.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:29 smithi122 ceph-mon[30925]: pgmap v786: 97 pgs: 97 active+clean; 502 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.0 MiB/s wr, 483 op/s 2024-09-12T02:01:30.051 INFO:tasks.workunit.client.0.smithi122.stdout: 10 28058 21.89 MB/sec warmup 77 sec latency 332.154 ms 2024-09-12T02:01:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:29 smithi151 ceph-mon[35229]: pgmap v786: 97 pgs: 97 active+clean; 502 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.0 MiB/s wr, 483 op/s 2024-09-12T02:01:31.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 28577 21.91 MB/sec warmup 78 sec latency 137.345 ms 2024-09-12T02:01:32.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 29351 22.18 MB/sec warmup 79 sec latency 50.280 ms 2024-09-12T02:01:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:31 smithi151 ceph-mon[35229]: pgmap v787: 97 pgs: 97 active+clean; 489 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.4 MiB/s wr, 477 op/s 2024-09-12T02:01:32.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:31 smithi183 ceph-mon[35127]: pgmap v787: 97 pgs: 97 active+clean; 489 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.4 MiB/s wr, 477 op/s 2024-09-12T02:01:32.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:31 smithi122 ceph-mon[30925]: pgmap v787: 97 pgs: 97 active+clean; 489 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.4 MiB/s wr, 477 op/s 2024-09-12T02:01:33.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 30068 22.38 MB/sec warmup 80 sec latency 85.577 ms 2024-09-12T02:01:34.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 30767 22.66 MB/sec warmup 81 sec latency 50.735 ms 2024-09-12T02:01:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:33 smithi151 ceph-mon[35229]: pgmap v788: 97 pgs: 97 active+clean; 499 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.2 MiB/s wr, 553 op/s 2024-09-12T02:01:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:33 smithi183 ceph-mon[35127]: pgmap v788: 97 pgs: 97 active+clean; 499 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.2 MiB/s wr, 553 op/s 2024-09-12T02:01:34.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:33 smithi122 ceph-mon[30925]: pgmap v788: 97 pgs: 97 active+clean; 499 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.2 MiB/s wr, 553 op/s 2024-09-12T02:01:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 -- 172.21.15.122:0/580564165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18102600 msgr2=0x7f8e18108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/580564165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18102600 0x7f8e18108e90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009ca0 tx=0x7f8e1002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 -- 172.21.15.122:0/580564165 shutdown_connections 2024-09-12T02:01:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/580564165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18102600 0x7f8e18108e90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/580564165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e18101c40 0x7f8e181020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/580564165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18100a50 0x7f8e18100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 -- 172.21.15.122:0/580564165 >> 172.21.15.122:0/580564165 conn(0x7f8e180fc1b0 msgr2=0x7f8e180fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 -- 172.21.15.122:0/580564165 shutdown_connections 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.288+0000 7f8e1ce67640 1 -- 172.21.15.122:0/580564165 wait complete. 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 Processor -- start 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 -- start start 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e18100a50 0x7f8e1806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 0x7f8e1819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18102600 0x7f8e1819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810c6d0 con 0x7f8e18101c40 2024-09-12T02:01:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810c550 con 0x7f8e18100a50 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1ce67640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1810c850 con 0x7f8e18102600 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 0x7f8e1819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 0x7f8e1819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37772/0 (socket says 172.21.15.122:37772) 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 -- 172.21.15.122:0/416848294 learned_addr learned my addr 172.21.15.122:0/416848294 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e1659c640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e18100a50 0x7f8e1806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18102600 0x7f8e1819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 -- 172.21.15.122:0/416848294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18102600 msgr2=0x7f8e1819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18102600 0x7f8e1819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 -- 172.21.15.122:0/416848294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e18100a50 msgr2=0x7f8e1806aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e18100a50 0x7f8e1806aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.289+0000 7f8e15d9b640 1 -- 172.21.15.122:0/416848294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1819f990 con 0x7f8e18101c40 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.290+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 0x7f8e1819ca80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9f0 tx=0x7f8e0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.290+0000 7f8e077fe640 1 -- 172.21.15.122:0/416848294 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00cd80 con 0x7f8e18101c40 2024-09-12T02:01:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.290+0000 7f8e077fe640 1 -- 172.21.15.122:0/416848294 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e0c010910 con 0x7f8e18101c40 2024-09-12T02:01:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.290+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1819fc80 con 0x7f8e18101c40 2024-09-12T02:01:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.290+0000 7f8e077fe640 1 -- 172.21.15.122:0/416848294 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c018990 con 0x7f8e18101c40 2024-09-12T02:01:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.290+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181a01d0 con 0x7f8e18101c40 2024-09-12T02:01:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.292+0000 7f8e077fe640 1 -- 172.21.15.122:0/416848294 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8e0c010ac0 con 0x7f8e18101c40 2024-09-12T02:01:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.292+0000 7f8e1fbff740 1 Processor -- start 2024-09-12T02:01:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 -- start start 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770edb0 0x55c8f770f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f77141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f7714780 0x55c8f7716c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c8f766e2f0 con 0x55c8f770f6f0 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c8f7616270 con 0x55c8f7714780 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8f766a2b0 con 0x55c8f770edb0 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e077fe640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8e000a2c30 0x7f8e000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f77141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f77141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37786/0 (socket says 172.21.15.122:37786) 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 -- 172.21.15.122:0/3072512566 learned_addr learned my addr 172.21.15.122:0/3072512566 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f7714780 0x55c8f7716c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770edb0 0x55c8f770f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8e000a2c30 0x7f8e000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e077fe640 1 -- 172.21.15.122:0/416848294 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8e0c0cf330 con 0x7f8e18101c40 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 -- 172.21.15.122:0/3072512566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770edb0 msgr2=0x55c8f770f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770edb0 0x55c8f770f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 -- 172.21.15.122:0/3072512566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f7714780 msgr2=0x55c8f7716c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f7714780 0x55c8f7716c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.293+0000 7f8e1659c640 1 -- 172.21.15.122:0/3072512566 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8f7615340 con 0x55c8f770f6f0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.294+0000 7f8e1659c640 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f77141b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009ca0 tx=0x7f8e08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d01808e6738d691 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.294+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8e000a2c30 0x7f8e000a50f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004af0 tx=0x7f8e10005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.294+0000 7f8e04ff9640 1 -- 172.21.15.122:0/3072512566 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08004220 con 0x55c8f770f6f0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.294+0000 7f8e04ff9640 1 -- 172.21.15.122:0/3072512566 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e080043c0 con 0x55c8f770f6f0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.294+0000 7f8e04ff9640 1 -- 172.21.15.122:0/3072512566 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08019970 con 0x55c8f770f6f0 2024-09-12T02:01:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 -- 172.21.15.122:0/3072512566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 msgr2=0x55c8f77141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f77141b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009ca0 tx=0x7f8e08011810 comp rx=0 tx=0).stop 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 -- 172.21.15.122:0/3072512566 shutdown_connections 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f7714780 0x55c8f7716c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f77141b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/3072512566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770edb0 0x55c8f770f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 -- 172.21.15.122:0/3072512566 >> 172.21.15.122:0/3072512566 conn(0x55c8f7706010 msgr2=0x55c8f7706410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 -- 172.21.15.122:0/3072512566 shutdown_connections 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 -- 172.21.15.122:0/3072512566 wait complete. 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.295+0000 7f8e1fbff740 1 -- start start 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 Processor -- start 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7718fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7720520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c8f7717750 con 0x55c8f7714780 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c8f75f7290 con 0x55c8f770edb0 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8f77178d0 con 0x55c8f770f6f0 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1fbff740 10 client.0 _ll_register_callbacks cb 0x55c8f7628140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e16d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7720520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e15d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7718fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e16d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7720520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37802/0 (socket says 172.21.15.122:37802) 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e16d9d640 1 -- 172.21.15.122:0/2343937691 learned_addr learned my addr 172.21.15.122:0/2343937691 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 -- 172.21.15.122:0/2343937691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 msgr2=0x55c8f77194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 -- 172.21.15.122:0/2343937691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 msgr2=0x55c8f7720520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7720520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 -- 172.21.15.122:0/2343937691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8f7381210 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7720520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8e1659c640 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7718fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e08011d10 tx=0x7f8e08004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3ca45ad7a23b8db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.296+0000 7f8dfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8dfdffb640 1 -- 172.21.15.122:0/2343937691 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08028070 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e080043c0 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8dfdffb640 1 -- 172.21.15.122:0/2343937691 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8e08003800 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8dfdffb640 1 -- 172.21.15.122:0/2343937691 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e08034050 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 10 client.26750 fetch_fsmap learned FSMap version 34 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8f744d620 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8dfdffb640 1 -- 172.21.15.122:0/2343937691 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e080043c0 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8dfdffb640 1 -- 172.21.15.122:0/2343937691 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f8e080049f0 con 0x55c8f770edb0 2024-09-12T02:01:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 10 client.26750 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 20 client.26750 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f7724a20 0x55c8f7726e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 4 client.26750 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.297+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8f767ba20 con 0x55c8f7724a20 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.298+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f7724a20 0x55c8f7726e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.298+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f7724a20 0x55c8f7726e10 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc96f82bc4a615f3 server_cookie=3a14ee8a801e9269 in_seq=0 out_seq=0 2024-09-12T02:01:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.298+0000 7f8dfdffb640 10 client.26750 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8dfdffb640 1 -- 172.21.15.122:0/2343937691 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8f767ba20 con 0x55c8f7724a20 2024-09-12T02:01:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8dfdffb640 10 client.26750 handle_command_reply: tid=0 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 client.26750 shutdown 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 msgr2=0x55c8f7718fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7718fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e08011d10 tx=0x7f8e08004600 comp rx=0 tx=0).stop 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 shutdown_connections 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f7724a20 0x55c8f7726e10 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7720520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8dfdffb640 0 client.26750 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/2343937691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7718fa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 >> 172.21.15.122:0/2343937691 conn(0x55c8f7706010 msgr2=0x55c8f7707820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 shutdown_connections 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 1 -- 172.21.15.122:0/2343937691 wait complete. 2024-09-12T02:01:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.299+0000 7f8e1fbff740 20 client.26750 trim_cache size 0 max 16384 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.316+0000 7f8e1fbff740 1 Processor -- start 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.316+0000 7f8e1fbff740 1 -- start start 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7719460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7713ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181a96c0 con 0x55c8f7714780 2024-09-12T02:01:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181a6f60 con 0x55c8f770edb0 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181a4d90 con 0x55c8f770f6f0 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7713ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7713ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37814/0 (socket says 172.21.15.122:37814) 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 -- 172.21.15.122:0/4083394404 learned_addr learned my addr 172.21.15.122:0/4083394404 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e1659c640 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7719460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 -- 172.21.15.122:0/4083394404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 msgr2=0x55c8f77204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 -- 172.21.15.122:0/4083394404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 msgr2=0x55c8f7719460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7719460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 -- 172.21.15.122:0/4083394404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8f7727180 con 0x55c8f7714780 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:01:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7713ea0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8e10005ff0 tx=0x7f8e10005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9f47df9bd32a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8dfffff640 1 -- 172.21.15.122:0/4083394404 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10057070 con 0x55c8f7714780 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8dfffff640 1 -- 172.21.15.122:0/4083394404 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e10036070 con 0x55c8f7714780 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.317+0000 7f8dfffff640 1 -- 172.21.15.122:0/4083394404 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10052430 con 0x55c8f7714780 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 -- 172.21.15.122:0/4083394404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 msgr2=0x55c8f7713ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7713ea0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8e10005ff0 tx=0x7f8e10005df0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 -- 172.21.15.122:0/4083394404 shutdown_connections 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f7714780 0x55c8f7713ea0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f770f6f0 0x55c8f77204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/4083394404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f7719460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 -- 172.21.15.122:0/4083394404 >> 172.21.15.122:0/4083394404 conn(0x55c8f7706010 msgr2=0x55c8f7718d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 -- 172.21.15.122:0/4083394404 shutdown_connections 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.318+0000 7f8e1fbff740 1 -- 172.21.15.122:0/4083394404 wait complete. 2024-09-12T02:01:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 -- start start 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 Processor -- start 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f771d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f771dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f7714780 0x55c8f771eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181a5420 con 0x55c8f770f6f0 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181a6890 con 0x55c8f770edb0 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181a5af0 con 0x55c8f7714780 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1fbff740 10 client.0 _ll_register_callbacks cb 0x7f8e0c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f771dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f771dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37830/0 (socket says 172.21.15.122:37830) 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 -- 172.21.15.122:0/465041476 learned_addr learned my addr 172.21.15.122:0/465041476 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1659c640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f771d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e16d9d640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f7714780 0x55c8f771eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 -- 172.21.15.122:0/465041476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f7714780 msgr2=0x55c8f771eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f7714780 0x55c8f771eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 -- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 msgr2=0x55c8f771d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f771d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 -- 172.21.15.122:0/465041476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8f771f180 con 0x55c8f770f6f0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e1659c640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f771d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8e15d9b640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f771dbb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9c0 tx=0x7f8e0c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c39ac1f6be694b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.319+0000 7f8dfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8dfdffb640 1 -- 172.21.15.122:0/465041476 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0b3030 con 0x55c8f770f6f0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8dfdffb640 1 -- 172.21.15.122:0/465041476 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8e0c086040 con 0x55c8f770f6f0 2024-09-12T02:01:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e10036070 con 0x55c8f770f6f0 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8dfdffb640 1 -- 172.21.15.122:0/465041476 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e0c0982b0 con 0x55c8f770f6f0 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 10 client.17643 fetch_fsmap learned FSMap version 34 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8f77204d0 con 0x55c8f770f6f0 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8dfdffb640 1 -- 172.21.15.122:0/465041476 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e10036070 con 0x55c8f770f6f0 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8dfdffb640 1 -- 172.21.15.122:0/465041476 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f8e0c0ce040 con 0x55c8f770f6f0 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 10 client.17643 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 20 client.17643 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f778fa90 0x55c8f7791eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 4 client.17643 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8f7675d70 con 0x55c8f778fa90 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.320+0000 7f8e1659c640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f778fa90 0x55c8f7791eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.321+0000 7f8dfdffb640 10 client.17643 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.321+0000 7f8e1659c640 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f778fa90 0x55c8f7791eb0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1402dadf3efaf3e server_cookie=4d9f79818b47ba27 in_seq=0 out_seq=0 2024-09-12T02:01:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.559+0000 7f8dfdffb640 1 -- 172.21.15.122:0/465041476 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8f7675d70 con 0x55c8f778fa90 2024-09-12T02:01:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.559+0000 7f8dfdffb640 10 client.17643 handle_command_reply: tid=0 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.559+0000 7f8e1fbff740 1 client.17643 shutdown 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 msgr2=0x55c8f771dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f771dbb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9c0 tx=0x7f8e0c018420 comp rx=0 tx=0).stop 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 shutdown_connections 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c8f778fa90 0x55c8f7791eb0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8f7714780 0x55c8f771eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8f770f6f0 0x55c8f771dbb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 --2- 172.21.15.122:0/465041476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f770edb0 0x55c8f771d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 >> 172.21.15.122:0/465041476 conn(0x55c8f7706010 msgr2=0x55c8f77049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8dfdffb640 0 client.17643 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 shutdown_connections 2024-09-12T02:01:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 1 -- 172.21.15.122:0/465041476 wait complete. 2024-09-12T02:01:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.560+0000 7f8e1fbff740 20 client.17643 trim_cache size 0 max 16384 2024-09-12T02:01:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:01:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:01:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:01:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8e000a2c30 msgr2=0x7f8e000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8e000a2c30 0x7f8e000a50f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004af0 tx=0x7f8e10005d80 comp rx=0 tx=0).stop 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 msgr2=0x7f8e1819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 0x7f8e1819ca80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9f0 tx=0x7f8e0c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 shutdown_connections 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8e000a2c30 0x7f8e000a50f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18102600 0x7f8e1819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18101c40 0x7f8e1819ca80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 --2- 172.21.15.122:0/416848294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e18100a50 0x7f8e1806aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 >> 172.21.15.122:0/416848294 conn(0x7f8e180fc1b0 msgr2=0x7f8e18104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 shutdown_connections 2024-09-12T02:01:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:34.561+0000 7f8e1ce67640 1 -- 172.21.15.122:0/416848294 wait complete. 2024-09-12T02:01:35.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 31233 22.67 MB/sec warmup 82 sec latency 174.228 ms 2024-09-12T02:01:35.702 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setuser ceph since I am not root 2024-09-12T02:01:35.702 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setgroup ceph since I am not root 2024-09-12T02:01:35.703 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:35 smithi183 ceph-mds[64439]: -- 172.21.15.183:0/1611770873 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d0f6b5c20 con 0x558d0f693400 2024-09-12T02:01:35.703 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: starting mds.f at 2024-09-12T02:01:36.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 31844 22.90 MB/sec warmup 83 sec latency 88.467 ms 2024-09-12T02:01:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:35 smithi151 ceph-mon[35229]: pgmap v789: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-09-12T02:01:36.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:35 smithi151 ceph-mon[35229]: daemon mds.f restarted 2024-09-12T02:01:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:35 smithi183 ceph-mon[35127]: pgmap v789: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-09-12T02:01:36.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:35 smithi183 ceph-mon[35127]: daemon mds.f restarted 2024-09-12T02:01:36.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:35 smithi122 ceph-mon[30925]: pgmap v789: 97 pgs: 97 active+clean; 494 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-09-12T02:01:36.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:35 smithi122 ceph-mon[30925]: daemon mds.f restarted 2024-09-12T02:01:37.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 32342 22.80 MB/sec warmup 84 sec latency 68.026 ms 2024-09-12T02:01:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:36 smithi151 ceph-mon[35229]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:01:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:36 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6834/3338703813,v1:172.21.15.183:6835/3338703813] up:boot 2024-09-12T02:01:37.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:36 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:37.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:36 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:37.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:36 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:37.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:36 smithi151 ceph-mon[35229]: pgmap v791: 97 pgs: 97 active+clean; 497 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 455 op/s 2024-09-12T02:01:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:36 smithi183 ceph-mon[35127]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:01:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:36 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6834/3338703813,v1:172.21.15.183:6835/3338703813] up:boot 2024-09-12T02:01:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:36 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:36 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:36 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:37.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:36 smithi183 ceph-mon[35127]: pgmap v791: 97 pgs: 97 active+clean; 497 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 455 op/s 2024-09-12T02:01:37.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:36 smithi122 ceph-mon[30925]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:01:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:36 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6834/3338703813,v1:172.21.15.183:6835/3338703813] up:boot 2024-09-12T02:01:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:36 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:36 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:36 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:36 smithi122 ceph-mon[30925]: pgmap v791: 97 pgs: 97 active+clean; 497 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 455 op/s 2024-09-12T02:01:38.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 33003 22.92 MB/sec warmup 85 sec latency 64.082 ms 2024-09-12T02:01:39.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 33667 23.09 MB/sec warmup 86 sec latency 60.922 ms 2024-09-12T02:01:40.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 34459 23.34 MB/sec warmup 87 sec latency 51.472 ms 2024-09-12T02:01:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:39 smithi151 ceph-mon[35229]: pgmap v792: 97 pgs: 97 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 607 op/s 2024-09-12T02:01:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:39 smithi183 ceph-mon[35127]: pgmap v792: 97 pgs: 97 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 607 op/s 2024-09-12T02:01:40.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:39 smithi122 ceph-mon[30925]: pgmap v792: 97 pgs: 97 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 607 op/s 2024-09-12T02:01:41.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 35098 23.46 MB/sec warmup 88 sec latency 48.106 ms 2024-09-12T02:01:42.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 35733 23.65 MB/sec warmup 89 sec latency 76.575 ms 2024-09-12T02:01:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:41 smithi151 ceph-mon[35229]: pgmap v793: 97 pgs: 97 active+clean; 507 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.9 MiB/s wr, 578 op/s 2024-09-12T02:01:42.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:41 smithi183 ceph-mon[35127]: pgmap v793: 97 pgs: 97 active+clean; 507 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.9 MiB/s wr, 578 op/s 2024-09-12T02:01:42.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:41 smithi122 ceph-mon[30925]: pgmap v793: 97 pgs: 97 active+clean; 507 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.9 MiB/s wr, 578 op/s 2024-09-12T02:01:43.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 36291 23.74 MB/sec warmup 90 sec latency 67.388 ms 2024-09-12T02:01:44.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 36873 23.75 MB/sec warmup 91 sec latency 64.606 ms 2024-09-12T02:01:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:43 smithi151 ceph-mon[35229]: pgmap v794: 97 pgs: 97 active+clean; 515 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-09-12T02:01:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:43 smithi183 ceph-mon[35127]: pgmap v794: 97 pgs: 97 active+clean; 515 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-09-12T02:01:44.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:43 smithi122 ceph-mon[30925]: pgmap v794: 97 pgs: 97 active+clean; 515 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-09-12T02:01:45.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 37375 23.77 MB/sec warmup 92 sec latency 94.266 ms 2024-09-12T02:01:46.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 38108 23.93 MB/sec warmup 93 sec latency 57.543 ms 2024-09-12T02:01:46.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:45 smithi151 ceph-mon[35229]: pgmap v795: 97 pgs: 97 active+clean; 517 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2024-09-12T02:01:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:45 smithi183 ceph-mon[35127]: pgmap v795: 97 pgs: 97 active+clean; 517 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2024-09-12T02:01:46.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:45 smithi122 ceph-mon[30925]: pgmap v795: 97 pgs: 97 active+clean; 517 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2024-09-12T02:01:47.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 38779 24.06 MB/sec warmup 94 sec latency 49.438 ms 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 -- 172.21.15.122:0/126597543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1724100a40 msgr2=0x7f1724100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 --2- 172.21.15.122:0/126597543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1724100a40 0x7f1724100e40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1720009900 tx=0x7f172002a410 comp rx=0 tx=0).stop 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 -- 172.21.15.122:0/126597543 shutdown_connections 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 --2- 172.21.15.122:0/126597543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17241025f0 0x7f1724108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 --2- 172.21.15.122:0/126597543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1724101c30 0x7f17241020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 --2- 172.21.15.122:0/126597543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1724100a40 0x7f1724100e40 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 -- 172.21.15.122:0/126597543 >> 172.21.15.122:0/126597543 conn(0x7f17240fc1e0 msgr2=0x7f17240fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 -- 172.21.15.122:0/126597543 shutdown_connections 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 -- 172.21.15.122:0/126597543 wait complete. 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.652+0000 7f172b8a6640 1 Processor -- start 2024-09-12T02:01:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 -- start start 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1724100a40 0x7f172406a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1724101c30 0x7f172419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 0x7f172419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f172410c710 con 0x7f17241025f0 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f172410c590 con 0x7f1724100a40 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f172410c890 con 0x7f1724101c30 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 0x7f172419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 0x7f172419f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59140/0 (socket says 172.21.15.122:59140) 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 -- 172.21.15.122:0/3771848721 learned_addr learned my addr 172.21.15.122:0/3771848721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1728e41640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1724101c30 0x7f172419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729642640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1724100a40 0x7f172406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 -- 172.21.15.122:0/3771848721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1724101c30 msgr2=0x7f172419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1724101c30 0x7f172419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 -- 172.21.15.122:0/3771848721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1724100a40 msgr2=0x7f172406a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1724100a40 0x7f172406a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 -- 172.21.15.122:0/3771848721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172419fa00 con 0x7f17241025f0 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f1729e43640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 0x7f172419f0f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f171c00cce0 tx=0x7f171c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f171a7fc640 1 -- 172.21.15.122:0/3771848721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c00f450 con 0x7f17241025f0 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f171a7fc640 1 -- 172.21.15.122:0/3771848721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171c00fad0 con 0x7f17241025f0 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172419fc90 con 0x7f17241025f0 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.653+0000 7f171a7fc640 1 -- 172.21.15.122:0/3771848721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c018c10 con 0x7f17241025f0 2024-09-12T02:01:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.654+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17241a0190 con 0x7f17241025f0 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f171a7fc640 1 -- 172.21.15.122:0/3771848721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f171c00f5f0 con 0x7f17241025f0 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 Processor -- start 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 -- start start 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b98b3102f0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b98b2b8270 con 0x55b98b3ad2e0 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f172e63a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b98b30c2b0 con 0x55b98b3b2370 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59146/0 (socket says 172.21.15.122:59146) 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.656+0000 7f1728e41640 1 -- 172.21.15.122:0/1193258273 learned_addr learned my addr 172.21.15.122:0/1193258273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1729642640 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f171a7fc640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17100a2c30 0x7f17100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 -- 172.21.15.122:0/1193258273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 msgr2=0x55b98b3b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 -- 172.21.15.122:0/1193258273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 msgr2=0x55b98b3b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 -- 172.21.15.122:0/1193258273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98b2b7340 con 0x55b98b3ac9a0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1729e43640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17100a2c30 0x7f17100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f171a7fc640 1 -- 172.21.15.122:0/3771848721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f171c0cf6c0 con 0x7f17241025f0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f1728e41640 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3acda0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1714013820 tx=0x7f1714009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cafcdbb81192c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f170f7fe640 1 -- 172.21.15.122:0/1193258273 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714013da0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f170f7fe640 1 -- 172.21.15.122:0/1193258273 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1714013120 con 0x55b98b3ac9a0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.657+0000 7f170f7fe640 1 -- 172.21.15.122:0/1193258273 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17140109e0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 -- 172.21.15.122:0/1193258273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 msgr2=0x55b98b3acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3acda0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1714013820 tx=0x7f1714009780 comp rx=0 tx=0).stop 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f1729e43640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17100a2c30 0x7f17100a50f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f171c098340 tx=0x7f171c0913c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 -- 172.21.15.122:0/1193258273 shutdown_connections 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 --2- 172.21.15.122:0/1193258273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3acda0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 -- 172.21.15.122:0/1193258273 >> 172.21.15.122:0/1193258273 conn(0x55b98b3a8010 msgr2=0x55b98b3a8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 -- 172.21.15.122:0/1193258273 shutdown_connections 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 -- 172.21.15.122:0/1193258273 wait complete. 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.658+0000 7f172e63a740 1 -- start start 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 Processor -- start 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b98b3b52e0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b98b299290 con 0x55b98b3ad2e0 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b98b3b5460 con 0x55b98b3b2370 2024-09-12T02:01:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3b6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59150/0 (socket says 172.21.15.122:59150) 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 -- 172.21.15.122:0/2692041284 learned_addr learned my addr 172.21.15.122:0/2692041284 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f172e63a740 10 client.0 _ll_register_callbacks cb 0x55b98b2ca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1729642640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 -- 172.21.15.122:0/2692041284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 msgr2=0x55b98b3be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3be090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 -- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 msgr2=0x55b98b3b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 -- 172.21.15.122:0/2692041284 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98b023210 con 0x55b98b3ac9a0 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1729642640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f1728e41640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3b6b10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f17140104e0 tx=0x7f1714010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3ff53a64fb37a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.659+0000 7f170cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f170cff9640 1 -- 172.21.15.122:0/2692041284 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714010760 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f170cff9640 1 -- 172.21.15.122:0/2692041284 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1714010900 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1714013120 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f170cff9640 1 -- 172.21.15.122:0/2692041284 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1714033050 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 10 client.17661 fetch_fsmap learned FSMap version 36 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b98b0ef620 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f170cff9640 1 -- 172.21.15.122:0/2692041284 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1714013120 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f170cff9640 1 -- 172.21.15.122:0/2692041284 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f1714003a00 con 0x55b98b3ac9a0 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 10 client.17661 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 10 client.17661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 20 client.17661 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b3c2520 0x55b98b3c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 4 client.17661 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98b31da20 con 0x55b98b3c2520 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.660+0000 7f1729642640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b3c2520 0x55b98b3c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.661+0000 7f170cff9640 10 client.17661 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.661+0000 7f1729642640 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b3c2520 0x55b98b3c4910 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829855af33e3a316 server_cookie=1979d86b36f7ed5c in_seq=0 out_seq=0 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.661+0000 7f170cff9640 1 -- 172.21.15.122:0/2692041284 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b98b31da20 con 0x55b98b3c2520 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.661+0000 7f170cff9640 10 client.17661 handle_command_reply: tid=0 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 client.17661 shutdown 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 msgr2=0x55b98b3b6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3b6b10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f17140104e0 tx=0x7f1714010430 comp rx=0 tx=0).stop 2024-09-12T02:01:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 shutdown_connections 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b3c2520 0x55b98b3c4910 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b3be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b3b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 --2- 172.21.15.122:0/2692041284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3b6b10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f170cff9640 0 client.17661 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 >> 172.21.15.122:0/2692041284 conn(0x55b98b3a8010 msgr2=0x55b98b3a9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 shutdown_connections 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 1 -- 172.21.15.122:0/2692041284 wait complete. 2024-09-12T02:01:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.662+0000 7f172e63a740 20 client.17661 trim_cache size 0 max 16384 2024-09-12T02:01:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 Processor -- start 2024-09-12T02:01:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 -- start start 2024-09-12T02:01:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3ad2e0 0x55b98b3be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3b2370 0x55b98b3a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17241ab790 con 0x55b98b3ac9a0 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17241a9030 con 0x55b98b3b2370 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f172e63a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17241a6e60 con 0x55b98b3ad2e0 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59166/0 (socket says 172.21.15.122:59166) 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f1728e41640 1 -- 172.21.15.122:0/1009131975 learned_addr learned my addr 172.21.15.122:0/1009131975 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f1729642640 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3ad2e0 0x55b98b3be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.677+0000 7f1728e41640 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3b2370 0x55b98b3a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f1728e41640 1 -- 172.21.15.122:0/1009131975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3ad2e0 msgr2=0x55b98b3be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f1728e41640 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3ad2e0 0x55b98b3be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f1728e41640 1 -- 172.21.15.122:0/1009131975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3b2370 msgr2=0x55b98b3a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f1728e41640 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3b2370 0x55b98b3a6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f1728e41640 1 -- 172.21.15.122:0/1009131975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98b3c4c40 con 0x55b98b3ac9a0 2024-09-12T02:01:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f1728e41640 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3a69a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1714009f30 tx=0x7f17140135f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6b0a94295bcba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f170effd640 1 -- 172.21.15.122:0/1009131975 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714013320 con 0x55b98b3ac9a0 2024-09-12T02:01:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f170effd640 1 -- 172.21.15.122:0/1009131975 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1714017070 con 0x55b98b3ac9a0 2024-09-12T02:01:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.678+0000 7f170effd640 1 -- 172.21.15.122:0/1009131975 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714021a70 con 0x55b98b3ac9a0 2024-09-12T02:01:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 -- 172.21.15.122:0/1009131975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 msgr2=0x55b98b3a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3a69a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1714009f30 tx=0x7f17140135f0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 -- 172.21.15.122:0/1009131975 shutdown_connections 2024-09-12T02:01:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3b2370 0x55b98b3a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3ad2e0 0x55b98b3be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 --2- 172.21.15.122:0/1009131975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b3a69a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 -- 172.21.15.122:0/1009131975 >> 172.21.15.122:0/1009131975 conn(0x55b98b3a8010 msgr2=0x55b98b3b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 -- 172.21.15.122:0/1009131975 shutdown_connections 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 -- 172.21.15.122:0/1009131975 wait complete. 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 -- start start 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 Processor -- start 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b428100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.679+0000 7f172e63a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b428640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b42f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17241a74f0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17241a8960 con 0x55b98b3ad2e0 2024-09-12T02:01:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17241a7bc0 con 0x55b98b3b2370 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 10 client.0 _ll_register_callbacks cb 0x7f1720009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b428100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b428100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59176/0 (socket says 172.21.15.122:59176) 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 -- 172.21.15.122:0/3379179954 learned_addr learned my addr 172.21.15.122:0/3379179954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1729642640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b428640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b42f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 -- 172.21.15.122:0/3379179954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 msgr2=0x55b98b42f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b42f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 -- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 msgr2=0x55b98b428640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b428640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 -- 172.21.15.122:0/3379179954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98b42fc20 con 0x55b98b3ac9a0 2024-09-12T02:01:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f1728e41640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b428100 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f17140059f0 tx=0x7f1714005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f7b4fd733a6b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f170cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f170cff9640 1 -- 172.21.15.122:0/3379179954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714021a10 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f170cff9640 1 -- 172.21.15.122:0/3379179954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f1714021bb0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1714017070 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f170cff9640 1 -- 172.21.15.122:0/3379179954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1714013330 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 10 client.17667 fetch_fsmap learned FSMap version 36 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b98b430f30 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.680+0000 7f170cff9640 1 -- 172.21.15.122:0/3379179954 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1714017070 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f170cff9640 1 -- 172.21.15.122:0/3379179954 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f17140052c0 con 0x55b98b3ac9a0 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f172e63a740 10 client.17667 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f172e63a740 10 client.17667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f172e63a740 20 client.17667 populate_metadata read hostname 'smithi122' 2024-09-12T02:01:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f172e63a740 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b431880 0x55b98b433ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:01:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f172e63a740 4 client.17667 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:01:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b98b317d70 con 0x55b98b431880 2024-09-12T02:01:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f1729642640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b431880 0x55b98b433ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:01:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f170cff9640 10 client.17667 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.681+0000 7f1729642640 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b431880 0x55b98b433ca0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c4458b90542c6e server_cookie=3e24f475dfc7cf27 in_seq=0 out_seq=0 2024-09-12T02:01:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.869+0000 7f170cff9640 1 -- 172.21.15.122:0/3379179954 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b98b317d70 con 0x55b98b431880 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.869+0000 7f170cff9640 10 client.17667 handle_command_reply: tid=0 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 client.17667 shutdown 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 msgr2=0x55b98b428100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b428100 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f17140059f0 tx=0x7f1714005f90 comp rx=0 tx=0).stop 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 shutdown_connections 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b98b431880 0x55b98b433ca0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f170cff9640 0 client.17667 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b98b3b2370 0x55b98b42f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b98b3ad2e0 0x55b98b428640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 --2- 172.21.15.122:0/3379179954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98b3ac9a0 0x55b98b428100 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 >> 172.21.15.122:0/3379179954 conn(0x55b98b3a8010 msgr2=0x55b98b3a8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 shutdown_connections 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 1 -- 172.21.15.122:0/3379179954 wait complete. 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.870+0000 7f172e63a740 20 client.17667 trim_cache size 0 max 16384 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:01:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:01:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17100a2c30 msgr2=0x7f17100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17100a2c30 0x7f17100a50f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f171c098340 tx=0x7f171c0913c0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 msgr2=0x7f172419f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:01:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 0x7f172419f0f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f171c00cce0 tx=0x7f171c00a620 comp rx=0 tx=0).stop 2024-09-12T02:01:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 shutdown_connections 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17100a2c30 0x7f17100a50f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17241025f0 0x7f172419f0f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1724101c30 0x7f172419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 --2- 172.21.15.122:0/3771848721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1724100a40 0x7f172406a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 >> 172.21.15.122:0/3771848721 conn(0x7f17240fc1e0 msgr2=0x7f1724104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 shutdown_connections 2024-09-12T02:01:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:01:47.871+0000 7f172b8a6640 1 -- 172.21.15.122:0/3771848721 wait complete. 2024-09-12T02:01:48.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 39478 24.20 MB/sec warmup 95 sec latency 133.035 ms 2024-09-12T02:01:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:47 smithi151 ceph-mon[35229]: pgmap v796: 97 pgs: 97 active+clean; 519 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-09-12T02:01:48.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:47 smithi183 ceph-mon[35127]: pgmap v796: 97 pgs: 97 active+clean; 519 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-09-12T02:01:48.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:47 smithi122 ceph-mon[30925]: pgmap v796: 97 pgs: 97 active+clean; 519 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 448 op/s 2024-09-12T02:01:48.452 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setuser ceph since I am not root 2024-09-12T02:01:48.453 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setgroup ceph since I am not root 2024-09-12T02:01:48.453 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:48 smithi183 ceph-mds[64439]: -- 172.21.15.183:0/929565992 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a381c8f860 con 0x55a381c6d400 2024-09-12T02:01:48.453 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:01:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: starting mds.f at 2024-09-12T02:01:49.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 39695 24.10 MB/sec warmup 96 sec latency 555.350 ms 2024-09-12T02:01:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:48 smithi151 ceph-mon[35229]: daemon mds.f restarted 2024-09-12T02:01:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:48 smithi183 ceph-mon[35127]: daemon mds.f restarted 2024-09-12T02:01:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:48 smithi122 ceph-mon[30925]: daemon mds.f restarted 2024-09-12T02:01:50.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 39713 23.85 MB/sec warmup 97 sec latency 1501.069 ms 2024-09-12T02:01:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:49 smithi151 ceph-mon[35229]: pgmap v797: 97 pgs: 97 active+clean; 528 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.8 MiB/s wr, 511 op/s 2024-09-12T02:01:50.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:49 smithi151 ceph-mon[35229]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:01:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:49 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6834/4050812174,v1:172.21.15.183:6835/4050812174] up:boot 2024-09-12T02:01:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:49 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:49 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:50.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:49 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:49 smithi183 ceph-mon[35127]: pgmap v797: 97 pgs: 97 active+clean; 528 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.8 MiB/s wr, 511 op/s 2024-09-12T02:01:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:49 smithi183 ceph-mon[35127]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:01:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:49 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6834/4050812174,v1:172.21.15.183:6835/4050812174] up:boot 2024-09-12T02:01:50.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:49 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:50.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:49 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:50.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:49 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:49 smithi122 ceph-mon[30925]: pgmap v797: 97 pgs: 97 active+clean; 528 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.8 MiB/s wr, 511 op/s 2024-09-12T02:01:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:49 smithi122 ceph-mon[30925]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:01:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:49 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6834/4050812174,v1:172.21.15.183:6835/4050812174] up:boot 2024-09-12T02:01:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:49 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:01:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:49 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:01:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:49 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:51.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 39889 23.70 MB/sec warmup 98 sec latency 1646.993 ms 2024-09-12T02:01:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:50 smithi151 ceph-mon[35229]: pgmap v799: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 564 op/s 2024-09-12T02:01:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:50 smithi183 ceph-mon[35127]: pgmap v799: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 564 op/s 2024-09-12T02:01:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:50 smithi122 ceph-mon[30925]: pgmap v799: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 564 op/s 2024-09-12T02:01:52.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 40458 23.83 MB/sec warmup 99 sec latency 103.644 ms 2024-09-12T02:01:53.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 40985 23.86 MB/sec warmup 100 sec latency 77.721 ms 2024-09-12T02:01:54.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 41651 23.95 MB/sec warmup 101 sec latency 71.147 ms 2024-09-12T02:01:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:53 smithi151 ceph-mon[35229]: pgmap v800: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 907 op/s 2024-09-12T02:01:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:53 smithi183 ceph-mon[35127]: pgmap v800: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 907 op/s 2024-09-12T02:01:54.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:53 smithi122 ceph-mon[30925]: pgmap v800: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 907 op/s 2024-09-12T02:01:55.052 INFO:tasks.workunit.client.0.smithi122.stdout: 10 42170 23.97 MB/sec warmup 102 sec latency 93.706 ms 2024-09-12T02:01:56.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 42797 24.01 MB/sec warmup 103 sec latency 121.595 ms 2024-09-12T02:01:56.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:55 smithi151 ceph-mon[35229]: pgmap v801: 97 pgs: 97 active+clean; 511 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-12T02:01:56.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:55 smithi183 ceph-mon[35127]: pgmap v801: 97 pgs: 97 active+clean; 511 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-12T02:01:56.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:55 smithi122 ceph-mon[30925]: pgmap v801: 97 pgs: 97 active+clean; 511 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-12T02:01:57.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 43415 24.15 MB/sec warmup 104 sec latency 81.429 ms 2024-09-12T02:01:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:01:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:01:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:01:57.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:01:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:01:57.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:01:57.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:01:57.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:01:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:01:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:01:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:01:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:01:58.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 44106 24.26 MB/sec warmup 105 sec latency 50.102 ms 2024-09-12T02:01:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:57 smithi151 ceph-mon[35229]: pgmap v802: 97 pgs: 97 active+clean; 486 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.13k op/s 2024-09-12T02:01:58.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:57 smithi183 ceph-mon[35127]: pgmap v802: 97 pgs: 97 active+clean; 486 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.13k op/s 2024-09-12T02:01:58.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:57 smithi122 ceph-mon[30925]: pgmap v802: 97 pgs: 97 active+clean; 486 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.13k op/s 2024-09-12T02:01:59.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 44771 24.45 MB/sec warmup 106 sec latency 66.991 ms 2024-09-12T02:01:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:58 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:01:58 smithi151 ceph-mon[35229]: pgmap v803: 97 pgs: 97 active+clean; 489 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-12T02:01:59.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:58 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:59.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:01:58 smithi183 ceph-mon[35127]: pgmap v803: 97 pgs: 97 active+clean; 489 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-12T02:01:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:58 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:01:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:01:58 smithi122 ceph-mon[30925]: pgmap v803: 97 pgs: 97 active+clean; 489 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-12T02:02:00.053 INFO:tasks.workunit.client.0.smithi122.stdout: 10 45348 24.50 MB/sec warmup 107 sec latency 87.334 ms 2024-09-12T02:02:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.965+0000 7fd604aaf640 1 -- 172.21.15.122:0/2422761669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600103710 msgr2=0x7fd600109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.965+0000 7fd604aaf640 1 --2- 172.21.15.122:0/2422761669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600103710 0x7fd600109fa0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ca0 tx=0x7fd5f802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 -- 172.21.15.122:0/2422761669 shutdown_connections 2024-09-12T02:02:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 --2- 172.21.15.122:0/2422761669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600103710 0x7fd600109fa0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 --2- 172.21.15.122:0/2422761669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600102d50 0x7fd6001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 --2- 172.21.15.122:0/2422761669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd600101b60 0x7fd600101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 -- 172.21.15.122:0/2422761669 >> 172.21.15.122:0/2422761669 conn(0x7fd6000fd2e0 msgr2=0x7fd6000ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 -- 172.21.15.122:0/2422761669 shutdown_connections 2024-09-12T02:02:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 -- 172.21.15.122:0/2422761669 wait complete. 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 Processor -- start 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.966+0000 7fd604aaf640 1 -- start start 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600101b60 0x7fd600197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 0x7fd600198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd600103710 0x7fd60019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010d7c0 con 0x7fd600102d50 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010d640 con 0x7fd600103710 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010d940 con 0x7fd600101b60 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 0x7fd600198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600101b60 0x7fd600197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 0x7fd600198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40020/0 (socket says 172.21.15.122:40020) 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd600103710 0x7fd60019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/4140449270 learned_addr learned my addr 172.21.15.122:0/4140449270 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600101b60 0x7fd600197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:44180/0 (socket says 172.21.15.122:44180) 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/4140449270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600101b60 msgr2=0x7fd600197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600101b60 0x7fd600197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/4140449270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd600103710 msgr2=0x7fd60019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd600103710 0x7fd60019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/4140449270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60019f960 con 0x7fd600102d50 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 0x7fd600198290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d950 tx=0x7fd5f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/4140449270 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4017070 con 0x7fd600102d50 2024-09-12T02:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/4140449270 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f4009c30 con 0x7fd600102d50 2024-09-12T02:02:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60019fc50 con 0x7fd600102d50 2024-09-12T02:02:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.967+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/4140449270 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400f6b0 con 0x7fd600102d50 2024-09-12T02:02:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.968+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001a01a0 con 0x7fd600102d50 2024-09-12T02:02:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/4140449270 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd5f400f850 con 0x7fd600102d50 2024-09-12T02:02:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 Processor -- start 2024-09-12T02:02:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 -- start start 2024-09-12T02:02:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0265da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02662e0 0x5612f026ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f026b370 0x5612f026d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612f01c92f0 con 0x5612f026b370 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5612f0171270 con 0x5612f02659a0 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.970+0000 7fd60783e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612f01c52b0 con 0x5612f02662e0 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0265da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f026b370 0x5612f026d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fe59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02662e0 0x5612f026ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0265da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:33870/0 (socket says 172.21.15.122:33870) 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 -- 172.21.15.122:0/1298887130 learned_addr learned my addr 172.21.15.122:0/1298887130 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 -- 172.21.15.122:0/1298887130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02662e0 msgr2=0x5612f026ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02662e0 0x5612f026ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5ef7fe640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e80a2c30 0x7fd5e80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 -- 172.21.15.122:0/1298887130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f026b370 msgr2=0x5612f026d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f026b370 0x5612f026d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 -- 172.21.15.122:0/1298887130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f0170340 con 0x5612f02659a0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f026b370 0x5612f026d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/4140449270 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f40cecb0 con 0x7fd600102d50 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e80a2c30 0x7fd5e80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0265da0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ca0 tx=0x7fd5f8009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba69c1660095214 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5ecff9640 1 -- 172.21.15.122:0/1298887130 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f804a070 con 0x5612f02659a0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5ecff9640 1 -- 172.21.15.122:0/1298887130 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8036040 con 0x5612f02659a0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.971+0000 7fd5ecff9640 1 -- 172.21.15.122:0/1298887130 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8045480 con 0x5612f02659a0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 -- 172.21.15.122:0/1298887130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 msgr2=0x5612f0265da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e80a2c30 0x7fd5e80a50f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009a60 tx=0x7fd5f8048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0265da0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ca0 tx=0x7fd5f8009360 comp rx=0 tx=0).stop 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 -- 172.21.15.122:0/1298887130 shutdown_connections 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f026b370 0x5612f026d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02662e0 0x5612f026ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 --2- 172.21.15.122:0/1298887130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0265da0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 -- 172.21.15.122:0/1298887130 >> 172.21.15.122:0/1298887130 conn(0x5612f0261010 msgr2=0x5612f0261410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 -- 172.21.15.122:0/1298887130 shutdown_connections 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.972+0000 7fd60783e740 1 -- 172.21.15.122:0/1298887130 wait complete. 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 -- start start 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 Processor -- start 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02659a0 0x5612f026fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f026ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f026b370 0x5612f0276f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612f026e160 con 0x5612f02662e0 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5612f0152290 con 0x5612f026b370 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612f026e2e0 con 0x5612f02659a0 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd60783e740 10 client.0 _ll_register_callbacks cb 0x5612f0183140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f026ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f026ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40040/0 (socket says 172.21.15.122:40040) 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1675375471 learned_addr learned my addr 172.21.15.122:0/1675375471 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02659a0 0x5612f026fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f026b370 0x5612f0276f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1675375471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02659a0 msgr2=0x5612f026fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02659a0 0x5612f026fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f026b370 msgr2=0x5612f0276f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f026b370 0x5612f0276f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1675375471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612efedc210 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f026ff50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4097fa0 tx=0x7fd5f400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cc8f98d8a03c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.973+0000 7fd5e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1675375471 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40d2020 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1675375471 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f4097520 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f8036040 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1675375471 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd5f4020070 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 10 client.17685 fetch_fsmap learned FSMap version 39 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5612effa8620 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1675375471 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8036040 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1675375471 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd5f409a520 con 0x5612f02662e0 2024-09-12T02:02:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 10 client.17685 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 20 client.17685 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f027b720 0x5612f027db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 4 client.17685 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f01d6a20 con 0x5612f027b720 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.974+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f027b720 0x5612f027db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.975+0000 7fd5e5ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.975+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f027b720 0x5612f027db10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5578e06e8fcac9b6 server_cookie=4167fa889d438ea5 in_seq=0 out_seq=0 2024-09-12T02:02:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1675375471 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612f01d6a20 con 0x5612f027b720 2024-09-12T02:02:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd5e5ffb640 10 client.17685 handle_command_reply: tid=0 2024-09-12T02:02:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 client.17685 shutdown 2024-09-12T02:02:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 msgr2=0x5612f026ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f026ff50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4097fa0 tx=0x7fd5f400bf60 comp rx=0 tx=0).stop 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 shutdown_connections 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f027b720 0x5612f027db10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f026b370 0x5612f0276f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f026ff50 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd5e5ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 --2- 172.21.15.122:0/1675375471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f02659a0 0x5612f026fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 >> 172.21.15.122:0/1675375471 conn(0x5612f0261010 msgr2=0x5612f0262820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 shutdown_connections 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 1 -- 172.21.15.122:0/1675375471 wait complete. 2024-09-12T02:02:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.979+0000 7fd60783e740 20 client.17685 trim_cache size 0 max 16384 2024-09-12T02:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 Processor -- start 2024-09-12T02:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 -- start start 2024-09-12T02:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0261ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f0276f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f0262400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001ab880 con 0x5612f02662e0 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001a9150 con 0x5612f02659a0 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd60783e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001a6f80 con 0x5612f026b370 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f0276f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f0276f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40056/0 (socket says 172.21.15.122:40056) 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/110380452 learned_addr learned my addr 172.21.15.122:0/110380452 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f0262400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0261ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/110380452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 msgr2=0x5612f0262400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f0262400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/110380452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 msgr2=0x5612f0261ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.996+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0261ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.997+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/110380452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f027e190 con 0x5612f02662e0 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.997+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f0276f50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd5f409aa40 tx=0x7fd5f4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb866c215e3429e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.997+0000 7fd5e7fff640 1 -- 172.21.15.122:0/110380452 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f409b1a0 con 0x5612f02662e0 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.997+0000 7fd5e7fff640 1 -- 172.21.15.122:0/110380452 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f40109d0 con 0x5612f02662e0 2024-09-12T02:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.997+0000 7fd5e7fff640 1 -- 172.21.15.122:0/110380452 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40997a0 con 0x5612f02662e0 2024-09-12T02:02:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- 172.21.15.122:0/110380452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 msgr2=0x5612f0276f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f0276f50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd5f409aa40 tx=0x7fd5f4010d70 comp rx=0 tx=0).stop 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- 172.21.15.122:0/110380452 shutdown_connections 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f0262400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f0276f50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- 172.21.15.122:0/110380452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0261ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- 172.21.15.122:0/110380452 >> 172.21.15.122:0/110380452 conn(0x5612f0261010 msgr2=0x5612f02674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- 172.21.15.122:0/110380452 shutdown_connections 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- 172.21.15.122:0/110380452 wait complete. 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- start start 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 Processor -- start 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0274bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f02750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f02760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001a7610 con 0x5612f02662e0 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001a8a80 con 0x5612f02659a0 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6001a7ce0 con 0x5612f026b370 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd60783e740 10 client.0 _ll_register_callbacks cb 0x7fd5f400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f02750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd5fdd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f02750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40058/0 (socket says 172.21.15.122:40058) 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1044284274 learned_addr learned my addr 172.21.15.122:0/1044284274 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd5fed9d640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f02760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.998+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0274bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1044284274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 msgr2=0x5612f02760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f02760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 msgr2=0x5612f0274bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0274bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5fdd9b640 1 -- 172.21.15.122:0/1044284274 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f02766c0 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5fdd9b640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f02750f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4010fd0 tx=0x7fd5f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f36f38e24ac324 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1044284274 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f409afc0 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1044284274 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f4097320 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f40109d0 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1044284274 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd5f409b1a0 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 10 client.17691 fetch_fsmap learned FSMap version 39 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5612f02e6860 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1044284274 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f40109d0 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1044284274 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd5f4010690 con 0x5612f02662e0 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 10 client.17691 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T02:02:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 10 client.17691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 20 client.17691 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f02e7020 0x5612f02e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 4 client.17691 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:00.999+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612f01d0d70 con 0x5612f02e7020 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.000+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f02e7020 0x5612f02e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.000+0000 7fd5e5ffb640 10 client.17691 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.000+0000 7fd5fe59c640 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f02e7020 0x5612f02e9440 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb07ec3f9e45968 server_cookie=7e5c4495bdc61da in_seq=0 out_seq=0 2024-09-12T02:02:01.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 45880 24.55 MB/sec warmup 108 sec latency 89.395 ms 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1044284274 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612f01d0d70 con 0x5612f02e7020 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd5e5ffb640 10 client.17691 handle_command_reply: tid=0 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 client.17691 shutdown 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 msgr2=0x5612f02750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f02750f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4010fd0 tx=0x7fd5f40046c0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 shutdown_connections 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5612f02e7020 0x5612f02e9440 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612f026b370 0x5612f02760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612f02662e0 0x5612f02750f0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd5e5ffb640 0 client.17691 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 --2- 172.21.15.122:0/1044284274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f02659a0 0x5612f0274bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 >> 172.21.15.122:0/1044284274 conn(0x5612f0261010 msgr2=0x5612f025f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 shutdown_connections 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.180+0000 7fd60783e740 1 -- 172.21.15.122:0/1044284274 wait complete. 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd60783e740 20 client.17691 trim_cache size 0 max 16384 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:02:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:02:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e80a2c30 msgr2=0x7fd5e80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e80a2c30 0x7fd5e80a50f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009a60 tx=0x7fd5f8048000 comp rx=0 tx=0).stop 2024-09-12T02:02:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 msgr2=0x7fd600198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 0x7fd600198290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d950 tx=0x7fd5f400de10 comp rx=0 tx=0).stop 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 shutdown_connections 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5e80a2c30 0x7fd5e80a50f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd600103710 0x7fd60019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd600102d50 0x7fd600198290 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 --2- 172.21.15.122:0/4140449270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd600101b60 0x7fd600197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 >> 172.21.15.122:0/4140449270 conn(0x7fd6000fd2e0 msgr2=0x7fd600106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 shutdown_connections 2024-09-12T02:02:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:01.181+0000 7fd604aaf640 1 -- 172.21.15.122:0/4140449270 wait complete. 2024-09-12T02:02:02.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 46294 24.49 MB/sec warmup 109 sec latency 171.507 ms 2024-09-12T02:02:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:01 smithi151 ceph-mon[35229]: pgmap v804: 97 pgs: 97 active+clean; 494 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 961 op/s 2024-09-12T02:02:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:01 smithi183 ceph-mon[35127]: pgmap v804: 97 pgs: 97 active+clean; 494 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 961 op/s 2024-09-12T02:02:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:01 smithi122 ceph-mon[30925]: pgmap v804: 97 pgs: 97 active+clean; 494 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 961 op/s 2024-09-12T02:02:03.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 46926 24.56 MB/sec warmup 110 sec latency 59.669 ms 2024-09-12T02:02:04.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 47533 24.63 MB/sec warmup 111 sec latency 55.730 ms 2024-09-12T02:02:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:03 smithi151 ceph-mon[35229]: pgmap v805: 97 pgs: 97 active+clean; 493 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-12T02:02:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:03 smithi183 ceph-mon[35127]: pgmap v805: 97 pgs: 97 active+clean; 493 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-12T02:02:04.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:03 smithi122 ceph-mon[30925]: pgmap v805: 97 pgs: 97 active+clean; 493 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-12T02:02:05.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 48241 24.72 MB/sec warmup 112 sec latency 49.027 ms 2024-09-12T02:02:06.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 48837 24.83 MB/sec warmup 113 sec latency 66.885 ms 2024-09-12T02:02:06.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:05 smithi151 ceph-mon[35229]: pgmap v806: 97 pgs: 97 active+clean; 500 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.0 MiB/s wr, 600 op/s 2024-09-12T02:02:06.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:05 smithi183 ceph-mon[35127]: pgmap v806: 97 pgs: 97 active+clean; 500 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.0 MiB/s wr, 600 op/s 2024-09-12T02:02:06.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:05 smithi122 ceph-mon[30925]: pgmap v806: 97 pgs: 97 active+clean; 500 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.0 MiB/s wr, 600 op/s 2024-09-12T02:02:07.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 49342 24.88 MB/sec warmup 114 sec latency 71.150 ms 2024-09-12T02:02:08.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 49995 24.99 MB/sec warmup 115 sec latency 106.452 ms 2024-09-12T02:02:08.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:07 smithi151 ceph-mon[35229]: pgmap v807: 97 pgs: 97 active+clean; 497 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 512 op/s 2024-09-12T02:02:08.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:07 smithi183 ceph-mon[35127]: pgmap v807: 97 pgs: 97 active+clean; 497 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 512 op/s 2024-09-12T02:02:08.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:07 smithi122 ceph-mon[30925]: pgmap v807: 97 pgs: 97 active+clean; 497 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.3 MiB/s wr, 512 op/s 2024-09-12T02:02:09.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 50494 24.96 MB/sec warmup 116 sec latency 85.564 ms 2024-09-12T02:02:10.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 51179 25.08 MB/sec warmup 117 sec latency 60.834 ms 2024-09-12T02:02:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:09 smithi151 ceph-mon[35229]: pgmap v808: 97 pgs: 97 active+clean; 492 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.3 MiB/s wr, 466 op/s 2024-09-12T02:02:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:09 smithi183 ceph-mon[35127]: pgmap v808: 97 pgs: 97 active+clean; 492 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.3 MiB/s wr, 466 op/s 2024-09-12T02:02:10.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:09 smithi122 ceph-mon[30925]: pgmap v808: 97 pgs: 97 active+clean; 492 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.3 MiB/s wr, 466 op/s 2024-09-12T02:02:11.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 51759 25.07 MB/sec warmup 118 sec latency 85.793 ms 2024-09-12T02:02:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:10 smithi151 ceph-mon[35229]: pgmap v809: 97 pgs: 97 active+clean; 491 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-09-12T02:02:11.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:10 smithi183 ceph-mon[35127]: pgmap v809: 97 pgs: 97 active+clean; 491 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-09-12T02:02:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:10 smithi122 ceph-mon[30925]: pgmap v809: 97 pgs: 97 active+clean; 491 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-09-12T02:02:12.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 52525 25.26 MB/sec warmup 119 sec latency 45.977 ms 2024-09-12T02:02:14.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 53874 43.73 MB/sec execute 1 sec latency 68.867 ms 2024-09-12T02:02:14.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:13 smithi151 ceph-mon[35229]: pgmap v810: 97 pgs: 97 active+clean; 502 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-09-12T02:02:14.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:13 smithi183 ceph-mon[35127]: pgmap v810: 97 pgs: 97 active+clean; 502 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-09-12T02:02:14.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:13 smithi122 ceph-mon[30925]: pgmap v810: 97 pgs: 97 active+clean; 502 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-09-12T02:02:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2544815498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc1025c0 msgr2=0x7f85cc1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2544815498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc1025c0 0x7f85cc1049b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009900 tx=0x7f85c802a410 comp rx=0 tx=0).stop 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2544815498 shutdown_connections 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2544815498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc107820 0x7f85cc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2544815498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85cc104ef0 0x7f85cc1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2544815498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc1025c0 0x7f85cc1049b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2544815498 >> 172.21.15.122:0/2544815498 conn(0x7f85cc0fc1e0 msgr2=0x7f85cc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2544815498 shutdown_connections 2024-09-12T02:02:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.280+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2544815498 wait complete. 2024-09-12T02:02:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 Processor -- start 2024-09-12T02:02:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 -- start start 2024-09-12T02:02:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc1025c0 0x7f85cc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85cc104ef0 0x7f85cc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 0x7f85cc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc10d830 con 0x7f85cc107820 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc10d6b0 con 0x7f85cc104ef0 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d3a3c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc10d9b0 con 0x7f85cc1025c0 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d1fd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 0x7f85cc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d1fd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 0x7f85cc19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36812/0 (socket says 172.21.15.122:36812) 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85cc104ef0 0x7f85cc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d1fd9640 1 -- 172.21.15.122:0/2107852682 learned_addr learned my addr 172.21.15.122:0/2107852682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.281+0000 7f85d17d8640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc1025c0 0x7f85cc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d1fd9640 1 -- 172.21.15.122:0/2107852682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc1025c0 msgr2=0x7f85cc197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d1fd9640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc1025c0 0x7f85cc197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d1fd9640 1 -- 172.21.15.122:0/2107852682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85cc104ef0 msgr2=0x7f85cc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d1fd9640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85cc104ef0 0x7f85cc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d1fd9640 1 -- 172.21.15.122:0/2107852682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc100430 con 0x7f85cc107820 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d1fd9640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 0x7f85cc19f3e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f85c400e9d0 tx=0x7f85c400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85c27fc640 1 -- 172.21.15.122:0/2107852682 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c400cd80 con 0x7f85cc107820 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85c27fc640 1 -- 172.21.15.122:0/2107852682 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4005020 con 0x7f85cc107820 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc100720 con 0x7f85cc107820 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85c27fc640 1 -- 172.21.15.122:0/2107852682 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40116d0 con 0x7f85cc107820 2024-09-12T02:02:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.282+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc100c70 con 0x7f85cc107820 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.284+0000 7f85c27fc640 1 -- 172.21.15.122:0/2107852682 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f85c40051d0 con 0x7f85cc107820 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.285+0000 7f85d67d5740 1 Processor -- start 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.285+0000 7f85d67d5740 1 -- start start 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.285+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d67d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f7c43332f0 con 0x55f7c43cf9a0 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d67d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f7c42db270 con 0x55f7c43d5370 2024-09-12T02:02:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85c27fc640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f85b80a2c30 0x7f85b80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d67d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c432f2b0 con 0x55f7c43d02e0 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36826/0 (socket says 172.21.15.122:36826) 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 -- 172.21.15.122:0/3362427641 learned_addr learned my addr 172.21.15.122:0/3362427641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d17d8640 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d1fd9640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f85b80a2c30 0x7f85b80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85c27fc640 1 -- 172.21.15.122:0/2107852682 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f85c40cf4d0 con 0x7f85cc107820 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 -- 172.21.15.122:0/3362427641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 msgr2=0x55f7c43d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 -- 172.21.15.122:0/3362427641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 msgr2=0x55f7c43d7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43d7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 -- 172.21.15.122:0/3362427641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c42da340 con 0x55f7c43cf9a0 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d17d8640 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cfda0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00be80 tx=0x7f85bc00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f731204597bd44d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85d1fd9640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f85b80a2c30 0x7f85b80a50f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f85c4085330 tx=0x7f85c4097a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85b77fe640 1 -- 172.21.15.122:0/3362427641 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc005020 con 0x55f7c43cf9a0 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85b77fe640 1 -- 172.21.15.122:0/3362427641 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc0056a0 con 0x55f7c43cf9a0 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.286+0000 7f85b77fe640 1 -- 172.21.15.122:0/3362427641 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00e690 con 0x55f7c43cf9a0 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 -- 172.21.15.122:0/3362427641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 msgr2=0x55f7c43cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cfda0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00be80 tx=0x7f85bc00b650 comp rx=0 tx=0).stop 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 -- 172.21.15.122:0/3362427641 shutdown_connections 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3362427641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cfda0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 -- 172.21.15.122:0/3362427641 >> 172.21.15.122:0/3362427641 conn(0x55f7c43cb010 msgr2=0x55f7c43cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 -- 172.21.15.122:0/3362427641 shutdown_connections 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.287+0000 7f85d67d5740 1 -- 172.21.15.122:0/3362427641 wait complete. 2024-09-12T02:02:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 -- start start 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 Processor -- start 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d02e0 0x55f7c43da080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d5370 0x55f7c43e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f7c43d8340 con 0x55f7c43d5370 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f7c42bc290 con 0x55f7c43d02e0 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c43d84c0 con 0x55f7c43cf9a0 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d67d5740 10 client.0 _ll_register_callbacks cb 0x55f7c42ed140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d0fd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d5370 0x55f7c43e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d17d8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d02e0 0x55f7c43da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d0fd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d5370 0x55f7c43e10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36836/0 (socket says 172.21.15.122:36836) 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.288+0000 7f85d0fd7640 1 -- 172.21.15.122:0/3272936604 learned_addr learned my addr 172.21.15.122:0/3272936604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d17d8640 1 -- 172.21.15.122:0/3272936604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 msgr2=0x55f7c43d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d17d8640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d17d8640 1 -- 172.21.15.122:0/3272936604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d5370 msgr2=0x55f7c43e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d17d8640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d5370 0x55f7c43e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d17d8640 1 -- 172.21.15.122:0/3272936604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c4046210 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85d17d8640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d02e0 0x55f7c43da080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85c8031440 tx=0x7f85c80318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc20cee06b5c20ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.289+0000 7f85b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85b4ff9640 1 -- 172.21.15.122:0/3272936604 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8035040 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85b4ff9640 1 -- 172.21.15.122:0/3272936604 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f85c8004240 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85bc0056a0 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85b4ff9640 1 -- 172.21.15.122:0/3272936604 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85c8004440 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 10 client.26813 fetch_fsmap learned FSMap version 39 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f7c4112620 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85b4ff9640 1 -- 172.21.15.122:0/3272936604 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc0056a0 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85b4ff9640 1 -- 172.21.15.122:0/3272936604 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f85c804f070 con 0x55f7c43d02e0 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 10 client.26813 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T02:02:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 10 client.26813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 20 client.26813 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c43e57b0 0x55f7c43e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 4 client.26813 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.290+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c4340a20 con 0x55f7c43e57b0 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.291+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c43e57b0 0x55f7c43e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.291+0000 7f85b4ff9640 10 client.26813 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.291+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c43e57b0 0x55f7c43e7bc0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8660720bb23f2f34 server_cookie=f826eca5a65d2aaa in_seq=0 out_seq=0 2024-09-12T02:02:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85b4ff9640 1 -- 172.21.15.122:0/3272936604 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7c4340a20 con 0x55f7c43e57b0 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85b4ff9640 10 client.26813 handle_command_reply: tid=0 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85d67d5740 1 client.26813 shutdown 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d02e0 msgr2=0x55f7c43da080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d02e0 0x55f7c43da080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85c8031440 tx=0x7f85c80318e0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 shutdown_connections 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.294+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c43e57b0 0x55f7c43e7bc0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d5370 0x55f7c43e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85b4ff9640 0 client.26813 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d02e0 0x55f7c43da080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 1 --2- 172.21.15.122:0/3272936604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 >> 172.21.15.122:0/3272936604 conn(0x55f7c43cb010 msgr2=0x55f7c43cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 shutdown_connections 2024-09-12T02:02:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 1 -- 172.21.15.122:0/3272936604 wait complete. 2024-09-12T02:02:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.295+0000 7f85d67d5740 20 client.26813 trim_cache size 0 max 16384 2024-09-12T02:02:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.311+0000 7f85d67d5740 1 Processor -- start 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.311+0000 7f85d67d5740 1 -- start start 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.311+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 0x55f7c43e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d67d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1adaf0 con 0x55f7c43d02e0 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d67d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1ab3c0 con 0x55f7c43d5370 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d67d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1a91f0 con 0x55f7c43cf9a0 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 0x55f7c43e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 0x55f7c43e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36844/0 (socket says 172.21.15.122:36844) 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 -- 172.21.15.122:0/4128584373 learned_addr learned my addr 172.21.15.122:0/4128584373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 -- 172.21.15.122:0/4128584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 msgr2=0x55f7c43cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 -- 172.21.15.122:0/4128584373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 msgr2=0x55f7c43cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 -- 172.21.15.122:0/4128584373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c43d4730 con 0x55f7c43d02e0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85d17d8640 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 0x55f7c43e1080 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c8032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65095792fcc92f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85b6ffd640 1 -- 172.21.15.122:0/4128584373 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8035030 con 0x55f7c43d02e0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85b6ffd640 1 -- 172.21.15.122:0/4128584373 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8004780 con 0x55f7c43d02e0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.312+0000 7f85b6ffd640 1 -- 172.21.15.122:0/4128584373 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80516f0 con 0x55f7c43d02e0 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 -- 172.21.15.122:0/4128584373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 msgr2=0x55f7c43e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 0x55f7c43e1080 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c8032d00 comp rx=0 tx=0).stop 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 -- 172.21.15.122:0/4128584373 shutdown_connections 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43d02e0 0x55f7c43e1080 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 --2- 172.21.15.122:0/4128584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43cf9a0 0x55f7c43cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 -- 172.21.15.122:0/4128584373 >> 172.21.15.122:0/4128584373 conn(0x55f7c43cb010 msgr2=0x55f7c43d9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 -- 172.21.15.122:0/4128584373 shutdown_connections 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.313+0000 7f85d67d5740 1 -- 172.21.15.122:0/4128584373 wait complete. 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 -- start start 2024-09-12T02:02:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 Processor -- start 2024-09-12T02:02:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43de880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1a9880 con 0x55f7c43cf9a0 2024-09-12T02:02:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1aacf0 con 0x55f7c43d5370 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1a9f50 con 0x55f7c43d02e0 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d67d5740 10 client.0 _ll_register_callbacks cb 0x7f85c8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d0fd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43de880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d0fd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43de880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36854/0 (socket says 172.21.15.122:36854) 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d0fd7640 1 -- 172.21.15.122:0/2272865977 learned_addr learned my addr 172.21.15.122:0/2272865977 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d17d8640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.314+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d0fd7640 1 -- 172.21.15.122:0/2272865977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 msgr2=0x55f7c43dedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43dedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d0fd7640 1 -- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 msgr2=0x55f7c43dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d0fd7640 1 -- 172.21.15.122:0/2272865977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c43e0390 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d17d8640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43dedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d0fd7640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43de880 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f85bc005d00 tx=0x7f85bc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d791b05fdd6419e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85b4ff9640 1 -- 172.21.15.122:0/2272865977 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc012270 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85b4ff9640 1 -- 172.21.15.122:0/2272865977 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f85bc004020 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c8004780 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85b4ff9640 1 -- 172.21.15.122:0/2272865977 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85bc004220 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d67d5740 10 client.17709 fetch_fsmap learned FSMap version 39 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f7c4450560 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.315+0000 7f85b4ff9640 1 -- 172.21.15.122:0/2272865977 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8004780 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85b4ff9640 1 -- 172.21.15.122:0/2272865977 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f85bc003680 con 0x55f7c43cf9a0 2024-09-12T02:02:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d67d5740 10 client.17709 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d67d5740 10 client.17709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d67d5740 20 client.17709 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d67d5740 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c4450d80 0x55f7c44531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d67d5740 4 client.17709 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7c433ad70 con 0x55f7c4450d80 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d17d8640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c4450d80 0x55f7c44531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85b4ff9640 10 client.17709 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.316+0000 7f85d17d8640 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c4450d80 0x55f7c44531a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c633762da070092 server_cookie=ba1b9f9b6eab818a in_seq=0 out_seq=0 2024-09-12T02:02:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85b4ff9640 1 -- 172.21.15.122:0/2272865977 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7c433ad70 con 0x55f7c4450d80 2024-09-12T02:02:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85b4ff9640 10 client.17709 handle_command_reply: tid=0 2024-09-12T02:02:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 client.17709 shutdown 2024-09-12T02:02:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 msgr2=0x55f7c43de880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43de880 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f85bc005d00 tx=0x7f85bc014040 comp rx=0 tx=0).stop 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 shutdown_connections 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f7c4450d80 0x55f7c44531a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c43d5370 0x55f7c43dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c43d02e0 0x55f7c43dedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 --2- 172.21.15.122:0/2272865977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f7c43cf9a0 0x55f7c43de880 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85b4ff9640 0 client.17709 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 >> 172.21.15.122:0/2272865977 conn(0x55f7c43cb010 msgr2=0x55f7c43c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 shutdown_connections 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.594+0000 7f85d67d5740 1 -- 172.21.15.122:0/2272865977 wait complete. 2024-09-12T02:02:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d67d5740 20 client.17709 trim_cache size 0 max 16384 2024-09-12T02:02:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:02:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:02:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:02:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f85b80a2c30 msgr2=0x7f85b80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f85b80a2c30 0x7f85b80a50f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f85c4085330 tx=0x7f85c4097a00 comp rx=0 tx=0).stop 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 msgr2=0x7f85cc19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 0x7f85cc19f3e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f85c400e9d0 tx=0x7f85c400ee90 comp rx=0 tx=0).stop 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 shutdown_connections 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f85b80a2c30 0x7f85b80a50f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85cc107820 0x7f85cc19f3e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85cc104ef0 0x7f85cc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 --2- 172.21.15.122:0/2107852682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85cc1025c0 0x7f85cc197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.595+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 >> 172.21.15.122:0/2107852682 conn(0x7f85cc0fc1e0 msgr2=0x7f85cc102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.596+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 shutdown_connections 2024-09-12T02:02:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:14.596+0000 7f85d3a3c640 1 -- 172.21.15.122:0/2107852682 wait complete. 2024-09-12T02:02:15.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 54384 36.88 MB/sec execute 2 sec latency 191.591 ms 2024-09-12T02:02:16.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 54991 34.12 MB/sec execute 3 sec latency 89.339 ms 2024-09-12T02:02:16.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:15 smithi151 ceph-mon[35229]: pgmap v811: 97 pgs: 97 active+clean; 503 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.6 MiB/s wr, 494 op/s 2024-09-12T02:02:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:15 smithi183 ceph-mon[35127]: pgmap v811: 97 pgs: 97 active+clean; 503 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.6 MiB/s wr, 494 op/s 2024-09-12T02:02:16.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:15 smithi122 ceph-mon[30925]: pgmap v811: 97 pgs: 97 active+clean; 503 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.6 MiB/s wr, 494 op/s 2024-09-12T02:02:17.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 55545 32.12 MB/sec execute 4 sec latency 66.953 ms 2024-09-12T02:02:18.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 56212 32.75 MB/sec execute 5 sec latency 60.081 ms 2024-09-12T02:02:18.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:17 smithi151 ceph-mon[35229]: pgmap v812: 97 pgs: 97 active+clean; 505 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-09-12T02:02:18.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:17 smithi183 ceph-mon[35127]: pgmap v812: 97 pgs: 97 active+clean; 505 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-09-12T02:02:18.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:17 smithi122 ceph-mon[30925]: pgmap v812: 97 pgs: 97 active+clean; 505 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-09-12T02:02:19.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 56861 32.90 MB/sec execute 6 sec latency 61.859 ms 2024-09-12T02:02:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:18 smithi151 ceph-mon[35229]: pgmap v813: 97 pgs: 97 active+clean; 508 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 467 op/s 2024-09-12T02:02:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:18 smithi183 ceph-mon[35127]: pgmap v813: 97 pgs: 97 active+clean; 508 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 467 op/s 2024-09-12T02:02:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:18 smithi122 ceph-mon[30925]: pgmap v813: 97 pgs: 97 active+clean; 508 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 467 op/s 2024-09-12T02:02:20.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 57591 33.77 MB/sec execute 7 sec latency 78.844 ms 2024-09-12T02:02:21.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 58121 34.19 MB/sec execute 8 sec latency 157.726 ms 2024-09-12T02:02:22.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 58734 34.35 MB/sec execute 9 sec latency 112.350 ms 2024-09-12T02:02:22.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:21 smithi151 ceph-mon[35229]: pgmap v814: 97 pgs: 97 active+clean; 516 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 591 op/s 2024-09-12T02:02:22.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:21 smithi183 ceph-mon[35127]: pgmap v814: 97 pgs: 97 active+clean; 516 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 591 op/s 2024-09-12T02:02:22.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:21 smithi122 ceph-mon[30925]: pgmap v814: 97 pgs: 97 active+clean; 516 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 591 op/s 2024-09-12T02:02:23.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 59206 33.18 MB/sec execute 10 sec latency 62.223 ms 2024-09-12T02:02:24.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 59730 32.49 MB/sec execute 11 sec latency 91.868 ms 2024-09-12T02:02:24.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:23 smithi151 ceph-mon[35229]: pgmap v815: 97 pgs: 97 active+clean; 520 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-09-12T02:02:24.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:23 smithi183 ceph-mon[35127]: pgmap v815: 97 pgs: 97 active+clean; 520 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-09-12T02:02:24.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:23 smithi122 ceph-mon[30925]: pgmap v815: 97 pgs: 97 active+clean; 520 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 543 op/s 2024-09-12T02:02:25.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 60030 31.01 MB/sec execute 12 sec latency 267.793 ms 2024-09-12T02:02:26.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 60438 30.19 MB/sec execute 13 sec latency 190.081 ms 2024-09-12T02:02:26.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:25 smithi151 ceph-mon[35229]: pgmap v816: 97 pgs: 97 active+clean; 524 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 457 op/s 2024-09-12T02:02:26.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:25 smithi183 ceph-mon[35127]: pgmap v816: 97 pgs: 97 active+clean; 524 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 457 op/s 2024-09-12T02:02:26.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:25 smithi122 ceph-mon[30925]: pgmap v816: 97 pgs: 97 active+clean; 524 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 457 op/s 2024-09-12T02:02:27.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 61134 30.76 MB/sec execute 14 sec latency 53.900 ms 2024-09-12T02:02:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:26 smithi151 ceph-mon[35229]: pgmap v817: 97 pgs: 97 active+clean; 529 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 470 op/s 2024-09-12T02:02:27.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:26 smithi183 ceph-mon[35127]: pgmap v817: 97 pgs: 97 active+clean; 529 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 470 op/s 2024-09-12T02:02:27.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:26 smithi122 ceph-mon[30925]: pgmap v817: 97 pgs: 97 active+clean; 529 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 470 op/s 2024-09-12T02:02:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 -- 172.21.15.122:0/543513618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 msgr2=0x7fb7581020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 --2- 172.21.15.122:0/543513618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 0x7fb7581020b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb748009900 tx=0x7fb74802a410 comp rx=0 tx=0).stop 2024-09-12T02:02:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 -- 172.21.15.122:0/543513618 shutdown_connections 2024-09-12T02:02:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 --2- 172.21.15.122:0/543513618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7581025f0 0x7fb758108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 --2- 172.21.15.122:0/543513618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 0x7fb7581020b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 --2- 172.21.15.122:0/543513618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 -- 172.21.15.122:0/543513618 >> 172.21.15.122:0/543513618 conn(0x7fb7580fc1e0 msgr2=0x7fb7580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 -- 172.21.15.122:0/543513618 shutdown_connections 2024-09-12T02:02:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.712+0000 7fb75f95f640 1 -- 172.21.15.122:0/543513618 wait complete. 2024-09-12T02:02:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 Processor -- start 2024-09-12T02:02:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 -- start start 2024-09-12T02:02:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 0x7fb758198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7581025f0 0x7fb75819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d870 con 0x7fb758101c30 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d6f0 con 0x7fb7581025f0 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75f95f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d9f0 con 0x7fb758100a40 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75d6fb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75d6fb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:49692/0 (socket says 172.21.15.122:49692) 2024-09-12T02:02:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75d6fb640 1 -- 172.21.15.122:0/1338760686 learned_addr learned my addr 172.21.15.122:0/1338760686 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75cefa640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 0x7fb758198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.713+0000 7fb75defc640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7581025f0 0x7fb75819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75d6fb640 1 -- 172.21.15.122:0/1338760686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7581025f0 msgr2=0x7fb75819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7581025f0 0x7fb75819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75d6fb640 1 -- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 msgr2=0x7fb758198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 0x7fb758198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75d6fb640 1 -- 172.21.15.122:0/1338760686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819f9f0 con 0x7fb758100a40 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758197dd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb75400e9c0 tx=0x7fb75400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb74e7fc640 1 -- 172.21.15.122:0/1338760686 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75400cd80 con 0x7fb758100a40 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819fce0 con 0x7fb758100a40 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb74e7fc640 1 -- 172.21.15.122:0/1338760686 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7540035f0 con 0x7fb758100a40 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb74e7fc640 1 -- 172.21.15.122:0/1338760686 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754010670 con 0x7fb758100a40 2024-09-12T02:02:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.714+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a01a0 con 0x7fb758100a40 2024-09-12T02:02:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.717+0000 7fb7627d6740 1 Processor -- start 2024-09-12T02:02:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.717+0000 7fb7627d6740 1 -- start start 2024-09-12T02:02:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 0x55d3076d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076df370 0x55d3076e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb7627d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d30763d2f0 con 0x55d3076da2e0 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb7627d6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3075e5270 con 0x55d3076d99a0 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb7627d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d3076392b0 con 0x55d3076df370 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb74e7fc640 1 -- 172.21.15.122:0/1338760686 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb7540108d0 con 0x7fb758100a40 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 0x55d3076d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 0x55d3076d9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:60796/0 (socket says 172.21.15.122:60796) 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 -- 172.21.15.122:0/1548916461 learned_addr learned my addr 172.21.15.122:0/1548916461 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75cefa640 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076df370 0x55d3076e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 -- 172.21.15.122:0/1548916461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076df370 msgr2=0x55d3076e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076df370 0x55d3076e1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 -- 172.21.15.122:0/1548916461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 msgr2=0x55d3076deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75defc640 1 -- 172.21.15.122:0/1548916461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3075e4340 con 0x55d3076d99a0 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.718+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb75defc640 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 0x55d3076d9da0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb750009fd0 tx=0x7fb7500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a773dc34b1458d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb74e7fc640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c30 0x7fb7440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c30 0x7fb7440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb7437fe640 1 -- 172.21.15.122:0/1548916461 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750004d20 con 0x55d3076d99a0 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb74e7fc640 1 -- 172.21.15.122:0/1338760686 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb7540cf400 con 0x7fb758100a40 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb7437fe640 1 -- 172.21.15.122:0/1548916461 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750004050 con 0x55d3076d99a0 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.719+0000 7fb7437fe640 1 -- 172.21.15.122:0/1548916461 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750005280 con 0x55d3076d99a0 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 -- 172.21.15.122:0/1548916461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 msgr2=0x55d3076d9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 0x55d3076d9da0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb750009fd0 tx=0x7fb7500118d0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb75d6fb640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c30 0x7fb7440a50f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb75400cfd0 tx=0x7fb7540d2e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 -- 172.21.15.122:0/1548916461 shutdown_connections 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076df370 0x55d3076e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 --2- 172.21.15.122:0/1548916461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076d99a0 0x55d3076d9da0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 -- 172.21.15.122:0/1548916461 >> 172.21.15.122:0/1548916461 conn(0x55d3076d5010 msgr2=0x55d3076d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 -- 172.21.15.122:0/1548916461 shutdown_connections 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 -- 172.21.15.122:0/1548916461 wait complete. 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 -- start start 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.720+0000 7fb7627d6740 1 Processor -- start 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 0x55d3076e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076da2e0 0x55d3076e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d3076e2340 con 0x55d3076d99a0 2024-09-12T02:02:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3075c6290 con 0x55d3076df370 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d3076e24c0 con 0x55d3076da2e0 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb7627d6740 10 client.0 _ll_register_callbacks cb 0x55d3075f7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 0x55d3076e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 0x55d3076e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51312/0 (socket says 172.21.15.122:51312) 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 -- 172.21.15.122:0/2744179945 learned_addr learned my addr 172.21.15.122:0/2744179945 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75cefa640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076da2e0 0x55d3076e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 -- 172.21.15.122:0/2744179945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076da2e0 msgr2=0x55d3076e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076da2e0 0x55d3076e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 -- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 msgr2=0x55d3076eb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076eb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.721+0000 7fb75defc640 1 -- 172.21.15.122:0/2744179945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d307350210 con 0x55d3076d99a0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb75defc640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 0x55d3076e3b60 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb750003fd0 tx=0x7fb750000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb33a5cb8adda8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb740ff9640 1 -- 172.21.15.122:0/2744179945 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000f4e0 con 0x55d3076d99a0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb740ff9640 1 -- 172.21.15.122:0/2744179945 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb75000f680 con 0x55d3076d99a0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb750004050 con 0x55d3076d99a0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb740ff9640 1 -- 172.21.15.122:0/2744179945 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb75000f880 con 0x55d3076d99a0 2024-09-12T02:02:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb7627d6740 10 client.17721 fetch_fsmap learned FSMap version 39 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d30741c620 con 0x55d3076d99a0 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb740ff9640 1 -- 172.21.15.122:0/2744179945 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750004050 con 0x55d3076d99a0 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb740ff9640 1 -- 172.21.15.122:0/2744179945 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb750028a90 con 0x55d3076d99a0 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb7627d6740 10 client.17721 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.722+0000 7fb7627d6740 10 client.17721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb7627d6740 20 client.17721 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d3076f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb7627d6740 4 client.17721 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30764aa20 con 0x55d3076ef5e0 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb75cefa640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d3076f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb740ff9640 10 client.17721 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.723+0000 7fb75cefa640 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d3076f19d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac9c38bdd647dc8 server_cookie=cfd089a948bced27 in_seq=0 out_seq=0 2024-09-12T02:02:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb740ff9640 1 -- 172.21.15.122:0/2744179945 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d30764aa20 con 0x55d3076ef5e0 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb740ff9640 10 client.17721 handle_command_reply: tid=0 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 client.17721 shutdown 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 msgr2=0x55d3076e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 0x55d3076e3b60 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb750003fd0 tx=0x7fb750000f40 comp rx=0 tx=0).stop 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 shutdown_connections 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d3076f19d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb740ff9640 0 client.17721 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076da2e0 0x55d3076e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2744179945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076d99a0 0x55d3076e3b60 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 >> 172.21.15.122:0/2744179945 conn(0x55d3076d5010 msgr2=0x55d3076d6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 shutdown_connections 2024-09-12T02:02:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 1 -- 172.21.15.122:0/2744179945 wait complete. 2024-09-12T02:02:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.724+0000 7fb7627d6740 20 client.17721 trim_cache size 0 max 16384 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.739+0000 7fb7627d6740 1 Processor -- start 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.739+0000 7fb7627d6740 1 -- start start 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb7627d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581ab8c0 con 0x55d3076da2e0 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb7627d6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581a9190 con 0x55d3076df370 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb7627d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581a6fc0 con 0x55d3076d99a0 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51318/0 (socket says 172.21.15.122:51318) 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 -- 172.21.15.122:0/983614048 learned_addr learned my addr 172.21.15.122:0/983614048 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75defc640 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75defc640 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 -- 172.21.15.122:0/983614048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 msgr2=0x55d3076d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076d6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 -- 172.21.15.122:0/983614048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 msgr2=0x55d3076d6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076d6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 -- 172.21.15.122:0/983614048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3076d95e0 con 0x55d3076da2e0 2024-09-12T02:02:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb75cefa640 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076eb0a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb748009900 tx=0x7fb7480315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdbd6ffac9194a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb742ffd640 1 -- 172.21.15.122:0/983614048 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748035040 con 0x55d3076da2e0 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb742ffd640 1 -- 172.21.15.122:0/983614048 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74804b070 con 0x55d3076da2e0 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.740+0000 7fb742ffd640 1 -- 172.21.15.122:0/983614048 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480316b0 con 0x55d3076da2e0 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 -- 172.21.15.122:0/983614048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 msgr2=0x55d3076eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076eb0a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb748009900 tx=0x7fb7480315b0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 -- 172.21.15.122:0/983614048 shutdown_connections 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076df370 0x55d3076d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076da2e0 0x55d3076eb0a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 --2- 172.21.15.122:0/983614048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 -- 172.21.15.122:0/983614048 >> 172.21.15.122:0/983614048 conn(0x55d3076d5010 msgr2=0x55d3076d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 -- 172.21.15.122:0/983614048 shutdown_connections 2024-09-12T02:02:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 -- 172.21.15.122:0/983614048 wait complete. 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 -- start start 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.741+0000 7fb7627d6740 1 Processor -- start 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076da2e0 0x55d3076e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 0x55d3076e9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581a7650 con 0x55d3076df370 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581a8ac0 con 0x55d3076da2e0 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581a7d20 con 0x55d3076d99a0 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb7627d6740 10 client.0 _ll_register_callbacks cb 0x7fb7540d2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 0x55d3076e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 0x55d3076e9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51320/0 (socket says 172.21.15.122:51320) 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 -- 172.21.15.122:0/2455411831 learned_addr learned my addr 172.21.15.122:0/2455411831 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75cefa640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076da2e0 0x55d3076e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 -- 172.21.15.122:0/2455411831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 msgr2=0x55d3076e83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076e83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 -- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076da2e0 msgr2=0x55d3076e8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076da2e0 0x55d3076e8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 -- 172.21.15.122:0/2455411831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3076e9e60 con 0x55d3076df370 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb75defc640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 0x55d3076e9850 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb750004bd0 tx=0x7fb750003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6668636d850f7771 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.742+0000 7fb740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb740ff9640 1 -- 172.21.15.122:0/2455411831 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000fa20 con 0x55d3076df370 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb740ff9640 1 -- 172.21.15.122:0/2455411831 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb75000fbc0 con 0x55d3076df370 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74804b070 con 0x55d3076df370 2024-09-12T02:02:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb740ff9640 1 -- 172.21.15.122:0/2455411831 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb75000fdc0 con 0x55d3076df370 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 10 client.17727 fetch_fsmap learned FSMap version 39 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d3076eb0e0 con 0x55d3076df370 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb740ff9640 1 -- 172.21.15.122:0/2455411831 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74804b070 con 0x55d3076df370 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb740ff9640 1 -- 172.21.15.122:0/2455411831 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb7500286e0 con 0x55d3076df370 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 10 client.17727 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 10 client.17727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 20 client.17727 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d30775c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 4 client.17727 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.743+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d307644d70 con 0x55d3076ef5e0 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.744+0000 7fb75cefa640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d30775c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.744+0000 7fb75cefa640 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d30775c710 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a73d5b06bd5734e server_cookie=bd6fa900e5c52ccb in_seq=0 out_seq=0 2024-09-12T02:02:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.744+0000 7fb740ff9640 10 client.17727 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.917+0000 7fb740ff9640 1 -- 172.21.15.122:0/2455411831 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d307644d70 con 0x55d3076ef5e0 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.917+0000 7fb740ff9640 10 client.17727 handle_command_reply: tid=0 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.917+0000 7fb7627d6740 1 client.17727 shutdown 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 msgr2=0x55d3076e9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 0x55d3076e9850 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb750004bd0 tx=0x7fb750003f10 comp rx=0 tx=0).stop 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 shutdown_connections 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d3076ef5e0 0x55d30775c710 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3076df370 0x55d3076e9850 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb740ff9640 0 client.17727 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3076da2e0 0x55d3076e8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 --2- 172.21.15.122:0/2455411831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d3076d99a0 0x55d3076e83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 >> 172.21.15.122:0/2455411831 conn(0x55d3076d5010 msgr2=0x55d3076d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 shutdown_connections 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 1 -- 172.21.15.122:0/2455411831 wait complete. 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.918+0000 7fb7627d6740 20 client.17727 trim_cache size 0 max 16384 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:02:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c30 msgr2=0x7fb7440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c30 0x7fb7440a50f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb75400cfd0 tx=0x7fb7540d2e40 comp rx=0 tx=0).stop 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 msgr2=0x7fb758197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758197dd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb75400e9c0 tx=0x7fb75400ee80 comp rx=0 tx=0).stop 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 shutdown_connections 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7440a2c30 0x7fb7440a50f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7581025f0 0x7fb75819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb758101c30 0x7fb758198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 --2- 172.21.15.122:0/1338760686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb758100a40 0x7fb758197dd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 >> 172.21.15.122:0/1338760686 conn(0x7fb7580fc1e0 msgr2=0x7fb758104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 shutdown_connections 2024-09-12T02:02:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:27.919+0000 7fb75f95f640 1 -- 172.21.15.122:0/1338760686 wait complete. 2024-09-12T02:02:28.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 61689 30.00 MB/sec execute 15 sec latency 124.831 ms 2024-09-12T02:02:29.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 62492 31.24 MB/sec execute 16 sec latency 46.035 ms 2024-09-12T02:02:30.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 62897 31.13 MB/sec execute 17 sec latency 333.960 ms 2024-09-12T02:02:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:29 smithi151 ceph-mon[35229]: pgmap v818: 97 pgs: 97 active+clean; 538 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-09-12T02:02:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:29 smithi183 ceph-mon[35127]: pgmap v818: 97 pgs: 97 active+clean; 538 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-09-12T02:02:30.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:29 smithi122 ceph-mon[30925]: pgmap v818: 97 pgs: 97 active+clean; 538 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-09-12T02:02:31.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 63378 30.92 MB/sec execute 18 sec latency 136.534 ms 2024-09-12T02:02:32.041 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:31 smithi122 ceph-mon[30925]: pgmap v819: 97 pgs: 97 active+clean; 532 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.3 MiB/s wr, 545 op/s 2024-09-12T02:02:32.055 INFO:tasks.workunit.client.0.smithi122.stdout: 10 63926 30.78 MB/sec execute 19 sec latency 100.037 ms 2024-09-12T02:02:32.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:31 smithi151 ceph-mon[35229]: pgmap v819: 97 pgs: 97 active+clean; 532 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.3 MiB/s wr, 545 op/s 2024-09-12T02:02:32.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:31 smithi183 ceph-mon[35127]: pgmap v819: 97 pgs: 97 active+clean; 532 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.3 MiB/s wr, 545 op/s 2024-09-12T02:02:33.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 64356 30.11 MB/sec execute 20 sec latency 112.456 ms 2024-09-12T02:02:34.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 64968 30.42 MB/sec execute 21 sec latency 92.384 ms 2024-09-12T02:02:34.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:33 smithi151 ceph-mon[35229]: pgmap v820: 97 pgs: 97 active+clean; 515 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-09-12T02:02:34.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:33 smithi183 ceph-mon[35127]: pgmap v820: 97 pgs: 97 active+clean; 515 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-09-12T02:02:34.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:33 smithi122 ceph-mon[30925]: pgmap v820: 97 pgs: 97 active+clean; 515 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-09-12T02:02:35.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 65417 29.91 MB/sec execute 22 sec latency 238.399 ms 2024-09-12T02:02:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:34 smithi151 ceph-mon[35229]: pgmap v821: 97 pgs: 97 active+clean; 513 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.7 MiB/s wr, 615 op/s 2024-09-12T02:02:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:34 smithi183 ceph-mon[35127]: pgmap v821: 97 pgs: 97 active+clean; 513 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.7 MiB/s wr, 615 op/s 2024-09-12T02:02:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:34 smithi122 ceph-mon[30925]: pgmap v821: 97 pgs: 97 active+clean; 513 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.7 MiB/s wr, 615 op/s 2024-09-12T02:02:36.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 66025 29.75 MB/sec execute 23 sec latency 91.797 ms 2024-09-12T02:02:37.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 66817 30.68 MB/sec execute 24 sec latency 48.305 ms 2024-09-12T02:02:38.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 67441 30.84 MB/sec execute 25 sec latency 60.906 ms 2024-09-12T02:02:38.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:37 smithi151 ceph-mon[35229]: pgmap v822: 97 pgs: 97 active+clean; 511 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 603 op/s 2024-09-12T02:02:38.153 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:02:38.153 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-12T02:02:38.154 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:02:38.179 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:37 smithi122 ceph-mon[30925]: pgmap v822: 97 pgs: 97 active+clean; 511 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 603 op/s 2024-09-12T02:02:38.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:37 smithi183 ceph-mon[35127]: pgmap v822: 97 pgs: 97 active+clean; 511 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 603 op/s 2024-09-12T02:02:38.438 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:38.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 -- 172.21.15.122:0/1626696865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4106f00 msgr2=0x7f70a410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 --2- 172.21.15.122:0/1626696865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4106f00 0x7f70a410ea80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f709c009ca0 tx=0x7f709c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 -- 172.21.15.122:0/1626696865 shutdown_connections 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 --2- 172.21.15.122:0/1626696865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4106f00 0x7f70a410ea80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 --2- 172.21.15.122:0/1626696865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4102780 0x7f70a41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 --2- 172.21.15.122:0/1626696865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4101dd0 0x7f70a41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 -- 172.21.15.122:0/1626696865 >> 172.21.15.122:0/1626696865 conn(0x7f70a40fd8b0 msgr2=0x7f70a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 -- 172.21.15.122:0/1626696865 shutdown_connections 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.912+0000 7f70ac39c640 1 -- 172.21.15.122:0/1626696865 wait complete. 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 Processor -- start 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 -- start start 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 0x7f70a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:38.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4102780 0x7f70a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4106f00 0x7f70a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4111f60 con 0x7f70a4102780 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4111de0 con 0x7f70a4106f00 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.913+0000 7f70ac39c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41120e0 con 0x7f70a4101dd0 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa939640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4106f00 0x7f70a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 0x7f70a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 0x7f70a40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51640/0 (socket says 172.21.15.122:51640) 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa939640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4106f00 0x7f70a4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:38408/0 (socket says 172.21.15.122:38408) 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 -- 172.21.15.122:0/458489818 learned_addr learned my addr 172.21.15.122:0/458489818 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:38.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70a9937640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4102780 0x7f70a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 -- 172.21.15.122:0/458489818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4106f00 msgr2=0x7f70a4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4106f00 0x7f70a4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 -- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4102780 msgr2=0x7f70a4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4102780 0x7f70a4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 -- 172.21.15.122:0/458489818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a41a68a0 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f70aa138640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 0x7f70a40744f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f70a001fa00 tx=0x7f70a001fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.914+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a000cd80 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.915+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a0021920 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.915+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a406bc70 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.915+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0029980 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.915+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a406c1b0 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.916+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a41021b0 con 0x7f70a4101dd0 2024-09-12T02:02:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.917+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f70a0021ad0 con 0x7f70a4101dd0 2024-09-12T02:02:38.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.918+0000 7f709b7fe640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f70900a2b80 0x7f70900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:38.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.918+0000 7f70a9937640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f70900a2b80 0x7f70900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:38.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.918+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f70a00e0510 con 0x7f70a4101dd0 2024-09-12T02:02:38.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.918+0000 7f70a9937640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f70900a2b80 0x7f70900a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70940095f0 tx=0x7f7094005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:38.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:38.922+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70a00ad470 con 0x7f70a4101dd0 2024-09-12T02:02:39.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68018 31.05 MB/sec execute 26 sec latency 54.137 ms 2024-09-12T02:02:39.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.173+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70a410a140 con 0x7f70a4101dd0 2024-09-12T02:02:39.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.174+0000 7f709b7fe640 1 -- 172.21.15.122:0/458489818 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11934 (secure 0 0 0) 0x7f70a00acd70 con 0x7f70a4101dd0 2024-09-12T02:02:39.174 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:02:39.174 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":39,"btime":"2024-09-12T02:01:57:759838+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:01:56.928841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:02:39.174 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 39 2024-09-12T02:02:39.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f70900a2b80 msgr2=0x7f70900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:39.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f70900a2b80 0x7f70900a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f70940095f0 tx=0x7f7094005f90 comp rx=0 tx=0).stop 2024-09-12T02:02:39.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 msgr2=0x7f70a40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:39.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 0x7f70a40744f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f70a001fa00 tx=0x7f70a001fec0 comp rx=0 tx=0).stop 2024-09-12T02:02:39.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 shutdown_connections 2024-09-12T02:02:39.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f70900a2b80 0x7f70900a5040 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:39.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70a4106f00 0x7f70a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:39.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4102780 0x7f70a4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:39.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 --2- 172.21.15.122:0/458489818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4101dd0 0x7f70a40744f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:39.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 >> 172.21.15.122:0/458489818 conn(0x7f70a40fd8b0 msgr2=0x7f70a40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:39.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 shutdown_connections 2024-09-12T02:02:39.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:39.176+0000 7f70ac39c640 1 -- 172.21.15.122:0/458489818 wait complete. 2024-09-12T02:02:39.949 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-12T02:02:39.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-12T02:02:40.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68385 30.56 MB/sec execute 27 sec latency 263.796 ms 2024-09-12T02:02:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:39 smithi151 ceph-mon[35229]: pgmap v823: 97 pgs: 97 active+clean; 513 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.8 MiB/s wr, 630 op/s 2024-09-12T02:02:40.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:39 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/458489818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:39 smithi183 ceph-mon[35127]: pgmap v823: 97 pgs: 97 active+clean; 513 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.8 MiB/s wr, 630 op/s 2024-09-12T02:02:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:39 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/458489818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:40.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:39 smithi122 ceph-mon[30925]: pgmap v823: 97 pgs: 97 active+clean; 513 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.8 MiB/s wr, 630 op/s 2024-09-12T02:02:40.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:39 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/458489818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:02:40.273 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:02:40.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 -- 172.21.15.122:0/729272554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 msgr2=0x7f6770103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:40.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 --2- 172.21.15.122:0/729272554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770103120 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f676c009900 tx=0x7f676c02a480 comp rx=0 tx=0).stop 2024-09-12T02:02:40.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 -- 172.21.15.122:0/729272554 shutdown_connections 2024-09-12T02:02:40.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 --2- 172.21.15.122:0/729272554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677010c170 0x7f677010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 --2- 172.21.15.122:0/729272554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6770103660 0x7f677010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 --2- 172.21.15.122:0/729272554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770103120 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 -- 172.21.15.122:0/729272554 >> 172.21.15.122:0/729272554 conn(0x7f67700fc7a0 msgr2=0x7f67700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 -- 172.21.15.122:0/729272554 shutdown_connections 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.836+0000 7f6777bf1640 1 -- 172.21.15.122:0/729272554 wait complete. 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 Processor -- start 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 -- start start 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6770103660 0x7f6770198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f677010c170 0x7f677019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6770111f70 con 0x7f6770102d40 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6770111df0 con 0x7f6770103660 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.837+0000 7f6777bf1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67701120f0 con 0x7f677010c170 2024-09-12T02:02:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39024/0 (socket says 172.21.15.122:39024) 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 -- 172.21.15.122:0/762653395 learned_addr learned my addr 172.21.15.122:0/762653395 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677518c640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6770103660 0x7f6770198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677618e640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f677010c170 0x7f677019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 -- 172.21.15.122:0/762653395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f677010c170 msgr2=0x7f677019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f677010c170 0x7f677019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 -- 172.21.15.122:0/762653395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6770103660 msgr2=0x7f6770198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6770103660 0x7f6770198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 -- 172.21.15.122:0/762653395 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677019d400 con 0x7f6770102d40 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677518c640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6770103660 0x7f6770198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.838+0000 7f677598d640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770198450 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f676c006010 tx=0x7f676c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:40.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.839+0000 7f6766ffd640 1 -- 172.21.15.122:0/762653395 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c03f030 con 0x7f6770102d40 2024-09-12T02:02:40.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.839+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677019d690 con 0x7f6770102d40 2024-09-12T02:02:40.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.839+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67701a4fd0 con 0x7f6770102d40 2024-09-12T02:02:40.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.839+0000 7f6766ffd640 1 -- 172.21.15.122:0/762653395 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c003ce0 con 0x7f6770102d40 2024-09-12T02:02:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.839+0000 7f6766ffd640 1 -- 172.21.15.122:0/762653395 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c035070 con 0x7f6770102d40 2024-09-12T02:02:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.841+0000 7f6766ffd640 1 -- 172.21.15.122:0/762653395 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f676c042030 con 0x7f6770102d40 2024-09-12T02:02:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.841+0000 7f6779fca740 1 Processor -- start 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 -- start start 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d07eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07f050 0x559d6d08d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6766ffd640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f675c0a2c00 0x7f675c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d08de40 0x559d6d090230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559d6cfe3160 con 0x559d6d07e730 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559d6cf8a720 con 0x559d6d07f050 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6779fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d6cfdf120 con 0x559d6d08de40 2024-09-12T02:02:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d07eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d07eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39036/0 (socket says 172.21.15.122:39036) 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 -- 172.21.15.122:0/951931025 learned_addr learned my addr 172.21.15.122:0/951931025 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f6766ffd640 1 -- 172.21.15.122:0/762653395 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f676c0ef4f0 con 0x7f6770102d40 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677518c640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f675c0a2c00 0x7f675c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677598d640 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07f050 0x559d6d08d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d08de40 0x559d6d090230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 -- 172.21.15.122:0/951931025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d08de40 msgr2=0x559d6d090230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d08de40 0x559d6d090230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 -- 172.21.15.122:0/951931025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07f050 msgr2=0x559d6d08d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07f050 0x559d6d08d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.842+0000 7f677618e640 1 -- 172.21.15.122:0/951931025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6cd54ba0 con 0x559d6d07e730 2024-09-12T02:02:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.843+0000 7f677618e640 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d07eb10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x559d6d07bed0 tx=0x7f6768009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79143932d91f662f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.843+0000 7f677518c640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f675c0a2c00 0x7f675c0a50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f677010d640 tx=0x7f6760005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.843+0000 7f675bfff640 1 -- 172.21.15.122:0/951931025 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676801a030 con 0x559d6d07e730 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.843+0000 7f675bfff640 1 -- 172.21.15.122:0/951931025 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6768014e00 con 0x559d6d07e730 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.843+0000 7f675bfff640 1 -- 172.21.15.122:0/951931025 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768010b90 con 0x559d6d07e730 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 -- 172.21.15.122:0/951931025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 msgr2=0x559d6d07eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d07eb10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x559d6d07bed0 tx=0x7f6768009ad0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 -- 172.21.15.122:0/951931025 shutdown_connections 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d08de40 0x559d6d090230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07f050 0x559d6d08d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 --2- 172.21.15.122:0/951931025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d07eb10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 -- 172.21.15.122:0/951931025 >> 172.21.15.122:0/951931025 conn(0x559d6d07ade0 msgr2=0x559d6d07b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 -- 172.21.15.122:0/951931025 shutdown_connections 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 -- 172.21.15.122:0/951931025 wait complete. 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 -- start start 2024-09-12T02:02:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.844+0000 7f6779fca740 1 Processor -- start 2024-09-12T02:02:40.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 0x559d6d08ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d08b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d08de40 0x559d6d08b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f677598d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 0x559d6d08ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f677598d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 0x559d6d08ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:46792/0 (socket says 172.21.15.122:46792) 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f677618e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d08de40 0x559d6d08b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f677618e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d08de40 0x559d6d08b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39038/0 (socket says 172.21.15.122:39038) 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f677598d640 1 -- 172.21.15.122:0/2784255518 learned_addr learned my addr 172.21.15.122:0/2784255518 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559d6d090a10 con 0x559d6d08de40 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559d6cf6b900 con 0x559d6d07e730 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d6d090b90 con 0x559d6d07f050 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f6779fca740 10 client.0 _ll_register_callbacks cb 0x559d6cf69780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.845+0000 7f677518c640 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d08b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f677598d640 1 -- 172.21.15.122:0/2784255518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 msgr2=0x559d6d08b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f677598d640 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d08b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f677598d640 1 -- 172.21.15.122:0/2784255518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d08de40 msgr2=0x559d6d08b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f677598d640 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d08de40 0x559d6d08b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f677598d640 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6cfabc50 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f677598d640 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 0x559d6d08ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f676c0bbe50 tx=0x7f676c0b87e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf86a337e2743b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f67597fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f67597fa640 1 -- 172.21.15.122:0/2784255518 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c049070 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6768014e00 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f67597fa640 1 -- 172.21.15.122:0/2784255518 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f676c0ba480 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f67597fa640 1 -- 172.21.15.122:0/2784255518 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f676c0bba00 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f6779fca740 10 client.26855 fetch_fsmap learned FSMap version 40 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559d6d09c6d0 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.846+0000 7f67597fa640 1 -- 172.21.15.122:0/2784255518 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6768014e00 con 0x559d6d07e730 2024-09-12T02:02:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f67597fa640 1 -- 172.21.15.122:0/2784255518 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f676c0a6bd0 con 0x559d6d07e730 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f6779fca740 10 client.26855 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f6779fca740 10 client.26855 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f6779fca740 20 client.26855 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f6779fca740 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d09fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f6779fca740 4 client.26855 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d6cff0890 con 0x559d6d09da70 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f677518c640 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d09fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f67597fa640 10 client.26855 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.847+0000 7f677518c640 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d09fea0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f51ea59f89b3463 server_cookie=5aec3571a7a22cd1 in_seq=0 out_seq=0 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f67597fa640 1 -- 172.21.15.122:0/2784255518 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d6cff0890 con 0x559d6d09da70 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f67597fa640 10 client.26855 handle_command_reply: tid=0 2024-09-12T02:02:40.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 client.26855 shutdown 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 msgr2=0x559d6d08ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 0x559d6d08ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f676c0bbe50 tx=0x7f676c0b87e0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 shutdown_connections 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d09fea0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d08de40 0x559d6d08b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f67597fa640 0 client.26855 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d08b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 --2- 172.21.15.122:0/2784255518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d07e730 0x559d6d08ad10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 >> 172.21.15.122:0/2784255518 conn(0x559d6d07ade0 msgr2=0x559d6d080b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 shutdown_connections 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 1 -- 172.21.15.122:0/2784255518 wait complete. 2024-09-12T02:02:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.849+0000 7f6779fca740 20 client.26855 trim_cache size 0 max 16384 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 Processor -- start 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 -- start start 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07e730 0x559d6d0a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 0x559d6d0a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d087f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67701adc80 con 0x559d6d07f050 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67701ab550 con 0x559d6d08de40 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f6779fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67701a9380 con 0x559d6d07e730 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f677518c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 0x559d6d0a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f677518c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 0x559d6d0a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39040/0 (socket says 172.21.15.122:39040) 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f677518c640 1 -- 172.21.15.122:0/2461534705 learned_addr learned my addr 172.21.15.122:0/2461534705 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.868+0000 7f677598d640 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07e730 0x559d6d0a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f677518c640 1 -- 172.21.15.122:0/2461534705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07e730 msgr2=0x559d6d0a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f677518c640 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07e730 0x559d6d0a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f677518c640 1 -- 172.21.15.122:0/2461534705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 msgr2=0x559d6d087f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f677518c640 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d087f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f677518c640 1 -- 172.21.15.122:0/2461534705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6d089cc0 con 0x559d6d07f050 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f677518c640 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 0x559d6d0a5cb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6760009e70 tx=0x7f6760014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81855cb82f014596 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f675b7fe640 1 -- 172.21.15.122:0/2461534705 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676002c070 con 0x559d6d07f050 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f675b7fe640 1 -- 172.21.15.122:0/2461534705 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67600277b0 con 0x559d6d07f050 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.869+0000 7f675b7fe640 1 -- 172.21.15.122:0/2461534705 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676000fa20 con 0x559d6d07f050 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 -- 172.21.15.122:0/2461534705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 msgr2=0x559d6d0a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 0x559d6d0a5cb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6760009e70 tx=0x7f6760014000 comp rx=0 tx=0).stop 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 -- 172.21.15.122:0/2461534705 shutdown_connections 2024-09-12T02:02:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d087f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07f050 0x559d6d0a5cb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 --2- 172.21.15.122:0/2461534705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07e730 0x559d6d0a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 -- 172.21.15.122:0/2461534705 >> 172.21.15.122:0/2461534705 conn(0x559d6d07ade0 msgr2=0x559d6d08cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 -- 172.21.15.122:0/2461534705 shutdown_connections 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 -- 172.21.15.122:0/2461534705 wait complete. 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.870+0000 7f6779fca740 1 -- start start 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 Processor -- start 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d092380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d0928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d107740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67701a9a10 con 0x559d6d07e730 2024-09-12T02:02:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67701aae80 con 0x559d6d08de40 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67701aa0e0 con 0x559d6d07f050 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f6779fca740 10 client.0 _ll_register_callbacks cb 0x7f676c003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d092380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d092380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39050/0 (socket says 172.21.15.122:39050) 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677618e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 -- 172.21.15.122:0/861729557 learned_addr learned my addr 172.21.15.122:0/861729557 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677518c640 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d0928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 -- 172.21.15.122:0/861729557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 msgr2=0x559d6d0928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d0928c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 -- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 msgr2=0x559d6d107740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d107740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.871+0000 7f677598d640 1 -- 172.21.15.122:0/861729557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6d07ee80 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f677598d640 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d092380 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ed0d0 tx=0x7f676c0c0340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e060f8c8fb5a3bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f67597fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f67597fa640 1 -- 172.21.15.122:0/861729557 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c049040 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f67597fa640 1 -- 172.21.15.122:0/861729557 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f676c03bdb0 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67600277b0 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f67597fa640 1 -- 172.21.15.122:0/861729557 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f676c0044e0 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 10 client.17748 fetch_fsmap learned FSMap version 40 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559d6d07ee80 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f67597fa640 1 -- 172.21.15.122:0/861729557 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67600277b0 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f67597fa640 1 -- 172.21.15.122:0/861729557 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f676c0ad020 con 0x559d6d07e730 2024-09-12T02:02:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 10 client.17748 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 10 client.17748 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 20 client.17748 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d10b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 4 client.17748 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.872+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x559d6cfeabe0 con 0x559d6d09da70 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.873+0000 7f677518c640 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d10b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.873+0000 7f677518c640 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d10b290 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de7c7b7603353ee server_cookie=40c2dfe570e83038 in_seq=0 out_seq=0 2024-09-12T02:02:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:40.873+0000 7f67597fa640 10 client.17748 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 -- 172.21.15.122:0/2601599588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c1036f0 msgr2=0x7f3d2c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 --2- 172.21.15.122:0/2601599588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c1036f0 0x7f3d2c109f80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ca0 tx=0x7f3d2402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 -- 172.21.15.122:0/2601599588 shutdown_connections 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 --2- 172.21.15.122:0/2601599588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c1036f0 0x7f3d2c109f80 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 --2- 172.21.15.122:0/2601599588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c102d30 0x7f3d2c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 --2- 172.21.15.122:0/2601599588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c101b40 0x7f3d2c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.040+0000 7f3d331e8640 1 -- 172.21.15.122:0/2601599588 >> 172.21.15.122:0/2601599588 conn(0x7f3d2c0fd2e0 msgr2=0x7f3d2c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.041+0000 7f3d331e8640 1 -- 172.21.15.122:0/2601599588 shutdown_connections 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.041+0000 7f3d331e8640 1 -- 172.21.15.122:0/2601599588 wait complete. 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.041+0000 7f3d331e8640 1 Processor -- start 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 -- start start 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 0x7f3d2c195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c102d30 0x7f3d2c19b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1036f0 0x7f3d2c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10d810 con 0x7f3d2c101b40 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10d690 con 0x7f3d2c102d30 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d331e8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c10d990 con 0x7f3d2c1036f0 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d30f84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 0x7f3d2c195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d30f84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 0x7f3d2c195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39082/0 (socket says 172.21.15.122:39082) 2024-09-12T02:02:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.042+0000 7f3d30f84640 1 -- 172.21.15.122:0/3391146986 learned_addr learned my addr 172.21.15.122:0/3391146986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d31785640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1036f0 0x7f3d2c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c102d30 0x7f3d2c19b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d30f84640 1 -- 172.21.15.122:0/3391146986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1036f0 msgr2=0x7f3d2c196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d30f84640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1036f0 0x7f3d2c196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d30f84640 1 -- 172.21.15.122:0/3391146986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c102d30 msgr2=0x7f3d2c19b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d30f84640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c102d30 0x7f3d2c19b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d30f84640 1 -- 172.21.15.122:0/3391146986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c1ae820 con 0x7f3d2c101b40 2024-09-12T02:02:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.043+0000 7f3d30f84640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 0x7f3d2c195be0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000e9c0 tx=0x7f3d2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.044+0000 7f3d29ffb640 1 -- 172.21.15.122:0/3391146986 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000cd80 con 0x7f3d2c101b40 2024-09-12T02:02:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.044+0000 7f3d29ffb640 1 -- 172.21.15.122:0/3391146986 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d200035f0 con 0x7f3d2c101b40 2024-09-12T02:02:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.044+0000 7f3d29ffb640 1 -- 172.21.15.122:0/3391146986 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20010690 con 0x7f3d2c101b40 2024-09-12T02:02:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.044+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c1aeb10 con 0x7f3d2c101b40 2024-09-12T02:02:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.044+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c1af010 con 0x7f3d2c101b40 2024-09-12T02:02:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 Processor -- start 2024-09-12T02:02:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 -- start start 2024-09-12T02:02:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 0x55b01698fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b016994da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b016995370 0x55b016997830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b0168f32f0 con 0x55b016995370 2024-09-12T02:02:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b01689b270 con 0x55b01698f9a0 2024-09-12T02:02:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d3605e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0168ef2b0 con 0x55b0169902e0 2024-09-12T02:02:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d31785640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 0x55b01698fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d31785640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 0x55b01698fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:46842/0 (socket says 172.21.15.122:46842) 2024-09-12T02:02:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d31785640 1 -- 172.21.15.122:0/2905242730 learned_addr learned my addr 172.21.15.122:0/2905242730 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b016995370 0x55b016997830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.048+0000 7f3d29ffb640 1 -- 172.21.15.122:0/3391146986 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3d200037a0 con 0x7f3d2c101b40 2024-09-12T02:02:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d31785640 1 -- 172.21.15.122:0/2905242730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 msgr2=0x55b016994da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d31785640 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b016994da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d31785640 1 -- 172.21.15.122:0/2905242730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b016995370 msgr2=0x55b016997830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d31785640 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b016995370 0x55b016997830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d31785640 1 -- 172.21.15.122:0/2905242730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01689a340 con 0x55b01698f9a0 2024-09-12T02:02:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d31785640 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 0x55b01698fda0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ca0 tx=0x7f3d24005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0ecd3aaf4ce27d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d16ffd640 1 -- 172.21.15.122:0/2905242730 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24031650 con 0x55b01698f9a0 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d16ffd640 1 -- 172.21.15.122:0/2905242730 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d24036030 con 0x55b01698f9a0 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d16ffd640 1 -- 172.21.15.122:0/2905242730 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24004830 con 0x55b01698f9a0 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.049+0000 7f3d29ffb640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d1c0a2da0 0x7f3d1c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d30f84640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d1c0a2da0 0x7f3d1c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d29ffb640 1 -- 172.21.15.122:0/3391146986 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f3d200cf310 con 0x7f3d2c101b40 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 -- 172.21.15.122:0/2905242730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 msgr2=0x55b01698fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 0x55b01698fda0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ca0 tx=0x7f3d24005fa0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d30f84640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d1c0a2da0 0x7f3d1c0a5260 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3d200965d0 tx=0x7f3d200c66c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 -- 172.21.15.122:0/2905242730 shutdown_connections 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b016995370 0x55b016997830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b016994da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 --2- 172.21.15.122:0/2905242730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01698f9a0 0x55b01698fda0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.050+0000 7f3d3605e740 1 -- 172.21.15.122:0/2905242730 >> 172.21.15.122:0/2905242730 conn(0x55b01698b010 msgr2=0x55b01698b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 -- 172.21.15.122:0/2905242730 shutdown_connections 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 -- 172.21.15.122:0/2905242730 wait complete. 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 -- start start 2024-09-12T02:02:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 Processor -- start 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 0x55b016999980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0169902e0 0x55b016999ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b0169a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b016998160 con 0x55b0169902e0 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b01687c290 con 0x55b016995370 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0169982e0 con 0x55b01698f9a0 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.051+0000 7f3d3605e740 10 client.0 _ll_register_callbacks cb 0x55b0168ad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 0x55b016999980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 0x55b016999980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:55234/0 (socket says 172.21.15.122:55234) 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 -- 172.21.15.122:0/3316827119 learned_addr learned my addr 172.21.15.122:0/3316827119 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0169902e0 0x55b016999ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b0169a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 -- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 msgr2=0x55b0169a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b0169a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 -- 172.21.15.122:0/3316827119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0169902e0 msgr2=0x55b016999ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0169902e0 0x55b016999ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 -- 172.21.15.122:0/3316827119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b016606210 con 0x55b01698f9a0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d31785640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 0x55b016999980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d2402ade0 tx=0x7f3d24004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c5f18d57d4a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d0bfff640 1 -- 172.21.15.122:0/3316827119 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24058060 con 0x55b01698f9a0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d24036030 con 0x55b01698f9a0 2024-09-12T02:02:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d0bfff640 1 -- 172.21.15.122:0/3316827119 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f3d24003b80 con 0x55b01698f9a0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d0bfff640 1 -- 172.21.15.122:0/3316827119 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3d24003d80 con 0x55b01698f9a0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d3605e740 10 client.26824 fetch_fsmap learned FSMap version 40 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.052+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b0166d2620 con 0x55b01698f9a0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d0bfff640 1 -- 172.21.15.122:0/3316827119 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d24036030 con 0x55b01698f9a0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d0bfff640 1 -- 172.21.15.122:0/3316827119 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f3d2405d070 con 0x55b01698f9a0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d3605e740 10 client.26824 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d3605e740 10 client.26824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d3605e740 20 client.26824 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0169a5310 0x55b0169a7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d3605e740 4 client.26824 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b016900a20 con 0x55b0169a5310 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.053+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0169a5310 0x55b0169a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.054+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0169a5310 0x55b0169a7700 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8d96b6cb0e4859 server_cookie=71604faa8e9d0353 in_seq=0 out_seq=0 2024-09-12T02:02:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.055+0000 7f3d0bfff640 10 client.26824 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d0bfff640 1 -- 172.21.15.122:0/3316827119 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b016900a20 con 0x55b0169a5310 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d0bfff640 10 client.26824 handle_command_reply: tid=0 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 client.26824 shutdown 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 msgr2=0x55b016999980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 0x55b016999980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d2402ade0 tx=0x7f3d24004040 comp rx=0 tx=0).stop 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 shutdown_connections 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b0169a5310 0x55b0169a7700 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b0169a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b0169902e0 0x55b016999ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3316827119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b01698f9a0 0x55b016999980 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 >> 172.21.15.122:0/3316827119 conn(0x55b01698b010 msgr2=0x55b01698c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d0bfff640 0 client.26824 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 shutdown_connections 2024-09-12T02:02:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 1 -- 172.21.15.122:0/3316827119 wait complete. 2024-09-12T02:02:41.062 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68812 30.17 MB/sec execute 28 sec latency 245.646 ms 2024-09-12T02:02:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.056+0000 7f3d3605e740 20 client.26824 trim_cache size 0 max 16384 2024-09-12T02:02:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 Processor -- start 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 -- start start 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01698a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0169902e0 0x55b0169a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b016995370 0x55b0169ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1ba5a0 con 0x55b01698f9a0 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b7e70 con 0x55b0169902e0 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.075+0000 7f3d3605e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b5ca0 con 0x55b016995370 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01698a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01698a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39108/0 (socket says 172.21.15.122:39108) 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 -- 172.21.15.122:0/3399765082 learned_addr learned my addr 172.21.15.122:0/3399765082 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0169902e0 0x55b0169a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b016995370 0x55b0169ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 -- 172.21.15.122:0/3399765082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b016995370 msgr2=0x55b0169ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b016995370 0x55b0169ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 -- 172.21.15.122:0/3399765082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0169902e0 msgr2=0x55b0169a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0169902e0 0x55b0169a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 -- 172.21.15.122:0/3399765082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0169977b0 con 0x55b01698f9a0 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d31785640 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01698a0b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3d240049d0 tx=0x7f3d2402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3476b72bea461dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.076+0000 7f3d167fc640 1 -- 172.21.15.122:0/3399765082 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24063070 con 0x55b01698f9a0 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d167fc640 1 -- 172.21.15.122:0/3399765082 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d24036070 con 0x55b01698f9a0 2024-09-12T02:02:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d167fc640 1 -- 172.21.15.122:0/3399765082 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24058030 con 0x55b01698f9a0 2024-09-12T02:02:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 -- 172.21.15.122:0/3399765082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 msgr2=0x55b01698a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01698a0b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3d240049d0 tx=0x7f3d2402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 -- 172.21.15.122:0/3399765082 shutdown_connections 2024-09-12T02:02:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b016995370 0x55b0169ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0169902e0 0x55b0169a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 --2- 172.21.15.122:0/3399765082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01698a0b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 -- 172.21.15.122:0/3399765082 >> 172.21.15.122:0/3399765082 conn(0x55b01698b010 msgr2=0x55b0169899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 -- 172.21.15.122:0/3399765082 shutdown_connections 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.077+0000 7f3d3605e740 1 -- 172.21.15.122:0/3399765082 wait complete. 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 -- start start 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 Processor -- start 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01699e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b01699e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b01699f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d31785640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01699e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b01699e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d31785640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01699e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39122/0 (socket says 172.21.15.122:39122) 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d31785640 1 -- 172.21.15.122:0/515327059 learned_addr learned my addr 172.21.15.122:0/515327059 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b6330 con 0x55b01698f9a0 2024-09-12T02:02:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b77a0 con 0x55b016995370 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.078+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b6a00 con 0x55b0169902e0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d3605e740 10 client.0 _ll_register_callbacks cb 0x7f3d1800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b01699f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 -- 172.21.15.122:0/515327059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 msgr2=0x55b01699e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b01699e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 -- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 msgr2=0x55b01699f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b01699f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b016990f70 con 0x55b01698f9a0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d31785640 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01699e420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3d24031c10 tx=0x7f3d24031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e3f10ecc412527 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d0bfff640 1 -- 172.21.15.122:0/515327059 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24063070 con 0x55b01698f9a0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d0bfff640 1 -- 172.21.15.122:0/515327059 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f3d24004340 con 0x55b01698f9a0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d24036070 con 0x55b01698f9a0 2024-09-12T02:02:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d0bfff640 1 -- 172.21.15.122:0/515327059 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3d240309c0 con 0x55b01698f9a0 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d3605e740 10 client.17766 fetch_fsmap learned FSMap version 40 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.079+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b0169a10a0 con 0x55b01698f9a0 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d0bfff640 1 -- 172.21.15.122:0/515327059 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d24036070 con 0x55b01698f9a0 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d0bfff640 1 -- 172.21.15.122:0/515327059 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f3d2405d040 con 0x55b01698f9a0 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d3605e740 10 client.17766 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d3605e740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d3605e740 20 client.17766 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d3605e740 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b016a103f0 0x55b016a12810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d3605e740 4 client.17766 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0168fad70 con 0x55b016a103f0 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.080+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b016a103f0 0x55b016a12810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.081+0000 7f3d2bfff640 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b016a103f0 0x55b016a12810 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42e7db8ffc95c33 server_cookie=3a81b23147ad961f in_seq=0 out_seq=0 2024-09-12T02:02:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.081+0000 7f3d0bfff640 10 client.17766 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:40 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:02:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:40 smithi183 ceph-mon[35127]: pgmap v824: 97 pgs: 97 active+clean; 521 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 627 op/s 2024-09-12T02:02:41.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:40 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:02:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:40 smithi122 ceph-mon[30925]: pgmap v824: 97 pgs: 97 active+clean; 521 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 627 op/s 2024-09-12T02:02:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.291+0000 7f3d0bfff640 1 -- 172.21.15.122:0/515327059 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0168fad70 con 0x55b016a103f0 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.291+0000 7f3d0bfff640 10 client.17766 handle_command_reply: tid=0 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 client.17766 shutdown 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 msgr2=0x55b01699e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01699e420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3d24031c10 tx=0x7f3d24031450 comp rx=0 tx=0).stop 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 shutdown_connections 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b016a103f0 0x55b016a12810 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b016995370 0x55b01699f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0169902e0 0x55b01699e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 --2- 172.21.15.122:0/515327059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b01698f9a0 0x55b01699e420 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 >> 172.21.15.122:0/515327059 conn(0x55b01698b010 msgr2=0x55b01698c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d0bfff640 0 client.17766 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 shutdown_connections 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 1 -- 172.21.15.122:0/515327059 wait complete. 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.292+0000 7f3d3605e740 20 client.17766 trim_cache size 0 max 16384 2024-09-12T02:02:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:02:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:02:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:02:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:02:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d1c0a2da0 msgr2=0x7f3d1c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d1c0a2da0 0x7f3d1c0a5260 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3d200965d0 tx=0x7f3d200c66c0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 msgr2=0x7f3d2c195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 0x7f3d2c195be0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000e9c0 tx=0x7f3d2000ee80 comp rx=0 tx=0).stop 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 shutdown_connections 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3d1c0a2da0 0x7f3d1c0a5260 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1036f0 0x7f3d2c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c102d30 0x7f3d2c19b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 --2- 172.21.15.122:0/3391146986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d2c101b40 0x7f3d2c195be0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 >> 172.21.15.122:0/3391146986 conn(0x7f3d2c0fd2e0 msgr2=0x7f3d2c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 shutdown_connections 2024-09-12T02:02:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:41.293+0000 7f3d331e8640 1 -- 172.21.15.122:0/3391146986 wait complete. 2024-09-12T02:02:41.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:40 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:02:41.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:40 smithi151 ceph-mon[35229]: pgmap v824: 97 pgs: 97 active+clean; 521 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 627 op/s 2024-09-12T02:02:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.807+0000 7f67597fa640 1 -- 172.21.15.122:0/861729557 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559d6cfeabe0 con 0x559d6d09da70 2024-09-12T02:02:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.807+0000 7f67597fa640 10 client.17748 handle_command_reply: tid=0 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.807+0000 7f6779fca740 1 client.17748 shutdown 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 msgr2=0x559d6d092380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d092380 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ed0d0 tx=0x7f676c0c0340 comp rx=0 tx=0).stop 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 shutdown_connections 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559d6d09da70 0x559d6d10b290 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d6d08de40 0x559d6d107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d6d07f050 0x559d6d0928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 --2- 172.21.15.122:0/861729557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d6d07e730 0x559d6d092380 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f67597fa640 0 client.17748 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 >> 172.21.15.122:0/861729557 conn(0x559d6d07ade0 msgr2=0x559d6d080240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 shutdown_connections 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 1 -- 172.21.15.122:0/861729557 wait complete. 2024-09-12T02:02:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.808+0000 7f6779fca740 20 client.17748 trim_cache size 0 max 16384 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 40, 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 58, 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "7fb8a6c9": { 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 58, 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:02:41.810 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.2, 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:02:41.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:02:41.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:02:41.812 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:02:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f675c0a2c00 msgr2=0x7f675c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f675c0a2c00 0x7f675c0a50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f677010d640 tx=0x7f6760005f90 comp rx=0 tx=0).stop 2024-09-12T02:02:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 msgr2=0x7f6770198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770198450 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f676c006010 tx=0x7f676c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 shutdown_connections 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f675c0a2c00 0x7f675c0a50c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f677010c170 0x7f677019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6770103660 0x7f6770198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 --2- 172.21.15.122:0/762653395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770102d40 0x7f6770198450 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 >> 172.21.15.122:0/762653395 conn(0x7f67700fc7a0 msgr2=0x7f67700fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 shutdown_connections 2024-09-12T02:02:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:02:41.809+0000 7f6777bf1640 1 -- 172.21.15.122:0/762653395 wait complete. 2024-09-12T02:02:42.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 29.14 MB/sec execute 29 sec latency 1245.728 ms 2024-09-12T02:02:42.604 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7fb8a6c9', quiesce took 4.5 seconds. Will release after: 19.75 2024-09-12T02:02:43.056 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 28.17 MB/sec execute 30 sec latency 2245.832 ms 2024-09-12T02:02:44.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 27.26 MB/sec execute 31 sec latency 3245.919 ms 2024-09-12T02:02:44.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:43 smithi151 ceph-mon[35229]: pgmap v825: 97 pgs: 97 active+clean; 523 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 648 op/s 2024-09-12T02:02:44.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:43 smithi183 ceph-mon[35127]: pgmap v825: 97 pgs: 97 active+clean; 523 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 648 op/s 2024-09-12T02:02:44.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:43 smithi122 ceph-mon[30925]: pgmap v825: 97 pgs: 97 active+clean; 523 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 648 op/s 2024-09-12T02:02:45.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 26.41 MB/sec execute 32 sec latency 4246.014 ms 2024-09-12T02:02:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:44 smithi151 ceph-mon[35229]: pgmap v826: 97 pgs: 97 active+clean; 517 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.7 MiB/s wr, 613 op/s 2024-09-12T02:02:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:44 smithi183 ceph-mon[35127]: pgmap v826: 97 pgs: 97 active+clean; 517 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.7 MiB/s wr, 613 op/s 2024-09-12T02:02:45.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:44 smithi122 ceph-mon[30925]: pgmap v826: 97 pgs: 97 active+clean; 517 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.7 MiB/s wr, 613 op/s 2024-09-12T02:02:46.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 25.61 MB/sec execute 33 sec latency 5246.107 ms 2024-09-12T02:02:47.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 24.85 MB/sec execute 34 sec latency 6246.234 ms 2024-09-12T02:02:48.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 24.14 MB/sec execute 35 sec latency 7246.311 ms 2024-09-12T02:02:48.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:47 smithi151 ceph-mon[35229]: pgmap v827: 97 pgs: 97 active+clean; 518 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 529 op/s 2024-09-12T02:02:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:47 smithi183 ceph-mon[35127]: pgmap v827: 97 pgs: 97 active+clean; 518 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 529 op/s 2024-09-12T02:02:48.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:47 smithi122 ceph-mon[30925]: pgmap v827: 97 pgs: 97 active+clean; 518 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 529 op/s 2024-09-12T02:02:49.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 23.47 MB/sec execute 36 sec latency 8246.409 ms 2024-09-12T02:02:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:48 smithi151 ceph-mon[35229]: pgmap v828: 97 pgs: 97 active+clean; 526 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 523 op/s 2024-09-12T02:02:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:48 smithi183 ceph-mon[35127]: pgmap v828: 97 pgs: 97 active+clean; 526 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 523 op/s 2024-09-12T02:02:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:48 smithi122 ceph-mon[30925]: pgmap v828: 97 pgs: 97 active+clean; 526 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 523 op/s 2024-09-12T02:02:50.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 22.84 MB/sec execute 37 sec latency 9246.456 ms 2024-09-12T02:02:51.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 22.24 MB/sec execute 38 sec latency 10246.560 ms 2024-09-12T02:02:51.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:50 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:02:51.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:50 smithi151 ceph-mon[35229]: pgmap v829: 97 pgs: 97 active+clean; 525 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.1 MiB/s wr, 483 op/s 2024-09-12T02:02:51.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:50 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:02:51.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:50 smithi183 ceph-mon[35127]: pgmap v829: 97 pgs: 97 active+clean; 525 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.1 MiB/s wr, 483 op/s 2024-09-12T02:02:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:50 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:02:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:50 smithi122 ceph-mon[30925]: pgmap v829: 97 pgs: 97 active+clean; 525 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.1 MiB/s wr, 483 op/s 2024-09-12T02:02:52.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 21.67 MB/sec execute 39 sec latency 11246.632 ms 2024-09-12T02:02:53.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 21.12 MB/sec execute 40 sec latency 12246.722 ms 2024-09-12T02:02:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:52 smithi151 ceph-mon[35229]: pgmap v830: 97 pgs: 97 active+clean; 511 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 406 op/s 2024-09-12T02:02:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:52 smithi183 ceph-mon[35127]: pgmap v830: 97 pgs: 97 active+clean; 511 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 406 op/s 2024-09-12T02:02:53.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:52 smithi122 ceph-mon[30925]: pgmap v830: 97 pgs: 97 active+clean; 511 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 406 op/s 2024-09-12T02:02:54.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 20.61 MB/sec execute 41 sec latency 13246.851 ms 2024-09-12T02:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 -- 172.21.15.122:0/3816833736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558102610 msgr2=0x7fe558108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 --2- 172.21.15.122:0/3816833736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558102610 0x7fe558108ea0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe550009ca0 tx=0x7fe55002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 -- 172.21.15.122:0/3816833736 shutdown_connections 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 --2- 172.21.15.122:0/3816833736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558102610 0x7fe558108ea0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 --2- 172.21.15.122:0/3816833736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe558101c50 0x7fe5581020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 --2- 172.21.15.122:0/3816833736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558100a60 0x7fe558100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.389+0000 7fe55eb33640 1 -- 172.21.15.122:0/3816833736 >> 172.21.15.122:0/3816833736 conn(0x7fe5580fc1e0 msgr2=0x7fe5580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 -- 172.21.15.122:0/3816833736 shutdown_connections 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 -- 172.21.15.122:0/3816833736 wait complete. 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 Processor -- start 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 -- start start 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 0x7fe55806ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe558101c50 0x7fe55819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558102610 0x7fe55819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe55810c6e0 con 0x7fe558100a60 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe55810c560 con 0x7fe558102610 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55eb33640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe55810c860 con 0x7fe558101c50 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55c8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 0x7fe55806ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55c8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 0x7fe55806ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37354/0 (socket says 172.21.15.122:37354) 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.390+0000 7fe55c8cf640 1 -- 172.21.15.122:0/4140672571 learned_addr learned my addr 172.21.15.122:0/4140672571 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe557fff640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe558101c50 0x7fe55819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558102610 0x7fe55819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55c8cf640 1 -- 172.21.15.122:0/4140672571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe558101c50 msgr2=0x7fe55819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe558101c50 0x7fe55819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55c8cf640 1 -- 172.21.15.122:0/4140672571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558102610 msgr2=0x7fe55819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558102610 0x7fe55819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55c8cf640 1 -- 172.21.15.122:0/4140672571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55819fa00 con 0x7fe558100a60 2024-09-12T02:02:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558102610 0x7fe55819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 0x7fe55806ae70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00e9c0 tx=0x7fe54c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe555ffb640 1 -- 172.21.15.122:0/4140672571 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00cd80 con 0x7fe558100a60 2024-09-12T02:02:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe555ffb640 1 -- 172.21.15.122:0/4140672571 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c0035f0 con 0x7fe558100a60 2024-09-12T02:02:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55819fc90 con 0x7fe558100a60 2024-09-12T02:02:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe555ffb640 1 -- 172.21.15.122:0/4140672571 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c010690 con 0x7fe558100a60 2024-09-12T02:02:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.391+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a0190 con 0x7fe558100a60 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.394+0000 7fe555ffb640 1 -- 172.21.15.122:0/4140672571 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe54c0037a0 con 0x7fe558100a60 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.394+0000 7fe5618c9740 1 Processor -- start 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.394+0000 7fe5618c9740 1 -- start start 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f25370 0x5596a1f27830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe5618c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5596a1e832f0 con 0x5596a1f202e0 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe5618c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5596a1e2b270 con 0x5596a1f25370 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe5618c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596a1e7f2b0 con 0x5596a1f1f9a0 2024-09-12T02:02:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe555ffb640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5480a2c30 0x7fe5480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f24da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37368/0 (socket says 172.21.15.122:37368) 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 -- 172.21.15.122:0/3055130342 learned_addr learned my addr 172.21.15.122:0/3055130342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe557fff640 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f25370 0x5596a1f27830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5480a2c30 0x7fe5480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe555ffb640 1 -- 172.21.15.122:0/4140672571 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe54c0cf510 con 0x7fe558100a60 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 -- 172.21.15.122:0/3055130342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f1f9a0 msgr2=0x5596a1f1fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 -- 172.21.15.122:0/3055130342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f25370 msgr2=0x5596a1f27830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f25370 0x5596a1f27830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 -- 172.21.15.122:0/3055130342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a1e2a340 con 0x5596a1f202e0 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe557fff640 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f25370 0x5596a1f27830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f24da0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c097970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68ae5ad37b1a99f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.395+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5480a2c30 0x7fe5480a50f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe550005cc0 tx=0x7fe5500095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe542ffd640 1 -- 172.21.15.122:0/3055130342 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c09c430 con 0x5596a1f202e0 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe542ffd640 1 -- 172.21.15.122:0/3055130342 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c09bd30 con 0x5596a1f202e0 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe542ffd640 1 -- 172.21.15.122:0/3055130342 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0983a0 con 0x5596a1f202e0 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 -- 172.21.15.122:0/3055130342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 msgr2=0x5596a1f24da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f24da0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c097970 comp rx=0 tx=0).stop 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 -- 172.21.15.122:0/3055130342 shutdown_connections 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f25370 0x5596a1f27830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f24da0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 --2- 172.21.15.122:0/3055130342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 -- 172.21.15.122:0/3055130342 >> 172.21.15.122:0/3055130342 conn(0x5596a1f1b010 msgr2=0x5596a1f1b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 -- 172.21.15.122:0/3055130342 shutdown_connections 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.396+0000 7fe5618c9740 1 -- 172.21.15.122:0/3055130342 wait complete. 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 -- start start 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 Processor -- start 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 0x5596a1f29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f202e0 0x5596a1f2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f31170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5596a1f28340 con 0x5596a1f1f9a0 2024-09-12T02:02:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5596a1e0c290 con 0x5596a1f202e0 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596a1f284c0 con 0x5596a1f25370 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe5618c9740 10 client.0 _ll_register_callbacks cb 0x5596a1e3d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe55c8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 0x5596a1f29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe55c8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 0x5596a1f29bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37370/0 (socket says 172.21.15.122:37370) 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe55c8cf640 1 -- 172.21.15.122:0/2965666618 learned_addr learned my addr 172.21.15.122:0/2965666618 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe557fff640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f202e0 0x5596a1f2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.397+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f31170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe55c8cf640 1 -- 172.21.15.122:0/2965666618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 msgr2=0x5596a1f31170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f31170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe55c8cf640 1 -- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f202e0 msgr2=0x5596a1f2a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f202e0 0x5596a1f2a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe55c8cf640 1 -- 172.21.15.122:0/2965666618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a1b96210 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 0x5596a1f29bf0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe54c09bcc0 tx=0x7fe54c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c11866d34db5302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe537fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe537fff640 1 -- 172.21.15.122:0/2965666618 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0b8070 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe537fff640 1 -- 172.21.15.122:0/2965666618 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe54c0a4710 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54c09bd30 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe537fff640 1 -- 172.21.15.122:0/2965666618 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe54c0a49a0 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 10 client.17778 fetch_fsmap learned FSMap version 41 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5596a1c62620 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe537fff640 1 -- 172.21.15.122:0/2965666618 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c09bd30 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe537fff640 1 -- 172.21.15.122:0/2965666618 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe54c09abf0 con 0x5596a1f1f9a0 2024-09-12T02:02:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 10 client.17778 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 10 client.17778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 20 client.17778 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1f35470 0x5596a1f37860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 4 client.17778 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.398+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596a1e90a20 con 0x5596a1f35470 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.399+0000 7fe557fff640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1f35470 0x5596a1f37860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.399+0000 7fe557fff640 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1f35470 0x5596a1f37860 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a109dd78993432e server_cookie=32af14cabe409af8 in_seq=0 out_seq=0 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.399+0000 7fe537fff640 10 client.17778 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe537fff640 1 -- 172.21.15.122:0/2965666618 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596a1e90a20 con 0x5596a1f35470 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe537fff640 10 client.17778 handle_command_reply: tid=0 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 client.17778 shutdown 2024-09-12T02:02:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 msgr2=0x5596a1f29bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 0x5596a1f29bf0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe54c09bcc0 tx=0x7fe54c00c840 comp rx=0 tx=0).stop 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 shutdown_connections 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1f35470 0x5596a1f37860 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f31170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f202e0 0x5596a1f2a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe537fff640 0 client.17778 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2965666618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f1f9a0 0x5596a1f29bf0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 >> 172.21.15.122:0/2965666618 conn(0x5596a1f1b010 msgr2=0x5596a1f1c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 shutdown_connections 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 1 -- 172.21.15.122:0/2965666618 wait complete. 2024-09-12T02:02:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.400+0000 7fe5618c9740 20 client.17778 trim_cache size 0 max 16384 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 Processor -- start 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 -- start start 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f202e0 0x5596a1f31130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 0x5596a1f3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581ab890 con 0x5596a1f25370 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581a9160 con 0x5596a1f1f9a0 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe5618c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581a6f90 con 0x5596a1f202e0 2024-09-12T02:02:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 0x5596a1f3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 0x5596a1f3bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37386/0 (socket says 172.21.15.122:37386) 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 -- 172.21.15.122:0/2469882278 learned_addr learned my addr 172.21.15.122:0/2469882278 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 -- 172.21.15.122:0/2469882278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f202e0 msgr2=0x5596a1f31130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe557fff640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f202e0 0x5596a1f31130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f202e0 0x5596a1f31130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 -- 172.21.15.122:0/2469882278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 msgr2=0x5596a1f1a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55d0d0640 1 -- 172.21.15.122:0/2469882278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a1f24780 con 0x5596a1f25370 2024-09-12T02:02:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe557fff640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f202e0 0x5596a1f31130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.415+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 0x5596a1f3bfe0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe550009a10 tx=0x7fe55003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc46a2a711e4846 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5427fc640 1 -- 172.21.15.122:0/2469882278 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550003970 con 0x5596a1f25370 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5427fc640 1 -- 172.21.15.122:0/2469882278 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe550036040 con 0x5596a1f25370 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5427fc640 1 -- 172.21.15.122:0/2469882278 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550030a60 con 0x5596a1f25370 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 -- 172.21.15.122:0/2469882278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 msgr2=0x5596a1f3bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 0x5596a1f3bfe0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe550009a10 tx=0x7fe55003a000 comp rx=0 tx=0).stop 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 -- 172.21.15.122:0/2469882278 shutdown_connections 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f25370 0x5596a1f3bfe0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f202e0 0x5596a1f31130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2469882278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f1a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 -- 172.21.15.122:0/2469882278 >> 172.21.15.122:0/2469882278 conn(0x5596a1f1b010 msgr2=0x5596a1f2a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 -- 172.21.15.122:0/2469882278 shutdown_connections 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.416+0000 7fe5618c9740 1 -- 172.21.15.122:0/2469882278 wait complete. 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 -- start start 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 Processor -- start 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f30ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f2f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f2fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581a7620 con 0x5596a1f202e0 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581a8a90 con 0x5596a1f1f9a0 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581a7cf0 con 0x5596a1f25370 2024-09-12T02:02:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe5618c9740 10 client.0 _ll_register_callbacks cb 0x7fe544009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe557fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f2f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe557fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f2f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37390/0 (socket says 172.21.15.122:37390) 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe557fff640 1 -- 172.21.15.122:0/2020313927 learned_addr learned my addr 172.21.15.122:0/2020313927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f30ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f2fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe557fff640 1 -- 172.21.15.122:0/2020313927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 msgr2=0x5596a1f2fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.417+0000 7fe557fff640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f2fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe557fff640 1 -- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 msgr2=0x5596a1f30ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe557fff640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f30ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe557fff640 1 -- 172.21.15.122:0/2020313927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a1f301c0 con 0x5596a1f202e0 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f30ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:02:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe55d0d0640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f2fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe557fff640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f2f570 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x5596a1f1be90 tx=0x7fe544004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2a489ac760ac59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe537fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe537fff640 1 -- 172.21.15.122:0/2020313927 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544022030 con 0x5596a1f202e0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe537fff640 1 -- 172.21.15.122:0/2020313927 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe54401f430 con 0x5596a1f202e0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe550036040 con 0x5596a1f202e0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe537fff640 1 -- 172.21.15.122:0/2020313927 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe54401f630 con 0x5596a1f202e0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 10 client.17784 fetch_fsmap learned FSMap version 41 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5596a1fa0f50 con 0x5596a1f202e0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe537fff640 1 -- 172.21.15.122:0/2020313927 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe550036040 con 0x5596a1f202e0 2024-09-12T02:02:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe537fff640 1 -- 172.21.15.122:0/2020313927 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe544010ab0 con 0x5596a1f202e0 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 10 client.17784 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 10 client.17784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 20 client.17784 populate_metadata read hostname 'smithi122' 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1fa1980 0x5596a1fa3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 4 client.17784 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.418+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596a1f20fd0 con 0x5596a1fa1980 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.419+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1fa1980 0x5596a1fa3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.419+0000 7fe537fff640 10 client.17784 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.419+0000 7fe55c8cf640 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1fa1980 0x5596a1fa3da0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728753265aac7a5a server_cookie=1c81bb452055117c in_seq=0 out_seq=0 2024-09-12T02:02:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.466+0000 7fe537fff640 1 -- 172.21.15.122:0/2020313927 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596a1f20fd0 con 0x5596a1fa1980 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.466+0000 7fe537fff640 10 client.17784 handle_command_reply: tid=0 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.466+0000 7fe5618c9740 1 client.17784 shutdown 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 msgr2=0x5596a1f2f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f2f570 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x5596a1f1be90 tx=0x7fe544004560 comp rx=0 tx=0).stop 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 shutdown_connections 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5596a1fa1980 0x5596a1fa3da0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596a1f25370 0x5596a1f2fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe537fff640 0 client.17784 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596a1f202e0 0x5596a1f2f570 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 --2- 172.21.15.122:0/2020313927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596a1f1f9a0 0x5596a1f30ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 >> 172.21.15.122:0/2020313927 conn(0x5596a1f1b010 msgr2=0x5596a1f1a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 shutdown_connections 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 1 -- 172.21.15.122:0/2020313927 wait complete. 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe5618c9740 20 client.17784 trim_cache size 0 max 16384 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:02:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:02:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5480a2c30 msgr2=0x7fe5480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe55eb33640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5480a2c30 0x7fe5480a50f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe550005cc0 tx=0x7fe5500095c0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 msgr2=0x7fe55806ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:02:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.467+0000 7fe55eb33640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 0x7fe55806ae70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00e9c0 tx=0x7fe54c00ee80 comp rx=0 tx=0).stop 2024-09-12T02:02:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 shutdown_connections 2024-09-12T02:02:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5480a2c30 0x7fe5480a50f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe558102610 0x7fe55819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe558101c50 0x7fe55819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 --2- 172.21.15.122:0/4140672571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe558100a60 0x7fe55806ae70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:02:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 >> 172.21.15.122:0/4140672571 conn(0x7fe5580fc1e0 msgr2=0x7fe558104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:02:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 shutdown_connections 2024-09-12T02:02:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:02:54.468+0000 7fe55eb33640 1 -- 172.21.15.122:0/4140672571 wait complete. 2024-09-12T02:02:55.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 20.12 MB/sec execute 42 sec latency 14246.927 ms 2024-09-12T02:02:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:54 smithi151 ceph-mon[35229]: pgmap v831: 97 pgs: 97 active+clean; 510 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 5.5 MiB/s wr, 310 op/s 2024-09-12T02:02:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:54 smithi183 ceph-mon[35127]: pgmap v831: 97 pgs: 97 active+clean; 510 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 5.5 MiB/s wr, 310 op/s 2024-09-12T02:02:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:54 smithi122 ceph-mon[30925]: pgmap v831: 97 pgs: 97 active+clean; 510 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 5.5 MiB/s wr, 310 op/s 2024-09-12T02:02:56.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 19.65 MB/sec execute 43 sec latency 15247.016 ms 2024-09-12T02:02:56.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:56 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:02:56.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:56 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:02:56.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:56 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:02:57.057 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 19.20 MB/sec execute 44 sec latency 16247.049 ms 2024-09-12T02:02:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:57 smithi183 ceph-mon[35127]: pgmap v832: 97 pgs: 97 active+clean; 512 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 5.0 MiB/s wr, 218 op/s 2024-09-12T02:02:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:02:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:02:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:02:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:57 smithi122 ceph-mon[30925]: pgmap v832: 97 pgs: 97 active+clean; 512 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 5.0 MiB/s wr, 218 op/s 2024-09-12T02:02:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:02:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:02:57.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:02:58.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 18.78 MB/sec execute 45 sec latency 17247.162 ms 2024-09-12T02:02:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:57 smithi151 ceph-mon[35229]: pgmap v832: 97 pgs: 97 active+clean; 512 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 5.0 MiB/s wr, 218 op/s 2024-09-12T02:02:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:02:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:02:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:02:59.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 18.37 MB/sec execute 46 sec latency 18247.264 ms 2024-09-12T02:02:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:02:58 smithi151 ceph-mon[35229]: pgmap v833: 97 pgs: 97 active+clean; 516 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 193 op/s 2024-09-12T02:02:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:02:58 smithi183 ceph-mon[35127]: pgmap v833: 97 pgs: 97 active+clean; 516 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 193 op/s 2024-09-12T02:02:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:02:58 smithi122 ceph-mon[30925]: pgmap v833: 97 pgs: 97 active+clean; 516 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 193 op/s 2024-09-12T02:03:00.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 17.98 MB/sec execute 47 sec latency 19247.361 ms 2024-09-12T02:03:01.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 17.60 MB/sec execute 48 sec latency 20247.457 ms 2024-09-12T02:03:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:00 smithi151 ceph-mon[35229]: pgmap v834: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-09-12T02:03:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:00 smithi183 ceph-mon[35127]: pgmap v834: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-09-12T02:03:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:00 smithi122 ceph-mon[30925]: pgmap v834: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-09-12T02:03:02.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 17.24 MB/sec execute 49 sec latency 21247.549 ms 2024-09-12T02:03:02.356 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:02.644 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:03.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:02 smithi122 ceph-mon[30925]: pgmap v835: 97 pgs: 97 active+clean; 513 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-12T02:03:03.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 16.90 MB/sec execute 50 sec latency 22247.593 ms 2024-09-12T02:03:03.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:02 smithi151 ceph-mon[35229]: pgmap v835: 97 pgs: 97 active+clean; 513 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-12T02:03:03.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 -- 172.21.15.122:0/1523754796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8106f10 msgr2=0x7f98f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 --2- 172.21.15.122:0/1523754796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8106f10 0x7f98f810ea90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ca0 tx=0x7f98f002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 -- 172.21.15.122:0/1523754796 shutdown_connections 2024-09-12T02:03:03.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 --2- 172.21.15.122:0/1523754796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8106f10 0x7f98f810ea90 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 --2- 172.21.15.122:0/1523754796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98f8102790 0x7f98f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 --2- 172.21.15.122:0/1523754796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8101de0 0x7f98f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 -- 172.21.15.122:0/1523754796 >> 172.21.15.122:0/1523754796 conn(0x7f98f80fd8c0 msgr2=0x7f98f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 -- 172.21.15.122:0/1523754796 shutdown_connections 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.121+0000 7f98feaf6640 1 -- 172.21.15.122:0/1523754796 wait complete. 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 Processor -- start 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 -- start start 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 0x7f98f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98f8102790 0x7f98f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8106f10 0x7f98f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8111fc0 con 0x7f98f8101de0 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8111e40 con 0x7f98f8102790 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98feaf6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8112140 con 0x7f98f8106f10 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98fc892640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 0x7f98f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98fc892640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 0x7f98f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55090/0 (socket says 172.21.15.122:55090) 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98fc892640 1 -- 172.21.15.122:0/143014872 learned_addr learned my addr 172.21.15.122:0/143014872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98fd093640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8106f10 0x7f98f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.122+0000 7f98f7fff640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98f8102790 0x7f98f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fc892640 1 -- 172.21.15.122:0/143014872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8106f10 msgr2=0x7f98f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fc892640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8106f10 0x7f98f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fc892640 1 -- 172.21.15.122:0/143014872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98f8102790 msgr2=0x7f98f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fc892640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98f8102790 0x7f98f8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fc892640 1 -- 172.21.15.122:0/143014872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f81a6890 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fd093640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8106f10 0x7f98f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98fc892640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 0x7f98f80744f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00e9f0 tx=0x7f98ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00cd80 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f81a6b80 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ec005500 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec010690 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.123+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81a7070 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.124+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f81021c0 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.126+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f98ec0046c0 con 0x7f98f8101de0 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.126+0000 7f98f5ffb640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98e80a2b80 0x7f98e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:03.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.127+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f98ec0cf580 con 0x7f98f8101de0 2024-09-12T02:03:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.127+0000 7f98f7fff640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98e80a2b80 0x7f98e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.127+0000 7f98f7fff640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98e80a2b80 0x7f98e80a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f98f8075f80 tx=0x7f98e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.131+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98ec09c4e0 con 0x7f98f8101de0 2024-09-12T02:03:03.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:02 smithi183 ceph-mon[35127]: pgmap v835: 97 pgs: 97 active+clean; 513 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.6 MiB/s wr, 210 op/s 2024-09-12T02:03:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.339+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98f802d7c0 con 0x7f98f8101de0 2024-09-12T02:03:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.340+0000 7f98f5ffb640 1 -- 172.21.15.122:0/143014872 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7f98ec09bde0 con 0x7f98f8101de0 2024-09-12T02:03:03.340 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:03:03.341 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:03:03.341 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:03:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98e80a2b80 msgr2=0x7f98e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98e80a2b80 0x7f98e80a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f98f8075f80 tx=0x7f98e4005f90 comp rx=0 tx=0).stop 2024-09-12T02:03:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 msgr2=0x7f98f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:03.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 0x7f98f80744f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00e9f0 tx=0x7f98ec00edf0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 shutdown_connections 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98e80a2b80 0x7f98e80a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8106f10 0x7f98f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98f8102790 0x7f98f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 --2- 172.21.15.122:0/143014872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98f8101de0 0x7f98f80744f0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.343+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 >> 172.21.15.122:0/143014872 conn(0x7f98f80fd8c0 msgr2=0x7f98f80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.344+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 shutdown_connections 2024-09-12T02:03:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:03.344+0000 7f98feaf6640 1 -- 172.21.15.122:0/143014872 wait complete. 2024-09-12T02:03:03.955 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 7fb8a6c9 --release --await' 2024-09-12T02:03:03.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 7fb8a6c9 --release --await 2024-09-12T02:03:04.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68826 16.57 MB/sec execute 51 sec latency 23247.624 ms 2024-09-12T02:03:04.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:03 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/143014872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:04.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:03 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/143014872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:04.203 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:04.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:03 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/143014872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:04.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 -- 172.21.15.122:0/32667105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8102d40 msgr2=0x7fcfa8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 --2- 172.21.15.122:0/32667105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8102d40 0x7fcfa8103120 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009900 tx=0x7fcf9802a480 comp rx=0 tx=0).stop 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 -- 172.21.15.122:0/32667105 shutdown_connections 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 --2- 172.21.15.122:0/32667105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa810c170 0x7fcfa810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 --2- 172.21.15.122:0/32667105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfa8103660 0x7fcfa810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 --2- 172.21.15.122:0/32667105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8102d40 0x7fcfa8103120 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 -- 172.21.15.122:0/32667105 >> 172.21.15.122:0/32667105 conn(0x7fcfa80fc7a0 msgr2=0x7fcfa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 -- 172.21.15.122:0/32667105 shutdown_connections 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.702+0000 7fcfad86c640 1 -- 172.21.15.122:0/32667105 wait complete. 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 Processor -- start 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 -- start start 2024-09-12T02:03:04.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8102d40 0x7fcfa8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfa8103660 0x7fcfa8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 0x7fcfa819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8111f70 con 0x7fcfa810c170 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8111df0 con 0x7fcfa8102d40 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfad86c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81120f0 con 0x7fcfa8103660 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfa77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 0x7fcfa819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfa6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8102d40 0x7fcfa8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfa77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 0x7fcfa819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55106/0 (socket says 172.21.15.122:55106) 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcfa77fe640 1 -- 172.21.15.122:0/1334160886 learned_addr learned my addr 172.21.15.122:0/1334160886 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.703+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfa8103660 0x7fcfa8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfa77fe640 1 -- 172.21.15.122:0/1334160886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfa8103660 msgr2=0x7fcfa8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfa77fe640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfa8103660 0x7fcfa8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfa77fe640 1 -- 172.21.15.122:0/1334160886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8102d40 msgr2=0x7fcfa8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfa77fe640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8102d40 0x7fcfa8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfa77fe640 1 -- 172.21.15.122:0/1334160886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa819d3d0 con 0x7fcfa810c170 2024-09-12T02:03:04.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfa77fe640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 0x7fcfa819ccf0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000eaa0 tx=0x7fcfa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:04.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfac86a640 1 -- 172.21.15.122:0/1334160886 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000cdb0 con 0x7fcfa810c170 2024-09-12T02:03:04.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa819d6c0 con 0x7fcfa810c170 2024-09-12T02:03:04.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa81a4fa0 con 0x7fcfa810c170 2024-09-12T02:03:04.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfac86a640 1 -- 172.21.15.122:0/1334160886 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa0005020 con 0x7fcfa810c170 2024-09-12T02:03:04.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.704+0000 7fcfac86a640 1 -- 172.21.15.122:0/1334160886 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa001f060 con 0x7fcfa810c170 2024-09-12T02:03:04.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfac86a640 1 -- 172.21.15.122:0/1334160886 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcfa0004a40 con 0x7fcfa810c170 2024-09-12T02:03:04.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 Processor -- start 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 -- start start 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca09440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca13d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca16630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5622cc96e620 con 0x5622cca09060 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622cc915e50 con 0x5622cca14240 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfafc45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622cc96a5e0 con 0x5622cca09980 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca09440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca09440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55110/0 (socket says 172.21.15.122:55110) 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/358015428 learned_addr learned my addr 172.21.15.122:0/358015428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfac86a640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcf940a2b80 0x7fcf940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfa77fe640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcf940a2b80 0x7fcf940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca13d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/358015428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 msgr2=0x5622cca13d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca13d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/358015428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 msgr2=0x5622cca16630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca16630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.707+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/358015428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cc906d40 con 0x5622cca09060 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcfac86a640 1 -- 172.21.15.122:0/1334160886 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fcfa00cefd0 con 0x7fcfa810c170 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca09440 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b780 tx=0x7fcf9000bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277797601b088d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcf9dffb640 1 -- 172.21.15.122:0/358015428 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90003dd0 con 0x5622cca09060 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcf9dffb640 1 -- 172.21.15.122:0/358015428 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90016030 con 0x5622cca09060 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcf9dffb640 1 -- 172.21.15.122:0/358015428 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004e00 con 0x5622cca09060 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcfa77fe640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcf940a2b80 0x7fcf940a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcfa009b910 tx=0x7fcfa006b830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:04.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcfafc45740 1 -- 172.21.15.122:0/358015428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 msgr2=0x5622cca09440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.708+0000 7fcfafc45740 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca09440 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b780 tx=0x7fcf9000bc40 comp rx=0 tx=0).stop 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- 172.21.15.122:0/358015428 shutdown_connections 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca16630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca13d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 --2- 172.21.15.122:0/358015428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca09440 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- 172.21.15.122:0/358015428 >> 172.21.15.122:0/358015428 conn(0x5622cca055f0 msgr2=0x5622cca059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- 172.21.15.122:0/358015428 shutdown_connections 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- 172.21.15.122:0/358015428 wait complete. 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- start start 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 Processor -- start 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca11b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5622cca16e10 con 0x5622cca09060 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622cc8f6f10 con 0x5622cca14240 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.709+0000 7fcfafc45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622cca16f90 con 0x5622cca09980 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcfafc45740 10 client.0 _ll_register_callbacks cb 0x5622cc91adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55112/0 (socket says 172.21.15.122:55112) 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/2724468484 learned_addr learned my addr 172.21.15.122:0/2724468484 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/2724468484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 msgr2=0x5622cca115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 msgr2=0x5622cca11b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca11b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/2724468484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cc67d580 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca110a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000bf50 tx=0x7fcf9000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4123f0ca3b2383 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf86ffd640 1 -- 172.21.15.122:0/2724468484 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004b80 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf86ffd640 1 -- 172.21.15.122:0/2724468484 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fcf90004d20 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90016030 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf86ffd640 1 -- 172.21.15.122:0/2724468484 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcf90004f20 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcfafc45740 10 client.17802 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5622cc747960 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.710+0000 7fcf86ffd640 1 -- 172.21.15.122:0/2724468484 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90016030 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcf86ffd640 1 -- 172.21.15.122:0/2724468484 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fcf9001bbf0 con 0x5622cca09060 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfafc45740 10 client.17802 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfafc45740 10 client.17802 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfafc45740 20 client.17802 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfafc45740 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca23c80 0x5622cca26070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfafc45740 4 client.17802 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622cc97bb90 con 0x5622cca23c80 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca23c80 0x5622cca26070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca23c80 0x5622cca26070 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82497436bfb5cc70 server_cookie=c85efabc85c64e02 in_seq=0 out_seq=0 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.711+0000 7fcf86ffd640 10 client.17802 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcf86ffd640 1 -- 172.21.15.122:0/2724468484 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622cc97bb90 con 0x5622cca23c80 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcf86ffd640 10 client.17802 handle_command_reply: tid=0 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 client.17802 shutdown 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 msgr2=0x5622cca110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca110a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000bf50 tx=0x7fcf9000bf80 comp rx=0 tx=0).stop 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 shutdown_connections 2024-09-12T02:03:04.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca23c80 0x5622cca26070 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca11b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 --2- 172.21.15.122:0/2724468484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca110a0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcf86ffd640 0 client.17802 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.712+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 >> 172.21.15.122:0/2724468484 conn(0x5622cca055f0 msgr2=0x5622cca0b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.713+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 shutdown_connections 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.713+0000 7fcfafc45740 1 -- 172.21.15.122:0/2724468484 wait complete. 2024-09-12T02:03:04.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.713+0000 7fcfafc45740 20 client.17802 trim_cache size 0 max 16384 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.730+0000 7fcfafc45740 1 Processor -- start 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 -- start start 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca0bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca2b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88010d60 con 0x5622cca09060 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8800e630 con 0x5622cca14240 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfafc45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8800c460 con 0x5622cca09980 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca0bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca0bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55116/0 (socket says 172.21.15.122:55116) 2024-09-12T02:03:04.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/1807936224 learned_addr learned my addr 172.21.15.122:0/1807936224 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca2b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/1807936224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 msgr2=0x5622cca2b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca2b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/1807936224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 msgr2=0x5622cca2bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca2bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.731+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/1807936224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cca05420 con 0x5622cca09060 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca0bac0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b570 tx=0x7fcf9000b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445b2b70699a458 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcf9d7fa640 1 -- 172.21.15.122:0/1807936224 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004220 con 0x5622cca09060 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcf9d7fa640 1 -- 172.21.15.122:0/1807936224 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90016040 con 0x5622cca09060 2024-09-12T02:03:04.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcf9d7fa640 1 -- 172.21.15.122:0/1807936224 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9002adf0 con 0x5622cca09060 2024-09-12T02:03:04.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 -- 172.21.15.122:0/1807936224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 msgr2=0x5622cca0bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca0bac0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b570 tx=0x7fcf9000b500 comp rx=0 tx=0).stop 2024-09-12T02:03:04.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 -- 172.21.15.122:0/1807936224 shutdown_connections 2024-09-12T02:03:04.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca2bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca2b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 --2- 172.21.15.122:0/1807936224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca0bac0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.732+0000 7fcfafc45740 1 -- 172.21.15.122:0/1807936224 >> 172.21.15.122:0/1807936224 conn(0x5622cca055f0 msgr2=0x5622cca04750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 -- 172.21.15.122:0/1807936224 shutdown_connections 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 -- 172.21.15.122:0/1807936224 wait complete. 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 -- start start 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 Processor -- start 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca1c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8800caf0 con 0x5622cca09060 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8800df60 con 0x5622cca14240 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.733+0000 7fcfafc45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8800d1c0 con 0x5622cca09980 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcfafc45740 10 client.0 _ll_register_callbacks cb 0x7fcf900044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:04.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca1c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca1c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55130/0 (socket says 172.21.15.122:55130) 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/3473402634 learned_addr learned my addr 172.21.15.122:0/3473402634 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca1c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/3473402634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 msgr2=0x5622cca1c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca1c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 msgr2=0x5622cca19210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca19210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 -- 172.21.15.122:0/3473402634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cca19850 con 0x5622cca09060 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf9e7fc640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca1c420 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcf900059f0 tx=0x7fcf90005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7613759416f95aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:04.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf86ffd640 1 -- 172.21.15.122:0/3473402634 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9002a910 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf86ffd640 1 -- 172.21.15.122:0/3473402634 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fcf9002a040 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90016040 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcf86ffd640 1 -- 172.21.15.122:0/3473402634 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcf9002a240 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcfafc45740 10 client.17808 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.734+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5622cca8f150 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcf86ffd640 1 -- 172.21.15.122:0/3473402634 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90016040 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcf86ffd640 1 -- 172.21.15.122:0/3473402634 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fcf9001b6b0 con 0x5622cca09060 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfafc45740 10 client.17808 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfafc45740 10 client.17808 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfafc45740 20 client.17808 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfafc45740 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca8f7b0 0x5622cca91bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfafc45740 4 client.17808 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7fb8a6c9", "release": true, "await": true}] 2024-09-12T02:03:04.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7fb8a6c9", "release": true, "await": true}) -- 0x5622cc9760a0 con 0x5622cca8f7b0 2024-09-12T02:03:04.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.735+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca8f7b0 0x5622cca91bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:04.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.736+0000 7fcf86ffd640 10 client.17808 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:04.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.736+0000 7fcfa6ffd640 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca8f7b0 0x5622cca91bd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1ef2ce013468ba server_cookie=754571966f34998a in_seq=0 out_seq=0 2024-09-12T02:03:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.814+0000 7fcf86ffd640 1 -- 172.21.15.122:0/3473402634 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5622cc9760a0 con 0x5622cca8f7b0 2024-09-12T02:03:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.814+0000 7fcf86ffd640 10 client.17808 handle_command_reply: tid=0 2024-09-12T02:03:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.814+0000 7fcfafc45740 1 client.17808 shutdown 2024-09-12T02:03:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 msgr2=0x5622cca1c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca1c420 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcf900059f0 tx=0x7fcf90005bb0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 shutdown_connections 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622cca8f7b0 0x5622cca91bd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622cca14240 0x5622cca19210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622cca09980 0x5622cca1c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcf86ffd640 0 client.17808 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 --2- 172.21.15.122:0/3473402634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622cca09060 0x5622cca1c420 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 >> 172.21.15.122:0/3473402634 conn(0x5622cca055f0 msgr2=0x5622cca1f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 shutdown_connections 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 1 -- 172.21.15.122:0/3473402634 wait complete. 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.815+0000 7fcfafc45740 20 client.17808 trim_cache size 0 max 16384 2024-09-12T02:03:04.816 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 60, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "7fb8a6c9": { 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 60, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:03:04.817 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.2, 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:04.818 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcf940a2b80 msgr2=0x7fcf940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcf940a2b80 0x7fcf940a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcfa009b910 tx=0x7fcfa006b830 comp rx=0 tx=0).stop 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 msgr2=0x7fcfa819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 0x7fcfa819ccf0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000eaa0 tx=0x7fcfa000ef60 comp rx=0 tx=0).stop 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 shutdown_connections 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcf940a2b80 0x7fcf940a5040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa810c170 0x7fcfa819ccf0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfa8103660 0x7fcfa8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 --2- 172.21.15.122:0/1334160886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8102d40 0x7fcfa8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 >> 172.21.15.122:0/1334160886 conn(0x7fcfa80fc7a0 msgr2=0x7fcfa80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 shutdown_connections 2024-09-12T02:03:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:04.816+0000 7fcfad86c640 1 -- 172.21.15.122:0/1334160886 wait complete. 2024-09-12T02:03:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:04 smithi122 ceph-mon[30925]: pgmap v836: 97 pgs: 97 active+clean; 507 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 5.4 MiB/s wr, 181 op/s 2024-09-12T02:03:05.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68950 16.44 MB/sec execute 52 sec latency 23971.098 ms 2024-09-12T02:03:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:04 smithi151 ceph-mon[35229]: pgmap v836: 97 pgs: 97 active+clean; 507 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 5.4 MiB/s wr, 181 op/s 2024-09-12T02:03:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:04 smithi183 ceph-mon[35127]: pgmap v836: 97 pgs: 97 active+clean; 507 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 5.4 MiB/s wr, 181 op/s 2024-09-12T02:03:05.490 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7fb8a6c9', total seconds elapsed: 27.3 2024-09-12T02:03:05.490 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-12T02:03:05.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:05.752 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:06.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 69285 16.41 MB/sec execute 53 sec latency 234.814 ms 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 -- 172.21.15.122:0/3705321828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c106a90 msgr2=0x7f1a6c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 --2- 172.21.15.122:0/3705321828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c106a90 0x7f1a6c106e70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009900 tx=0x7f1a5c02a410 comp rx=0 tx=0).stop 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 -- 172.21.15.122:0/3705321828 shutdown_connections 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 --2- 172.21.15.122:0/3705321828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c100860 0x7f1a6c1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 --2- 172.21.15.122:0/3705321828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a6c107440 0x7f1a6c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 --2- 172.21.15.122:0/3705321828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c106a90 0x7f1a6c106e70 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.245+0000 7f1a71514640 1 -- 172.21.15.122:0/3705321828 >> 172.21.15.122:0/3705321828 conn(0x7f1a6c077230 msgr2=0x7f1a6c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 -- 172.21.15.122:0/3705321828 shutdown_connections 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 -- 172.21.15.122:0/3705321828 wait complete. 2024-09-12T02:03:06.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 Processor -- start 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 -- start start 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a6c100860 0x7f1a6c1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c106a90 0x7f1a6c1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 0x7f1a6c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c118590 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c118410 con 0x7f1a6c100860 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a71514640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c118710 con 0x7f1a6c106a90 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 0x7f1a6c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 0x7f1a6c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55148/0 (socket says 172.21.15.122:55148) 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a6b7fe640 1 -- 172.21.15.122:0/3468112282 learned_addr learned my addr 172.21.15.122:0/3468112282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a6affd640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a6c100860 0x7f1a6c1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.246+0000 7f1a63fff640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c106a90 0x7f1a6c1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a6b7fe640 1 -- 172.21.15.122:0/3468112282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c106a90 msgr2=0x7f1a6c1a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a6b7fe640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c106a90 0x7f1a6c1a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a6b7fe640 1 -- 172.21.15.122:0/3468112282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a6c100860 msgr2=0x7f1a6c1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a6b7fe640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a6c100860 0x7f1a6c1a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a6b7fe640 1 -- 172.21.15.122:0/3468112282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c19f3c0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a63fff640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c106a90 0x7f1a6c1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a6b7fe640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 0x7f1a6c19ed50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400eb10 tx=0x7f1a6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004b20 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a64004cc0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64010760 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c19f6b0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.247+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c1ab8f0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.249+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c106e70 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.251+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1a64005000 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.251+0000 7f1a68ff9640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a580a2c30 0x7f1a580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:06.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.251+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f1a640cf740 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.251+0000 7f1a6affd640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a580a2c30 0x7f1a580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:06.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.252+0000 7f1a6affd640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a580a2c30 0x7f1a580a50f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c19f9e0 tx=0x7f1a5c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:06.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.255+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a640541b0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.482+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a6c19fdd0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.482+0000 7f1a68ff9640 1 -- 172.21.15.122:0/3468112282 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7f1a6c19fdd0 con 0x7f1a6c19e8a0 2024-09-12T02:03:06.483 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:03:06.483 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:03:06.483 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a580a2c30 msgr2=0x7f1a580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a580a2c30 0x7f1a580a50f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c19f9e0 tx=0x7f1a5c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 msgr2=0x7f1a6c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 0x7f1a6c19ed50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400eb10 tx=0x7f1a6400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 shutdown_connections 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a580a2c30 0x7f1a580a50f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a6c19e8a0 0x7f1a6c19ed50 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c106a90 0x7f1a6c1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 --2- 172.21.15.122:0/3468112282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a6c100860 0x7f1a6c1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:06.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.484+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 >> 172.21.15.122:0/3468112282 conn(0x7f1a6c077230 msgr2=0x7f1a6c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:06.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.485+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 shutdown_connections 2024-09-12T02:03:06.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:06.485+0000 7f1a71514640 1 -- 172.21.15.122:0/3468112282 wait complete. 2024-09-12T02:03:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:06 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3468112282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:06 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3468112282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:06 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3468112282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:07.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 69958 16.68 MB/sec execute 54 sec latency 80.166 ms 2024-09-12T02:03:07.218 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-12T02:03:07.218 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-12T02:03:07.513 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 -- 172.21.15.122:0/2714332293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9540ffe30 msgr2=0x7fb954100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 --2- 172.21.15.122:0/2714332293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9540ffe30 0x7fb954100230 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb94402a410 comp rx=0 tx=0).stop 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 -- 172.21.15.122:0/2714332293 shutdown_connections 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 --2- 172.21.15.122:0/2714332293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb954103860 0x7fb95410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 --2- 172.21.15.122:0/2714332293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb954102ea0 0x7fb954103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 --2- 172.21.15.122:0/2714332293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9540ffe30 0x7fb954100230 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 -- 172.21.15.122:0/2714332293 >> 172.21.15.122:0/2714332293 conn(0x7fb9540fb970 msgr2=0x7fb9540fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 -- 172.21.15.122:0/2714332293 shutdown_connections 2024-09-12T02:03:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.574+0000 7fb958f52640 1 -- 172.21.15.122:0/2714332293 wait complete. 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.575+0000 7fb958f52640 1 Processor -- start 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.575+0000 7fb958f52640 1 -- start start 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.575+0000 7fb958f52640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9540ffe30 0x7fb954197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.575+0000 7fb958f52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 0x7fb954198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.575+0000 7fb958f52640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb954103860 0x7fb95419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.575+0000 7fb958f52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb95410d830 con 0x7fb954102ea0 2024-09-12T02:03:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb958f52640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb95410d6b0 con 0x7fb9540ffe30 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb958f52640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb95410d9b0 con 0x7fb954103860 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 0x7fb954198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 0x7fb954198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55164/0 (socket says 172.21.15.122:55164) 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 -- 172.21.15.122:0/612972769 learned_addr learned my addr 172.21.15.122:0/612972769 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb95259c640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9540ffe30 0x7fb954197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb952d9d640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb954103860 0x7fb95419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 -- 172.21.15.122:0/612972769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb954103860 msgr2=0x7fb95419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb954103860 0x7fb95419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 -- 172.21.15.122:0/612972769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9540ffe30 msgr2=0x7fb954197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9540ffe30 0x7fb954197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 -- 172.21.15.122:0/612972769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95419f9d0 con 0x7fb954102ea0 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb951d9b640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 0x7fb954198300 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d950 tx=0x7fb94800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb9437fe640 1 -- 172.21.15.122:0/612972769 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948017030 con 0x7fb954102ea0 2024-09-12T02:03:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95419fcc0 con 0x7fb954102ea0 2024-09-12T02:03:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb9437fe640 1 -- 172.21.15.122:0/612972769 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948004020 con 0x7fb954102ea0 2024-09-12T02:03:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb9437fe640 1 -- 172.21.15.122:0/612972769 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480047a0 con 0x7fb954102ea0 2024-09-12T02:03:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.576+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9541a0180 con 0x7fb954102ea0 2024-09-12T02:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.579+0000 7fb9437fe640 1 -- 172.21.15.122:0/612972769 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb9480041d0 con 0x7fb954102ea0 2024-09-12T02:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.579+0000 7fb95bdca740 1 Processor -- start 2024-09-12T02:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.579+0000 7fb95bdca740 1 -- start start 2024-09-12T02:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be63da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be66830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95bdca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563d7bdc22f0 con 0x563d7be5e9a0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95bdca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d7bd6a270 con 0x563d7be5f2e0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95bdca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d7bdbe2b0 con 0x563d7be64370 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb9437fe640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb93c0a2c30 0x7fb93c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be5eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55172/0 (socket says 172.21.15.122:55172) 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be63da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb952d9d640 1 -- 172.21.15.122:0/2864882694 learned_addr learned my addr 172.21.15.122:0/2864882694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb951d9b640 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be66830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb952d9d640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb93c0a2c30 0x7fb93c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb9437fe640 1 -- 172.21.15.122:0/612972769 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb9480ceaf0 con 0x7fb954102ea0 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 -- 172.21.15.122:0/2864882694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 msgr2=0x563d7be66830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be66830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 -- 172.21.15.122:0/2864882694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 msgr2=0x563d7be5eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be5eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 -- 172.21.15.122:0/2864882694 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7bd69340 con 0x563d7be5f2e0 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb952d9d640 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.580+0000 7fb95259c640 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be63da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb944009230 tx=0x7fb944005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc69b9f330d62d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.581+0000 7fb952d9d640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb93c0a2c30 0x7fb93c0a50f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0049d0 tx=0x7fb94c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.581+0000 7fb940ff9640 1 -- 172.21.15.122:0/2864882694 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944035040 con 0x563d7be5f2e0 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.581+0000 7fb940ff9640 1 -- 172.21.15.122:0/2864882694 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944005520 con 0x563d7be5f2e0 2024-09-12T02:03:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.581+0000 7fb940ff9640 1 -- 172.21.15.122:0/2864882694 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440316b0 con 0x563d7be5f2e0 2024-09-12T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.581+0000 7fb95bdca740 1 -- 172.21.15.122:0/2864882694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 msgr2=0x563d7be63da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be63da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb944009230 tx=0x7fb944005c00 comp rx=0 tx=0).stop 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- 172.21.15.122:0/2864882694 shutdown_connections 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be66830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be63da0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- 172.21.15.122:0/2864882694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- 172.21.15.122:0/2864882694 >> 172.21.15.122:0/2864882694 conn(0x563d7be5a010 msgr2=0x563d7be5a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- 172.21.15.122:0/2864882694 shutdown_connections 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- 172.21.15.122:0/2864882694 wait complete. 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- start start 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 Processor -- start 2024-09-12T02:03:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563d7be672e0 con 0x563d7be5e9a0 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d7bd4b290 con 0x563d7be5f2e0 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d7be67460 con 0x563d7be64370 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.582+0000 7fb95bdca740 10 client.0 _ll_register_callbacks cb 0x563d7bd7c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55186/0 (socket says 172.21.15.122:55186) 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb951d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 -- 172.21.15.122:0/1014407451 learned_addr learned my addr 172.21.15.122:0/1014407451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 -- 172.21.15.122:0/1014407451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 msgr2=0x563d7be788d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be788d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 -- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 msgr2=0x563d7be71890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be71890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 -- 172.21.15.122:0/1014407451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7bad5210 con 0x563d7be5e9a0 2024-09-12T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb951d9b640 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95259c640 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71350 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb94402aa00 tx=0x7fb9440045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cfab2948a12147 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb939ffb640 1 -- 172.21.15.122:0/1014407451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944053060 con 0x563d7be5e9a0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb944005520 con 0x563d7be5e9a0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb939ffb640 1 -- 172.21.15.122:0/1014407451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb944004970 con 0x563d7be5e9a0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb939ffb640 1 -- 172.21.15.122:0/1014407451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb944035050 con 0x563d7be5e9a0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95bdca740 10 client.17826 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.583+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563d7bba1620 con 0x563d7be5e9a0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb939ffb640 1 -- 172.21.15.122:0/1014407451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944005520 con 0x563d7be5e9a0 2024-09-12T02:03:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb939ffb640 1 -- 172.21.15.122:0/1014407451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb944058070 con 0x563d7be5e9a0 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb95bdca740 10 client.17826 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb95bdca740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb95bdca740 20 client.17826 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb95bdca740 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7be7efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb95bdca740 4 client.17826 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d7bdcfa20 con 0x563d7be7cbd0 2024-09-12T02:03:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb951d9b640 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7be7efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb951d9b640 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7be7efe0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809a08007ea0e11a server_cookie=d23b7fbab7930531 in_seq=0 out_seq=0 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.584+0000 7fb939ffb640 10 client.17826 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb939ffb640 1 -- 172.21.15.122:0/1014407451 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d7bdcfa20 con 0x563d7be7cbd0 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb939ffb640 10 client.17826 handle_command_reply: tid=0 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb95bdca740 1 client.17826 shutdown 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 msgr2=0x563d7be71350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb95bdca740 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71350 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb94402aa00 tx=0x7fb9440045c0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 shutdown_connections 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb95bdca740 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7be7efe0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.586+0000 7fb95bdca740 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be788d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb939ffb640 0 client.17826 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb95bdca740 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb95bdca740 1 --2- 172.21.15.122:0/1014407451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71350 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 >> 172.21.15.122:0/1014407451 conn(0x563d7be5a010 msgr2=0x563d7be5b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 shutdown_connections 2024-09-12T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb95bdca740 1 -- 172.21.15.122:0/1014407451 wait complete. 2024-09-12T02:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.587+0000 7fb95bdca740 20 client.17826 trim_cache size 0 max 16384 2024-09-12T02:03:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 Processor -- start 2024-09-12T02:03:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 -- start start 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a9590 con 0x563d7be5e9a0 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a6e60 con 0x563d7be5f2e0 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95bdca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a4c90 con 0x563d7be64370 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55198/0 (socket says 172.21.15.122:55198) 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 -- 172.21.15.122:0/900119499 learned_addr learned my addr 172.21.15.122:0/900119499 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb951d9b640 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb952d9d640 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 -- 172.21.15.122:0/900119499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 msgr2=0x563d7be590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 -- 172.21.15.122:0/900119499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 msgr2=0x563d7be78890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be78890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 -- 172.21.15.122:0/900119499 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7be59e30 con 0x563d7be5e9a0 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.604+0000 7fb95259c640 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71920 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb9440314e0 tx=0x7fb944003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b3283deb62c6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb93bfff640 1 -- 172.21.15.122:0/900119499 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94405e070 con 0x563d7be5e9a0 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb93bfff640 1 -- 172.21.15.122:0/900119499 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94404a030 con 0x563d7be5e9a0 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb93bfff640 1 -- 172.21.15.122:0/900119499 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94405a460 con 0x563d7be5e9a0 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 -- 172.21.15.122:0/900119499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 msgr2=0x563d7be71920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71920 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb9440314e0 tx=0x7fb944003800 comp rx=0 tx=0).stop 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 -- 172.21.15.122:0/900119499 shutdown_connections 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 --2- 172.21.15.122:0/900119499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be71920 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 -- 172.21.15.122:0/900119499 >> 172.21.15.122:0/900119499 conn(0x563d7be5a010 msgr2=0x563d7be589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.605+0000 7fb95bdca740 1 -- 172.21.15.122:0/900119499 shutdown_connections 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 -- 172.21.15.122:0/900119499 wait complete. 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 -- start start 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 Processor -- start 2024-09-12T02:03:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be75f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be76490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be774a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a5320 con 0x563d7be5e9a0 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a6790 con 0x563d7be5f2e0 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a59f0 con 0x563d7be64370 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95bdca740 10 client.0 _ll_register_callbacks cb 0x7fb94c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be75f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be75f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55214/0 (socket says 172.21.15.122:55214) 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb95259c640 1 -- 172.21.15.122:0/3317252881 learned_addr learned my addr 172.21.15.122:0/3317252881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb951d9b640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be76490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.606+0000 7fb952d9d640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be774a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95259c640 1 -- 172.21.15.122:0/3317252881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 msgr2=0x563d7be774a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95259c640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be774a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95259c640 1 -- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 msgr2=0x563d7be76490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95259c640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be76490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95259c640 1 -- 172.21.15.122:0/3317252881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7be77ab0 con 0x563d7be5e9a0 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb951d9b640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95259c640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be75f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb944005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b225e2e8e22bb14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb939ffb640 1 -- 172.21.15.122:0/3317252881 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94405e040 con 0x563d7be5e9a0 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb939ffb640 1 -- 172.21.15.122:0/3317252881 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb94405a460 con 0x563d7be5e9a0 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb94404a030 con 0x563d7be5e9a0 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb939ffb640 1 -- 172.21.15.122:0/3317252881 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb94405a660 con 0x563d7be5e9a0 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95bdca740 10 client.17832 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.607+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563d7bee78c0 con 0x563d7be5e9a0 2024-09-12T02:03:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb939ffb640 1 -- 172.21.15.122:0/3317252881 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94404a030 con 0x563d7be5e9a0 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb939ffb640 1 -- 172.21.15.122:0/3317252881 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb944058070 con 0x563d7be5e9a0 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb95bdca740 10 client.17832 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb95bdca740 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb95bdca740 20 client.17832 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb95bdca740 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7beea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb95bdca740 4 client.17832 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.608+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d7bdc9d70 con 0x563d7be7cbd0 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.609+0000 7fb951d9b640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7beea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.609+0000 7fb939ffb640 10 client.17832 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.609+0000 7fb951d9b640 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7beea110 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2aa0d0fe9e3174 server_cookie=1f5e9685785e34eb in_seq=0 out_seq=0 2024-09-12T02:03:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb939ffb640 1 -- 172.21.15.122:0/3317252881 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d7bdc9d70 con 0x563d7be7cbd0 2024-09-12T02:03:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb939ffb640 10 client.17832 handle_command_reply: tid=0 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 client.17832 shutdown 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 msgr2=0x563d7be75f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be75f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb944005990 comp rx=0 tx=0).stop 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 shutdown_connections 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x563d7be7cbd0 0x563d7beea110 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7be64370 0x563d7be774a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d7be5f2e0 0x563d7be76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 --2- 172.21.15.122:0/3317252881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d7be5e9a0 0x563d7be75f50 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.725+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 >> 172.21.15.122:0/3317252881 conn(0x563d7be5a010 msgr2=0x563d7be591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb939ffb640 0 client.17832 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 shutdown_connections 2024-09-12T02:03:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb95bdca740 1 -- 172.21.15.122:0/3317252881 wait complete. 2024-09-12T02:03:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb95bdca740 20 client.17832 trim_cache size 0 max 16384 2024-09-12T02:03:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:03:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:03:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:03:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:03:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb93c0a2c30 msgr2=0x7fb93c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb93c0a2c30 0x7fb93c0a50f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0049d0 tx=0x7fb94c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 msgr2=0x7fb954198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 0x7fb954198300 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d950 tx=0x7fb94800de10 comp rx=0 tx=0).stop 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 shutdown_connections 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb93c0a2c30 0x7fb93c0a50f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb954103860 0x7fb95419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb954102ea0 0x7fb954198300 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 --2- 172.21.15.122:0/612972769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9540ffe30 0x7fb954197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 >> 172.21.15.122:0/612972769 conn(0x7fb9540fb970 msgr2=0x7fb954106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.726+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 shutdown_connections 2024-09-12T02:03:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:07.727+0000 7fb958f52640 1 -- 172.21.15.122:0/612972769 wait complete. 2024-09-12T02:03:07.789 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:07 smithi122 ceph-mon[30925]: pgmap v837: 97 pgs: 97 active+clean; 502 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 5.3 MiB/s wr, 187 op/s 2024-09-12T02:03:07.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:07 smithi151 ceph-mon[35229]: pgmap v837: 97 pgs: 97 active+clean; 502 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 5.3 MiB/s wr, 187 op/s 2024-09-12T02:03:07.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:07 smithi183 ceph-mon[35127]: pgmap v837: 97 pgs: 97 active+clean; 502 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 5.3 MiB/s wr, 187 op/s 2024-09-12T02:03:08.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/1232876034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b74102d30 msgr2=0x7f4b74103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/1232876034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b74102d30 0x7f4b74103110 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009930 tx=0x7f4b6402a420 comp rx=0 tx=0).stop 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/1232876034 shutdown_connections 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/1232876034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b741082a0 0x7f4b7410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/1232876034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741036e0 0x7f4b74107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/1232876034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b74102d30 0x7f4b74103110 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/1232876034 >> 172.21.15.122:0/1232876034 conn(0x7f4b740fc7d0 msgr2=0x7f4b740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/1232876034 shutdown_connections 2024-09-12T02:03:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.043+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/1232876034 wait complete. 2024-09-12T02:03:08.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.044+0000 7f4b7a2e8640 1 Processor -- start 2024-09-12T02:03:08.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.044+0000 7f4b7a2e8640 1 -- start start 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.044+0000 7f4b7a2e8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b74102d30 0x7f4b741a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b7a2e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b741036e0 0x7f4b741a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b7a2e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 0x7f4b7419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b7a2e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741120a0 con 0x7f4b741036e0 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b7a2e8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74111f20 con 0x7f4b74102d30 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b7a2e8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74112220 con 0x7f4b741082a0 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 0x7f4b7419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b741036e0 0x7f4b741a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 0x7f4b7419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:54396/0 (socket says 172.21.15.122:54396) 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 -- 172.21.15.122:0/207568985 learned_addr learned my addr 172.21.15.122:0/207568985 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 -- 172.21.15.122:0/207568985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b74102d30 msgr2=0x7f4b741a0450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b74102d30 0x7f4b741a0450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 -- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b741036e0 msgr2=0x7f4b741a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b741036e0 0x7f4b741a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b78885640 1 -- 172.21.15.122:0/207568985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7419ad40 con 0x7f4b741082a0 2024-09-12T02:03:08.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.045+0000 7f4b737fe640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b741036e0 0x7f4b741a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:08.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.046+0000 7f4b78885640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 0x7f4b7419a610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00eb10 tx=0x7f4b6c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.046+0000 7f4b717fa640 1 -- 172.21.15.122:0/207568985 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c004b20 con 0x7f4b741082a0 2024-09-12T02:03:08.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.046+0000 7f4b717fa640 1 -- 172.21.15.122:0/207568985 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c004cc0 con 0x7f4b741082a0 2024-09-12T02:03:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.046+0000 7f4b717fa640 1 -- 172.21.15.122:0/207568985 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0107b0 con 0x7f4b741082a0 2024-09-12T02:03:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.046+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7419b030 con 0x7f4b741082a0 2024-09-12T02:03:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.046+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b741a7320 con 0x7f4b741082a0 2024-09-12T02:03:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.049+0000 7f4b717fa640 1 -- 172.21.15.122:0/207568985 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4b6c005000 con 0x7f4b741082a0 2024-09-12T02:03:08.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 Processor -- start 2024-09-12T02:03:08.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 -- start start 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7ed319160 con 0x55c7ed3bfa30 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7ed2c0720 con 0x55c7ed3b5170 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c7ed315120 con 0x55c7ed3b4850 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b73fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b73fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:32814/0 (socket says 172.21.15.122:32814) 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b73fff640 1 -- 172.21.15.122:0/4160742787 learned_addr learned my addr 172.21.15.122:0/4160742787 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.050+0000 7f4b737fe640 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b737fe640 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b717fa640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b5c0a2c00 0x7f4b5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b73fff640 1 -- 172.21.15.122:0/4160742787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b4850 msgr2=0x55c7ed3b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b73fff640 1 -- 172.21.15.122:0/4160742787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3bfa30 msgr2=0x55c7ed3c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b78885640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b5c0a2c00 0x7f4b5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b717fa640 1 -- 172.21.15.122:0/207568985 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4b6c0cf5d0 con 0x7f4b741082a0 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.051+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b73fff640 1 -- 172.21.15.122:0/4160742787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ed08aba0 con 0x55c7ed3b5170 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b737fe640 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bf4f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006010 tx=0x7f4b64031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b06b8366036d948 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b78885640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b5c0a2c00 0x7f4b5c0a50c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00efd0 tx=0x7f4b6c066a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b5a7fc640 1 -- 172.21.15.122:0/4160742787 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6403e070 con 0x55c7ed3b5170 2024-09-12T02:03:08.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b5a7fc640 1 -- 172.21.15.122:0/4160742787 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b64031870 con 0x55c7ed3b5170 2024-09-12T02:03:08.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.052+0000 7f4b5a7fc640 1 -- 172.21.15.122:0/4160742787 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64035040 con 0x55c7ed3b5170 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.053+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4160742787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 msgr2=0x55c7ed3bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.053+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bf4f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006010 tx=0x7f4b64031510 comp rx=0 tx=0).stop 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4160742787 shutdown_connections 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bf4f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4160742787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4160742787 >> 172.21.15.122:0/4160742787 conn(0x55c7ed3b0de0 msgr2=0x55c7ed3b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4160742787 shutdown_connections 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4160742787 wait complete. 2024-09-12T02:03:08.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.054+0000 7f4b7c6c1740 1 -- start start 2024-09-12T02:03:08.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 Processor -- start 2024-09-12T02:03:08.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 0x55c7ed3bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3bd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7ed3c2600 con 0x55c7ed3b5170 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7ed2a1900 con 0x55c7ed3b4850 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c7ed3c2780 con 0x55c7ed3bfa30 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b7c6c1740 10 client.0 _ll_register_callbacks cb 0x55c7ed29f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.055+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 0x55c7ed3bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 0x55c7ed3bc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:32826/0 (socket says 172.21.15.122:32826) 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 -- 172.21.15.122:0/2622273110 learned_addr learned my addr 172.21.15.122:0/2622273110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3bd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b73fff640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 -- 172.21.15.122:0/2622273110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3bfa30 msgr2=0x55c7ed3bd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3bd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 -- 172.21.15.122:0/2622273110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b5170 msgr2=0x55c7ed3bce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.056+0000 7f4b737fe640 1 -- 172.21.15.122:0/2622273110 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ed2e1c50 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b737fe640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 0x55c7ed3bc900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b740697d0 tx=0x7f4b68005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f215fc193a75c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b537fe640 1 -- 172.21.15.122:0/2622273110 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68004260 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b64031870 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b537fe640 1 -- 172.21.15.122:0/2622273110 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4b68004400 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b537fe640 1 -- 172.21.15.122:0/2622273110 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4b68029050 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b7c6c1740 10 client.26942 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c7ed3d6b50 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.057+0000 7f4b537fe640 1 -- 172.21.15.122:0/2622273110 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b68003ad0 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b537fe640 1 -- 172.21.15.122:0/2622273110 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4b680215b0 con 0x55c7ed3b4850 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b7c6c1740 10 client.26942 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:08.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b7c6c1740 10 client.26942 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:03:08.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b7c6c1740 20 client.26942 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:08.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed3d8330 0x55c7ed3da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b7c6c1740 4 client.26942 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:08.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b64031870 con 0x55c7ed3d8330 2024-09-12T02:03:08.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.058+0000 7f4b73fff640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed3d8330 0x55c7ed3da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.061 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70554 16.97 MB/sec execute 55 sec latency 86.945 ms 2024-09-12T02:03:08.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.059+0000 7f4b537fe640 10 client.26942 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.059+0000 7f4b73fff640 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed3d8330 0x55c7ed3da7d0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1612997de85a92 server_cookie=d5cd940b8032199f in_seq=0 out_seq=0 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.059+0000 7f4b537fe640 1 -- 172.21.15.122:0/2622273110 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4b64031870 con 0x55c7ed3d8330 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.059+0000 7f4b537fe640 10 client.26942 handle_command_reply: tid=0 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.059+0000 7f4b7c6c1740 1 client.26942 shutdown 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 msgr2=0x55c7ed3bc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 0x55c7ed3bc900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b740697d0 tx=0x7f4b68005f90 comp rx=0 tx=0).stop 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 shutdown_connections 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed3d8330 0x55c7ed3da7d0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3bd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b5170 0x55c7ed3bce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/2622273110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3b4850 0x55c7ed3bc900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 >> 172.21.15.122:0/2622273110 conn(0x55c7ed3b0de0 msgr2=0x55c7ed3b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b537fe640 0 client.26942 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:08.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 shutdown_connections 2024-09-12T02:03:08.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/2622273110 wait complete. 2024-09-12T02:03:08.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.060+0000 7f4b7c6c1740 20 client.26942 trim_cache size 0 max 16384 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.083+0000 7f4b7c6c1740 1 Processor -- start 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.083+0000 7f4b7c6c1740 1 -- start start 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed3be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741adec0 con 0x55c7ed3b4850 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741ab790 con 0x55c7ed3bfa30 2024-09-12T02:03:08.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741a9620 con 0x55c7ed3b5170 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55254/0 (socket says 172.21.15.122:55254) 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 -- 172.21.15.122:0/1880518269 learned_addr learned my addr 172.21.15.122:0/1880518269 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b73fff640 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed3be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 -- 172.21.15.122:0/1880518269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 msgr2=0x55c7ed3be870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed3be870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 -- 172.21.15.122:0/1880518269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 msgr2=0x55c7ed3e00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3e00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 -- 172.21.15.122:0/1880518269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ed3bb8b0 con 0x55c7ed3b4850 2024-09-12T02:03:08.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b73fff640 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed3be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:08.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.084+0000 7f4b737fe640 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b72b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x55c7ed3b78a0 tx=0x7f4b68010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203cf68603c6791a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b59ffb640 1 -- 172.21.15.122:0/1880518269 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68026020 con 0x55c7ed3b4850 2024-09-12T02:03:08.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b59ffb640 1 -- 172.21.15.122:0/1880518269 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b68021b50 con 0x55c7ed3b4850 2024-09-12T02:03:08.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b59ffb640 1 -- 172.21.15.122:0/1880518269 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680035f0 con 0x55c7ed3b4850 2024-09-12T02:03:08.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/1880518269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 msgr2=0x55c7ed3b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b72b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x55c7ed3b78a0 tx=0x7f4b68010e50 comp rx=0 tx=0).stop 2024-09-12T02:03:08.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/1880518269 shutdown_connections 2024-09-12T02:03:08.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed3be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/1880518269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed3b72b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.085+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/1880518269 >> 172.21.15.122:0/1880518269 conn(0x55c7ed3b0de0 msgr2=0x55c7ed3be160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.086+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/1880518269 shutdown_connections 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.086+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/1880518269 wait complete. 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.086+0000 7f4b7c6c1740 1 -- start start 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.086+0000 7f4b7c6c1740 1 Processor -- start 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed43b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed43b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741a9c70 con 0x55c7ed3b4850 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed43b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed43b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55268/0 (socket says 172.21.15.122:55268) 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b737fe640 1 -- 172.21.15.122:0/4215331419 learned_addr learned my addr 172.21.15.122:0/4215331419 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741ab0c0 con 0x55c7ed3bfa30 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b741aa320 con 0x55c7ed3b5170 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed43b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b7c6c1740 10 client.0 _ll_register_callbacks cb 0x7f4b680104d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b737fe640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 -- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 msgr2=0x55c7ed3c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 -- 172.21.15.122:0/4215331419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 msgr2=0x55c7ed43b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed43b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 -- 172.21.15.122:0/4215331419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ed3c3760 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b73fff640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed43b8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b640096c0 tx=0x7f4b640058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c19bded93a1acccf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.087+0000 7f4b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b537fe640 1 -- 172.21.15.122:0/4215331419 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6403e040 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b537fe640 1 -- 172.21.15.122:0/4215331419 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4b64004240 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b68021b50 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b537fe640 1 -- 172.21.15.122:0/4215331419 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4b64035040 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 10 client.26899 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c7ed442b40 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b537fe640 1 -- 172.21.15.122:0/4215331419 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b68021b50 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b537fe640 1 -- 172.21.15.122:0/4215331419 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4b64004a90 con 0x55c7ed3b5170 2024-09-12T02:03:08.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 10 client.26899 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 10 client.26899 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 20 client.26899 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed442e50 0x55c7ed445270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 4 client.26899 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x55c7ed326890 con 0x55c7ed442e50 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.088+0000 7f4b737fe640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed442e50 0x55c7ed445270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.089+0000 7f4b737fe640 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed442e50 0x55c7ed445270 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc89f57d189ce3e server_cookie=41613a9fa71f2fc in_seq=0 out_seq=0 2024-09-12T02:03:08.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.089+0000 7f4b537fe640 10 client.26899 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:08.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b537fe640 1 -- 172.21.15.122:0/4215331419 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c7ed326890 con 0x55c7ed442e50 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b537fe640 10 client.26899 handle_command_reply: tid=0 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 client.26899 shutdown 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 msgr2=0x55c7ed43b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed43b8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b640096c0 tx=0x7f4b640058e0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 shutdown_connections 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c7ed442e50 0x55c7ed445270 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7ed3bfa30 0x55c7ed3c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c7ed3b5170 0x55c7ed43b8e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b537fe640 0 client.26899 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 --2- 172.21.15.122:0/4215331419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7ed3b4850 0x55c7ed43b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 >> 172.21.15.122:0/4215331419 conn(0x55c7ed3b0de0 msgr2=0x55c7ed3cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 shutdown_connections 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 1 -- 172.21.15.122:0/4215331419 wait complete. 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.665+0000 7f4b7c6c1740 20 client.26899 trim_cache size 0 max 16384 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 62, 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "32aa008e": { 2024-09-12T02:03:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 62, 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.2, 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:03:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:03:08.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b5c0a2c00 msgr2=0x7f4b5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b5c0a2c00 0x7f4b5c0a50c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00efd0 tx=0x7f4b6c066a00 comp rx=0 tx=0).stop 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 msgr2=0x7f4b7419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 0x7f4b7419a610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00eb10 tx=0x7f4b6c00c2d0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 shutdown_connections 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4b5c0a2c00 0x7f4b5c0a50c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b741082a0 0x7f4b7419a610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b741036e0 0x7f4b741a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 --2- 172.21.15.122:0/207568985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b74102d30 0x7f4b741a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 >> 172.21.15.122:0/207568985 conn(0x7f4b740fc7d0 msgr2=0x7f4b7410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 shutdown_connections 2024-09-12T02:03:08.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:08.666+0000 7f4b7a2e8640 1 -- 172.21.15.122:0/207568985 wait complete. 2024-09-12T02:03:08.983 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:08 smithi122 ceph-mon[30925]: pgmap v838: 97 pgs: 97 active+clean; 495 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 224 op/s 2024-09-12T02:03:09.058 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 16.73 MB/sec execute 56 sec latency 973.591 ms 2024-09-12T02:03:09.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:08 smithi151 ceph-mon[35229]: pgmap v838: 97 pgs: 97 active+clean; 495 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 224 op/s 2024-09-12T02:03:09.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:08 smithi183 ceph-mon[35127]: pgmap v838: 97 pgs: 97 active+clean; 495 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 224 op/s 2024-09-12T02:03:09.337 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '32aa008e', quiesce took 3.8 seconds. Will release after: 20.45 2024-09-12T02:03:10.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 16.44 MB/sec execute 57 sec latency 1973.706 ms 2024-09-12T02:03:11.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 16.15 MB/sec execute 58 sec latency 2973.780 ms 2024-09-12T02:03:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:10 smithi151 ceph-mon[35229]: pgmap v839: 97 pgs: 97 active+clean; 484 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 331 op/s 2024-09-12T02:03:11.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:10 smithi183 ceph-mon[35127]: pgmap v839: 97 pgs: 97 active+clean; 484 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 331 op/s 2024-09-12T02:03:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:10 smithi122 ceph-mon[30925]: pgmap v839: 97 pgs: 97 active+clean; 484 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 331 op/s 2024-09-12T02:03:12.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 15.88 MB/sec execute 59 sec latency 3973.857 ms 2024-09-12T02:03:13.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 15.61 MB/sec execute 60 sec latency 4973.908 ms 2024-09-12T02:03:13.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:12 smithi151 ceph-mon[35229]: pgmap v840: 97 pgs: 97 active+clean; 497 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.6 MiB/s wr, 396 op/s 2024-09-12T02:03:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:12 smithi183 ceph-mon[35127]: pgmap v840: 97 pgs: 97 active+clean; 497 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.6 MiB/s wr, 396 op/s 2024-09-12T02:03:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:12 smithi122 ceph-mon[30925]: pgmap v840: 97 pgs: 97 active+clean; 497 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.6 MiB/s wr, 396 op/s 2024-09-12T02:03:14.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 15.36 MB/sec execute 61 sec latency 5973.941 ms 2024-09-12T02:03:15.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 15.11 MB/sec execute 62 sec latency 6973.978 ms 2024-09-12T02:03:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:14 smithi151 ceph-mon[35229]: pgmap v841: 97 pgs: 97 active+clean; 496 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 366 op/s 2024-09-12T02:03:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:14 smithi183 ceph-mon[35127]: pgmap v841: 97 pgs: 97 active+clean; 496 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 366 op/s 2024-09-12T02:03:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:14 smithi122 ceph-mon[30925]: pgmap v841: 97 pgs: 97 active+clean; 496 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.8 MiB/s wr, 366 op/s 2024-09-12T02:03:16.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 14.87 MB/sec execute 63 sec latency 7974.063 ms 2024-09-12T02:03:17.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 14.64 MB/sec execute 64 sec latency 8974.164 ms 2024-09-12T02:03:17.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:16 smithi151 ceph-mon[35229]: pgmap v842: 97 pgs: 97 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-09-12T02:03:17.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:16 smithi183 ceph-mon[35127]: pgmap v842: 97 pgs: 97 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-09-12T02:03:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:16 smithi122 ceph-mon[30925]: pgmap v842: 97 pgs: 97 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.9 MiB/s wr, 350 op/s 2024-09-12T02:03:18.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 14.41 MB/sec execute 65 sec latency 9974.265 ms 2024-09-12T02:03:18.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:18 smithi122 ceph-mon[30925]: pgmap v843: 97 pgs: 97 active+clean; 479 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 327 op/s 2024-09-12T02:03:19.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 14.20 MB/sec execute 66 sec latency 10974.368 ms 2024-09-12T02:03:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:18 smithi151 ceph-mon[35229]: pgmap v843: 97 pgs: 97 active+clean; 479 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 327 op/s 2024-09-12T02:03:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:18 smithi183 ceph-mon[35127]: pgmap v843: 97 pgs: 97 active+clean; 479 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 327 op/s 2024-09-12T02:03:20.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 13.98 MB/sec execute 67 sec latency 11974.471 ms 2024-09-12T02:03:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.822+0000 7f6df4a65640 1 -- 172.21.15.122:0/3617620046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0101c40 msgr2=0x7f6df01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.822+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3617620046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0101c40 0x7f6df01020c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009900 tx=0x7f6de402a410 comp rx=0 tx=0).stop 2024-09-12T02:03:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 -- 172.21.15.122:0/3617620046 shutdown_connections 2024-09-12T02:03:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3617620046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6df0102600 0x7f6df0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3617620046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0101c40 0x7f6df01020c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3617620046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df0100a50 0x7f6df0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 -- 172.21.15.122:0/3617620046 >> 172.21.15.122:0/3617620046 conn(0x7f6df00fc1b0 msgr2=0x7f6df00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 -- 172.21.15.122:0/3617620046 shutdown_connections 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 -- 172.21.15.122:0/3617620046 wait complete. 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 Processor -- start 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.823+0000 7f6df4a65640 1 -- start start 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 0x7f6df0196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6df0101c40 0x7f6df0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df0102600 0x7f6df019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010c6d0 con 0x7f6df0100a50 2024-09-12T02:03:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010c550 con 0x7f6df0101c40 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010c850 con 0x7f6df0102600 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 0x7f6df0196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 0x7f6df0196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42198/0 (socket says 172.21.15.122:42198) 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 -- 172.21.15.122:0/3548000848 learned_addr learned my addr 172.21.15.122:0/3548000848 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6df0101c40 0x7f6df0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 -- 172.21.15.122:0/3548000848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df0102600 msgr2=0x7f6df019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df0102600 0x7f6df019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 -- 172.21.15.122:0/3548000848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6df0101c40 msgr2=0x7f6df0197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6df0101c40 0x7f6df0197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 -- 172.21.15.122:0/3548000848 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df019e900 con 0x7f6df0100a50 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6dee59c640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 0x7f6df0196cc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6de000d950 tx=0x7f6de000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6ddf7fe640 1 -- 172.21.15.122:0/3548000848 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0017070 con 0x7f6df0100a50 2024-09-12T02:03:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6ddf7fe640 1 -- 172.21.15.122:0/3548000848 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de0009c30 con 0x7f6df0100a50 2024-09-12T02:03:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df019ebf0 con 0x7f6df0100a50 2024-09-12T02:03:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6ddf7fe640 1 -- 172.21.15.122:0/3548000848 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de000f600 con 0x7f6df0100a50 2024-09-12T02:03:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.824+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df019f0c0 con 0x7f6df0100a50 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6ddf7fe640 1 -- 172.21.15.122:0/3548000848 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6de001a070 con 0x7f6df0100a50 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 Processor -- start 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 -- start start 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd5aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffcc32f0 con 0x55b3ffd5b700 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffc6b270 con 0x55b3ffd5c760 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.827+0000 7f6df7807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffcbf2b0 con 0x55b3ffd5c360 2024-09-12T02:03:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6deed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd5aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42202/0 (socket says 172.21.15.122:42202) 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 -- 172.21.15.122:0/750466185 learned_addr learned my addr 172.21.15.122:0/750466185 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6ddf7fe640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd80a2c30 0x7f6dd80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dee59c640 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6deed9d640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd80a2c30 0x7f6dd80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 -- 172.21.15.122:0/750466185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 msgr2=0x55b3ffd5aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd5aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 -- 172.21.15.122:0/750466185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 msgr2=0x55b3ffd5b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 -- 172.21.15.122:0/750466185 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ffc6a340 con 0x55b3ffd5b700 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6ddf7fe640 1 -- 172.21.15.122:0/3548000848 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f6de00ceb90 con 0x7f6df0100a50 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6bc90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6de402a930 tx=0x7f6de4032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df014b0a1fe0588 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6deed9d640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd80a2c30 0x7f6dd80a50f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6de80135e0 tx=0x7f6de8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6ddcff9640 1 -- 172.21.15.122:0/750466185 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de403f070 con 0x55b3ffd5b700 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6ddcff9640 1 -- 172.21.15.122:0/750466185 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de40318b0 con 0x55b3ffd5b700 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.828+0000 7f6ddcff9640 1 -- 172.21.15.122:0/750466185 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4035070 con 0x55b3ffd5b700 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 -- 172.21.15.122:0/750466185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 msgr2=0x55b3ffd6bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6bc90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6de402a930 tx=0x7f6de4032d60 comp rx=0 tx=0).stop 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 -- 172.21.15.122:0/750466185 shutdown_connections 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6bc90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 --2- 172.21.15.122:0/750466185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd5aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 -- 172.21.15.122:0/750466185 >> 172.21.15.122:0/750466185 conn(0x55b3ffd5d220 msgr2=0x55b3ffd5d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 -- 172.21.15.122:0/750466185 shutdown_connections 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.829+0000 7f6df7807740 1 -- 172.21.15.122:0/750466185 wait complete. 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 -- start start 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 Processor -- start 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd6e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd754c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffd6c700 con 0x55b3ffd5b700 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffc4c290 con 0x55b3ffd5c360 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffd6c880 con 0x55b3ffd5c760 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6df7807740 10 client.0 _ll_register_callbacks cb 0x55b3ffc7d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42206/0 (socket says 172.21.15.122:42206) 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 -- 172.21.15.122:0/3678541810 learned_addr learned my addr 172.21.15.122:0/3678541810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 -- 172.21.15.122:0/3678541810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 msgr2=0x55b3ffd754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd6e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6deed9d640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd754c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 -- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 msgr2=0x55b3ffd6e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd6e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 -- 172.21.15.122:0/3678541810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ff9d6210 con 0x55b3ffd5b700 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dee59c640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6dfb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6de00cc690 tx=0x7f6de009baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed72a452cd3dbeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.830+0000 7f6dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/3678541810 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0021050 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/3678541810 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f6de00ccb50 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de40318b0 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/3678541810 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6de00b43b0 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 10 client.17862 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b3ffaa2620 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/3678541810 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de40318b0 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/3678541810 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6de00cc100 con 0x55b3ffd5b700 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 10 client.17862 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 20 client.17862 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffd79c00 0x55b3ffd7c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 4 client.17862 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3ffcd0a20 con 0x55b3ffd79c00 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.831+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffd79c00 0x55b3ffd7c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.832+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffd79c00 0x55b3ffd7c010 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd70a0ece4eba19c server_cookie=cde52554c65a6b9c in_seq=0 out_seq=0 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.832+0000 7f6dd5ffb640 10 client.17862 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.832+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/3678541810 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3ffcd0a20 con 0x55b3ffd79c00 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.832+0000 7f6dd5ffb640 10 client.17862 handle_command_reply: tid=0 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 client.17862 shutdown 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 msgr2=0x55b3ffd6dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6dfb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6de00cc690 tx=0x7f6de009baf0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 shutdown_connections 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffd79c00 0x55b3ffd7c010 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd6e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6dd5ffb640 0 client.17862 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 --2- 172.21.15.122:0/3678541810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6dfb0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 >> 172.21.15.122:0/3678541810 conn(0x55b3ffd5d220 msgr2=0x55b3ffd5e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 shutdown_connections 2024-09-12T02:03:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 1 -- 172.21.15.122:0/3678541810 wait complete. 2024-09-12T02:03:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.833+0000 7f6df7807740 20 client.17862 trim_cache size 0 max 16384 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 Processor -- start 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 -- start start 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd75480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01a95c0 con 0x55b3ffd5b700 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01a6e90 con 0x55b3ffd5c360 2024-09-12T02:03:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.847+0000 7f6df7807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01a4cc0 con 0x55b3ffd5c760 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42216/0 (socket says 172.21.15.122:42216) 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 -- 172.21.15.122:0/2713803968 learned_addr learned my addr 172.21.15.122:0/2713803968 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6deed9d640 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd75480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 -- 172.21.15.122:0/2713803968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 msgr2=0x55b3ffd5a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 -- 172.21.15.122:0/2713803968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 msgr2=0x55b3ffd75480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd75480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 -- 172.21.15.122:0/2713803968 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ffd5d040 con 0x55b3ffd5b700 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dee59c640 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6e580 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6de0097dc0 tx=0x7f6de009b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee2945d1f6b7f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dd7fff640 1 -- 172.21.15.122:0/2713803968 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0021070 con 0x55b3ffd5b700 2024-09-12T02:03:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dd7fff640 1 -- 172.21.15.122:0/2713803968 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de009f7a0 con 0x55b3ffd5b700 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.848+0000 7f6dd7fff640 1 -- 172.21.15.122:0/2713803968 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de00ccb50 con 0x55b3ffd5b700 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 -- 172.21.15.122:0/2713803968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 msgr2=0x55b3ffd6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6e580 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6de0097dc0 tx=0x7f6de009b060 comp rx=0 tx=0).stop 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 -- 172.21.15.122:0/2713803968 shutdown_connections 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c760 0x55b3ffd5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c360 0x55b3ffd75480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 --2- 172.21.15.122:0/2713803968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd6e580 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 -- 172.21.15.122:0/2713803968 >> 172.21.15.122:0/2713803968 conn(0x55b3ffd5d220 msgr2=0x55b3ffd5e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 -- 172.21.15.122:0/2713803968 shutdown_connections 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 -- 172.21.15.122:0/2713803968 wait complete. 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 -- start start 2024-09-12T02:03:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 Processor -- start 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd72df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.849+0000 7f6df7807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01a5350 con 0x55b3ffd5b700 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01a67c0 con 0x55b3ffd5c760 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01a5a20 con 0x55b3ffd5c360 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 10 client.0 _ll_register_callbacks cb 0x7f6de0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd72df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd72df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42222/0 (socket says 172.21.15.122:42222) 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 -- 172.21.15.122:0/1249557411 learned_addr learned my addr 172.21.15.122:0/1249557411 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6deed9d640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 -- 172.21.15.122:0/1249557411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 msgr2=0x55b3ffd73330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd73330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 -- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 msgr2=0x55b3ffd742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd742f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 -- 172.21.15.122:0/1249557411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ffd74900 con 0x55b3ffd5b700 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6deed9d640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dee59c640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd72df0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6de00970d0 tx=0x7f6de009baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31585162752da3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/1249557411 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0021070 con 0x55b3ffd5b700 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/1249557411 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f6de00ccd50 con 0x55b3ffd5b700 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de009f7a0 con 0x55b3ffd5b700 2024-09-12T02:03:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/1249557411 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6de009b3f0 con 0x55b3ffd5b700 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 10 client.17868 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.850+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b3ffde4a30 con 0x55b3ffd5b700 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/1249557411 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de009f7a0 con 0x55b3ffd5b700 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/1249557411 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f6de00a0930 con 0x55b3ffd5b700 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6df7807740 10 client.17868 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6df7807740 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6df7807740 20 client.17868 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6df7807740 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffde5330 0x55b3ffde7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6df7807740 4 client.17868 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3ffccad70 con 0x55b3ffde5330 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffde5330 0x55b3ffde7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6dedd9b640 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffde5330 0x55b3ffde7750 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd750181239b8a0e server_cookie=dae29a7f3a93585e in_seq=0 out_seq=0 2024-09-12T02:03:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.851+0000 7f6dd5ffb640 10 client.17868 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6dd5ffb640 1 -- 172.21.15.122:0/1249557411 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3ffccad70 con 0x55b3ffde5330 2024-09-12T02:03:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6dd5ffb640 10 client.17868 handle_command_reply: tid=0 2024-09-12T02:03:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 client.17868 shutdown 2024-09-12T02:03:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 msgr2=0x55b3ffd72df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd72df0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6de00970d0 tx=0x7f6de009baf0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 shutdown_connections 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b3ffde5330 0x55b3ffde7750 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3ffd5c760 0x55b3ffd742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3ffd5c360 0x55b3ffd73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 --2- 172.21.15.122:0/1249557411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ffd5b700 0x55b3ffd72df0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6dd5ffb640 0 client.17868 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 >> 172.21.15.122:0/1249557411 conn(0x55b3ffd5d220 msgr2=0x55b3ffd5e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 shutdown_connections 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 1 -- 172.21.15.122:0/1249557411 wait complete. 2024-09-12T02:03:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.900+0000 7f6df7807740 20 client.17868 trim_cache size 0 max 16384 2024-09-12T02:03:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:03:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:03:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:03:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd80a2c30 msgr2=0x7f6dd80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd80a2c30 0x7f6dd80a50f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6de80135e0 tx=0x7f6de8004a40 comp rx=0 tx=0).stop 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 msgr2=0x7f6df0196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 0x7f6df0196cc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6de000d950 tx=0x7f6de000de10 comp rx=0 tx=0).stop 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 shutdown_connections 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6dd80a2c30 0x7f6dd80a50f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df0102600 0x7f6df019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6df0101c40 0x7f6df0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 --2- 172.21.15.122:0/3548000848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6df0100a50 0x7f6df0196cc0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 >> 172.21.15.122:0/3548000848 conn(0x7f6df00fc1b0 msgr2=0x7f6df00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 shutdown_connections 2024-09-12T02:03:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:20.901+0000 7f6df4a65640 1 -- 172.21.15.122:0/3548000848 wait complete. 2024-09-12T02:03:21.059 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 13.78 MB/sec execute 68 sec latency 12974.515 ms 2024-09-12T02:03:21.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:21 smithi122 ceph-mon[30925]: pgmap v844: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 272 op/s 2024-09-12T02:03:21.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:21 smithi151 ceph-mon[35229]: pgmap v844: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 272 op/s 2024-09-12T02:03:21.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:21 smithi183 ceph-mon[35127]: pgmap v844: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 272 op/s 2024-09-12T02:03:22.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 13.58 MB/sec execute 69 sec latency 13974.600 ms 2024-09-12T02:03:23.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 13.38 MB/sec execute 70 sec latency 14974.636 ms 2024-09-12T02:03:23.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:22 smithi151 ceph-mon[35229]: pgmap v845: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-12T02:03:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:22 smithi183 ceph-mon[35127]: pgmap v845: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-12T02:03:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:22 smithi122 ceph-mon[30925]: pgmap v845: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-12T02:03:24.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 13.20 MB/sec execute 71 sec latency 15974.752 ms 2024-09-12T02:03:25.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 13.01 MB/sec execute 72 sec latency 16974.867 ms 2024-09-12T02:03:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:24 smithi151 ceph-mon[35229]: pgmap v846: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 762 KiB/s wr, 70 op/s 2024-09-12T02:03:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:24 smithi183 ceph-mon[35127]: pgmap v846: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 762 KiB/s wr, 70 op/s 2024-09-12T02:03:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:24 smithi122 ceph-mon[30925]: pgmap v846: 97 pgs: 97 active+clean; 478 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 762 KiB/s wr, 70 op/s 2024-09-12T02:03:26.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 12.83 MB/sec execute 73 sec latency 17974.915 ms 2024-09-12T02:03:27.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 12.66 MB/sec execute 74 sec latency 18974.991 ms 2024-09-12T02:03:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:26 smithi151 ceph-mon[35229]: pgmap v847: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 247 KiB/s wr, 53 op/s 2024-09-12T02:03:27.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:26 smithi183 ceph-mon[35127]: pgmap v847: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 247 KiB/s wr, 53 op/s 2024-09-12T02:03:27.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:26 smithi122 ceph-mon[30925]: pgmap v847: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 247 KiB/s wr, 53 op/s 2024-09-12T02:03:28.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 12.49 MB/sec execute 75 sec latency 19975.071 ms 2024-09-12T02:03:29.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 12.33 MB/sec execute 76 sec latency 20975.170 ms 2024-09-12T02:03:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:28 smithi151 ceph-mon[35229]: pgmap v848: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 179 KiB/s wr, 42 op/s 2024-09-12T02:03:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:28 smithi183 ceph-mon[35127]: pgmap v848: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 179 KiB/s wr, 42 op/s 2024-09-12T02:03:29.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:28 smithi122 ceph-mon[30925]: pgmap v848: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 179 KiB/s wr, 42 op/s 2024-09-12T02:03:29.789 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:03:30.048 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:30.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 12.17 MB/sec execute 77 sec latency 21975.256 ms 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 -- 172.21.15.122:0/1839039874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584106a80 msgr2=0x7f0584106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 --2- 172.21.15.122:0/1839039874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584106a80 0x7f0584106e60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0578009930 tx=0x7f057802a420 comp rx=0 tx=0).stop 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 -- 172.21.15.122:0/1839039874 shutdown_connections 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 --2- 172.21.15.122:0/1839039874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584101420 0x7f058410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 --2- 172.21.15.122:0/1839039874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584100a80 0x7f0584100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 --2- 172.21.15.122:0/1839039874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584106a80 0x7f0584106e60 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 -- 172.21.15.122:0/1839039874 >> 172.21.15.122:0/1839039874 conn(0x7f05840fc7a0 msgr2=0x7f05840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 -- 172.21.15.122:0/1839039874 shutdown_connections 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.533+0000 7f058b164640 1 -- 172.21.15.122:0/1839039874 wait complete. 2024-09-12T02:03:30.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 Processor -- start 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 -- start start 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584100a80 0x7f05840fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 0x7f05840fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584106a80 0x7f05840fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0584110ea0 con 0x7f0584101420 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0584110d20 con 0x7f0584100a80 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f058b164640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0584111020 con 0x7f0584106a80 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 0x7f05840fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 0x7f05840fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34346/0 (socket says 172.21.15.122:34346) 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 -- 172.21.15.122:0/2354220385 learned_addr learned my addr 172.21.15.122:0/2354220385 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:30.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0588f00640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584100a80 0x7f05840fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0589701640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584106a80 0x7f05840fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 -- 172.21.15.122:0/2354220385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584106a80 msgr2=0x7f05840fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584106a80 0x7f05840fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 -- 172.21.15.122:0/2354220385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584100a80 msgr2=0x7f05840fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584100a80 0x7f05840fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0583fff640 1 -- 172.21.15.122:0/2354220385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05841a6980 con 0x7f0584101420 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.534+0000 7f0589701640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584106a80 0x7f05840fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.535+0000 7f0583fff640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 0x7f05840fd2d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f057000b4a0 tx=0x7f057000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.535+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570014070 con 0x7f0584101420 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.535+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0570010af0 con 0x7f0584101420 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.535+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841a6c70 con 0x7f0584101420 2024-09-12T02:03:30.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.535+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570017d10 con 0x7f0584101420 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.535+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841a7160 con 0x7f0584101420 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.536+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0584100ee0 con 0x7f0584101420 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.537+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0570017070 con 0x7f0584101420 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.538+0000 7f0581ffb640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05740a2b80 0x7f05740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.538+0000 7f0588f00640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05740a2b80 0x7f05740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.538+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f05700ce410 con 0x7f0584101420 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.538+0000 7f0588f00640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05740a2b80 0x7f05740a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0578009900 tx=0x7f05780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:30.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.542+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f057009b370 con 0x7f0584101420 2024-09-12T02:03:30.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.743+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0584109a70 con 0x7f0584101420 2024-09-12T02:03:30.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.744+0000 7f0581ffb640 1 -- 172.21.15.122:0/2354220385 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7f057009ac70 con 0x7f0584101420 2024-09-12T02:03:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:03:30.745 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:03:30.746 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:03:30.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05740a2b80 msgr2=0x7f05740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05740a2b80 0x7f05740a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0578009900 tx=0x7f05780058d0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 msgr2=0x7f05840fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:30.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 0x7f05840fd2d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f057000b4a0 tx=0x7f057000b960 comp rx=0 tx=0).stop 2024-09-12T02:03:30.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 shutdown_connections 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f05740a2b80 0x7f05740a5040 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0584106a80 0x7f05840fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0584101420 0x7f05840fd2d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 --2- 172.21.15.122:0/2354220385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0584100a80 0x7f05840fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 >> 172.21.15.122:0/2354220385 conn(0x7f05840fc7a0 msgr2=0x7f058406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 shutdown_connections 2024-09-12T02:03:30.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:30.746+0000 7f058b164640 1 -- 172.21.15.122:0/2354220385 wait complete. 2024-09-12T02:03:31.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 12.01 MB/sec execute 78 sec latency 22975.328 ms 2024-09-12T02:03:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:30 smithi151 ceph-mon[35229]: pgmap v849: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 97 KiB/s wr, 30 op/s 2024-09-12T02:03:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:30 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2354220385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:31.188 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:30 smithi122 ceph-mon[30925]: pgmap v849: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 97 KiB/s wr, 30 op/s 2024-09-12T02:03:31.188 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:30 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2354220385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:30 smithi183 ceph-mon[35127]: pgmap v849: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 97 KiB/s wr, 30 op/s 2024-09-12T02:03:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:30 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2354220385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:03:31.492 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 32aa008e --release --await' 2024-09-12T02:03:31.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 32aa008e --release --await 2024-09-12T02:03:31.736 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:03:32.060 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70599 11.86 MB/sec execute 79 sec latency 23975.394 ms 2024-09-12T02:03:32.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 -- 172.21.15.122:0/2871377188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8101420 msgr2=0x7f15a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/2871377188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8101420 0x7f15a810d950 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009cd0 tx=0x7f15a002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 -- 172.21.15.122:0/2871377188 shutdown_connections 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/2871377188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8101420 0x7f15a810d950 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/2871377188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8100a80 0x7f15a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/2871377188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8106a80 0x7f15a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 -- 172.21.15.122:0/2871377188 >> 172.21.15.122:0/2871377188 conn(0x7f15a80fc7a0 msgr2=0x7f15a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 -- 172.21.15.122:0/2871377188 shutdown_connections 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.216+0000 7f15ad6a0640 1 -- 172.21.15.122:0/2871377188 wait complete. 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 Processor -- start 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 -- start start 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 0x7f15a8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8101420 0x7f15a8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8106a80 0x7f15a819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15a806b070 con 0x7f15a8100a80 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15a806aef0 con 0x7f15a8101420 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15ad6a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a806b1f0 con 0x7f15a8106a80 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 0x7f15a8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 0x7f15a8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34368/0 (socket says 172.21.15.122:34368) 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 -- 172.21.15.122:0/509739428 learned_addr learned my addr 172.21.15.122:0/509739428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a67fc640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8101420 0x7f15a8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a77fe640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8106a80 0x7f15a819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 -- 172.21.15.122:0/509739428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8106a80 msgr2=0x7f15a819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8106a80 0x7f15a819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 -- 172.21.15.122:0/509739428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8101420 msgr2=0x7f15a8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8101420 0x7f15a8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a6ffd640 1 -- 172.21.15.122:0/509739428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a819d3b0 con 0x7f15a8100a80 2024-09-12T02:03:32.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a67fc640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8101420 0x7f15a8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.217+0000 7f15a77fe640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8106a80 0x7f15a819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.218+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 0x7f15a8198400 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f159800c920 tx=0x7f159800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.218+0000 7f158ffff640 1 -- 172.21.15.122:0/509739428 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598007c70 con 0x7f15a8100a80 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.218+0000 7f158ffff640 1 -- 172.21.15.122:0/509739428 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1598007e10 con 0x7f15a8100a80 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.218+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a819d6a0 con 0x7f15a8100a80 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.218+0000 7f158ffff640 1 -- 172.21.15.122:0/509739428 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598018890 con 0x7f15a8100a80 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.218+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81a4f80 con 0x7f15a8100a80 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.220+0000 7f158ffff640 1 -- 172.21.15.122:0/509739428 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1598018a70 con 0x7f15a8100a80 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.220+0000 7f15afa79740 1 Processor -- start 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.220+0000 7f15afa79740 1 -- start start 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15afa79740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15afa79740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15afa79740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b67f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15afa79740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5577b674e620 con 0x5577b67f4240 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15afa79740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577b66f5e50 con 0x5577b67e9060 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15afa79740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5577b674a5e0 con 0x5577b67e9980 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b67f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b67f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34378/0 (socket says 172.21.15.122:34378) 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 -- 172.21.15.122:0/877214190 learned_addr learned my addr 172.21.15.122:0/877214190 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f158ffff640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15900a2b80 0x7f15900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a77fe640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a77fe640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15900a2b80 0x7f15900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 -- 172.21.15.122:0/877214190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 msgr2=0x5577b67f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f158ffff640 1 -- 172.21.15.122:0/509739428 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f15980cf280 con 0x7f15a8100a80 2024-09-12T02:03:32.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 -- 172.21.15.122:0/877214190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 msgr2=0x5577b67e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 -- 172.21.15.122:0/877214190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b66e6d40 con 0x5577b67f4240 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a77fe640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a67fc640 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b67f6630 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f159c007800 tx=0x7f159c007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cebfc927eab1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f158d7fa640 1 -- 172.21.15.122:0/877214190 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c00d950 con 0x5577b67f4240 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.221+0000 7f15a77fe640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15900a2b80 0x7f15900a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009a60 tx=0x7f15a00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f158d7fa640 1 -- 172.21.15.122:0/877214190 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c018070 con 0x5577b67f4240 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f158d7fa640 1 -- 172.21.15.122:0/877214190 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0123e0 con 0x5577b67f4240 2024-09-12T02:03:32.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 -- 172.21.15.122:0/877214190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 msgr2=0x5577b67f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b67f6630 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f159c007800 tx=0x7f159c007cc0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 -- 172.21.15.122:0/877214190 shutdown_connections 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b67f6630 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 --2- 172.21.15.122:0/877214190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 -- 172.21.15.122:0/877214190 >> 172.21.15.122:0/877214190 conn(0x5577b67e55f0 msgr2=0x5577b67e59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 -- 172.21.15.122:0/877214190 shutdown_connections 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.222+0000 7f15afa79740 1 -- 172.21.15.122:0/877214190 wait complete. 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 -- start start 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 Processor -- start 2024-09-12T02:03:32.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 0x5577b67f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67f4240 0x5577b67f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5577b67f6e10 con 0x5577b67e9980 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577b66d6f10 con 0x5577b67e9060 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5577b67f6f90 con 0x5577b67f4240 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15afa79740 10 client.0 _ll_register_callbacks cb 0x5577b66fadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 0x5577b67f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 0x5577b67f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34382/0 (socket says 172.21.15.122:34382) 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 -- 172.21.15.122:0/1923464729 learned_addr learned my addr 172.21.15.122:0/1923464729 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 -- 172.21.15.122:0/1923464729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67f4240 msgr2=0x5577b67f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67f4240 0x5577b67f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 -- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 msgr2=0x5577b67f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a67fc640 1 -- 172.21.15.122:0/1923464729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b645d580 con 0x5577b67e9980 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.223+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15a67fc640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 0x5577b67f15b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f159c004700 tx=0x7f159c013870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee52af2bb38dabab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f158aff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f158aff5640 1 -- 172.21.15.122:0/1923464729 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c00de20 con 0x5577b67e9980 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f158aff5640 1 -- 172.21.15.122:0/1923464729 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f159c01c3f0 con 0x5577b67e9980 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159c018070 con 0x5577b67e9980 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f158aff5640 1 -- 172.21.15.122:0/1923464729 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f159c01c5f0 con 0x5577b67e9980 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 10 client.17886 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:32.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5577b6527960 con 0x5577b67e9980 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f158aff5640 1 -- 172.21.15.122:0/1923464729 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c018070 con 0x5577b67e9980 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f158aff5640 1 -- 172.21.15.122:0/1923464729 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f159c02d050 con 0x5577b67e9980 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 10 client.17886 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 10 client.17886 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 20 client.17886 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b6803bf0 0x5577b6805fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 4 client.17886 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.224+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b675bb90 con 0x5577b6803bf0 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.225+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b6803bf0 0x5577b6805fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.225+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b6803bf0 0x5577b6805fe0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad9b4a5525efbad6 server_cookie=dd81156695475879 in_seq=0 out_seq=0 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.225+0000 7f158aff5640 10 client.17886 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f158aff5640 1 -- 172.21.15.122:0/1923464729 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577b675bb90 con 0x5577b6803bf0 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f158aff5640 10 client.17886 handle_command_reply: tid=0 2024-09-12T02:03:32.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 client.17886 shutdown 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 msgr2=0x5577b67f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 0x5577b67f15b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f159c004700 tx=0x7f159c013870 comp rx=0 tx=0).stop 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 shutdown_connections 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b6803bf0 0x5577b6805fe0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67f4240 0x5577b67f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9980 0x5577b67f15b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 --2- 172.21.15.122:0/1923464729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f158aff5640 0 client.17886 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 >> 172.21.15.122:0/1923464729 conn(0x5577b67e55f0 msgr2=0x5577b67eb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 shutdown_connections 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 1 -- 172.21.15.122:0/1923464729 wait complete. 2024-09-12T02:03:32.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.226+0000 7f15afa79740 20 client.17886 trim_cache size 0 max 16384 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 Processor -- start 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 -- start start 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 0x5577b680b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b680bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67f4240 0x5577b67ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81afd50 con 0x5577b67e9060 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81ad620 con 0x5577b67f4240 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15afa79740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81ab450 con 0x5577b67e9980 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 0x5577b680b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 0x5577b680b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34396/0 (socket says 172.21.15.122:34396) 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 -- 172.21.15.122:0/2094158910 learned_addr learned my addr 172.21.15.122:0/2094158910 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 -- 172.21.15.122:0/2094158910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 msgr2=0x5577b680bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b680bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 -- 172.21.15.122:0/2094158910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67f4240 msgr2=0x5577b67ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67f4240 0x5577b67ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.243+0000 7f15a6ffd640 1 -- 172.21.15.122:0/2094158910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b67e5420 con 0x5577b67e9060 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 0x5577b680b690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1598007590 tx=0x7f15980076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febb23c7656b4598 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f158cff9640 1 -- 172.21.15.122:0/2094158910 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980a4710 con 0x5577b67e9060 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f158cff9640 1 -- 172.21.15.122:0/2094158910 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1598099da0 con 0x5577b67e9060 2024-09-12T02:03:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f158cff9640 1 -- 172.21.15.122:0/2094158910 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980a4280 con 0x5577b67e9060 2024-09-12T02:03:32.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 -- 172.21.15.122:0/2094158910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 msgr2=0x5577b680b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 0x5577b680b690 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1598007590 tx=0x7f15980076c0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 -- 172.21.15.122:0/2094158910 shutdown_connections 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67f4240 0x5577b67ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b680bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 --2- 172.21.15.122:0/2094158910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67e9060 0x5577b680b690 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 -- 172.21.15.122:0/2094158910 >> 172.21.15.122:0/2094158910 conn(0x5577b67e55f0 msgr2=0x5577b67f28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 -- 172.21.15.122:0/2094158910 shutdown_connections 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.244+0000 7f15afa79740 1 -- 172.21.15.122:0/2094158910 wait complete. 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 -- start start 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 Processor -- start 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b686d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81abae0 con 0x5577b67f4240 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81acf50 con 0x5577b67e9060 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81ac1b0 con 0x5577b67e9980 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15afa79740 10 client.0 _ll_register_callbacks cb 0x7f159c013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b686d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b686d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34402/0 (socket says 172.21.15.122:34402) 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 -- 172.21.15.122:0/1254280255 learned_addr learned my addr 172.21.15.122:0/1254280255 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a67fc640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 -- 172.21.15.122:0/1254280255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 msgr2=0x5577b67f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 -- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 msgr2=0x5577b67f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a77fe640 1 -- 172.21.15.122:0/1254280255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b67f9060 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.245+0000 7f15a67fc640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15a77fe640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b686d660 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f15a0003bf0 tx=0x7f15a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21805d525edc955d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f158aff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f158aff5640 1 -- 172.21.15.122:0/1254280255 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a004d030 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1598099da0 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f158aff5640 1 -- 172.21.15.122:0/1254280255 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f15a00044c0 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f158aff5640 1 -- 172.21.15.122:0/1254280255 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f15a00046c0 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 10 client.17892 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5577b686ecb0 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f158aff5640 1 -- 172.21.15.122:0/1254280255 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1598099da0 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f158aff5640 1 -- 172.21.15.122:0/1254280255 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f15a0030070 con 0x5577b67f4240 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 10 client.17892 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:32.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 10 client.17892 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 20 client.17892 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b686f480 0x5577b68718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 4 client.17892 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "32aa008e", "release": true, "await": true}] 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.246+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "32aa008e", "release": true, "await": true}) -- 0x5577b67ecc90 con 0x5577b686f480 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.247+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b686f480 0x5577b68718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.247+0000 7f15a6ffd640 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b686f480 0x5577b68718a0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233e45248d625180 server_cookie=346262467cbb34e9 in_seq=0 out_seq=0 2024-09-12T02:03:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.247+0000 7f158aff5640 10 client.17892 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:32.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.307+0000 7f158aff5640 1 -- 172.21.15.122:0/1254280255 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5577b67ecc90 con 0x5577b686f480 2024-09-12T02:03:32.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.307+0000 7f158aff5640 10 client.17892 handle_command_reply: tid=0 2024-09-12T02:03:32.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.307+0000 7f15afa79740 1 client.17892 shutdown 2024-09-12T02:03:32.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.307+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 msgr2=0x5577b686d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.307+0000 7f15afa79740 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b686d660 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f15a0003bf0 tx=0x7f15a0003c20 comp rx=0 tx=0).stop 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 shutdown_connections 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5577b686f480 0x5577b68718a0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577b67f4240 0x5577b686d660 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f158aff5640 0 client.17892 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577b67e9980 0x5577b67f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 --2- 172.21.15.122:0/1254280255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577b67e9060 0x5577b67f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 >> 172.21.15.122:0/1254280255 conn(0x5577b67e55f0 msgr2=0x5577b67eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 shutdown_connections 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 1 -- 172.21.15.122:0/1254280255 wait complete. 2024-09-12T02:03:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15afa79740 20 client.17892 trim_cache size 0 max 16384 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 64, 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "32aa008e": { 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 64, 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:03:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 144.2, 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:03:32.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:32.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:32.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:32.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:32.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:03:32.312 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:03:32.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15900a2b80 msgr2=0x7f15900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15900a2b80 0x7f15900a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009a60 tx=0x7f15a00095c0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 msgr2=0x7f15a8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.308+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 0x7f15a8198400 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f159800c920 tx=0x7f159800cde0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 shutdown_connections 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f15900a2b80 0x7f15900a5040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a8106a80 0x7f15a819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15a8101420 0x7f15a8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 --2- 172.21.15.122:0/509739428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a8100a80 0x7f15a8198400 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 >> 172.21.15.122:0/509739428 conn(0x7f15a80fc7a0 msgr2=0x7f15a80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 shutdown_connections 2024-09-12T02:03:32.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:03:32.309+0000 7f15ad6a0640 1 -- 172.21.15.122:0/509739428 wait complete. 2024-09-12T02:03:32.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:32 smithi122 ceph-mon[30925]: pgmap v850: 97 pgs: 97 active+clean; 475 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 106 KiB/s wr, 37 op/s 2024-09-12T02:03:32.999 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '32aa008e', total seconds elapsed: 27.5 2024-09-12T02:03:33.061 INFO:tasks.workunit.client.0.smithi122.stdout: 10 71241 12.14 MB/sec execute 80 sec latency 24249.769 ms 2024-09-12T02:03:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:32 smithi151 ceph-mon[35229]: pgmap v850: 97 pgs: 97 active+clean; 475 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 106 KiB/s wr, 37 op/s 2024-09-12T02:03:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:32 smithi183 ceph-mon[35127]: pgmap v850: 97 pgs: 97 active+clean; 475 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 106 KiB/s wr, 37 op/s 2024-09-12T02:03:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 -- 172.21.15.122:0/997249159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca341025f0 msgr2=0x7fca34108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 --2- 172.21.15.122:0/997249159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca341025f0 0x7fca34108e80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ca0 tx=0x7fca2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 -- 172.21.15.122:0/997249159 shutdown_connections 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 --2- 172.21.15.122:0/997249159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca341025f0 0x7fca34108e80 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 --2- 172.21.15.122:0/997249159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca34101c30 0x7fca341020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 --2- 172.21.15.122:0/997249159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 -- 172.21.15.122:0/997249159 >> 172.21.15.122:0/997249159 conn(0x7fca340fc1e0 msgr2=0x7fca340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 -- 172.21.15.122:0/997249159 shutdown_connections 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.993+0000 7fca39c65640 1 -- 172.21.15.122:0/997249159 wait complete. 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 Processor -- start 2024-09-12T02:03:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 -- start start 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 0x7fca341982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca341025f0 0x7fca3419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410d810 con 0x7fca34101c30 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410d690 con 0x7fca34100a40 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca39c65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410d990 con 0x7fca341025f0 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca32ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 0x7fca341982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca337fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca337fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:50782/0 (socket says 172.21.15.122:50782) 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca32ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 0x7fca341982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34418/0 (socket says 172.21.15.122:34418) 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca337fe640 1 -- 172.21.15.122:0/2828009008 learned_addr learned my addr 172.21.15.122:0/2828009008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.994+0000 7fca33fff640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca341025f0 0x7fca3419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca337fe640 1 -- 172.21.15.122:0/2828009008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca341025f0 msgr2=0x7fca3419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca337fe640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca341025f0 0x7fca3419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca337fe640 1 -- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 msgr2=0x7fca341982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca337fe640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 0x7fca341982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca337fe640 1 -- 172.21.15.122:0/2828009008 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3419f9c0 con 0x7fca34100a40 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca32ffd640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 0x7fca341982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca337fe640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34197db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9c0 tx=0x7fca2400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca30ff9640 1 -- 172.21.15.122:0/2828009008 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400cd80 con 0x7fca34100a40 2024-09-12T02:03:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca30ff9640 1 -- 172.21.15.122:0/2828009008 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca240035f0 con 0x7fca34100a40 2024-09-12T02:03:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3419fcb0 con 0x7fca34100a40 2024-09-12T02:03:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca30ff9640 1 -- 172.21.15.122:0/2828009008 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24010690 con 0x7fca34100a40 2024-09-12T02:03:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.995+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341a01b0 con 0x7fca34100a40 2024-09-12T02:03:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca30ff9640 1 -- 172.21.15.122:0/2828009008 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fca240037a0 con 0x7fca34100a40 2024-09-12T02:03:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 Processor -- start 2024-09-12T02:03:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 -- start start 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbe2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbe7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56184cb462f0 con 0x56184cbe29a0 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56184caee270 con 0x56184cbe8370 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca3ca04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56184cb422b0 con 0x56184cbe32e0 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca33fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbe2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca337fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbe7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca33fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbe2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34424/0 (socket says 172.21.15.122:34424) 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca30ff9640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca1c0a2c30 0x7fca1c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca33fff640 1 -- 172.21.15.122:0/2728655387 learned_addr learned my addr 172.21.15.122:0/2728655387 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.998+0000 7fca32ffd640 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca32ffd640 1 -- 172.21.15.122:0/2728655387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 msgr2=0x56184cbe7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca33fff640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca1c0a2c30 0x7fca1c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca32ffd640 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbe7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca32ffd640 1 -- 172.21.15.122:0/2728655387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 msgr2=0x56184cbe2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca32ffd640 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbe2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca32ffd640 1 -- 172.21.15.122:0/2728655387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56184caed340 con 0x56184cbe8370 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca30ff9640 1 -- 172.21.15.122:0/2828009008 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fca240cf4a0 con 0x7fca34100a40 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca32ffd640 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbea830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca280103d0 tx=0x7fca28010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1e864d758cd85f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca33fff640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca1c0a2c30 0x7fca1c0a50f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ca0 tx=0x7fca2c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca19ffb640 1 -- 172.21.15.122:0/2728655387 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28011700 con 0x56184cbe8370 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca19ffb640 1 -- 172.21.15.122:0/2728655387 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca28011d80 con 0x56184cbe8370 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:33.999+0000 7fca19ffb640 1 -- 172.21.15.122:0/2728655387 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2801a380 con 0x56184cbe8370 2024-09-12T02:03:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 -- 172.21.15.122:0/2728655387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 msgr2=0x56184cbea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbea830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca280103d0 tx=0x7fca28010890 comp rx=0 tx=0).stop 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 -- 172.21.15.122:0/2728655387 shutdown_connections 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbea830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbe7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2728655387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbe2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 -- 172.21.15.122:0/2728655387 >> 172.21.15.122:0/2728655387 conn(0x56184cbde010 msgr2=0x56184cbde410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 -- 172.21.15.122:0/2728655387 shutdown_connections 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 -- 172.21.15.122:0/2728655387 wait complete. 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 -- start start 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.000+0000 7fca3ca04740 1 Processor -- start 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe29a0 0x56184cbecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbf4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56184cbeb340 con 0x56184cbe8370 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56184cacf290 con 0x56184cbe29a0 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56184cbeb4c0 con 0x56184cbe32e0 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca3ca04740 10 client.0 _ll_register_callbacks cb 0x56184cb00140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbf4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbf4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34430/0 (socket says 172.21.15.122:34430) 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca337fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe29a0 0x56184cbecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 -- 172.21.15.122:0/2882218205 learned_addr learned my addr 172.21.15.122:0/2882218205 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca32ffd640 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 -- 172.21.15.122:0/2882218205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 msgr2=0x56184cbed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 -- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe29a0 msgr2=0x56184cbecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe29a0 0x56184cbecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 -- 172.21.15.122:0/2882218205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56184c859210 con 0x56184cbe8370 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca33fff640 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbf4050 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fca2c031ae0 tx=0x7fca2c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d095aba6e2371e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.001+0000 7fca12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca12ffd640 1 -- 172.21.15.122:0/2882218205 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c04e070 con 0x56184cbe8370 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca12ffd640 1 -- 172.21.15.122:0/2882218205 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fca2c0307c0 con 0x56184cbe8370 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca28011d80 con 0x56184cbe8370 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca12ffd640 1 -- 172.21.15.122:0/2882218205 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fca2c05a050 con 0x56184cbe8370 2024-09-12T02:03:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 10 client.17904 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56184c925620 con 0x56184cbe8370 2024-09-12T02:03:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca12ffd640 1 -- 172.21.15.122:0/2882218205 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca28011d80 con 0x56184cbe8370 2024-09-12T02:03:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca12ffd640 1 -- 172.21.15.122:0/2882218205 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fca2c005520 con 0x56184cbe8370 2024-09-12T02:03:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 10 client.17904 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 20 client.17904 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cbf8740 0x56184cbfab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 4 client.17904 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56184cb53a20 con 0x56184cbf8740 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.002+0000 7fca337fe640 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cbf8740 0x56184cbfab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.003+0000 7fca12ffd640 10 client.17904 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.003+0000 7fca337fe640 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cbf8740 0x56184cbfab70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66be0704f0874814 server_cookie=3a1ecdb5902c15e1 in_seq=0 out_seq=0 2024-09-12T02:03:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca12ffd640 1 -- 172.21.15.122:0/2882218205 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56184cb53a20 con 0x56184cbf8740 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca12ffd640 10 client.17904 handle_command_reply: tid=0 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 client.17904 shutdown 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 msgr2=0x56184cbf4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbf4050 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fca2c031ae0 tx=0x7fca2c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 shutdown_connections 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cbf8740 0x56184cbfab70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbf4050 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 --2- 172.21.15.122:0/2882218205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe29a0 0x56184cbecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca12ffd640 0 client.17904 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 >> 172.21.15.122:0/2882218205 conn(0x56184cbde010 msgr2=0x56184cbdf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 shutdown_connections 2024-09-12T02:03:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 1 -- 172.21.15.122:0/2882218205 wait complete. 2024-09-12T02:03:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.004+0000 7fca3ca04740 20 client.17904 trim_cache size 0 max 16384 2024-09-12T02:03:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.019+0000 7fca3ca04740 1 Processor -- start 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.019+0000 7fca3ca04740 1 -- start start 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe29a0 0x56184cbdeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe32e0 0x56184cbf4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbdf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca3ca04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca341ab890 con 0x56184cbe8370 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca3ca04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a9160 con 0x56184cbe32e0 2024-09-12T02:03:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca3ca04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a6f90 con 0x56184cbe29a0 2024-09-12T02:03:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbdf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbdf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34434/0 (socket says 172.21.15.122:34434) 2024-09-12T02:03:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 -- 172.21.15.122:0/1866940916 learned_addr learned my addr 172.21.15.122:0/1866940916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca337fe640 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe29a0 0x56184cbdeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca32ffd640 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe32e0 0x56184cbf4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 -- 172.21.15.122:0/1866940916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe29a0 msgr2=0x56184cbdeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe29a0 0x56184cbdeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 -- 172.21.15.122:0/1866940916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe32e0 msgr2=0x56184cbf4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe32e0 0x56184cbf4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 -- 172.21.15.122:0/1866940916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56184cbdde30 con 0x56184cbe8370 2024-09-12T02:03:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca33fff640 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbdf400 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005920 tx=0x7fca2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6293a89a0e389d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca197fa640 1 -- 172.21.15.122:0/1866940916 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c04e040 con 0x56184cbe8370 2024-09-12T02:03:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca197fa640 1 -- 172.21.15.122:0/1866940916 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2c036040 con 0x56184cbe8370 2024-09-12T02:03:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.020+0000 7fca197fa640 1 -- 172.21.15.122:0/1866940916 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c005300 con 0x56184cbe8370 2024-09-12T02:03:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 -- 172.21.15.122:0/1866940916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 msgr2=0x56184cbdf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbdf400 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005920 tx=0x7fca2c004a40 comp rx=0 tx=0).stop 2024-09-12T02:03:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 -- 172.21.15.122:0/1866940916 shutdown_connections 2024-09-12T02:03:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe8370 0x56184cbdf400 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe32e0 0x56184cbf4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 --2- 172.21.15.122:0/1866940916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe29a0 0x56184cbdeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 -- 172.21.15.122:0/1866940916 >> 172.21.15.122:0/1866940916 conn(0x56184cbde010 msgr2=0x56184cbecad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 -- 172.21.15.122:0/1866940916 shutdown_connections 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 -- 172.21.15.122:0/1866940916 wait complete. 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.021+0000 7fca3ca04740 1 -- start start 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 Processor -- start 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbf1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbf1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbf2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a7620 con 0x56184cbe29a0 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a8a90 con 0x56184cbe8370 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca341a7cf0 con 0x56184cbe32e0 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca3ca04740 10 client.0 _ll_register_callbacks cb 0x7fca2c004f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbf1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbf1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34444/0 (socket says 172.21.15.122:34444) 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 -- 172.21.15.122:0/3859494653 learned_addr learned my addr 172.21.15.122:0/3859494653 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca32ffd640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbf1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca33fff640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbf2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 -- 172.21.15.122:0/3859494653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 msgr2=0x56184cbf1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbf1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 -- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 msgr2=0x56184cbf2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbf2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 -- 172.21.15.122:0/3859494653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56184cbf32f0 con 0x56184cbe29a0 2024-09-12T02:03:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.022+0000 7fca337fe640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbf1790 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fca2400c6a0 tx=0x7fca2409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca83f1da8eea18e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca12ffd640 1 -- 172.21.15.122:0/3859494653 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2409bd00 con 0x56184cbe29a0 2024-09-12T02:03:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca12ffd640 1 -- 172.21.15.122:0/3859494653 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fca240982e0 con 0x56184cbe29a0 2024-09-12T02:03:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c036040 con 0x56184cbe29a0 2024-09-12T02:03:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca12ffd640 1 -- 172.21.15.122:0/3859494653 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fca240a4970 con 0x56184cbe29a0 2024-09-12T02:03:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 10 client.17910 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56184cc635e0 con 0x56184cbe29a0 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca12ffd640 1 -- 172.21.15.122:0/3859494653 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2c036040 con 0x56184cbe29a0 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca12ffd640 1 -- 172.21.15.122:0/3859494653 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fca240a4570 con 0x56184cbe29a0 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 10 client.17910 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 10 client.17910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 20 client.17910 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cc63f40 0x56184cc66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 4 client.17910 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56184cb4dd70 con 0x56184cc63f40 2024-09-12T02:03:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.023+0000 7fca32ffd640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cc63f40 0x56184cc66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.024+0000 7fca32ffd640 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cc63f40 0x56184cc66360 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61b809736c37a48 server_cookie=5fb2fe7ec1f9f679 in_seq=0 out_seq=0 2024-09-12T02:03:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.024+0000 7fca12ffd640 10 client.17910 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:34.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 71834 12.47 MB/sec execute 81 sec latency 51.931 ms 2024-09-12T02:03:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca12ffd640 1 -- 172.21.15.122:0/3859494653 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56184cb4dd70 con 0x56184cc63f40 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca12ffd640 10 client.17910 handle_command_reply: tid=0 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 client.17910 shutdown 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 msgr2=0x56184cbf1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbf1790 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fca2400c6a0 tx=0x7fca2409f040 comp rx=0 tx=0).stop 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 shutdown_connections 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56184cc63f40 0x56184cc66360 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56184cbe8370 0x56184cbf2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca12ffd640 0 client.17910 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56184cbe32e0 0x56184cbf1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 --2- 172.21.15.122:0/3859494653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56184cbe29a0 0x56184cbf1790 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 >> 172.21.15.122:0/3859494653 conn(0x56184cbde010 msgr2=0x56184cbdc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 shutdown_connections 2024-09-12T02:03:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 1 -- 172.21.15.122:0/3859494653 wait complete. 2024-09-12T02:03:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.117+0000 7fca3ca04740 20 client.17910 trim_cache size 0 max 16384 2024-09-12T02:03:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:03:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:03:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:03:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:03:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca1c0a2c30 msgr2=0x7fca1c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca1c0a2c30 0x7fca1c0a50f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ca0 tx=0x7fca2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 msgr2=0x7fca34197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34197db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9c0 tx=0x7fca2400ee80 comp rx=0 tx=0).stop 2024-09-12T02:03:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 shutdown_connections 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca1c0a2c30 0x7fca1c0a50f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca341025f0 0x7fca3419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca34101c30 0x7fca341982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 --2- 172.21.15.122:0/2828009008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100a40 0x7fca34197db0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 >> 172.21.15.122:0/2828009008 conn(0x7fca340fc1e0 msgr2=0x7fca34104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 shutdown_connections 2024-09-12T02:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:34.118+0000 7fca39c65640 1 -- 172.21.15.122:0/2828009008 wait complete. 2024-09-12T02:03:35.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 72390 12.71 MB/sec execute 82 sec latency 72.969 ms 2024-09-12T02:03:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:34 smithi151 ceph-mon[35229]: pgmap v851: 97 pgs: 97 active+clean; 479 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 514 KiB/s wr, 96 op/s 2024-09-12T02:03:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:34 smithi183 ceph-mon[35127]: pgmap v851: 97 pgs: 97 active+clean; 479 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 514 KiB/s wr, 96 op/s 2024-09-12T02:03:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:34 smithi122 ceph-mon[30925]: pgmap v851: 97 pgs: 97 active+clean; 479 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 514 KiB/s wr, 96 op/s 2024-09-12T02:03:36.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 72960 12.93 MB/sec execute 83 sec latency 83.405 ms 2024-09-12T02:03:37.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 73401 13.04 MB/sec execute 84 sec latency 94.094 ms 2024-09-12T02:03:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:36 smithi151 ceph-mon[35229]: pgmap v852: 97 pgs: 97 active+clean; 480 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 642 KiB/s wr, 109 op/s 2024-09-12T02:03:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:36 smithi183 ceph-mon[35127]: pgmap v852: 97 pgs: 97 active+clean; 480 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 642 KiB/s wr, 109 op/s 2024-09-12T02:03:37.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:36 smithi122 ceph-mon[30925]: pgmap v852: 97 pgs: 97 active+clean; 480 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 642 KiB/s wr, 109 op/s 2024-09-12T02:03:38.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 74030 13.26 MB/sec execute 85 sec latency 77.240 ms 2024-09-12T02:03:39.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 74622 13.38 MB/sec execute 86 sec latency 86.654 ms 2024-09-12T02:03:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:38 smithi151 ceph-mon[35229]: pgmap v853: 97 pgs: 97 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-12T02:03:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:38 smithi183 ceph-mon[35127]: pgmap v853: 97 pgs: 97 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-12T02:03:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:38 smithi122 ceph-mon[30925]: pgmap v853: 97 pgs: 97 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.3 MiB/s wr, 174 op/s 2024-09-12T02:03:40.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 75369 13.75 MB/sec execute 87 sec latency 88.744 ms 2024-09-12T02:03:41.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 75987 13.92 MB/sec execute 88 sec latency 66.813 ms 2024-09-12T02:03:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:40 smithi151 ceph-mon[35229]: pgmap v854: 97 pgs: 97 active+clean; 490 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-09-12T02:03:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:40 smithi183 ceph-mon[35127]: pgmap v854: 97 pgs: 97 active+clean; 490 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-09-12T02:03:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:40 smithi122 ceph-mon[30925]: pgmap v854: 97 pgs: 97 active+clean; 490 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-09-12T02:03:42.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 76352 13.99 MB/sec execute 89 sec latency 338.823 ms 2024-09-12T02:03:43.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 76975 14.33 MB/sec execute 90 sec latency 63.991 ms 2024-09-12T02:03:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:42 smithi151 ceph-mon[35229]: pgmap v855: 97 pgs: 97 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 406 op/s 2024-09-12T02:03:43.189 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:42 smithi122 ceph-mon[30925]: pgmap v855: 97 pgs: 97 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 406 op/s 2024-09-12T02:03:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:42 smithi183 ceph-mon[35127]: pgmap v855: 97 pgs: 97 active+clean; 499 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 406 op/s 2024-09-12T02:03:44.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 77627 14.52 MB/sec execute 91 sec latency 60.620 ms 2024-09-12T02:03:45.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 78106 14.65 MB/sec execute 92 sec latency 56.542 ms 2024-09-12T02:03:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:44 smithi151 ceph-mon[35229]: pgmap v856: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-09-12T02:03:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:44 smithi183 ceph-mon[35127]: pgmap v856: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-09-12T02:03:45.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:44 smithi122 ceph-mon[30925]: pgmap v856: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-09-12T02:03:46.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 78767 14.82 MB/sec execute 93 sec latency 75.026 ms 2024-09-12T02:03:47.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 79393 15.03 MB/sec execute 94 sec latency 120.243 ms 2024-09-12T02:03:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:46 smithi151 ceph-mon[35229]: pgmap v857: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-09-12T02:03:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:46 smithi183 ceph-mon[35127]: pgmap v857: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-09-12T02:03:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 -- 172.21.15.122:0/4265202217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80100a40 msgr2=0x7f4d80100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 --2- 172.21.15.122:0/4265202217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80100a40 0x7f4d80100e40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009900 tx=0x7f4d7c02a410 comp rx=0 tx=0).stop 2024-09-12T02:03:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 -- 172.21.15.122:0/4265202217 shutdown_connections 2024-09-12T02:03:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 --2- 172.21.15.122:0/4265202217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d801025f0 0x7f4d80108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 --2- 172.21.15.122:0/4265202217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80101c30 0x7f4d801020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 --2- 172.21.15.122:0/4265202217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80100a40 0x7f4d80100e40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 -- 172.21.15.122:0/4265202217 >> 172.21.15.122:0/4265202217 conn(0x7f4d800fc1e0 msgr2=0x7f4d800fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 -- 172.21.15.122:0/4265202217 shutdown_connections 2024-09-12T02:03:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.210+0000 7f4d87c42640 1 -- 172.21.15.122:0/4265202217 wait complete. 2024-09-12T02:03:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 Processor -- start 2024-09-12T02:03:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 -- start start 2024-09-12T02:03:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 0x7f4d80199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d80101c30 0x7f4d8019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d801025f0 0x7f4d801a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010c6c0 con 0x7f4d801025f0 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010c540 con 0x7f4d80100a40 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d87c42640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010c840 con 0x7f4d80101c30 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d859de640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 0x7f4d80199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d859de640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 0x7f4d80199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:48250/0 (socket says 172.21.15.122:48250) 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d859de640 1 -- 172.21.15.122:0/2869019184 learned_addr learned my addr 172.21.15.122:0/2869019184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d851dd640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d80101c30 0x7f4d8019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.211+0000 7f4d861df640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d801025f0 0x7f4d801a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d859de640 1 -- 172.21.15.122:0/2869019184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d80101c30 msgr2=0x7f4d8019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d859de640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d80101c30 0x7f4d8019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d859de640 1 -- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d801025f0 msgr2=0x7f4d801a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d859de640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d801025f0 0x7f4d801a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d859de640 1 -- 172.21.15.122:0/2869019184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801a1c00 con 0x7f4d80100a40 2024-09-12T02:03:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d859de640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 0x7f4d80199fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c031610 tx=0x7f4d7c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d76ffd640 1 -- 172.21.15.122:0/2869019184 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c03f030 con 0x7f4d80100a40 2024-09-12T02:03:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d76ffd640 1 -- 172.21.15.122:0/2869019184 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c003ad0 con 0x7f4d80100a40 2024-09-12T02:03:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a1e90 con 0x7f4d80100a40 2024-09-12T02:03:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d76ffd640 1 -- 172.21.15.122:0/2869019184 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c035070 con 0x7f4d80100a40 2024-09-12T02:03:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.212+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a22f0 con 0x7f4d80100a40 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 Processor -- start 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d76ffd640 1 -- 172.21.15.122:0/2869019184 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4d7c042030 con 0x7f4d80100a40 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 -- start start 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac80fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac814da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac817830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5ac7732f0 con 0x55e5ac8102e0 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5ac71b270 con 0x55e5ac815370 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5ac76f2b0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d859de640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac814da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d851dd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac817830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d859de640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac814da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33728/0 (socket says 172.21.15.122:33728) 2024-09-12T02:03:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d859de640 1 -- 172.21.15.122:0/1025107189 learned_addr learned my addr 172.21.15.122:0/1025107189 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.216+0000 7f4d861df640 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac80fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d76ffd640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.218 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d76ffd640 1 -- 172.21.15.122:0/2869019184 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4d7c0ef1e0 con 0x7f4d80100a40 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d859de640 1 -- 172.21.15.122:0/1025107189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 msgr2=0x55e5ac80fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d859de640 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac80fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d859de640 1 -- 172.21.15.122:0/1025107189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 msgr2=0x55e5ac817830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d859de640 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac817830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d859de640 1 -- 172.21.15.122:0/1025107189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ac71a340 con 0x55e5ac8102e0 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.217+0000 7f4d861df640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.218+0000 7f4d859de640 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac814da0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0bba80 tx=0x7f4d7c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ee79773518b12c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.218+0000 7f4d6bfff640 1 -- 172.21.15.122:0/1025107189 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c049070 con 0x55e5ac8102e0 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.218+0000 7f4d861df640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009fd0 tx=0x7f4d78004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.218+0000 7f4d6bfff640 1 -- 172.21.15.122:0/1025107189 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c03be40 con 0x55e5ac8102e0 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.218+0000 7f4d6bfff640 1 -- 172.21.15.122:0/1025107189 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0c46b0 con 0x55e5ac8102e0 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1025107189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 msgr2=0x55e5ac814da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac814da0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0bba80 tx=0x7f4d7c02a410 comp rx=0 tx=0).stop 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1025107189 shutdown_connections 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac817830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac814da0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1025107189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac80fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1025107189 >> 172.21.15.122:0/1025107189 conn(0x55e5ac80b010 msgr2=0x55e5ac80b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1025107189 shutdown_connections 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1025107189 wait complete. 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 -- start start 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.219+0000 7f4d8a9d3740 1 Processor -- start 2024-09-12T02:03:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 0x55e5ac819890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac8102e0 0x55e5ac819dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac820e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5ac7732f0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5ac6fc290 con 0x55e5ac815370 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5ac818100 con 0x55e5ac8102e0 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d8a9d3740 10 client.0 _ll_register_callbacks cb 0x55e5ac72d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 0x55e5ac819890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d861df640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac820e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d851dd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac8102e0 0x55e5ac819dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 0x55e5ac819890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33732/0 (socket says 172.21.15.122:33732) 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 -- 172.21.15.122:0/3796160364 learned_addr learned my addr 172.21.15.122:0/3796160364 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 -- 172.21.15.122:0/3796160364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac8102e0 msgr2=0x55e5ac819dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac8102e0 0x55e5ac819dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 -- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 msgr2=0x55e5ac820e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac820e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.220+0000 7f4d859de640 1 -- 172.21.15.122:0/3796160364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ac486210 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d859de640 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 0x55e5ac819890 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b84c0 tx=0x7f4d7c0bb4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c616202b14dea91b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d697fa640 1 -- 172.21.15.122:0/3796160364 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c049040 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d697fa640 1 -- 172.21.15.122:0/3796160364 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4d7c003c80 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5ac780a20 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d697fa640 1 -- 172.21.15.122:0/3796160364 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c0c46b0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d8a9d3740 10 client.17922 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e5ac552620 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d697fa640 1 -- 172.21.15.122:0/3796160364 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e5ac780a20 con 0x55e5ac80f9a0 2024-09-12T02:03:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d697fa640 1 -- 172.21.15.122:0/3796160364 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4d7c0bbd50 con 0x55e5ac80f9a0 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.221+0000 7f4d8a9d3740 10 client.17922 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d8a9d3740 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d8a9d3740 20 client.17922 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac8251b0 0x55e5ac8275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d8a9d3740 4 client.17922 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ac77ad70 con 0x55e5ac8251b0 2024-09-12T02:03:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d851dd640 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac8251b0 0x55e5ac8275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d851dd640 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac8251b0 0x55e5ac8275a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdaeb5753cb55 server_cookie=383ef95b7bcea3f1 in_seq=0 out_seq=0 2024-09-12T02:03:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.222+0000 7f4d697fa640 10 client.17922 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d697fa640 1 -- 172.21.15.122:0/3796160364 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5ac77ad70 con 0x55e5ac8251b0 2024-09-12T02:03:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d697fa640 10 client.17922 handle_command_reply: tid=0 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 client.17922 shutdown 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 msgr2=0x55e5ac819890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 0x55e5ac819890 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b84c0 tx=0x7f4d7c0bb4d0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 shutdown_connections 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac8251b0 0x55e5ac8275a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac820e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d697fa640 0 client.17922 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac8102e0 0x55e5ac819dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/3796160364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac80f9a0 0x55e5ac819890 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.223+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 >> 172.21.15.122:0/3796160364 conn(0x55e5ac80b010 msgr2=0x55e5ac80c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.224+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 shutdown_connections 2024-09-12T02:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.224+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/3796160364 wait complete. 2024-09-12T02:03:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.224+0000 7f4d8a9d3740 20 client.17922 trim_cache size 0 max 16384 2024-09-12T02:03:47.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:46 smithi122 ceph-mon[30925]: pgmap v857: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-09-12T02:03:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 Processor -- start 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 -- start start 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac8099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac820dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac809ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801ada80 con 0x55e5ac8102e0 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801ab350 con 0x55e5ac815370 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.240+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801a9180 con 0x55e5ac80f9a0 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac820dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac820dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33740/0 (socket says 172.21.15.122:33740) 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 -- 172.21.15.122:0/1835131000 learned_addr learned my addr 172.21.15.122:0/1835131000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d859de640 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac8099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d861df640 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac809ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 -- 172.21.15.122:0/1835131000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 msgr2=0x55e5ac8099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac8099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 -- 172.21.15.122:0/1835131000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 msgr2=0x55e5ac809ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac809ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 -- 172.21.15.122:0/1835131000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ac552620 con 0x55e5ac8102e0 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d859de640 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac8099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d851dd640 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac820dd0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009dd0 tx=0x7f4d70005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873e845f15d136bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d6b7fe640 1 -- 172.21.15.122:0/1835131000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70022030 con 0x55e5ac8102e0 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d6b7fe640 1 -- 172.21.15.122:0/1835131000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70005500 con 0x55e5ac8102e0 2024-09-12T02:03:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.241+0000 7f4d6b7fe640 1 -- 172.21.15.122:0/1835131000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7001e680 con 0x55e5ac8102e0 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1835131000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 msgr2=0x55e5ac820dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac820dd0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009dd0 tx=0x7f4d70005f90 comp rx=0 tx=0).stop 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1835131000 shutdown_connections 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac815370 0x55e5ac809ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac8102e0 0x55e5ac820dd0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/1835131000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac8099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1835131000 >> 172.21.15.122:0/1835131000 conn(0x55e5ac80b010 msgr2=0x55e5ac821880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1835131000 shutdown_connections 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/1835131000 wait complete. 2024-09-12T02:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.242+0000 7f4d8a9d3740 1 -- start start 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 Processor -- start 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac81d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac8102e0 0x55e5ac81df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 0x55e5ac88de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801a9810 con 0x55e5ac815370 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801aac80 con 0x55e5ac8102e0 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801a9ee0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d8a9d3740 10 client.0 _ll_register_callbacks cb 0x7f4d7c0a6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d861df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 0x55e5ac88de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d851dd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac8102e0 0x55e5ac81df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d859de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac81d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d861df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 0x55e5ac88de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33756/0 (socket says 172.21.15.122:33756) 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d861df640 1 -- 172.21.15.122:0/2080998199 learned_addr learned my addr 172.21.15.122:0/2080998199 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d859de640 1 -- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac8102e0 msgr2=0x55e5ac81df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d859de640 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac8102e0 0x55e5ac81df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d859de640 1 -- 172.21.15.122:0/2080998199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 msgr2=0x55e5ac88de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d859de640 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 0x55e5ac88de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d859de640 1 -- 172.21.15.122:0/2080998199 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ac820fd0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.243+0000 7f4d861df640 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 0x55e5ac88de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d859de640 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac81d9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0bb7f0 tx=0x7f4d7c031c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40570cc3a4b3039b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d697fa640 1 -- 172.21.15.122:0/2080998199 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c049040 con 0x55e5ac80f9a0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d697fa640 1 -- 172.21.15.122:0/2080998199 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4d7c0035f0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d70005500 con 0x55e5ac80f9a0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d697fa640 1 -- 172.21.15.122:0/2080998199 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c0bb630 con 0x55e5ac80f9a0 2024-09-12T02:03:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 10 client.26950 fetch_fsmap learned FSMap version 41 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e5ac820fd0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d697fa640 1 -- 172.21.15.122:0/2080998199 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70005500 con 0x55e5ac80f9a0 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d697fa640 1 -- 172.21.15.122:0/2080998199 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4d7c0da1b0 con 0x55e5ac80f9a0 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 10 client.26950 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 10 client.26950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 20 client.26950 populate_metadata read hostname 'smithi122' 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac88f8e0 0x55e5ac891d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 4 client.26950 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.244+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5ac6f7900 con 0x55e5ac88f8e0 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.245+0000 7f4d851dd640 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac88f8e0 0x55e5ac891d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.245+0000 7f4d851dd640 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac88f8e0 0x55e5ac891d00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37f38fadcf26366 server_cookie=3ee78daf2c70870e in_seq=0 out_seq=0 2024-09-12T02:03:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.245+0000 7f4d697fa640 10 client.26950 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.439+0000 7f4d697fa640 1 -- 172.21.15.122:0/2080998199 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5ac6f7900 con 0x55e5ac88f8e0 2024-09-12T02:03:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.439+0000 7f4d697fa640 10 client.26950 handle_command_reply: tid=0 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.439+0000 7f4d8a9d3740 1 client.26950 shutdown 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 msgr2=0x55e5ac81d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac81d9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0bb7f0 tx=0x7f4d7c031c20 comp rx=0 tx=0).stop 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 shutdown_connections 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5ac88f8e0 0x55e5ac891d00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ac815370 0x55e5ac88de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d697fa640 0 client.26950 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5ac8102e0 0x55e5ac81df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 --2- 172.21.15.122:0/2080998199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5ac80f9a0 0x55e5ac81d9f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 >> 172.21.15.122:0/2080998199 conn(0x55e5ac80b010 msgr2=0x55e5ac80bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 shutdown_connections 2024-09-12T02:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 1 -- 172.21.15.122:0/2080998199 wait complete. 2024-09-12T02:03:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d8a9d3740 20 client.26950 trim_cache size 0 max 16384 2024-09-12T02:03:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:03:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:03:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:03:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d6c0a2cb0 msgr2=0x7f4d6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009fd0 tx=0x7f4d78004a40 comp rx=0 tx=0).stop 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 msgr2=0x7f4d80199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 0x7f4d80199fe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c031610 tx=0x7f4d7c004270 comp rx=0 tx=0).stop 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 shutdown_connections 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d801025f0 0x7f4d801a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d80101c30 0x7f4d8019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 --2- 172.21.15.122:0/2869019184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d80100a40 0x7f4d80199fe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:03:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.440+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 >> 172.21.15.122:0/2869019184 conn(0x7f4d800fc1e0 msgr2=0x7f4d800698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:03:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.441+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 shutdown_connections 2024-09-12T02:03:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:03:47.441+0000 7f4d87c42640 1 -- 172.21.15.122:0/2869019184 wait complete. 2024-09-12T02:03:48.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 80119 15.24 MB/sec execute 95 sec latency 150.806 ms 2024-09-12T02:03:49.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 80800 15.50 MB/sec execute 96 sec latency 55.174 ms 2024-09-12T02:03:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:48 smithi151 ceph-mon[35229]: pgmap v858: 97 pgs: 97 active+clean; 506 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-09-12T02:03:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:48 smithi183 ceph-mon[35127]: pgmap v858: 97 pgs: 97 active+clean; 506 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-09-12T02:03:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:48 smithi122 ceph-mon[30925]: pgmap v858: 97 pgs: 97 active+clean; 506 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-09-12T02:03:50.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 81462 15.76 MB/sec execute 97 sec latency 61.475 ms 2024-09-12T02:03:51.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 82042 15.94 MB/sec execute 98 sec latency 71.556 ms 2024-09-12T02:03:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:50 smithi151 ceph-mon[35229]: pgmap v859: 97 pgs: 97 active+clean; 509 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-09-12T02:03:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:50 smithi183 ceph-mon[35127]: pgmap v859: 97 pgs: 97 active+clean; 509 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-09-12T02:03:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:50 smithi122 ceph-mon[30925]: pgmap v859: 97 pgs: 97 active+clean; 509 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.3 MiB/s wr, 577 op/s 2024-09-12T02:03:52.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 82455 15.99 MB/sec execute 99 sec latency 149.773 ms 2024-09-12T02:03:53.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 83082 16.17 MB/sec execute 100 sec latency 56.341 ms 2024-09-12T02:03:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:52 smithi151 ceph-mon[35229]: pgmap v860: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-09-12T02:03:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:52 smithi183 ceph-mon[35127]: pgmap v860: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-09-12T02:03:53.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:52 smithi122 ceph-mon[30925]: pgmap v860: 97 pgs: 97 active+clean; 503 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-09-12T02:03:54.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 83624 16.24 MB/sec execute 101 sec latency 109.278 ms 2024-09-12T02:03:55.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 84395 16.48 MB/sec execute 102 sec latency 51.889 ms 2024-09-12T02:03:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:54 smithi151 ceph-mon[35229]: pgmap v861: 97 pgs: 97 active+clean; 502 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 516 op/s 2024-09-12T02:03:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:54 smithi183 ceph-mon[35127]: pgmap v861: 97 pgs: 97 active+clean; 502 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 516 op/s 2024-09-12T02:03:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:54 smithi122 ceph-mon[30925]: pgmap v861: 97 pgs: 97 active+clean; 502 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 516 op/s 2024-09-12T02:03:56.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 85083 16.65 MB/sec execute 103 sec latency 79.975 ms 2024-09-12T02:03:57.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 85787 16.98 MB/sec execute 104 sec latency 52.410 ms 2024-09-12T02:03:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:56 smithi151 ceph-mon[35229]: pgmap v862: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.5 MiB/s wr, 478 op/s 2024-09-12T02:03:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:56 smithi183 ceph-mon[35127]: pgmap v862: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.5 MiB/s wr, 478 op/s 2024-09-12T02:03:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:56 smithi122 ceph-mon[30925]: pgmap v862: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.5 MiB/s wr, 478 op/s 2024-09-12T02:03:58.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 86283 17.05 MB/sec execute 105 sec latency 71.822 ms 2024-09-12T02:03:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: Unable to set osd_memory_target on smithi183 to 184571699: error parsing value: Value '184571699' is below minimum 939524096 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi122 to 175.9M 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: Unable to set osd_memory_target on smithi122 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: Unable to set osd_memory_target on smithi151 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:03:58.117 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:03:58.118 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: Unable to set osd_memory_target on smithi183 to 184571699: error parsing value: Value '184571699' is below minimum 939524096 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi122 to 175.9M 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: Unable to set osd_memory_target on smithi122 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: Unable to set osd_memory_target on smithi151 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:03:58.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:03:58.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: Unable to set osd_memory_target on smithi183 to 184571699: error parsing value: Value '184571699' is below minimum 939524096 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi122 to 175.9M 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: Unable to set osd_memory_target on smithi122 to 184520806: error parsing value: Value '184520806' is below minimum 939524096 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: Unable to set osd_memory_target on smithi151 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:03:58.241 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:03:59.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 86816 17.17 MB/sec execute 106 sec latency 95.864 ms 2024-09-12T02:03:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:03:58 smithi151 ceph-mon[35229]: pgmap v863: 97 pgs: 97 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.7 MiB/s wr, 496 op/s 2024-09-12T02:03:59.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:03:58 smithi183 ceph-mon[35127]: pgmap v863: 97 pgs: 97 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.7 MiB/s wr, 496 op/s 2024-09-12T02:03:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:03:58 smithi122 ceph-mon[30925]: pgmap v863: 97 pgs: 97 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.7 MiB/s wr, 496 op/s 2024-09-12T02:04:00.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 87264 17.23 MB/sec execute 107 sec latency 66.235 ms 2024-09-12T02:04:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 -- 172.21.15.122:0/2052612499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c101c30 msgr2=0x7f739c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 --2- 172.21.15.122:0/2052612499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c101c30 0x7f739c1020b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7390009900 tx=0x7f739002a410 comp rx=0 tx=0).stop 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 -- 172.21.15.122:0/2052612499 shutdown_connections 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 --2- 172.21.15.122:0/2052612499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f739c1025f0 0x7f739c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 --2- 172.21.15.122:0/2052612499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c101c30 0x7f739c1020b0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 --2- 172.21.15.122:0/2052612499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739c100a40 0x7f739c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 -- 172.21.15.122:0/2052612499 >> 172.21.15.122:0/2052612499 conn(0x7f739c0fc1e0 msgr2=0x7f739c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 -- 172.21.15.122:0/2052612499 shutdown_connections 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.530+0000 7f73a1065640 1 -- 172.21.15.122:0/2052612499 wait complete. 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 Processor -- start 2024-09-12T02:04:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 -- start start 2024-09-12T02:04:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f739c100a40 0x7f739c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739c101c30 0x7f739c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 0x7f739c19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f739c10c6c0 con 0x7f739c1025f0 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f739c10c540 con 0x7f739c100a40 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f73a1065640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f739c10c840 con 0x7f739c101c30 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 0x7f739c19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739c101c30 0x7f739c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 0x7f739c19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51680/0 (socket says 172.21.15.122:51680) 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739ad9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f739c100a40 0x7f739c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 -- 172.21.15.122:0/2394047823 learned_addr learned my addr 172.21.15.122:0/2394047823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 -- 172.21.15.122:0/2394047823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739c101c30 msgr2=0x7f739c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739c101c30 0x7f739c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 -- 172.21.15.122:0/2394047823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f739c100a40 msgr2=0x7f739c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f739c100a40 0x7f739c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.531+0000 7f739b59e640 1 -- 172.21.15.122:0/2394047823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c19fa30 con 0x7f739c1025f0 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.532+0000 7f739b59e640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 0x7f739c19f3c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f739400eb10 tx=0x7f739400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.532+0000 7f7383fff640 1 -- 172.21.15.122:0/2394047823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394004b20 con 0x7f739c1025f0 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.532+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739c19fd20 con 0x7f739c1025f0 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.532+0000 7f7383fff640 1 -- 172.21.15.122:0/2394047823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7394004cc0 con 0x7f739c1025f0 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.532+0000 7f7383fff640 1 -- 172.21.15.122:0/2394047823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394010760 con 0x7f739c1025f0 2024-09-12T02:04:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.532+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739c1a0190 con 0x7f739c1025f0 2024-09-12T02:04:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.534+0000 7f7383fff640 1 -- 172.21.15.122:0/2394047823 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7394005020 con 0x7f739c1025f0 2024-09-12T02:04:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.534+0000 7f73a3e02740 1 Processor -- start 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.534+0000 7f73a3e02740 1 -- start start 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73a3e02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556f968492f0 con 0x556f968e59a0 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73a3e02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556f967f1270 con 0x556f968eb370 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73a3e02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f968452b0 con 0x556f968e62e0 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968e5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51692/0 (socket says 172.21.15.122:51692) 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739a59c640 1 -- 172.21.15.122:0/2872035470 learned_addr learned my addr 172.21.15.122:0/2872035470 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f7383fff640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73840a2c30 0x7f73840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739a59c640 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 -- 172.21.15.122:0/2872035470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 msgr2=0x556f968ed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739b59e640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73840a2c30 0x7f73840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968ed830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 -- 172.21.15.122:0/2872035470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 msgr2=0x556f968e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 -- 172.21.15.122:0/2872035470 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f967f0340 con 0x556f968e62e0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739a59c640 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f7383fff640 1 -- 172.21.15.122:0/2394047823 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f73940cf630 con 0x7f739c1025f0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739ad9d640 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968eada0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f739c101a90 tx=0x7f738c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4511562ed61e0262 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f739b59e640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73840a2c30 0x7f73840a50f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7394098820 tx=0x7f739406de60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.535+0000 7f73817fa640 1 -- 172.21.15.122:0/2872035470 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c01c480 con 0x556f968e62e0 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73817fa640 1 -- 172.21.15.122:0/2872035470 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738c011920 con 0x556f968e62e0 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73817fa640 1 -- 172.21.15.122:0/2872035470 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c01a850 con 0x556f968e62e0 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 -- 172.21.15.122:0/2872035470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 msgr2=0x556f968eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968eada0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f739c101a90 tx=0x7f738c0107a0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 -- 172.21.15.122:0/2872035470 shutdown_connections 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968eada0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 --2- 172.21.15.122:0/2872035470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 -- 172.21.15.122:0/2872035470 >> 172.21.15.122:0/2872035470 conn(0x556f968e1010 msgr2=0x556f968e1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.536+0000 7f73a3e02740 1 -- 172.21.15.122:0/2872035470 shutdown_connections 2024-09-12T02:04:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 -- 172.21.15.122:0/2872035470 wait complete. 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 -- start start 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 Processor -- start 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968efba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556f968ee340 con 0x556f968e59a0 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556f967d2290 con 0x556f968eb370 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f968ee4c0 con 0x556f968e62e0 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f73a3e02740 10 client.0 _ll_register_callbacks cb 0x556f96803140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f739a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f739a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968efba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51694/0 (socket says 172.21.15.122:51694) 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f739a59c640 1 -- 172.21.15.122:0/185626730 learned_addr learned my addr 172.21.15.122:0/185626730 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f739ad9d640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.537+0000 7f739a59c640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f739a59c640 1 -- 172.21.15.122:0/185626730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 msgr2=0x556f968f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f739a59c640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f739a59c640 1 -- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 msgr2=0x556f968f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f739a59c640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f739a59c640 1 -- 172.21.15.122:0/185626730 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f9655c210 con 0x556f968e59a0 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f739a59c640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968efba0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7390009230 tx=0x7f7390005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8ee91b589b007 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f737eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:04:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f737eff5640 1 -- 172.21.15.122:0/185626730 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390035040 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f737eff5640 1 -- 172.21.15.122:0/185626730 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f7390004780 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738c011920 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f737eff5640 1 -- 172.21.15.122:0/185626730 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7390004980 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f73a3e02740 10 client.17940 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556f96628620 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.538+0000 7f737eff5640 1 -- 172.21.15.122:0/185626730 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738c011920 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f737eff5640 1 -- 172.21.15.122:0/185626730 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7390046a70 con 0x556f968e59a0 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f73a3e02740 10 client.17940 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f73a3e02740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f73a3e02740 20 client.17940 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f73a3e02740 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f968fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f73a3e02740 4 client.17940 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f96856a20 con 0x556f968fb840 2024-09-12T02:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f739ad9d640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f968fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f739ad9d640 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f968fdc90 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb04d3ff67e6538 server_cookie=ce2d1ed92fd759b3 in_seq=0 out_seq=0 2024-09-12T02:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.539+0000 7f737eff5640 10 client.17940 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f737eff5640 1 -- 172.21.15.122:0/185626730 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f96856a20 con 0x556f968fb840 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f737eff5640 10 client.17940 handle_command_reply: tid=0 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 client.17940 shutdown 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 msgr2=0x556f968efba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968efba0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7390009230 tx=0x7f7390005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 shutdown_connections 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f968fdc90 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f737eff5640 0 client.17940 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e62e0 0x556f968f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 --2- 172.21.15.122:0/185626730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968efba0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 >> 172.21.15.122:0/185626730 conn(0x556f968e1010 msgr2=0x556f968e2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 shutdown_connections 2024-09-12T02:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 1 -- 172.21.15.122:0/185626730 wait complete. 2024-09-12T02:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.542+0000 7f73a3e02740 20 client.17940 trim_cache size 0 max 16384 2024-09-12T02:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 Processor -- start 2024-09-12T02:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 -- start start 2024-09-12T02:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e59a0 0x556f968f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 0x556f968f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968e00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1a95d0 con 0x556f968e62e0 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1a6ea0 con 0x556f968eb370 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f73a3e02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1a4cd0 con 0x556f968e59a0 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f739ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 0x556f968f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f739ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 0x556f968f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51706/0 (socket says 172.21.15.122:51706) 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f739ad9d640 1 -- 172.21.15.122:0/1183662780 learned_addr learned my addr 172.21.15.122:0/1183662780 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.558+0000 7f739a59c640 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e59a0 0x556f968f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739a59c640 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968e00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739ad9d640 1 -- 172.21.15.122:0/1183662780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e59a0 msgr2=0x556f968f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739ad9d640 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e59a0 0x556f968f0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739ad9d640 1 -- 172.21.15.122:0/1183662780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 msgr2=0x556f968e00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739ad9d640 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968e00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739ad9d640 1 -- 172.21.15.122:0/1183662780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f968e67c0 con 0x556f968e62e0 2024-09-12T02:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f739ad9d640 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 0x556f968f70e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x556f968e6d10 tx=0x7f738c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3fa1a17a1e3210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f7380ff9640 1 -- 172.21.15.122:0/1183662780 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c005240 con 0x556f968e62e0 2024-09-12T02:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f7380ff9640 1 -- 172.21.15.122:0/1183662780 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738c0053e0 con 0x556f968e62e0 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.559+0000 7f7380ff9640 1 -- 172.21.15.122:0/1183662780 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c011960 con 0x556f968e62e0 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- 172.21.15.122:0/1183662780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 msgr2=0x556f968f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 0x556f968f70e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x556f968e6d10 tx=0x7f738c00b390 comp rx=0 tx=0).stop 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- 172.21.15.122:0/1183662780 shutdown_connections 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968eb370 0x556f968e00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e62e0 0x556f968f70e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- 172.21.15.122:0/1183662780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968e59a0 0x556f968f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- 172.21.15.122:0/1183662780 >> 172.21.15.122:0/1183662780 conn(0x556f968e1010 msgr2=0x556f968df9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- 172.21.15.122:0/1183662780 shutdown_connections 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- 172.21.15.122:0/1183662780 wait complete. 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- start start 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 Processor -- start 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968e62e0 0x556f968f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968eb370 0x556f968f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1a5360 con 0x556f968e59a0 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1a67d0 con 0x556f968e62e0 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1a5a30 con 0x556f968eb370 2024-09-12T02:04:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.560+0000 7f73a3e02740 10 client.0 _ll_register_callbacks cb 0x7f73900319d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51708/0 (socket says 172.21.15.122:51708) 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 -- 172.21.15.122:0/3642380335 learned_addr learned my addr 172.21.15.122:0/3642380335 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739ad9d640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968e62e0 0x556f968f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968eb370 0x556f968f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 -- 172.21.15.122:0/3642380335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968eb370 msgr2=0x556f968f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968eb370 0x556f968f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 -- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968e62e0 msgr2=0x556f968f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968e62e0 0x556f968f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 -- 172.21.15.122:0/3642380335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f968f5cf0 con 0x556f968e59a0 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f739a59c640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968f4190 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7390046e90 tx=0x7f7390005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca3b25c4ba4b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f737eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f737eff5640 1 -- 172.21.15.122:0/3642380335 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390035040 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f737eff5640 1 -- 172.21.15.122:0/3642380335 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f7390004d20 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738c0053e0 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f737eff5640 1 -- 172.21.15.122:0/3642380335 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7390004240 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f73a3e02740 10 client.17946 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.561+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556f968f7000 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f737eff5640 1 -- 172.21.15.122:0/3642380335 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738c0053e0 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f737eff5640 1 -- 172.21.15.122:0/3642380335 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f73900466a0 con 0x556f968e59a0 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f73a3e02740 10 client.17946 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f73a3e02740 10 client.17946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f73a3e02740 20 client.17946 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f73a3e02740 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f96968870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f73a3e02740 4 client.17946 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f96850d70 con 0x556f968fb840 2024-09-12T02:04:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f739ad9d640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f96968870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f739ad9d640 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f96968870 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c683edf0e4e83522 server_cookie=8178043dd6df029f in_seq=0 out_seq=0 2024-09-12T02:04:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.562+0000 7f737eff5640 10 client.17946 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.778+0000 7f737eff5640 1 -- 172.21.15.122:0/3642380335 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f96850d70 con 0x556f968fb840 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.778+0000 7f737eff5640 10 client.17946 handle_command_reply: tid=0 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.778+0000 7f73a3e02740 1 client.17946 shutdown 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 msgr2=0x556f968f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968f4190 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7390046e90 tx=0x7f7390005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 shutdown_connections 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556f968fb840 0x556f96968870 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f968eb370 0x556f968f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f968e62e0 0x556f968f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f737eff5640 0 client.17946 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 --2- 172.21.15.122:0/3642380335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f968e59a0 0x556f968f4190 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 >> 172.21.15.122:0/3642380335 conn(0x556f968e1010 msgr2=0x556f968df9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 shutdown_connections 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 1 -- 172.21.15.122:0/3642380335 wait complete. 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a3e02740 20 client.17946 trim_cache size 0 max 16384 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:04:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:04:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:04:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73840a2c30 msgr2=0x7f73840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a1065640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73840a2c30 0x7f73840a50f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7394098820 tx=0x7f739406de60 comp rx=0 tx=0).stop 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 msgr2=0x7f739c19f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.779+0000 7f73a1065640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 0x7f739c19f3c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f739400eb10 tx=0x7f739400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 shutdown_connections 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73840a2c30 0x7f73840a50f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f739c1025f0 0x7f739c19f3c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f739c101c30 0x7f739c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 --2- 172.21.15.122:0/2394047823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f739c100a40 0x7f739c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 >> 172.21.15.122:0/2394047823 conn(0x7f739c0fc1e0 msgr2=0x7f739c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 shutdown_connections 2024-09-12T02:04:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:00.780+0000 7f73a1065640 1 -- 172.21.15.122:0/2394047823 wait complete. 2024-09-12T02:04:01.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 87758 17.25 MB/sec execute 108 sec latency 135.078 ms 2024-09-12T02:04:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:00 smithi151 ceph-mon[35229]: pgmap v864: 97 pgs: 97 active+clean; 509 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.7 MiB/s wr, 468 op/s 2024-09-12T02:04:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:00 smithi183 ceph-mon[35127]: pgmap v864: 97 pgs: 97 active+clean; 509 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.7 MiB/s wr, 468 op/s 2024-09-12T02:04:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:00 smithi122 ceph-mon[30925]: pgmap v864: 97 pgs: 97 active+clean; 509 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.7 MiB/s wr, 468 op/s 2024-09-12T02:04:02.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 88492 17.42 MB/sec execute 109 sec latency 72.477 ms 2024-09-12T02:04:03.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 89257 17.63 MB/sec execute 110 sec latency 86.311 ms 2024-09-12T02:04:03.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:02 smithi151 ceph-mon[35229]: pgmap v865: 97 pgs: 97 active+clean; 513 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-09-12T02:04:03.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:02 smithi183 ceph-mon[35127]: pgmap v865: 97 pgs: 97 active+clean; 513 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-09-12T02:04:03.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:02 smithi122 ceph-mon[30925]: pgmap v865: 97 pgs: 97 active+clean; 513 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.4 MiB/s wr, 458 op/s 2024-09-12T02:04:04.065 INFO:tasks.workunit.client.0.smithi122.stdout: 10 89971 17.88 MB/sec execute 111 sec latency 64.906 ms 2024-09-12T02:04:05.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 90612 18.05 MB/sec execute 112 sec latency 62.112 ms 2024-09-12T02:04:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:04 smithi151 ceph-mon[35229]: pgmap v866: 97 pgs: 97 active+clean; 522 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-09-12T02:04:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:04 smithi183 ceph-mon[35127]: pgmap v866: 97 pgs: 97 active+clean; 522 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-09-12T02:04:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:04 smithi122 ceph-mon[30925]: pgmap v866: 97 pgs: 97 active+clean; 522 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-09-12T02:04:06.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 91125 18.14 MB/sec execute 113 sec latency 60.987 ms 2024-09-12T02:04:07.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 91465 18.13 MB/sec execute 114 sec latency 329.262 ms 2024-09-12T02:04:07.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:06 smithi151 ceph-mon[35229]: pgmap v867: 97 pgs: 97 active+clean; 520 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 459 op/s 2024-09-12T02:04:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:06 smithi183 ceph-mon[35127]: pgmap v867: 97 pgs: 97 active+clean; 520 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 459 op/s 2024-09-12T02:04:07.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:06 smithi122 ceph-mon[30925]: pgmap v867: 97 pgs: 97 active+clean; 520 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 459 op/s 2024-09-12T02:04:08.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 91992 18.23 MB/sec execute 115 sec latency 421.437 ms 2024-09-12T02:04:09.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 92470 18.24 MB/sec execute 116 sec latency 80.755 ms 2024-09-12T02:04:09.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:08 smithi151 ceph-mon[35229]: pgmap v868: 97 pgs: 97 active+clean; 530 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 501 op/s 2024-09-12T02:04:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:08 smithi183 ceph-mon[35127]: pgmap v868: 97 pgs: 97 active+clean; 530 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 501 op/s 2024-09-12T02:04:09.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:08 smithi122 ceph-mon[30925]: pgmap v868: 97 pgs: 97 active+clean; 530 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 501 op/s 2024-09-12T02:04:10.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 93233 18.39 MB/sec execute 117 sec latency 74.204 ms 2024-09-12T02:04:11.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 94000 18.62 MB/sec execute 118 sec latency 55.409 ms 2024-09-12T02:04:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:10 smithi151 ceph-mon[35229]: pgmap v869: 97 pgs: 97 active+clean; 531 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-09-12T02:04:11.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:10 smithi183 ceph-mon[35127]: pgmap v869: 97 pgs: 97 active+clean; 531 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-09-12T02:04:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:10 smithi122 ceph-mon[30925]: pgmap v869: 97 pgs: 97 active+clean; 531 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-09-12T02:04:12.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 94539 18.74 MB/sec execute 119 sec latency 223.148 ms 2024-09-12T02:04:13.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 95041 18.86 MB/sec execute 120 sec latency 335.039 ms 2024-09-12T02:04:13.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:12 smithi151 ceph-mon[35229]: pgmap v870: 97 pgs: 97 active+clean; 544 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.0 MiB/s wr, 647 op/s 2024-09-12T02:04:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:12 smithi183 ceph-mon[35127]: pgmap v870: 97 pgs: 97 active+clean; 544 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.0 MiB/s wr, 647 op/s 2024-09-12T02:04:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:12 smithi122 ceph-mon[30925]: pgmap v870: 97 pgs: 97 active+clean; 544 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.0 MiB/s wr, 647 op/s 2024-09-12T02:04:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 -- 172.21.15.122:0/2469592360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88104f50 msgr2=0x7efc88107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 --2- 172.21.15.122:0/2469592360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88104f50 0x7efc88107340 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efc78009900 tx=0x7efc7802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 -- 172.21.15.122:0/2469592360 shutdown_connections 2024-09-12T02:04:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 --2- 172.21.15.122:0/2469592360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc88107880 0x7efc88109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 --2- 172.21.15.122:0/2469592360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88104f50 0x7efc88107340 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 --2- 172.21.15.122:0/2469592360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88102620 0x7efc88104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 -- 172.21.15.122:0/2469592360 >> 172.21.15.122:0/2469592360 conn(0x7efc880fc1e0 msgr2=0x7efc880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 -- 172.21.15.122:0/2469592360 shutdown_connections 2024-09-12T02:04:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 -- 172.21.15.122:0/2469592360 wait complete. 2024-09-12T02:04:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.878+0000 7efc8f50a640 1 Processor -- start 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 -- start start 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 0x7efc88101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc88104f50 0x7efc881005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88107880 0x7efc88100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810d850 con 0x7efc88102620 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810d6d0 con 0x7efc88104f50 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8f50a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810d9d0 con 0x7efc88107880 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 0x7efc88101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8caa5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc88104f50 0x7efc881005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 0x7efc88101f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46740/0 (socket says 172.21.15.122:46740) 2024-09-12T02:04:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 -- 172.21.15.122:0/3867464767 learned_addr learned my addr 172.21.15.122:0/3867464767 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88107880 0x7efc88100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 -- 172.21.15.122:0/3867464767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88107880 msgr2=0x7efc88100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88107880 0x7efc88100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 -- 172.21.15.122:0/3867464767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc88104f50 msgr2=0x7efc881005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc88104f50 0x7efc881005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.879+0000 7efc8d2a6640 1 -- 172.21.15.122:0/3867464767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc881013a0 con 0x7efc88102620 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc8d2a6640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 0x7efc88101f90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7efc8400e9f0 tx=0x7efc8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88107880 0x7efc88100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc7e7fc640 1 -- 172.21.15.122:0/3867464767 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400cd80 con 0x7efc88102620 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc881a1fc0 con 0x7efc88102620 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc7e7fc640 1 -- 172.21.15.122:0/3867464767 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc84005500 con 0x7efc88102620 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc7e7fc640 1 -- 172.21.15.122:0/3867464767 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84010690 con 0x7efc88102620 2024-09-12T02:04:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.880+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc881a23c0 con 0x7efc88102620 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 Processor -- start 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 -- start start 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f134ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134b2e0 0x5582f134fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f1350370 0x5582f1352830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582f12ae2f0 con 0x5582f134a9a0 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5582f1256270 con 0x5582f134b2e0 2024-09-12T02:04:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc922a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582f12aa2b0 con 0x5582f1350370 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc8daa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f134ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc8daa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f134ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46742/0 (socket says 172.21.15.122:46742) 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.883+0000 7efc8daa7640 1 -- 172.21.15.122:0/1101343988 learned_addr learned my addr 172.21.15.122:0/1101343988 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8caa5640 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f1350370 0x5582f1352830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8daa7640 1 -- 172.21.15.122:0/1101343988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f1350370 msgr2=0x5582f1352830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f1350370 0x5582f1352830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8daa7640 1 -- 172.21.15.122:0/1101343988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134b2e0 msgr2=0x5582f134fda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134b2e0 0x5582f134fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8daa7640 1 -- 172.21.15.122:0/1101343988 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f1255340 con 0x5582f134a9a0 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc7e7fc640 1 -- 172.21.15.122:0/3867464767 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efc84004f50 con 0x7efc88102620 2024-09-12T02:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f134ada0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efc800117a0 tx=0x7efc80011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba16b9b36bd2fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc737fe640 1 -- 172.21.15.122:0/1101343988 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80011220 con 0x5582f134a9a0 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc737fe640 1 -- 172.21.15.122:0/1101343988 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8000f9a0 con 0x5582f134a9a0 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.884+0000 7efc737fe640 1 -- 172.21.15.122:0/1101343988 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800046b0 con 0x5582f134a9a0 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc7e7fc640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc740a2c30 0x7efc740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc8d2a6640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc740a2c30 0x7efc740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc7e7fc640 1 -- 172.21.15.122:0/3867464767 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7efc840cf3c0 con 0x7efc88102620 2024-09-12T02:04:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 -- 172.21.15.122:0/1101343988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 msgr2=0x5582f134ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc8d2a6640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc740a2c30 0x7efc740a50f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efc840a4360 tx=0x7efc84091100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f134ada0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efc800117a0 tx=0x7efc80011c60 comp rx=0 tx=0).stop 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 -- 172.21.15.122:0/1101343988 shutdown_connections 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f1350370 0x5582f1352830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134b2e0 0x5582f134fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 --2- 172.21.15.122:0/1101343988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f134ada0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 -- 172.21.15.122:0/1101343988 >> 172.21.15.122:0/1101343988 conn(0x5582f1346010 msgr2=0x5582f1346410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 -- 172.21.15.122:0/1101343988 shutdown_connections 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.885+0000 7efc922a2740 1 -- 172.21.15.122:0/1101343988 wait complete. 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 -- start start 2024-09-12T02:04:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 Processor -- start 2024-09-12T02:04:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1354b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f13550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582f1353340 con 0x5582f1350370 2024-09-12T02:04:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5582f1237290 con 0x5582f134a9a0 2024-09-12T02:04:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.886+0000 7efc922a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582f13534c0 con 0x5582f134b2e0 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc922a2740 10 client.0 _ll_register_callbacks cb 0x5582f1268140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f13550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f13550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:35152/0 (socket says 172.21.15.122:35152) 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 -- 172.21.15.122:0/3507886284 learned_addr learned my addr 172.21.15.122:0/3507886284 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1354b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 -- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 msgr2=0x5582f1354b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1354b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 -- 172.21.15.122:0/3507886284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 msgr2=0x5582f135c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8caa5640 1 -- 172.21.15.122:0/3507886284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f0fc1210 con 0x5582f134b2e0 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.887+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:04:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.888+0000 7efc8caa5640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f13550a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc78000c00 tx=0x7efc78043e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51af16583536e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.888+0000 7efc70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.888+0000 7efc70ff9640 1 -- 172.21.15.122:0/3507886284 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78035040 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.888+0000 7efc70ff9640 1 -- 172.21.15.122:0/3507886284 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7efc78005030 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.888+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc8000f9a0 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.888+0000 7efc70ff9640 1 -- 172.21.15.122:0/3507886284 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efc78005230 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 10 client.26980 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5582f108d620 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc70ff9640 1 -- 172.21.15.122:0/3507886284 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc78004c50 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc70ff9640 1 -- 172.21.15.122:0/3507886284 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7efc7804a070 con 0x5582f134b2e0 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 10 client.26980 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 10 client.26980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 20 client.26980 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f1360820 0x5582f1362c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 4 client.26980 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.889+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efc8000f9a0 con 0x5582f1360820 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.890+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f1360820 0x5582f1362c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.890+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f1360820 0x5582f1362c50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eac0be2e7d348a server_cookie=2de212b3f2ec3665 in_seq=0 out_seq=0 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.890+0000 7efc70ff9640 10 client.26980 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc70ff9640 1 -- 172.21.15.122:0/3507886284 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efc8000f9a0 con 0x5582f1360820 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc70ff9640 10 client.26980 handle_command_reply: tid=0 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 client.26980 shutdown 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 msgr2=0x5582f13550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f13550a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc78000c00 tx=0x7efc78043e80 comp rx=0 tx=0).stop 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 shutdown_connections 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f1360820 0x5582f1362c50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc70ff9640 0 client.26980 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f13550a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 --2- 172.21.15.122:0/3507886284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1354b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 >> 172.21.15.122:0/3507886284 conn(0x5582f1346010 msgr2=0x5582f1347820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 shutdown_connections 2024-09-12T02:04:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 1 -- 172.21.15.122:0/3507886284 wait complete. 2024-09-12T02:04:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.892+0000 7efc922a2740 20 client.26980 trim_cache size 0 max 16384 2024-09-12T02:04:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 Processor -- start 2024-09-12T02:04:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 -- start start 2024-09-12T02:04:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f13450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f135c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f1350370 0x5582f1347400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a9760 con 0x5582f134a9a0 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a7030 con 0x5582f1350370 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc922a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a4e60 con 0x5582f134b2e0 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc8daa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f13450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8daa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f13450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46772/0 (socket says 172.21.15.122:46772) 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8daa7640 1 -- 172.21.15.122:0/3914726421 learned_addr learned my addr 172.21.15.122:0/3914726421 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.909+0000 7efc8caa5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f135c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8daa7640 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f1350370 0x5582f1347400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8caa5640 1 -- 172.21.15.122:0/3914726421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f1350370 msgr2=0x5582f1347400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8caa5640 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f1350370 0x5582f1347400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8caa5640 1 -- 172.21.15.122:0/3914726421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 msgr2=0x5582f13450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8caa5640 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f13450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8caa5640 1 -- 172.21.15.122:0/3914726421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f1345e30 con 0x5582f134b2e0 2024-09-12T02:04:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc8caa5640 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f135c0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc78006010 tx=0x7efc7802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5402dac5bef9289 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc72ffd640 1 -- 172.21.15.122:0/3914726421 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7805b030 con 0x5582f134b2e0 2024-09-12T02:04:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc72ffd640 1 -- 172.21.15.122:0/3914726421 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc780316b0 con 0x5582f134b2e0 2024-09-12T02:04:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.910+0000 7efc72ffd640 1 -- 172.21.15.122:0/3914726421 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78035030 con 0x5582f134b2e0 2024-09-12T02:04:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 -- 172.21.15.122:0/3914726421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 msgr2=0x5582f135c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f135c0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc78006010 tx=0x7efc7802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 -- 172.21.15.122:0/3914726421 shutdown_connections 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f1350370 0x5582f1347400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f135c0a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 --2- 172.21.15.122:0/3914726421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f134a9a0 0x5582f13450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 -- 172.21.15.122:0/3914726421 >> 172.21.15.122:0/3914726421 conn(0x5582f1346010 msgr2=0x5582f13449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 -- 172.21.15.122:0/3914726421 shutdown_connections 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 -- 172.21.15.122:0/3914726421 wait complete. 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.911+0000 7efc922a2740 1 -- start start 2024-09-12T02:04:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 Processor -- start 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1359940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f1359e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a54f0 con 0x5582f1350370 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a6960 con 0x5582f134a9a0 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a5bc0 con 0x5582f134b2e0 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc922a2740 10 client.0 _ll_register_callbacks cb 0x7efc84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46776/0 (socket says 172.21.15.122:46776) 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 -- 172.21.15.122:0/1289073690 learned_addr learned my addr 172.21.15.122:0/1289073690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8caa5640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f1359e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1359940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 -- 172.21.15.122:0/1289073690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 msgr2=0x5582f1359e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f1359e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 -- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 msgr2=0x5582f1359940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1359940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.912+0000 7efc8daa7640 1 -- 172.21.15.122:0/1289073690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f135b4a0 con 0x5582f1350370 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc8caa5640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f1359e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc8daa7640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135ae90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efc80006010 tx=0x7efc80000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da43b901c09d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc70ff9640 1 -- 172.21.15.122:0/1289073690 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800052d0 con 0x5582f1350370 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc70ff9640 1 -- 172.21.15.122:0/1289073690 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7efc80005470 con 0x5582f1350370 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc780316b0 con 0x5582f1350370 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc70ff9640 1 -- 172.21.15.122:0/1289073690 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efc80005670 con 0x5582f1350370 2024-09-12T02:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc922a2740 10 client.17961 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5582f13cb6e0 con 0x5582f1350370 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc70ff9640 1 -- 172.21.15.122:0/1289073690 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc780316b0 con 0x5582f1350370 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.913+0000 7efc70ff9640 1 -- 172.21.15.122:0/1289073690 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7efc80018070 con 0x5582f1350370 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc922a2740 10 client.17961 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc922a2740 10 client.17961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc922a2740 20 client.17961 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc922a2740 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f13cbe60 0x5582f13ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc922a2740 4 client.17961 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582f12bba20 con 0x5582f13cbe60 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc8caa5640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f13cbe60 0x5582f13ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc70ff9640 10 client.17961 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:13.914+0000 7efc8caa5640 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f13cbe60 0x5582f13ce280 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9285a184bc4a2149 server_cookie=ab41dc9c34f055f3 in_seq=0 out_seq=0 2024-09-12T02:04:14.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 95449 18.88 MB/sec execute 121 sec latency 91.334 ms 2024-09-12T02:04:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.125+0000 7efc70ff9640 1 -- 172.21.15.122:0/1289073690 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582f12bba20 con 0x5582f13cbe60 2024-09-12T02:04:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.125+0000 7efc70ff9640 10 client.17961 handle_command_reply: tid=0 2024-09-12T02:04:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.125+0000 7efc922a2740 1 client.17961 shutdown 2024-09-12T02:04:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.125+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 msgr2=0x5582f135ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.125+0000 7efc922a2740 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135ae90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efc80006010 tx=0x7efc80000f40 comp rx=0 tx=0).stop 2024-09-12T02:04:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 shutdown_connections 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5582f13cbe60 0x5582f13ce280 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582f1350370 0x5582f135ae90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582f134b2e0 0x5582f1359e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc70ff9640 0 client.17961 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 --2- 172.21.15.122:0/1289073690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5582f134a9a0 0x5582f1359940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 >> 172.21.15.122:0/1289073690 conn(0x5582f1346010 msgr2=0x5582f13451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 shutdown_connections 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 1 -- 172.21.15.122:0/1289073690 wait complete. 2024-09-12T02:04:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc922a2740 20 client.17961 trim_cache size 0 max 16384 2024-09-12T02:04:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:04:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:04:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:04:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc740a2c30 msgr2=0x7efc740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc8f50a640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc740a2c30 0x7efc740a50f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efc840a4360 tx=0x7efc84091100 comp rx=0 tx=0).stop 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 msgr2=0x7efc88101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.126+0000 7efc8f50a640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 0x7efc88101f90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7efc8400e9f0 tx=0x7efc8400edf0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 shutdown_connections 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7efc740a2c30 0x7efc740a50f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc88107880 0x7efc88100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc88104f50 0x7efc881005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 --2- 172.21.15.122:0/3867464767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc88102620 0x7efc88101f90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 >> 172.21.15.122:0/3867464767 conn(0x7efc880fc1e0 msgr2=0x7efc88102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 shutdown_connections 2024-09-12T02:04:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:14.127+0000 7efc8f50a640 1 -- 172.21.15.122:0/3867464767 wait complete. 2024-09-12T02:04:14.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:14 smithi122 ceph-mon[30925]: pgmap v871: 97 pgs: 97 active+clean; 530 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 577 op/s 2024-09-12T02:04:15.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 95978 18.94 MB/sec execute 122 sec latency 123.240 ms 2024-09-12T02:04:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:14 smithi151 ceph-mon[35229]: pgmap v871: 97 pgs: 97 active+clean; 530 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 577 op/s 2024-09-12T02:04:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:14 smithi183 ceph-mon[35127]: pgmap v871: 97 pgs: 97 active+clean; 530 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 577 op/s 2024-09-12T02:04:16.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 96508 18.99 MB/sec execute 123 sec latency 62.630 ms 2024-09-12T02:04:17.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 97260 19.08 MB/sec execute 124 sec latency 102.392 ms 2024-09-12T02:04:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:16 smithi151 ceph-mon[35229]: pgmap v872: 97 pgs: 97 active+clean; 523 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.0 MiB/s wr, 510 op/s 2024-09-12T02:04:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:16 smithi183 ceph-mon[35127]: pgmap v872: 97 pgs: 97 active+clean; 523 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.0 MiB/s wr, 510 op/s 2024-09-12T02:04:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:16 smithi122 ceph-mon[30925]: pgmap v872: 97 pgs: 97 active+clean; 523 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.0 MiB/s wr, 510 op/s 2024-09-12T02:04:18.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 97821 19.22 MB/sec execute 125 sec latency 351.194 ms 2024-09-12T02:04:19.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 98606 19.40 MB/sec execute 126 sec latency 85.312 ms 2024-09-12T02:04:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:18 smithi151 ceph-mon[35229]: pgmap v873: 97 pgs: 97 active+clean; 517 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.0 MiB/s wr, 584 op/s 2024-09-12T02:04:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:18 smithi183 ceph-mon[35127]: pgmap v873: 97 pgs: 97 active+clean; 517 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.0 MiB/s wr, 584 op/s 2024-09-12T02:04:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:18 smithi122 ceph-mon[30925]: pgmap v873: 97 pgs: 97 active+clean; 517 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.0 MiB/s wr, 584 op/s 2024-09-12T02:04:20.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 99175 19.58 MB/sec execute 127 sec latency 55.607 ms 2024-09-12T02:04:21.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 99787 19.67 MB/sec execute 128 sec latency 49.043 ms 2024-09-12T02:04:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:20 smithi151 ceph-mon[35229]: pgmap v874: 97 pgs: 97 active+clean; 526 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.7 MiB/s wr, 663 op/s 2024-09-12T02:04:21.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:20 smithi183 ceph-mon[35127]: pgmap v874: 97 pgs: 97 active+clean; 526 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.7 MiB/s wr, 663 op/s 2024-09-12T02:04:21.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:20 smithi122 ceph-mon[30925]: pgmap v874: 97 pgs: 97 active+clean; 526 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.7 MiB/s wr, 663 op/s 2024-09-12T02:04:22.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 100227 19.69 MB/sec execute 129 sec latency 109.878 ms 2024-09-12T02:04:23.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 100543 19.66 MB/sec execute 130 sec latency 346.249 ms 2024-09-12T02:04:23.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:22 smithi151 ceph-mon[35229]: pgmap v875: 97 pgs: 97 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.4 MiB/s wr, 694 op/s 2024-09-12T02:04:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:22 smithi183 ceph-mon[35127]: pgmap v875: 97 pgs: 97 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.4 MiB/s wr, 694 op/s 2024-09-12T02:04:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:22 smithi122 ceph-mon[30925]: pgmap v875: 97 pgs: 97 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.4 MiB/s wr, 694 op/s 2024-09-12T02:04:24.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 101106 19.71 MB/sec execute 131 sec latency 97.339 ms 2024-09-12T02:04:25.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 101688 19.74 MB/sec execute 132 sec latency 77.972 ms 2024-09-12T02:04:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:24 smithi151 ceph-mon[35229]: pgmap v876: 97 pgs: 97 active+clean; 533 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 625 op/s 2024-09-12T02:04:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:24 smithi183 ceph-mon[35127]: pgmap v876: 97 pgs: 97 active+clean; 533 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 625 op/s 2024-09-12T02:04:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:24 smithi122 ceph-mon[30925]: pgmap v876: 97 pgs: 97 active+clean; 533 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 625 op/s 2024-09-12T02:04:26.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 102472 19.93 MB/sec execute 133 sec latency 52.819 ms 2024-09-12T02:04:27.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 103179 20.14 MB/sec execute 134 sec latency 77.377 ms 2024-09-12T02:04:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:26 smithi151 ceph-mon[35229]: pgmap v877: 97 pgs: 97 active+clean; 533 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.6 MiB/s wr, 585 op/s 2024-09-12T02:04:27.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:26 smithi183 ceph-mon[35127]: pgmap v877: 97 pgs: 97 active+clean; 533 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.6 MiB/s wr, 585 op/s 2024-09-12T02:04:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.240+0000 7fb786b59640 1 -- 172.21.15.122:0/2196873871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb780076230 msgr2=0x7fb780074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.240+0000 7fb786b59640 1 --2- 172.21.15.122:0/2196873871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb780076230 0x7fb780074680 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb774009900 tx=0x7fb77402a410 comp rx=0 tx=0).stop 2024-09-12T02:04:27.242 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:26 smithi122 ceph-mon[30925]: pgmap v877: 97 pgs: 97 active+clean; 533 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.6 MiB/s wr, 585 op/s 2024-09-12T02:04:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.242+0000 7fb786b59640 1 -- 172.21.15.122:0/2196873871 shutdown_connections 2024-09-12T02:04:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.242+0000 7fb786b59640 1 --2- 172.21.15.122:0/2196873871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb78010e130 0x7fb780110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.242+0000 7fb786b59640 1 --2- 172.21.15.122:0/2196873871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780074c50 0x7fb7800750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.242+0000 7fb786b59640 1 --2- 172.21.15.122:0/2196873871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb780076230 0x7fb780074680 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.242+0000 7fb786b59640 1 -- 172.21.15.122:0/2196873871 >> 172.21.15.122:0/2196873871 conn(0x7fb7800fd2c0 msgr2=0x7fb7800ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.249+0000 7fb786b59640 1 -- 172.21.15.122:0/2196873871 shutdown_connections 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.250+0000 7fb786b59640 1 -- 172.21.15.122:0/2196873871 wait complete. 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 Processor -- start 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 -- start start 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb780074c50 0x7fb7800720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 0x7fb7800725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb78010e130 0x7fb780073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb78006c620 con 0x7fb78010e130 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb78006c4a0 con 0x7fb780076230 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.251+0000 7fb786b59640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb78006c7a0 con 0x7fb780074c50 2024-09-12T02:04:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.252+0000 7fb7848f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb780074c50 0x7fb7800720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 0x7fb7800725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 0x7fb7800725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:56670/0 (socket says 172.21.15.122:56670) 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 -- 172.21.15.122:0/1829632225 learned_addr learned my addr 172.21.15.122:0/1829632225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb7850f6640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb78010e130 0x7fb780073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 -- 172.21.15.122:0/1829632225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb780074c50 msgr2=0x7fb7800720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb780074c50 0x7fb7800720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 -- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb78010e130 msgr2=0x7fb780073500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb78010e130 0x7fb780073500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.253+0000 7fb77ffff640 1 -- 172.21.15.122:0/1829632225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb780073c30 con 0x7fb780076230 2024-09-12T02:04:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.254+0000 7fb77ffff640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 0x7fb7800725f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb77000cce0 tx=0x7fb77000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.254+0000 7fb77dffb640 1 -- 172.21.15.122:0/1829632225 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770004b20 con 0x7fb780076230 2024-09-12T02:04:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.254+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb780073f20 con 0x7fb780076230 2024-09-12T02:04:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.254+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb780119460 con 0x7fb780076230 2024-09-12T02:04:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.256+0000 7fb77dffb640 1 -- 172.21.15.122:0/1829632225 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb770004cc0 con 0x7fb780076230 2024-09-12T02:04:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.256+0000 7fb77dffb640 1 -- 172.21.15.122:0/1829632225 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77000f870 con 0x7fb780076230 2024-09-12T02:04:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.263+0000 7fb7899d1740 1 Processor -- start 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 -- start start 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc47830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac5cba32f0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac5cb4b270 con 0x55ac5cc45370 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.266+0000 7fb7899d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac5cb9f2b0 con 0x55ac5cc402e0 2024-09-12T02:04:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.267+0000 7fb7848f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.268+0000 7fb77dffb640 1 -- 172.21.15.122:0/1829632225 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb770005020 con 0x7fb780076230 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.268+0000 7fb77dffb640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7680a2d80 0x7fb7680a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.269+0000 7fb7850f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.269+0000 7fb7850f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc3fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49680/0 (socket says 172.21.15.122:49680) 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.269+0000 7fb7850f6640 1 -- 172.21.15.122:0/2647328989 learned_addr learned my addr 172.21.15.122:0/2647328989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.269+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc47830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.273+0000 7fb7850f6640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7680a2d80 0x7fb7680a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.273+0000 7fb7848f5640 1 -- 172.21.15.122:0/2647328989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 msgr2=0x55ac5cc47830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.273+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc47830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.274+0000 7fb7848f5640 1 -- 172.21.15.122:0/2647328989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 msgr2=0x55ac5cc3fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.274+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc3fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.274+0000 7fb7848f5640 1 -- 172.21.15.122:0/2647328989 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5cb4a340 con 0x55ac5cc402e0 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.273+0000 7fb77dffb640 1 -- 172.21.15.122:0/1829632225 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb7700cf0a0 con 0x7fb780076230 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.274+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc47830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.274+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc44da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb774033040 tx=0x7fb774004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88aa01697fdfbcb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.276+0000 7fb766ffd640 1 -- 172.21.15.122:0/2647328989 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774040070 con 0x55ac5cc402e0 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.276+0000 7fb766ffd640 1 -- 172.21.15.122:0/2647328989 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7740054e0 con 0x55ac5cc402e0 2024-09-12T02:04:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.276+0000 7fb766ffd640 1 -- 172.21.15.122:0/2647328989 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774045960 con 0x55ac5cc402e0 2024-09-12T02:04:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.279+0000 7fb7899d1740 1 -- 172.21.15.122:0/2647328989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 msgr2=0x55ac5cc44da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.279+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc44da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb774033040 tx=0x7fb774004de0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.281+0000 7fb7850f6640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7680a2d80 0x7fb7680a5240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb778005fd0 tx=0x7fb778004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.285+0000 7fb7899d1740 1 -- 172.21.15.122:0/2647328989 shutdown_connections 2024-09-12T02:04:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.285+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc47830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.285+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc44da0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.285+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2647328989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc3fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.285+0000 7fb7899d1740 1 -- 172.21.15.122:0/2647328989 >> 172.21.15.122:0/2647328989 conn(0x55ac5cc3b010 msgr2=0x55ac5cc3b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.291+0000 7fb7899d1740 1 -- 172.21.15.122:0/2647328989 shutdown_connections 2024-09-12T02:04:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.295+0000 7fb7899d1740 1 -- 172.21.15.122:0/2647328989 wait complete. 2024-09-12T02:04:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.296+0000 7fb7899d1740 1 -- start start 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.296+0000 7fb7899d1740 1 Processor -- start 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc49bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc45370 0x55ac5cc51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac5cc482e0 con 0x55ac5cc402e0 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac5cb2c290 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac5cc48460 con 0x55ac5cc45370 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.297+0000 7fb7899d1740 10 client.0 _ll_register_callbacks cb 0x55ac5cb5d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.298+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.298+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49682/0 (socket says 172.21.15.122:49682) 2024-09-12T02:04:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.298+0000 7fb77ffff640 1 -- 172.21.15.122:0/763466391 learned_addr learned my addr 172.21.15.122:0/763466391 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.298+0000 7fb7848f5640 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc49bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.300+0000 7fb7848f5640 1 -- 172.21.15.122:0/763466391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc45370 msgr2=0x55ac5cc51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.300+0000 7fb7848f5640 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc45370 0x55ac5cc51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.300+0000 7fb7848f5640 1 -- 172.21.15.122:0/763466391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 msgr2=0x55ac5cc4a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.300+0000 7fb7848f5640 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.300+0000 7fb7848f5640 1 -- 172.21.15.122:0/763466391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac5c8b6210 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.305+0000 7fb7848f5640 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc49bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb774005ab0 tx=0x7fb77402a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b94d83ae3c54bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.305+0000 7fb75ffff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:04:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.306+0000 7fb75ffff640 1 -- 172.21.15.122:0/763466391 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774040070 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.306+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7740054e0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.306+0000 7fb75ffff640 1 -- 172.21.15.122:0/763466391 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb7740036b0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.307+0000 7fb75ffff640 1 -- 172.21.15.122:0/763466391 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb774009520 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.307+0000 7fb7899d1740 10 client.27050 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.307+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac5c982620 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb75ffff640 1 -- 172.21.15.122:0/763466391 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7740054e0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb75ffff640 1 -- 172.21.15.122:0/763466391 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb7740317b0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb7899d1740 10 client.27050 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb7899d1740 10 client.27050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb7899d1740 20 client.27050 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb7899d1740 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5cc66990 0x55ac5cc68e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb7899d1740 4 client.27050 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.308+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac5cbb0a20 con 0x55ac5cc66990 2024-09-12T02:04:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.309+0000 7fb77ffff640 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5cc66990 0x55ac5cc68e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.314+0000 7fb77ffff640 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5cc66990 0x55ac5cc68e30 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cb7651c05e9e49 server_cookie=e2d6a87e6a48e93c in_seq=0 out_seq=0 2024-09-12T02:04:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.314+0000 7fb75ffff640 10 client.27050 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.319+0000 7fb75ffff640 1 -- 172.21.15.122:0/763466391 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac5cbb0a20 con 0x55ac5cc66990 2024-09-12T02:04:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.319+0000 7fb75ffff640 10 client.27050 handle_command_reply: tid=0 2024-09-12T02:04:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.319+0000 7fb7899d1740 1 client.27050 shutdown 2024-09-12T02:04:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.325+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc3f9a0 msgr2=0x55ac5cc49bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.325+0000 7fb7899d1740 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc49bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb774005ab0 tx=0x7fb77402a910 comp rx=0 tx=0).stop 2024-09-12T02:04:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.329+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 shutdown_connections 2024-09-12T02:04:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.329+0000 7fb7899d1740 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5cc66990 0x55ac5cc68e30 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.329+0000 7fb7899d1740 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc45370 0x55ac5cc51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.329+0000 7fb7899d1740 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.329+0000 7fb7899d1740 1 --2- 172.21.15.122:0/763466391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc49bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.329+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 >> 172.21.15.122:0/763466391 conn(0x55ac5cc3b010 msgr2=0x55ac5cc3c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.330+0000 7fb75ffff640 0 client.27050 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.336+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 shutdown_connections 2024-09-12T02:04:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.337+0000 7fb7899d1740 1 -- 172.21.15.122:0/763466391 wait complete. 2024-09-12T02:04:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.339+0000 7fb7899d1740 20 client.27050 trim_cache size 0 max 16384 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.359+0000 7fb7899d1740 1 Processor -- start 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 -- start start 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc41520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc51130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc41a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c010e10 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c00e6e0 con 0x55ac5cc45370 2024-09-12T02:04:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7899d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c00c510 con 0x55ac5cc402e0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc41520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc41520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49692/0 (socket says 172.21.15.122:49692) 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 -- 172.21.15.122:0/2340366884 learned_addr learned my addr 172.21.15.122:0/2340366884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc51130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7850f6640 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc41a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 -- 172.21.15.122:0/2340366884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 msgr2=0x55ac5cc51130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc51130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 -- 172.21.15.122:0/2340366884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 msgr2=0x55ac5cc41a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc41a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 -- 172.21.15.122:0/2340366884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5cc3bb00 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.360+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc41520 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb77402a980 tx=0x7fb774005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a121e840eda8d69d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.361+0000 7fb7667fc640 1 -- 172.21.15.122:0/2340366884 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774040040 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.361+0000 7fb7667fc640 1 -- 172.21.15.122:0/2340366884 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7740036b0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.361+0000 7fb7667fc640 1 -- 172.21.15.122:0/2340366884 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77404fa40 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.361+0000 7fb7899d1740 1 -- 172.21.15.122:0/2340366884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 msgr2=0x55ac5cc41520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.361+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc41520 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb77402a980 tx=0x7fb774005e70 comp rx=0 tx=0).stop 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 -- 172.21.15.122:0/2340366884 shutdown_connections 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc41a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc402e0 0x55ac5cc51130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2340366884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc41520 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 -- 172.21.15.122:0/2340366884 >> 172.21.15.122:0/2340366884 conn(0x55ac5cc3b010 msgr2=0x55ac5cc3b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 -- 172.21.15.122:0/2340366884 shutdown_connections 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 -- 172.21.15.122:0/2340366884 wait complete. 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 -- start start 2024-09-12T02:04:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 Processor -- start 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.362+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7899d1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:43770/0 (socket says 172.21.15.122:43770) 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 -- 172.21.15.122:0/2852069043 learned_addr learned my addr 172.21.15.122:0/2852069043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc4e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c00cba0 con 0x55ac5cc402e0 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c00e010 con 0x55ac5cc45370 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c00d270 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7899d1740 10 client.0 _ll_register_callbacks cb 0x7fb77800c3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7850f6640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc4e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 -- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 msgr2=0x55ac5cc4e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc4e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 -- 172.21.15.122:0/2852069043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 msgr2=0x55ac5cc4db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac5cccf740 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb7848f5640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc510d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb774031b80 tx=0x7fb774031ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552ad6510a0ec45f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.363+0000 7fb75ffff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb75ffff640 1 -- 172.21.15.122:0/2852069043 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774040030 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb75ffff640 1 -- 172.21.15.122:0/2852069043 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fb77404b070 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7740036b0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb75ffff640 1 -- 172.21.15.122:0/2852069043 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb774005490 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 10 client.26995 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac5ccd0a50 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb75ffff640 1 -- 172.21.15.122:0/2852069043 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7740036b0 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb75ffff640 1 -- 172.21.15.122:0/2852069043 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb774066030 con 0x55ac5cc3f9a0 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 10 client.26995 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 10 client.26995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 20 client.26995 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5ccd1340 0x55ac5ccd3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 4 client.26995 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac5cbaad70 con 0x55ac5ccd1340 2024-09-12T02:04:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.364+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5ccd1340 0x55ac5ccd3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.367+0000 7fb75ffff640 10 client.26995 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.367+0000 7fb77ffff640 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5ccd1340 0x55ac5ccd3760 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cf96e268389188 server_cookie=b294310bf3f215d6 in_seq=0 out_seq=0 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb75ffff640 1 -- 172.21.15.122:0/2852069043 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac5cbaad70 con 0x55ac5ccd1340 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb75ffff640 10 client.26995 handle_command_reply: tid=0 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 client.26995 shutdown 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 msgr2=0x55ac5cc510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc510d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb774031b80 tx=0x7fb774031ab0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 shutdown_connections 2024-09-12T02:04:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ac5ccd1340 0x55ac5ccd3760 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb75ffff640 0 client.26995 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac5cc45370 0x55ac5cc4e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac5cc402e0 0x55ac5cc4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 --2- 172.21.15.122:0/2852069043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac5cc3f9a0 0x55ac5cc510d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 >> 172.21.15.122:0/2852069043 conn(0x55ac5cc3b010 msgr2=0x55ac5cc41520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.819+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 shutdown_connections 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.820+0000 7fb7899d1740 1 -- 172.21.15.122:0/2852069043 wait complete. 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.820+0000 7fb7899d1740 20 client.26995 trim_cache size 0 max 16384 2024-09-12T02:04:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:04:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:04:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:04:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7680a2d80 msgr2=0x7fb7680a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7680a2d80 0x7fb7680a5240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb778005fd0 tx=0x7fb778004730 comp rx=0 tx=0).stop 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 msgr2=0x7fb7800725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 0x7fb7800725f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb77000cce0 tx=0x7fb77000a620 comp rx=0 tx=0).stop 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 shutdown_connections 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb7680a2d80 0x7fb7680a5240 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb78010e130 0x7fb780073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb780076230 0x7fb7800725f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 --2- 172.21.15.122:0/1829632225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb780074c50 0x7fb7800720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 >> 172.21.15.122:0/1829632225 conn(0x7fb7800fd2c0 msgr2=0x7fb7800ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 shutdown_connections 2024-09-12T02:04:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:27.821+0000 7fb786b59640 1 -- 172.21.15.122:0/1829632225 wait complete. 2024-09-12T02:04:28.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 103457 20.11 MB/sec execute 135 sec latency 475.747 ms 2024-09-12T02:04:29.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 103957 20.20 MB/sec execute 136 sec latency 117.144 ms 2024-09-12T02:04:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:28 smithi151 ceph-mon[35229]: pgmap v878: 97 pgs: 97 active+clean; 545 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.7 MiB/s wr, 668 op/s 2024-09-12T02:04:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:28 smithi183 ceph-mon[35127]: pgmap v878: 97 pgs: 97 active+clean; 545 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.7 MiB/s wr, 668 op/s 2024-09-12T02:04:29.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:28 smithi122 ceph-mon[30925]: pgmap v878: 97 pgs: 97 active+clean; 545 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.7 MiB/s wr, 668 op/s 2024-09-12T02:04:30.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 104400 20.22 MB/sec execute 137 sec latency 66.716 ms 2024-09-12T02:04:31.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 104937 20.26 MB/sec execute 138 sec latency 84.655 ms 2024-09-12T02:04:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:30 smithi151 ceph-mon[35229]: pgmap v879: 97 pgs: 97 active+clean; 543 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.6 MiB/s wr, 719 op/s 2024-09-12T02:04:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:30 smithi183 ceph-mon[35127]: pgmap v879: 97 pgs: 97 active+clean; 543 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.6 MiB/s wr, 719 op/s 2024-09-12T02:04:31.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:30 smithi122 ceph-mon[30925]: pgmap v879: 97 pgs: 97 active+clean; 543 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.6 MiB/s wr, 719 op/s 2024-09-12T02:04:32.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 105527 20.28 MB/sec execute 139 sec latency 71.108 ms 2024-09-12T02:04:33.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 106055 20.29 MB/sec execute 140 sec latency 270.327 ms 2024-09-12T02:04:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:32 smithi151 ceph-mon[35229]: pgmap v880: 97 pgs: 97 active+clean; 539 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.3 MiB/s wr, 787 op/s 2024-09-12T02:04:33.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:32 smithi183 ceph-mon[35127]: pgmap v880: 97 pgs: 97 active+clean; 539 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.3 MiB/s wr, 787 op/s 2024-09-12T02:04:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:32 smithi122 ceph-mon[30925]: pgmap v880: 97 pgs: 97 active+clean; 539 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.3 MiB/s wr, 787 op/s 2024-09-12T02:04:34.066 INFO:tasks.workunit.client.0.smithi122.stdout: 10 106637 20.38 MB/sec execute 141 sec latency 69.087 ms 2024-09-12T02:04:35.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 107473 20.58 MB/sec execute 142 sec latency 54.948 ms 2024-09-12T02:04:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:34 smithi151 ceph-mon[35229]: pgmap v881: 97 pgs: 97 active+clean; 537 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.5 MiB/s wr, 784 op/s 2024-09-12T02:04:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:34 smithi183 ceph-mon[35127]: pgmap v881: 97 pgs: 97 active+clean; 537 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.5 MiB/s wr, 784 op/s 2024-09-12T02:04:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:34 smithi122 ceph-mon[30925]: pgmap v881: 97 pgs: 97 active+clean; 537 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.5 MiB/s wr, 784 op/s 2024-09-12T02:04:36.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 108037 20.70 MB/sec execute 143 sec latency 93.059 ms 2024-09-12T02:04:37.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 108530 20.76 MB/sec execute 144 sec latency 55.810 ms 2024-09-12T02:04:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:36 smithi151 ceph-mon[35229]: pgmap v882: 97 pgs: 97 active+clean; 531 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.9 MiB/s wr, 742 op/s 2024-09-12T02:04:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:36 smithi183 ceph-mon[35127]: pgmap v882: 97 pgs: 97 active+clean; 531 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.9 MiB/s wr, 742 op/s 2024-09-12T02:04:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:36 smithi122 ceph-mon[30925]: pgmap v882: 97 pgs: 97 active+clean; 531 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.9 MiB/s wr, 742 op/s 2024-09-12T02:04:38.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 108967 20.81 MB/sec execute 145 sec latency 262.257 ms 2024-09-12T02:04:39.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 109319 20.75 MB/sec execute 146 sec latency 113.179 ms 2024-09-12T02:04:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:38 smithi151 ceph-mon[35229]: pgmap v883: 97 pgs: 97 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 771 op/s 2024-09-12T02:04:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:38 smithi183 ceph-mon[35127]: pgmap v883: 97 pgs: 97 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 771 op/s 2024-09-12T02:04:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:38 smithi122 ceph-mon[30925]: pgmap v883: 97 pgs: 97 active+clean; 525 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 771 op/s 2024-09-12T02:04:40.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 110001 20.84 MB/sec execute 147 sec latency 54.795 ms 2024-09-12T02:04:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 -- 172.21.15.122:0/1032480750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10100a40 msgr2=0x7f3f10100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1032480750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10100a40 0x7f3f10100e40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009900 tx=0x7f3f0c02a410 comp rx=0 tx=0).stop 2024-09-12T02:04:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 -- 172.21.15.122:0/1032480750 shutdown_connections 2024-09-12T02:04:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1032480750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 0x7f3f10108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1032480750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10101c30 0x7f3f101020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1032480750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10100a40 0x7f3f10100e40 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 -- 172.21.15.122:0/1032480750 >> 172.21.15.122:0/1032480750 conn(0x7f3f100fc1e0 msgr2=0x7f3f100fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 -- 172.21.15.122:0/1032480750 shutdown_connections 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 -- 172.21.15.122:0/1032480750 wait complete. 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 Processor -- start 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.917+0000 7f3f176ac640 1 -- start start 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f176ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10100a40 0x7f3f10197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f176ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 0x7f3f10198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f176ac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 0x7f3f1019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f176ac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010c6c0 con 0x7f3f10101c30 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f176ac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010c540 con 0x7f3f101025f0 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f176ac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010c840 con 0x7f3f10100a40 2024-09-12T02:04:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 0x7f3f10198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 0x7f3f10198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55668/0 (socket says 172.21.15.122:55668) 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 -- 172.21.15.122:0/1140468796 learned_addr learned my addr 172.21.15.122:0/1140468796 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f15448640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10100a40 0x7f3f10197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f15c49640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 0x7f3f1019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 -- 172.21.15.122:0/1140468796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10100a40 msgr2=0x7f3f10197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10100a40 0x7f3f10197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 -- 172.21.15.122:0/1140468796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 msgr2=0x7f3f1019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 0x7f3f1019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 -- 172.21.15.122:0/1140468796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1019fa40 con 0x7f3f10101c30 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.918+0000 7f3f14c47640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 0x7f3f10198360 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000e9f0 tx=0x7f3f0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.919+0000 7f3f067fc640 1 -- 172.21.15.122:0/1140468796 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000cd80 con 0x7f3f10101c30 2024-09-12T02:04:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.919+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1019fd30 con 0x7f3f10101c30 2024-09-12T02:04:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.919+0000 7f3f067fc640 1 -- 172.21.15.122:0/1140468796 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f00005500 con 0x7f3f10101c30 2024-09-12T02:04:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.919+0000 7f3f067fc640 1 -- 172.21.15.122:0/1140468796 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00010690 con 0x7f3f10101c30 2024-09-12T02:04:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.919+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f10102e50 con 0x7f3f10101c30 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.921+0000 7f3f067fc640 1 -- 172.21.15.122:0/1140468796 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3f000046c0 con 0x7f3f10101c30 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f067fc640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a7040 0x7f3efc0a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 Processor -- start 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 -- start start 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f15448640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a7040 0x7f3efc0a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f067fc640 1 -- 172.21.15.122:0/1140468796 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f3f000cf420 con 0x7f3f10101c30 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6ceedda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b6ce4c2f0 con 0x557b6ceee370 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b6cdf4270 con 0x557b6cee89a0 2024-09-12T02:04:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.922+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b6ce482b0 con 0x557b6cee92e0 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f15c49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f15c49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55674/0 (socket says 172.21.15.122:55674) 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f15c49640 1 -- 172.21.15.122:0/2459216428 learned_addr learned my addr 172.21.15.122:0/2459216428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6ceedda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f15448640 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f15448640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a7040 0x7f3efc0a9500 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c005ab0 tx=0x7f3f0c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 -- 172.21.15.122:0/2459216428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 msgr2=0x557b6cee8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 -- 172.21.15.122:0/2459216428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 msgr2=0x557b6cef0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 -- 172.21.15.122:0/2459216428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6cdf3340 con 0x557b6cee92e0 2024-09-12T02:04:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f15c49640 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3f14c47640 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6ceedda0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c6a0 tx=0x7f3f0009b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872c23faa92ef2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3efb7fe640 1 -- 172.21.15.122:0/2459216428 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f000a0030 con 0x557b6cee92e0 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3efb7fe640 1 -- 172.21.15.122:0/2459216428 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f000a48f0 con 0x557b6cee92e0 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.923+0000 7f3efb7fe640 1 -- 172.21.15.122:0/2459216428 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f000a46f0 con 0x557b6cee92e0 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2459216428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 msgr2=0x557b6ceedda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6ceedda0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c6a0 tx=0x7f3f0009b5e0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2459216428 shutdown_connections 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6ceedda0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2459216428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2459216428 >> 172.21.15.122:0/2459216428 conn(0x557b6cee4010 msgr2=0x557b6cee4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2459216428 shutdown_connections 2024-09-12T02:04:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2459216428 wait complete. 2024-09-12T02:04:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.924+0000 7f3f1a44b740 1 -- start start 2024-09-12T02:04:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 Processor -- start 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 0x557b6cef3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6ceee370 0x557b6cefa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b6cef1340 con 0x557b6cee92e0 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b6cdd5290 con 0x557b6ceee370 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b6cef14c0 con 0x557b6cee89a0 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f1a44b740 10 client.0 _ll_register_callbacks cb 0x557b6ce06140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f14c47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 0x557b6cef3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f14c47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 0x557b6cef3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55682/0 (socket says 172.21.15.122:55682) 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f14c47640 1 -- 172.21.15.122:0/4112534455 learned_addr learned my addr 172.21.15.122:0/4112534455 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15448640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15c49640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6ceee370 0x557b6cefa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15448640 1 -- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6ceee370 msgr2=0x557b6cefa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15448640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6ceee370 0x557b6cefa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15448640 1 -- 172.21.15.122:0/4112534455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 msgr2=0x557b6cef3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15448640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 0x557b6cef3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f15448640 1 -- 172.21.15.122:0/4112534455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6cb5f210 con 0x557b6cee89a0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.925+0000 7f3f14c47640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 0x557b6cef3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3f15448640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef2c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009230 tx=0x7f3f0c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66eaf2d97a31312f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3ef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/4112534455 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c035040 con 0x557b6cee89a0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f000a48f0 con 0x557b6cee89a0 2024-09-12T02:04:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/4112534455 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f3f0c005040 con 0x557b6cee89a0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/4112534455 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c005240 con 0x557b6cee89a0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3f1a44b740 10 client.27007 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557b6cc2b620 con 0x557b6cee89a0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/4112534455 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f000a48f0 con 0x557b6cee89a0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.926+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/4112534455 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3f0c0037b0 con 0x557b6cee89a0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f1a44b740 10 client.27007 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f1a44b740 10 client.27007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f1a44b740 20 client.27007 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cefe8c0 0x557b6cf00cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f1a44b740 4 client.27007 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6ce59a20 con 0x557b6cefe8c0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f14c47640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cefe8c0 0x557b6cf00cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3f14c47640 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cefe8c0 0x557b6cf00cd0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226bd04e9ec308cc server_cookie=9f3bc6099da2141 in_seq=0 out_seq=0 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.927+0000 7f3ef8ff9640 10 client.27007 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/4112534455 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b6ce59a20 con 0x557b6cefe8c0 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3ef8ff9640 10 client.27007 handle_command_reply: tid=0 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 client.27007 shutdown 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 msgr2=0x557b6cef2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef2c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009230 tx=0x7f3f0c02a410 comp rx=0 tx=0).stop 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 shutdown_connections 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cefe8c0 0x557b6cf00cd0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6ceee370 0x557b6cefa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3ef8ff9640 0 client.27007 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6cee92e0 0x557b6cef3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/4112534455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef2c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 >> 172.21.15.122:0/4112534455 conn(0x557b6cee4010 msgr2=0x557b6cee5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 shutdown_connections 2024-09-12T02:04:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 1 -- 172.21.15.122:0/4112534455 wait complete. 2024-09-12T02:04:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.929+0000 7f3f1a44b740 20 client.27007 trim_cache size 0 max 16384 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 Processor -- start 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 -- start start 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6cefa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cee5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a9730 con 0x557b6ceee370 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a7000 con 0x557b6cee89a0 2024-09-12T02:04:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.945+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a4e30 con 0x557b6cee92e0 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cee5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f14c47640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6cefa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cee5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55690/0 (socket says 172.21.15.122:55690) 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 -- 172.21.15.122:0/1377899416 learned_addr learned my addr 172.21.15.122:0/1377899416 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 -- 172.21.15.122:0/1377899416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 msgr2=0x557b6cefa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6cefa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 -- 172.21.15.122:0/1377899416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 msgr2=0x557b6cee4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 -- 172.21.15.122:0/1377899416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6cef0760 con 0x557b6ceee370 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3f15c49640 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cee5400 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3f08003ce0 tx=0x7f3f08004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4209efc3ac91a359 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3efaffd640 1 -- 172.21.15.122:0/1377899416 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08021070 con 0x557b6ceee370 2024-09-12T02:04:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3efaffd640 1 -- 172.21.15.122:0/1377899416 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0801cb40 con 0x557b6ceee370 2024-09-12T02:04:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.946+0000 7f3efaffd640 1 -- 172.21.15.122:0/1377899416 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08010d30 con 0x557b6ceee370 2024-09-12T02:04:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 -- 172.21.15.122:0/1377899416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 msgr2=0x557b6cee5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cee5400 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3f08003ce0 tx=0x7f3f08004410 comp rx=0 tx=0).stop 2024-09-12T02:04:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 -- 172.21.15.122:0/1377899416 shutdown_connections 2024-09-12T02:04:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cee5400 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee92e0 0x557b6cefa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/1377899416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee89a0 0x557b6cee4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 -- 172.21.15.122:0/1377899416 >> 172.21.15.122:0/1377899416 conn(0x557b6cee4010 msgr2=0x557b6cef2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 -- 172.21.15.122:0/1377899416 shutdown_connections 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 -- 172.21.15.122:0/1377899416 wait complete. 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 -- start start 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.947+0000 7f3f1a44b740 1 Processor -- start 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee92e0 0x557b6cef8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a54c0 con 0x557b6ceee370 2024-09-12T02:04:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a6930 con 0x557b6cee92e0 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101a5b90 con 0x557b6cee89a0 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 10 client.0 _ll_register_callbacks cb 0x7f3f0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55692/0 (socket says 172.21.15.122:55692) 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 -- 172.21.15.122:0/2912329143 learned_addr learned my addr 172.21.15.122:0/2912329143 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f14c47640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee92e0 0x557b6cef8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15448640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 -- 172.21.15.122:0/2912329143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 msgr2=0x557b6cef7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 -- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee92e0 msgr2=0x557b6cef8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee92e0 0x557b6cef8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 -- 172.21.15.122:0/2912329143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6cef96e0 con 0x557b6ceee370 2024-09-12T02:04:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f14c47640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee92e0 0x557b6cef8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:04:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f15c49640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef90d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3f08000f80 tx=0x7f3f08003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ca2f79f0be9125 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3ef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:04:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/2912329143 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08021070 con 0x557b6ceee370 2024-09-12T02:04:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/2912329143 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f3f08004a40 con 0x557b6ceee370 2024-09-12T02:04:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.948+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0801cb40 con 0x557b6ceee370 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/2912329143 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3f08004c40 con 0x557b6ceee370 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 10 client.17997 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557b6cf698b0 con 0x557b6ceee370 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/2912329143 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0801cb40 con 0x557b6ceee370 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/2912329143 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f3f08027070 con 0x557b6ceee370 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 10 client.17997 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 10 client.17997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 20 client.17997 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cf6a280 0x557b6cf6c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 4 client.17997 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b6ce53d70 con 0x557b6cf6a280 2024-09-12T02:04:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f15448640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cf6a280 0x557b6cf6c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3ef8ff9640 10 client.17997 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:40.949+0000 7f3f15448640 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cf6a280 0x557b6cf6c6a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15b917cbfee1a50 server_cookie=8307ef656fce3424 in_seq=0 out_seq=0 2024-09-12T02:04:41.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 110616 20.87 MB/sec execute 148 sec latency 58.305 ms 2024-09-12T02:04:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:40 smithi151 ceph-mon[35229]: pgmap v884: 97 pgs: 97 active+clean; 528 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-09-12T02:04:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:40 smithi183 ceph-mon[35127]: pgmap v884: 97 pgs: 97 active+clean; 528 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-09-12T02:04:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:40 smithi122 ceph-mon[30925]: pgmap v884: 97 pgs: 97 active+clean; 528 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-09-12T02:04:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.270+0000 7f3ef8ff9640 1 -- 172.21.15.122:0/2912329143 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b6ce53d70 con 0x557b6cf6a280 2024-09-12T02:04:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.270+0000 7f3ef8ff9640 10 client.17997 handle_command_reply: tid=0 2024-09-12T02:04:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.270+0000 7f3f1a44b740 1 client.17997 shutdown 2024-09-12T02:04:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 msgr2=0x557b6cef90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef90d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3f08000f80 tx=0x7f3f08003970 comp rx=0 tx=0).stop 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 shutdown_connections 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x557b6cf6a280 0x557b6cf6c6a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b6ceee370 0x557b6cef90d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b6cee92e0 0x557b6cef8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 --2- 172.21.15.122:0/2912329143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b6cee89a0 0x557b6cef7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 >> 172.21.15.122:0/2912329143 conn(0x557b6cee4010 msgr2=0x557b6cee29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3ef8ff9640 0 client.17997 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 shutdown_connections 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 1 -- 172.21.15.122:0/2912329143 wait complete. 2024-09-12T02:04:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.271+0000 7f3f1a44b740 20 client.17997 trim_cache size 0 max 16384 2024-09-12T02:04:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:04:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:04:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:04:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a7040 msgr2=0x7f3efc0a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a7040 0x7f3efc0a9500 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c005ab0 tx=0x7f3f0c005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 msgr2=0x7f3f10198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 0x7f3f10198360 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000e9f0 tx=0x7f3f0000edf0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 shutdown_connections 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f3efc0a7040 0x7f3efc0a9500 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f101025f0 0x7f3f1019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f10101c30 0x7f3f10198360 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 --2- 172.21.15.122:0/1140468796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f10100a40 0x7f3f10197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 >> 172.21.15.122:0/1140468796 conn(0x7f3f100fc1e0 msgr2=0x7f3f100fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 shutdown_connections 2024-09-12T02:04:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:41.272+0000 7f3f176ac640 1 -- 172.21.15.122:0/1140468796 wait complete. 2024-09-12T02:04:42.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 111226 20.98 MB/sec execute 149 sec latency 163.965 ms 2024-09-12T02:04:43.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 111847 21.12 MB/sec execute 150 sec latency 221.043 ms 2024-09-12T02:04:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:42 smithi151 ceph-mon[35229]: pgmap v885: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 724 op/s 2024-09-12T02:04:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:42 smithi183 ceph-mon[35127]: pgmap v885: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 724 op/s 2024-09-12T02:04:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:42 smithi122 ceph-mon[30925]: pgmap v885: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 724 op/s 2024-09-12T02:04:44.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 112289 21.11 MB/sec execute 151 sec latency 101.102 ms 2024-09-12T02:04:45.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 112764 21.15 MB/sec execute 152 sec latency 58.342 ms 2024-09-12T02:04:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:44 smithi151 ceph-mon[35229]: pgmap v886: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 627 op/s 2024-09-12T02:04:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:44 smithi183 ceph-mon[35127]: pgmap v886: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 627 op/s 2024-09-12T02:04:45.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:44 smithi122 ceph-mon[30925]: pgmap v886: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 627 op/s 2024-09-12T02:04:46.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 113214 21.18 MB/sec execute 153 sec latency 123.109 ms 2024-09-12T02:04:47.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 113632 21.20 MB/sec execute 154 sec latency 60.005 ms 2024-09-12T02:04:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:46 smithi151 ceph-mon[35229]: pgmap v887: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.1 MiB/s wr, 643 op/s 2024-09-12T02:04:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:46 smithi183 ceph-mon[35127]: pgmap v887: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.1 MiB/s wr, 643 op/s 2024-09-12T02:04:47.238 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:46 smithi122 ceph-mon[30925]: pgmap v887: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.1 MiB/s wr, 643 op/s 2024-09-12T02:04:48.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 114089 21.18 MB/sec execute 155 sec latency 215.877 ms 2024-09-12T02:04:49.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 114670 21.20 MB/sec execute 156 sec latency 108.357 ms 2024-09-12T02:04:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:48 smithi151 ceph-mon[35229]: pgmap v888: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2024-09-12T02:04:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:48 smithi183 ceph-mon[35127]: pgmap v888: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2024-09-12T02:04:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:48 smithi122 ceph-mon[30925]: pgmap v888: 97 pgs: 97 active+clean; 531 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2024-09-12T02:04:50.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 115401 21.34 MB/sec execute 157 sec latency 76.650 ms 2024-09-12T02:04:51.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 116140 21.50 MB/sec execute 158 sec latency 49.422 ms 2024-09-12T02:04:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:50 smithi151 ceph-mon[35229]: pgmap v889: 97 pgs: 97 active+clean; 535 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 669 op/s 2024-09-12T02:04:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:50 smithi183 ceph-mon[35127]: pgmap v889: 97 pgs: 97 active+clean; 535 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 669 op/s 2024-09-12T02:04:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:50 smithi122 ceph-mon[30925]: pgmap v889: 97 pgs: 97 active+clean; 535 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 669 op/s 2024-09-12T02:04:52.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 116814 21.56 MB/sec execute 159 sec latency 46.566 ms 2024-09-12T02:04:53.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 117131 21.59 MB/sec execute 160 sec latency 256.637 ms 2024-09-12T02:04:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:52 smithi151 ceph-mon[35229]: pgmap v890: 97 pgs: 97 active+clean; 538 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.6 MiB/s wr, 656 op/s 2024-09-12T02:04:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:52 smithi183 ceph-mon[35127]: pgmap v890: 97 pgs: 97 active+clean; 538 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.6 MiB/s wr, 656 op/s 2024-09-12T02:04:53.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:52 smithi122 ceph-mon[30925]: pgmap v890: 97 pgs: 97 active+clean; 538 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.6 MiB/s wr, 656 op/s 2024-09-12T02:04:54.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 117589 21.59 MB/sec execute 161 sec latency 202.983 ms 2024-09-12T02:04:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 -- 172.21.15.122:0/1528682175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4102d50 msgr2=0x7f01d41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 --2- 172.21.15.122:0/1528682175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4102d50 0x7f01d41031d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009900 tx=0x7f01c802a410 comp rx=0 tx=0).stop 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 -- 172.21.15.122:0/1528682175 shutdown_connections 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 --2- 172.21.15.122:0/1528682175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01d4103710 0x7f01d4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 --2- 172.21.15.122:0/1528682175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4102d50 0x7f01d41031d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 --2- 172.21.15.122:0/1528682175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 0x7f01d4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 -- 172.21.15.122:0/1528682175 >> 172.21.15.122:0/1528682175 conn(0x7f01d40fd2e0 msgr2=0x7f01d40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 -- 172.21.15.122:0/1528682175 shutdown_connections 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 -- 172.21.15.122:0/1528682175 wait complete. 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.382+0000 7f01da566640 1 Processor -- start 2024-09-12T02:04:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 -- start start 2024-09-12T02:04:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 0x7f01d41079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01d4102d50 0x7f01d4107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 0x7f01d4104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01d410d810 con 0x7f01d4103710 2024-09-12T02:04:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01d410d690 con 0x7f01d4101b60 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01da566640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01d410d990 con 0x7f01d4102d50 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d8b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 0x7f01d4104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d8b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 0x7f01d4104520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46902/0 (socket says 172.21.15.122:46902) 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d8b03640 1 -- 172.21.15.122:0/1757985838 learned_addr learned my addr 172.21.15.122:0/1757985838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 0x7f01d41079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01d4102d50 0x7f01d4107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d3fff640 1 -- 172.21.15.122:0/1757985838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01d4102d50 msgr2=0x7f01d4107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01d4102d50 0x7f01d4107ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d3fff640 1 -- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 msgr2=0x7f01d4104520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 0x7f01d4104520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d3fff640 1 -- 172.21.15.122:0/1757985838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4104de0 con 0x7f01d4101b60 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.383+0000 7f01d8b03640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 0x7f01d4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.384+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 0x7f01d41079b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e9c0 tx=0x7f01c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.384+0000 7f01d17fa640 1 -- 172.21.15.122:0/1757985838 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c400ccc0 con 0x7f01d4101b60 2024-09-12T02:04:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.384+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d41a6440 con 0x7f01d4101b60 2024-09-12T02:04:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.384+0000 7f01d17fa640 1 -- 172.21.15.122:0/1757985838 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c40035f0 con 0x7f01d4101b60 2024-09-12T02:04:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.384+0000 7f01d17fa640 1 -- 172.21.15.122:0/1757985838 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c400ce60 con 0x7f01d4101b60 2024-09-12T02:04:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.384+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d41a6840 con 0x7f01d4101b60 2024-09-12T02:04:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.387+0000 7f01d17fa640 1 -- 172.21.15.122:0/1757985838 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f01c40037a0 con 0x7f01d4101b60 2024-09-12T02:04:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 Processor -- start 2024-09-12T02:04:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 -- start start 2024-09-12T02:04:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f609a0 0x559a12f60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 0x559a12f68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559a12ec42f0 con 0x559a12f609a0 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559a12e6c270 con 0x559a12f612e0 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01dd3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a12ec02b0 con 0x559a12f66370 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d8b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f609a0 0x559a12f60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 0x559a12f68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 0x559a12f68830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33514/0 (socket says 172.21.15.122:33514) 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d8b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f609a0 0x559a12f60da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46914/0 (socket says 172.21.15.122:46914) 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d37fe640 1 -- 172.21.15.122:0/2775696427 learned_addr learned my addr 172.21.15.122:0/2775696427 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d3fff640 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.388+0000 7f01d17fa640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d37fe640 1 -- 172.21.15.122:0/2775696427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 msgr2=0x559a12f65da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d37fe640 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f65da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d37fe640 1 -- 172.21.15.122:0/2775696427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f609a0 msgr2=0x559a12f60da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d37fe640 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f609a0 0x559a12f60da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d37fe640 1 -- 172.21.15.122:0/2775696427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a12e6b340 con 0x559a12f66370 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d17fa640 1 -- 172.21.15.122:0/1757985838 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f01c40cf450 con 0x7f01d4101b60 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d8b03640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01d37fe640 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 0x559a12f68830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c00 tx=0x7f01c8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38088ea0937618ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01ba7fc640 1 -- 172.21.15.122:0/2775696427 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035040 con 0x559a12f66370 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01ba7fc640 1 -- 172.21.15.122:0/2775696427 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c8005520 con 0x559a12f66370 2024-09-12T02:04:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.389+0000 7f01ba7fc640 1 -- 172.21.15.122:0/2775696427 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8031910 con 0x559a12f66370 2024-09-12T02:04:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 -- 172.21.15.122:0/2775696427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 msgr2=0x559a12f68830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01d8b03640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005c40 tx=0x7f01cc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 0x559a12f68830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c00 tx=0x7f01c8004040 comp rx=0 tx=0).stop 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 -- 172.21.15.122:0/2775696427 shutdown_connections 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f66370 0x559a12f68830 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 --2- 172.21.15.122:0/2775696427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f609a0 0x559a12f60da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 -- 172.21.15.122:0/2775696427 >> 172.21.15.122:0/2775696427 conn(0x559a12f5c010 msgr2=0x559a12f5c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 -- 172.21.15.122:0/2775696427 shutdown_connections 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 -- 172.21.15.122:0/2775696427 wait complete. 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.390+0000 7f01dd3df740 1 -- start start 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 Processor -- start 2024-09-12T02:04:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f6b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559a12f692e0 con 0x559a12f66370 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559a12e4d290 con 0x559a12f612e0 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a12f69460 con 0x559a12f609a0 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01d3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01d8b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01d8b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f72110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46928/0 (socket says 172.21.15.122:46928) 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01d8b03640 1 -- 172.21.15.122:0/1658118424 learned_addr learned my addr 172.21.15.122:0/1658118424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f6b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.391+0000 7f01dd3df740 10 client.0 _ll_register_callbacks cb 0x559a12e7e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01d3fff640 1 -- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 msgr2=0x559a12f6b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f6b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01d3fff640 1 -- 172.21.15.122:0/1658118424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 msgr2=0x559a12f72110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f72110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01d3fff640 1 -- 172.21.15.122:0/1658118424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a12bd7210 con 0x559a12f609a0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6ab90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x559a12f61d10 tx=0x7f01c409b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afdc4b1e902b143 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01b37fe640 1 -- 172.21.15.122:0/1658118424 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c409bcb0 con 0x559a12f609a0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01b37fe640 1 -- 172.21.15.122:0/1658118424 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f01c4003b10 con 0x559a12f609a0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c8005520 con 0x559a12f609a0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01b37fe640 1 -- 172.21.15.122:0/1658118424 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f01c40a0060 con 0x559a12f609a0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01dd3df740 10 client.27025 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559a12ca3620 con 0x559a12f609a0 2024-09-12T02:04:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.392+0000 7f01b37fe640 1 -- 172.21.15.122:0/1658118424 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c8005520 con 0x559a12f609a0 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01b37fe640 1 -- 172.21.15.122:0/1658118424 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f01c409b100 con 0x559a12f609a0 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01dd3df740 10 client.27025 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01dd3df740 10 client.27025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01dd3df740 20 client.27025 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12f767b0 0x559a12f78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01dd3df740 4 client.27025 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a12ed1a20 con 0x559a12f767b0 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12f767b0 0x559a12f78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01b37fe640 10 client.27025 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.393+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12f767b0 0x559a12f78ba0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e32d4873d75337c server_cookie=9d499f67a3dcb8f2 in_seq=0 out_seq=0 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.394+0000 7f01b37fe640 1 -- 172.21.15.122:0/1658118424 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a12ed1a20 con 0x559a12f767b0 2024-09-12T02:04:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.394+0000 7f01b37fe640 10 client.27025 handle_command_reply: tid=0 2024-09-12T02:04:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 client.27025 shutdown 2024-09-12T02:04:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 msgr2=0x559a12f6ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6ab90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x559a12f61d10 tx=0x7f01c409b8f0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 shutdown_connections 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12f767b0 0x559a12f78ba0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f72110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f612e0 0x559a12f6b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1658118424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6ab90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 >> 172.21.15.122:0/1658118424 conn(0x559a12f5c010 msgr2=0x559a12f5d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.395+0000 7f01b37fe640 0 client.27025 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.396+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 shutdown_connections 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.396+0000 7f01dd3df740 1 -- 172.21.15.122:0/1658118424 wait complete. 2024-09-12T02:04:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.396+0000 7f01dd3df740 20 client.27025 trim_cache size 0 max 16384 2024-09-12T02:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 Processor -- start 2024-09-12T02:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 -- start start 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 0x559a12f720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f66370 0x559a12f5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41b1dd0 con 0x559a12f612e0 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41af6a0 con 0x559a12f66370 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01dd3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41ad4d0 con 0x559a12f609a0 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 0x559a12f720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 0x559a12f720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46934/0 (socket says 172.21.15.122:46934) 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 -- 172.21.15.122:0/3205828181 learned_addr learned my addr 172.21.15.122:0/3205828181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d3fff640 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d8b03640 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f66370 0x559a12f5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 -- 172.21.15.122:0/3205828181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 msgr2=0x559a12f6b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 -- 172.21.15.122:0/3205828181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f66370 msgr2=0x559a12f5cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f66370 0x559a12f5cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d37fe640 1 -- 172.21.15.122:0/3205828181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a12f69560 con 0x559a12f612e0 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.413+0000 7f01d3fff640 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.414+0000 7f01d37fe640 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 0x559a12f720d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x559a12f5ce90 tx=0x7f01c8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29a01d8ef79581e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.414+0000 7f01b9ffb640 1 -- 172.21.15.122:0/3205828181 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035040 con 0x559a12f612e0 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.414+0000 7f01b9ffb640 1 -- 172.21.15.122:0/3205828181 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c8031d90 con 0x559a12f612e0 2024-09-12T02:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.414+0000 7f01b9ffb640 1 -- 172.21.15.122:0/3205828181 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8003de0 con 0x559a12f612e0 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 -- 172.21.15.122:0/3205828181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 msgr2=0x559a12f720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 0x559a12f720d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x559a12f5ce90 tx=0x7f01c8005fd0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 -- 172.21.15.122:0/3205828181 shutdown_connections 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f66370 0x559a12f5cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f612e0 0x559a12f720d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- 172.21.15.122:0/3205828181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f609a0 0x559a12f6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 -- 172.21.15.122:0/3205828181 >> 172.21.15.122:0/3205828181 conn(0x559a12f5c010 msgr2=0x559a12f6a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 -- 172.21.15.122:0/3205828181 shutdown_connections 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 -- 172.21.15.122:0/3205828181 wait complete. 2024-09-12T02:04:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 -- start start 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 Processor -- start 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f609a0 0x559a12f6f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 0x559a12f6fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.415+0000 7f01dd3df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f70e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01dd3df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41adb60 con 0x559a12f66370 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01dd3df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41aefd0 con 0x559a12f609a0 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01dd3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41ae230 con 0x559a12f612e0 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01dd3df740 10 client.0 _ll_register_callbacks cb 0x7f01c40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 0x559a12f6fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f609a0 0x559a12f6f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 0x559a12f6fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33536/0 (socket says 172.21.15.122:33536) 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 -- 172.21.15.122:0/1109684578 learned_addr learned my addr 172.21.15.122:0/1109684578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d8b03640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f70e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 -- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f609a0 msgr2=0x559a12f6f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f609a0 0x559a12f6f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 -- 172.21.15.122:0/1109684578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 msgr2=0x559a12f70e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f70e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 -- 172.21.15.122:0/1109684578 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a12f71430 con 0x559a12f612e0 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f609a0 0x559a12f6f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:04:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01d37fe640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 0x559a12f6fe10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f01c8031440 tx=0x7f01c80095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33de0c871e996c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01b37fe640 1 -- 172.21.15.122:0/1109684578 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035050 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.416+0000 7f01b37fe640 1 -- 172.21.15.122:0/1109684578 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f01c8003d20 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c8031d90 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01b37fe640 1 -- 172.21.15.122:0/1109684578 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f01c80316b0 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 10 client.27031 fetch_fsmap learned FSMap version 41 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559a12fe15c0 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01b37fe640 1 -- 172.21.15.122:0/1109684578 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c8031d90 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01b37fe640 1 -- 172.21.15.122:0/1109684578 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f01c805f020 con 0x559a12f612e0 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 10 client.27031 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:04:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 10 client.27031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 20 client.27031 populate_metadata read hostname 'smithi122' 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12fe1af0 0x559a12fe3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 4 client.27031 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a12f660f0 con 0x559a12fe1af0 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.417+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12fe1af0 0x559a12fe3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.418+0000 7f01b37fe640 10 client.27031 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.418+0000 7f01d3fff640 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12fe1af0 0x559a12fe3ee0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c782eb86aa614584 server_cookie=4735c7561a8df080 in_seq=0 out_seq=0 2024-09-12T02:04:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.670+0000 7f01b37fe640 1 -- 172.21.15.122:0/1109684578 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a12f660f0 con 0x559a12fe1af0 2024-09-12T02:04:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.670+0000 7f01b37fe640 10 client.27031 handle_command_reply: tid=0 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.670+0000 7f01dd3df740 1 client.27031 shutdown 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 msgr2=0x559a12f6fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 0x559a12f6fe10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f01c8031440 tx=0x7f01c80095b0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 shutdown_connections 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559a12fe1af0 0x559a12fe3ee0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a12f66370 0x559a12f70e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01b37fe640 0 client.27031 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a12f612e0 0x559a12f6fe10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 --2- 172.21.15.122:0/1109684578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a12f609a0 0x559a12f6f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 >> 172.21.15.122:0/1109684578 conn(0x559a12f5c010 msgr2=0x559a12f5a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 shutdown_connections 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 1 -- 172.21.15.122:0/1109684578 wait complete. 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.671+0000 7f01dd3df740 20 client.27031 trim_cache size 0 max 16384 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:04:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:04:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01bc0a2c30 msgr2=0x7f01bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005c40 tx=0x7f01cc004ab0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 msgr2=0x7f01d41079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 0x7f01d41079b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e9c0 tx=0x7f01c400ee80 comp rx=0 tx=0).stop 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 shutdown_connections 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01d4103710 0x7f01d4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01d4102d50 0x7f01d4107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 --2- 172.21.15.122:0/1757985838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01d4101b60 0x7f01d41079b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 >> 172.21.15.122:0/1757985838 conn(0x7f01d40fd2e0 msgr2=0x7f01d4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 shutdown_connections 2024-09-12T02:04:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:04:54.672+0000 7f01da566640 1 -- 172.21.15.122:0/1757985838 wait complete. 2024-09-12T02:04:55.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 118154 21.68 MB/sec execute 162 sec latency 173.822 ms 2024-09-12T02:04:55.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:54 smithi151 ceph-mon[35229]: pgmap v891: 97 pgs: 97 active+clean; 544 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 641 op/s 2024-09-12T02:04:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:54 smithi183 ceph-mon[35127]: pgmap v891: 97 pgs: 97 active+clean; 544 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 641 op/s 2024-09-12T02:04:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:54 smithi122 ceph-mon[30925]: pgmap v891: 97 pgs: 97 active+clean; 544 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 641 op/s 2024-09-12T02:04:56.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 118685 21.66 MB/sec execute 163 sec latency 63.538 ms 2024-09-12T02:04:57.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 119397 21.75 MB/sec execute 164 sec latency 73.598 ms 2024-09-12T02:04:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:56 smithi151 ceph-mon[35229]: pgmap v892: 97 pgs: 97 active+clean; 540 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 564 op/s 2024-09-12T02:04:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:56 smithi183 ceph-mon[35127]: pgmap v892: 97 pgs: 97 active+clean; 540 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 564 op/s 2024-09-12T02:04:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:56 smithi122 ceph-mon[30925]: pgmap v892: 97 pgs: 97 active+clean; 540 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 564 op/s 2024-09-12T02:04:58.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 119888 21.81 MB/sec execute 165 sec latency 141.688 ms 2024-09-12T02:04:58.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:04:58.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:04:58.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:04:59.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 120294 21.79 MB/sec execute 166 sec latency 415.720 ms 2024-09-12T02:04:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:04:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:04:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:04:59.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:04:58 smithi151 ceph-mon[35229]: pgmap v893: 97 pgs: 97 active+clean; 542 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2024-09-12T02:04:59.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:04:59.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:04:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:04:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:04:58 smithi183 ceph-mon[35127]: pgmap v893: 97 pgs: 97 active+clean; 542 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2024-09-12T02:04:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:04:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:04:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:04:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:04:58 smithi122 ceph-mon[30925]: pgmap v893: 97 pgs: 97 active+clean; 542 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2024-09-12T02:05:00.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 120911 21.87 MB/sec execute 167 sec latency 73.347 ms 2024-09-12T02:05:01.067 INFO:tasks.workunit.client.0.smithi122.stdout: 10 121502 21.92 MB/sec execute 168 sec latency 74.318 ms 2024-09-12T02:05:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:00 smithi151 ceph-mon[35229]: pgmap v894: 97 pgs: 97 active+clean; 515 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 668 op/s 2024-09-12T02:05:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:00 smithi183 ceph-mon[35127]: pgmap v894: 97 pgs: 97 active+clean; 515 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 668 op/s 2024-09-12T02:05:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:00 smithi122 ceph-mon[30925]: pgmap v894: 97 pgs: 97 active+clean; 515 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 668 op/s 2024-09-12T02:05:02.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 122102 22.03 MB/sec execute 169 sec latency 77.955 ms 2024-09-12T02:05:03.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 122716 22.06 MB/sec execute 170 sec latency 58.496 ms 2024-09-12T02:05:03.071 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:05:03.071 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-09-12T02:05:03.072 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:03.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:02 smithi151 ceph-mon[35229]: pgmap v895: 97 pgs: 97 active+clean; 515 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 607 op/s 2024-09-12T02:05:03.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:02 smithi122 ceph-mon[30925]: pgmap v895: 97 pgs: 97 active+clean; 515 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 607 op/s 2024-09-12T02:05:03.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:02 smithi183 ceph-mon[35127]: pgmap v895: 97 pgs: 97 active+clean; 515 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 607 op/s 2024-09-12T02:05:03.383 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:04.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 -- 172.21.15.122:0/249366752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 msgr2=0x7fe0c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 --2- 172.21.15.122:0/249366752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 0x7fe0c8101200 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009900 tx=0x7fe0c402a480 comp rx=0 tx=0).stop 2024-09-12T02:05:04.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 -- 172.21.15.122:0/249366752 shutdown_connections 2024-09-12T02:05:04.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 --2- 172.21.15.122:0/249366752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 --2- 172.21.15.122:0/249366752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c8102570 0x7fe0c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 --2- 172.21.15.122:0/249366752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 0x7fe0c8101200 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 -- 172.21.15.122:0/249366752 >> 172.21.15.122:0/249366752 conn(0x7fe0c80fc7d0 msgr2=0x7fe0c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 -- 172.21.15.122:0/249366752 shutdown_connections 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.013+0000 7fe0d0635640 1 -- 172.21.15.122:0/249366752 wait complete. 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.014+0000 7fe0d0635640 1 Processor -- start 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.014+0000 7fe0d0635640 1 -- start start 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.014+0000 7fe0d0635640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 0x7fe0c81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.014+0000 7fe0d0635640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c8102570 0x7fe0c81a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0d0635640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0d0635640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8110f80 con 0x7fe0c8100e20 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0d0635640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8110e00 con 0x7fe0c8102570 2024-09-12T02:05:04.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0d0635640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8111100 con 0x7fe0c8102ef0 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:48934/0 (socket says 172.21.15.122:48934) 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 -- 172.21.15.122:0/3555393512 learned_addr learned my addr 172.21.15.122:0/3555393512 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 -- 172.21.15.122:0/3555393512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c8102570 msgr2=0x7fe0c81a0960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c8102570 0x7fe0c81a0960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 -- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 msgr2=0x7fe0c81a0420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:05:04.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 0x7fe0c81a0420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.015+0000 7fe0cebd2640 1 -- 172.21.15.122:0/3555393512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c819ad10 con 0x7fe0c8102ef0 2024-09-12T02:05:04.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.016+0000 7fe0cebd2640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c819a5e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000eb10 tx=0x7fe0c000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.016+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0004b20 con 0x7fe0c8102ef0 2024-09-12T02:05:04.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.017+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0c0004cc0 con 0x7fe0c8102ef0 2024-09-12T02:05:04.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.017+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00107b0 con 0x7fe0c8102ef0 2024-09-12T02:05:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.017+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c819b000 con 0x7fe0c8102ef0 2024-09-12T02:05:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.017+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c806dc20 con 0x7fe0c8102ef0 2024-09-12T02:05:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.018+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c81021b0 con 0x7fe0c8102ef0 2024-09-12T02:05:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.019+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe0c0005020 con 0x7fe0c8102ef0 2024-09-12T02:05:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.020+0000 7fe0bf7fe640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe0b40a2cd0 0x7fe0b40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.020+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe0c00cf640 con 0x7fe0c8102ef0 2024-09-12T02:05:04.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.020+0000 7fe0ce3d1640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe0b40a2cd0 0x7fe0b40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:04.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.021+0000 7fe0ce3d1640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe0b40a2cd0 0x7fe0b40a5190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40314e0 tx=0x7fe0c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:04.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.025+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0c009c5a0 con 0x7fe0c8102ef0 2024-09-12T02:05:04.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 123144 22.08 MB/sec execute 171 sec latency 222.902 ms 2024-09-12T02:05:04.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.266+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0c806dee0 con 0x7fe0c8102ef0 2024-09-12T02:05:04.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.266+0000 7fe0bf7fe640 1 -- 172.21.15.122:0/3555393512 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7fe0c009bea0 con 0x7fe0c8102ef0 2024-09-12T02:05:04.267 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:05:04.267 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:05:04.267 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:05:04.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe0b40a2cd0 msgr2=0x7fe0b40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe0b40a2cd0 0x7fe0b40a5190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40314e0 tx=0x7fe0c4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 msgr2=0x7fe0c819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c819a5e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000eb10 tx=0x7fe0c000c2d0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 shutdown_connections 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe0b40a2cd0 0x7fe0b40a5190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c8102ef0 0x7fe0c819a5e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c8102570 0x7fe0c81a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 --2- 172.21.15.122:0/3555393512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0c8100e20 0x7fe0c81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 >> 172.21.15.122:0/3555393512 conn(0x7fe0c80fc7d0 msgr2=0x7fe0c80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 shutdown_connections 2024-09-12T02:05:04.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:04.268+0000 7fe0d0635640 1 -- 172.21.15.122:0/3555393512 wait complete. 2024-09-12T02:05:04.596 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:04 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3555393512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:04.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:04 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3555393512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:04 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3555393512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:05.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 123760 22.11 MB/sec execute 172 sec latency 58.427 ms 2024-09-12T02:05:05.078 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-09-12T02:05:05.078 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-09-12T02:05:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:05.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:05 smithi122 ceph-mon[30925]: pgmap v896: 97 pgs: 97 active+clean; 525 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 737 op/s 2024-09-12T02:05:05.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:05 smithi151 ceph-mon[35229]: pgmap v896: 97 pgs: 97 active+clean; 525 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 737 op/s 2024-09-12T02:05:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:05 smithi183 ceph-mon[35127]: pgmap v896: 97 pgs: 97 active+clean; 525 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.1 MiB/s wr, 737 op/s 2024-09-12T02:05:05.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 -- 172.21.15.122:0/2971247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0102ef0 msgr2=0x7f81e010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:05.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/2971247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0102ef0 0x7f81e010da40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009ca0 tx=0x7f81d802a950 comp rx=0 tx=0).stop 2024-09-12T02:05:05.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 -- 172.21.15.122:0/2971247 shutdown_connections 2024-09-12T02:05:05.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/2971247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0102ef0 0x7f81e010da40 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/2971247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81e0102570 0x7f81e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/2971247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0100e20 0x7f81e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 -- 172.21.15.122:0/2971247 >> 172.21.15.122:0/2971247 conn(0x7f81e00fc7d0 msgr2=0x7f81e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 -- 172.21.15.122:0/2971247 shutdown_connections 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 -- 172.21.15.122:0/2971247 wait complete. 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 Processor -- start 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.928+0000 7f81e6f4f640 1 -- start start 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e6f4f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 0x7f81e01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e6f4f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0102570 0x7f81e0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e6f4f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81e0102ef0 0x7f81e019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e6f4f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0110f80 con 0x7f81e0100e20 2024-09-12T02:05:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e6f4f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0110e00 con 0x7f81e0102570 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e6f4f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0111100 con 0x7f81e0102ef0 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e4ceb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 0x7f81e01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0102570 0x7f81e0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e4ceb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 0x7f81e01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55448/0 (socket says 172.21.15.122:55448) 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e4ceb640 1 -- 172.21.15.122:0/3201259901 learned_addr learned my addr 172.21.15.122:0/3201259901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e54ec640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81e0102ef0 0x7f81e019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 -- 172.21.15.122:0/3201259901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81e0102ef0 msgr2=0x7f81e019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81e0102ef0 0x7f81e019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 -- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 msgr2=0x7f81e01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 0x7f81e01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 -- 172.21.15.122:0/3201259901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e019d3e0 con 0x7f81e0102570 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 0x7f81e01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:05.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.929+0000 7f81dffff640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0102570 0x7f81e0198d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00e9f0 tx=0x7f81cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:05.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.930+0000 7f81ddffb640 1 -- 172.21.15.122:0/3201259901 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00ccc0 con 0x7f81e0102570 2024-09-12T02:05:05.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.930+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e019d6d0 con 0x7f81e0102570 2024-09-12T02:05:05.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.930+0000 7f81ddffb640 1 -- 172.21.15.122:0/3201259901 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81cc010910 con 0x7f81e0102570 2024-09-12T02:05:05.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.930+0000 7f81ddffb640 1 -- 172.21.15.122:0/3201259901 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc018960 con 0x7f81e0102570 2024-09-12T02:05:05.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.930+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e01116e0 con 0x7f81e0102570 2024-09-12T02:05:05.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.933+0000 7f81ddffb640 1 -- 172.21.15.122:0/3201259901 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f81cc010ac0 con 0x7f81e0102570 2024-09-12T02:05:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.933+0000 7f81e9328740 1 Processor -- start 2024-09-12T02:05:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.933+0000 7f81e9328740 1 -- start start 2024-09-12T02:05:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e9328740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188874ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e9328740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874b080 0x561888755400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e9328740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x561888757d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e9328740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5618886b02a0 con 0x561888755940 2024-09-12T02:05:05.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e9328740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561888657b10 con 0x56188874a760 2024-09-12T02:05:05.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e9328740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5618886ac260 con 0x56188874b080 2024-09-12T02:05:05.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e54ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188874ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e54ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188874ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37580/0 (socket says 172.21.15.122:37580) 2024-09-12T02:05:05.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874b080 0x561888755400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e54ec640 1 -- 172.21.15.122:0/1217969463 learned_addr learned my addr 172.21.15.122:0/1217969463 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:05.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81dffff640 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x561888757d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 -- 172.21.15.122:0/1217969463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 msgr2=0x56188874ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188874ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 -- 172.21.15.122:0/1217969463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 msgr2=0x561888757d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x561888757d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 -- 172.21.15.122:0/1217969463 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561888421b90 con 0x56188874b080 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.934+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874b080 0x561888755400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f81d40114f0 tx=0x7f81d40119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc5ff15a0c784cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.935+0000 7f81ddffb640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81d00a7010 0x7f81d00a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.935+0000 7f81caffd640 1 -- 172.21.15.122:0/1217969463 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d401e070 con 0x56188874b080 2024-09-12T02:05:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.935+0000 7f81e54ec640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81d00a7010 0x7f81d00a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.935+0000 7f81caffd640 1 -- 172.21.15.122:0/1217969463 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d40035f0 con 0x56188874b080 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.935+0000 7f81ddffb640 1 -- 172.21.15.122:0/3201259901 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f81cc0cf370 con 0x7f81e0102570 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.935+0000 7f81caffd640 1 -- 172.21.15.122:0/1217969463 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d400f820 con 0x56188874b080 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 -- 172.21.15.122:0/1217969463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874b080 msgr2=0x561888755400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e54ec640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81d00a7010 0x7f81d00a94d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009ca0 tx=0x7f81d8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874b080 0x561888755400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f81d40114f0 tx=0x7f81d40119b0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 -- 172.21.15.122:0/1217969463 shutdown_connections 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x561888757d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874b080 0x561888755400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 --2- 172.21.15.122:0/1217969463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188874ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 -- 172.21.15.122:0/1217969463 >> 172.21.15.122:0/1217969463 conn(0x561888746cf0 msgr2=0x5618887470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 -- 172.21.15.122:0/1217969463 shutdown_connections 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.936+0000 7f81e9328740 1 -- 172.21.15.122:0/1217969463 wait complete. 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 -- start start 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 Processor -- start 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874a760 0x5618887527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x561888752d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x561888753270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561888758510 con 0x56188874a760 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5618886386e0 con 0x56188874b080 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.937+0000 7f81e9328740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561888758690 con 0x561888755940 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81e9328740 10 client.0 _ll_register_callbacks cb 0x56188865ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x561888752d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81e4ceb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874a760 0x5618887527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81e54ec640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x561888753270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x561888752d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37586/0 (socket says 172.21.15.122:37586) 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 -- 172.21.15.122:0/2869507924 learned_addr learned my addr 172.21.15.122:0/2869507924 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874a760 0x5618887527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55474/0 (socket says 172.21.15.122:55474) 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 -- 172.21.15.122:0/2869507924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 msgr2=0x561888753270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x561888753270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 -- 172.21.15.122:0/2869507924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874a760 msgr2=0x5618887527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874a760 0x5618887527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 -- 172.21.15.122:0/2869507924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618887619d0 con 0x56188874b080 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81dffff640 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x561888752d30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0a42d0 tx=0x7f81cc010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a0891cba954986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81bffff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:05:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81bffff640 1 -- 172.21.15.122:0/2869507924 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc09b5a0 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81bffff640 1 -- 172.21.15.122:0/2869507924 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f81cc09b090 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.938+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d40035f0 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81bffff640 1 -- 172.21.15.122:0/2869507924 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f81cc09bc40 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 10 client.27104 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561888764360 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81bffff640 1 -- 172.21.15.122:0/2869507924 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81cc09ff40 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81bffff640 1 -- 172.21.15.122:0/2869507924 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f81cc0ce030 con 0x56188874b080 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 10 client.27104 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 10 client.27104 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 20 client.27104 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561888765770 0x561888767ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 4 client.27104 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.939+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f81d40035f0 con 0x561888765770 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.940+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561888765770 0x561888767ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.940+0000 7f81bffff640 10 client.27104 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.940+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561888765770 0x561888767ba0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110e720152840c15 server_cookie=b43b9e211b295e9a in_seq=0 out_seq=0 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81bffff640 1 -- 172.21.15.122:0/2869507924 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f81d40035f0 con 0x561888765770 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81bffff640 10 client.27104 handle_command_reply: tid=0 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 client.27104 shutdown 2024-09-12T02:05:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 msgr2=0x561888752d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:05.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x561888752d30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0a42d0 tx=0x7f81cc010ee0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 shutdown_connections 2024-09-12T02:05:05.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561888765770 0x561888767ba0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x561888753270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x561888752d30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 --2- 172.21.15.122:0/2869507924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874a760 0x5618887527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 >> 172.21.15.122:0/2869507924 conn(0x561888746cf0 msgr2=0x561888745e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:05.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81bffff640 0 client.27104 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:05.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 shutdown_connections 2024-09-12T02:05:05.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 1 -- 172.21.15.122:0/2869507924 wait complete. 2024-09-12T02:05:05.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.941+0000 7f81e9328740 20 client.27104 trim_cache size 0 max 16384 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 Processor -- start 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 -- start start 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874a760 0x56188876d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x56188876d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x56188874fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01adc70 con 0x561888755940 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01ab540 con 0x56188874b080 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e9328740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01a9370 con 0x56188874a760 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e54ec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x56188874fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e54ec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x56188874fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55488/0 (socket says 172.21.15.122:55488) 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e54ec640 1 -- 172.21.15.122:0/2980615079 learned_addr learned my addr 172.21.15.122:0/2980615079 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81dffff640 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x56188876d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.961+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874a760 0x56188876d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81e54ec640 1 -- 172.21.15.122:0/2980615079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874a760 msgr2=0x56188876d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81e54ec640 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874a760 0x56188876d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81e54ec640 1 -- 172.21.15.122:0/2980615079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 msgr2=0x56188876d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81e54ec640 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x56188876d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81e54ec640 1 -- 172.21.15.122:0/2980615079 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618887517a0 con 0x561888755940 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81e54ec640 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x56188874fa70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f81d8035000 tx=0x7f81d8042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce1ba6c5ba5c824 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81ca7fc640 1 -- 172.21.15.122:0/2980615079 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8034040 con 0x561888755940 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81ca7fc640 1 -- 172.21.15.122:0/2980615079 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d8003cf0 con 0x561888755940 2024-09-12T02:05:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.962+0000 7f81ca7fc640 1 -- 172.21.15.122:0/2980615079 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8030900 con 0x561888755940 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 -- 172.21.15.122:0/2980615079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 msgr2=0x56188874fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x56188874fa70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f81d8035000 tx=0x7f81d8042000 comp rx=0 tx=0).stop 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 -- 172.21.15.122:0/2980615079 shutdown_connections 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561888755940 0x56188874fa70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874b080 0x56188876d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 --2- 172.21.15.122:0/2980615079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56188874a760 0x56188876d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 -- 172.21.15.122:0/2980615079 >> 172.21.15.122:0/2980615079 conn(0x561888746cf0 msgr2=0x561888754870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 -- 172.21.15.122:0/2980615079 shutdown_connections 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 -- 172.21.15.122:0/2980615079 wait complete. 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 -- start start 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.963+0000 7f81e9328740 1 Processor -- start 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188875d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 0x56188875de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x56188875a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01a9a00 con 0x56188874b080 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01aae70 con 0x56188874a760 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01aa0d0 con 0x561888755940 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e9328740 10 client.0 _ll_register_callbacks cb 0x7f81cc010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 0x56188875de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e4ceb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188875d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 0x56188875de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55492/0 (socket says 172.21.15.122:55492) 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 -- 172.21.15.122:0/3379146019 learned_addr learned my addr 172.21.15.122:0/3379146019 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e54ec640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x56188875a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 -- 172.21.15.122:0/3379146019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 msgr2=0x56188875a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x56188875a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 -- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 msgr2=0x56188875d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188875d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 -- 172.21.15.122:0/3379146019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188875adf0 con 0x56188874b080 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188875d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81e54ec640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x56188875a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81dffff640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 0x56188875de20 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c6a0 tx=0x7f81cc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33270bdd445c7e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81bffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.964+0000 7f81bffff640 1 -- 172.21.15.122:0/3379146019 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc09b5a0 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81bffff640 1 -- 172.21.15.122:0/3379146019 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f81cc0b3030 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d8003cf0 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81bffff640 1 -- 172.21.15.122:0/3379146019 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f81cc09ba10 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 10 client.18036 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5618887d0bd0 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81bffff640 1 -- 172.21.15.122:0/3379146019 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d8003cf0 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81bffff640 1 -- 172.21.15.122:0/3379146019 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f81cc0ce030 con 0x56188874b080 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 10 client.18036 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 10 client.18036 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 20 client.18036 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:05.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5618887d1310 0x5618887d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:05.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 4 client.18036 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-09-12T02:05:05.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x5618886bd9d0 con 0x5618887d1310 2024-09-12T02:05:05.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.965+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5618887d1310 0x5618887d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:05.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.966+0000 7f81bffff640 10 client.18036 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:05.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:05.966+0000 7f81e4ceb640 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5618887d1310 0x5618887d3730 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3250d38fe9f403 server_cookie=96c40971e583bf7 in_seq=0 out_seq=0 2024-09-12T02:05:06.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124462 22.24 MB/sec execute 173 sec latency 105.525 ms 2024-09-12T02:05:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81bffff640 1 -- 172.21.15.122:0/3379146019 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5618886bd9d0 con 0x5618887d1310 2024-09-12T02:05:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81bffff640 10 client.18036 handle_command_reply: tid=0 2024-09-12T02:05:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 client.18036 shutdown 2024-09-12T02:05:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 msgr2=0x56188875de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 0x56188875de20 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c6a0 tx=0x7f81cc00c7d0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 shutdown_connections 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5618887d1310 0x5618887d3730 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81bffff640 0 client.18036 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561888755940 0x56188875a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56188874b080 0x56188875de20 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 --2- 172.21.15.122:0/3379146019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56188874a760 0x56188875d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 >> 172.21.15.122:0/3379146019 conn(0x561888746cf0 msgr2=0x5618887527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 shutdown_connections 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.867+0000 7f81e9328740 1 -- 172.21.15.122:0/3379146019 wait complete. 2024-09-12T02:05:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e9328740 20 client.18036 trim_cache size 0 max 16384 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 66, 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "5d8f4fa9": { 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 66, 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:05:06.870 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 141.8, 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:05:06.871 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:06.872 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:05:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81d00a7010 msgr2=0x7f81d00a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81d00a7010 0x7f81d00a94d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009ca0 tx=0x7f81d8005c50 comp rx=0 tx=0).stop 2024-09-12T02:05:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0102570 msgr2=0x7f81e0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0102570 0x7f81e0198d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00e9f0 tx=0x7f81cc00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 shutdown_connections 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f81d00a7010 0x7f81d00a94d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81e0102ef0 0x7f81e019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e0102570 0x7f81e0198d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 --2- 172.21.15.122:0/3201259901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81e0100e20 0x7f81e01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.868+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 >> 172.21.15.122:0/3201259901 conn(0x7f81e00fc7d0 msgr2=0x7f81e01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.869+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 shutdown_connections 2024-09-12T02:05:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:06.869+0000 7f81e6f4f640 1 -- 172.21.15.122:0/3201259901 wait complete. 2024-09-12T02:05:07.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 22.11 MB/sec execute 174 sec latency 1091.997 ms 2024-09-12T02:05:07.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:06 smithi151 ceph-mon[35229]: pgmap v897: 97 pgs: 97 active+clean; 525 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-09-12T02:05:07.189 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:06 smithi122 ceph-mon[30925]: pgmap v897: 97 pgs: 97 active+clean; 525 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-09-12T02:05:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:06 smithi183 ceph-mon[35127]: pgmap v897: 97 pgs: 97 active+clean; 525 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-09-12T02:05:07.501 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5d8f4fa9', quiesce took 4.4 seconds. Will release after: 17.4 2024-09-12T02:05:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 -- 172.21.15.122:0/1772845274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c101b60 msgr2=0x7fc68c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 --2- 172.21.15.122:0/1772845274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c101b60 0x7fc68c101f60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009900 tx=0x7fc67c02a410 comp rx=0 tx=0).stop 2024-09-12T02:05:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 -- 172.21.15.122:0/1772845274 shutdown_connections 2024-09-12T02:05:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 --2- 172.21.15.122:0/1772845274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc68c103710 0x7fc68c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 --2- 172.21.15.122:0/1772845274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc68c102d50 0x7fc68c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 --2- 172.21.15.122:0/1772845274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c101b60 0x7fc68c101f60 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 -- 172.21.15.122:0/1772845274 >> 172.21.15.122:0/1772845274 conn(0x7fc68c0fd2e0 msgr2=0x7fc68c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 -- 172.21.15.122:0/1772845274 shutdown_connections 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 -- 172.21.15.122:0/1772845274 wait complete. 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 Processor -- start 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.772+0000 7fc6918ba640 1 -- start start 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc68c101b60 0x7fc68c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 0x7fc68c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc68c103710 0x7fc68c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10d8c0 con 0x7fc68c102d50 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10d740 con 0x7fc68c103710 2024-09-12T02:05:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10da40 con 0x7fc68c101b60 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 0x7fc68c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 0x7fc68c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55510/0 (socket says 172.21.15.122:55510) 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 -- 172.21.15.122:0/112848846 learned_addr learned my addr 172.21.15.122:0/112848846 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc68affd640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc68c101b60 0x7fc68c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc68c103710 0x7fc68c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 -- 172.21.15.122:0/112848846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc68c101b60 msgr2=0x7fc68c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc68c101b60 0x7fc68c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 -- 172.21.15.122:0/112848846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc68c103710 msgr2=0x7fc68c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc68c103710 0x7fc68c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 -- 172.21.15.122:0/112848846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1a1d80 con 0x7fc68c102d50 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6827fc640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 0x7fc68c19a6a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b4a0 tx=0x7fc67400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6908b8640 1 -- 172.21.15.122:0/112848846 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674014070 con 0x7fc68c102d50 2024-09-12T02:05:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6908b8640 1 -- 172.21.15.122:0/112848846 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc674010af0 con 0x7fc68c102d50 2024-09-12T02:05:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1a2070 con 0x7fc68c102d50 2024-09-12T02:05:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6908b8640 1 -- 172.21.15.122:0/112848846 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674017d10 con 0x7fc68c102d50 2024-09-12T02:05:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.773+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a2640 con 0x7fc68c102d50 2024-09-12T02:05:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.775+0000 7fc6908b8640 1 -- 172.21.15.122:0/112848846 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc674017070 con 0x7fc68c102d50 2024-09-12T02:05:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 Processor -- start 2024-09-12T02:05:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 -- start start 2024-09-12T02:05:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 0x5622e2dbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dc3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5622e2d1f2f0 con 0x5622e2dbb9a0 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622e2cc7270 con 0x5622e2dc1370 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc69465a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622e2d1b2b0 con 0x5622e2dbc2e0 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 0x5622e2dbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 0x5622e2dbbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55514/0 (socket says 172.21.15.122:55514) 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 -- 172.21.15.122:0/1758955384 learned_addr learned my addr 172.21.15.122:0/1758955384 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc6908b8640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6780a2cb0 0x7fc6780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68affd640 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc6827fc640 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dc3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6780a2cb0 0x7fc6780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc6908b8640 1 -- 172.21.15.122:0/112848846 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fc6740ce450 con 0x7fc68c102d50 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 -- 172.21.15.122:0/1758955384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbc2e0 msgr2=0x5622e2dc0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 -- 172.21.15.122:0/1758955384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 msgr2=0x5622e2dc3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dc3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.776+0000 7fc68b7fe640 1 -- 172.21.15.122:0/1758955384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e2cc6340 con 0x5622e2dbb9a0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 0x5622e2dbbda0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc68400cb20 tx=0x7fc6840072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a8d286be8ab373 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc681ffb640 1 -- 172.21.15.122:0/1758955384 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684015070 con 0x5622e2dbb9a0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc681ffb640 1 -- 172.21.15.122:0/1758955384 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684016040 con 0x5622e2dbb9a0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc681ffb640 1 -- 172.21.15.122:0/1758955384 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68401e020 con 0x5622e2dbb9a0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6780a2cb0 0x7fc6780a5170 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc68401f000 tx=0x7fc68401b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc69465a740 1 -- 172.21.15.122:0/1758955384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 msgr2=0x5622e2dbbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc69465a740 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 0x5622e2dbbda0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc68400cb20 tx=0x7fc6840072b0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.777+0000 7fc69465a740 1 -- 172.21.15.122:0/1758955384 shutdown_connections 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dc3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 --2- 172.21.15.122:0/1758955384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbb9a0 0x5622e2dbbda0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- 172.21.15.122:0/1758955384 >> 172.21.15.122:0/1758955384 conn(0x5622e2db7010 msgr2=0x5622e2db7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- 172.21.15.122:0/1758955384 shutdown_connections 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- 172.21.15.122:0/1758955384 wait complete. 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- start start 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 Processor -- start 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dc5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dcd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5622e2dc4340 con 0x5622e2dbc2e0 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622e2ca8290 con 0x5622e2dc1370 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622e2dc44c0 con 0x5622e2dbb9a0 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc69465a740 10 client.0 _ll_register_callbacks cb 0x5622e2cd9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc6827fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc6827fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55522/0 (socket says 172.21.15.122:55522) 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc6827fc640 1 -- 172.21.15.122:0/2405990128 learned_addr learned my addr 172.21.15.122:0/2405990128 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.778+0000 7fc68affd640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dc5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dcd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc6827fc640 1 -- 172.21.15.122:0/2405990128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 msgr2=0x5622e2dc5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc6827fc640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dc5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc6827fc640 1 -- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 msgr2=0x5622e2dcd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc6827fc640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dcd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc6827fc640 1 -- 172.21.15.122:0/2405990128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e2a32210 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc6827fc640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc60d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc674004700 tx=0x7fc674017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504fe5ee99c2db6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc66affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc66affd640 1 -- 172.21.15.122:0/2405990128 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67409acb0 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc66affd640 1 -- 172.21.15.122:0/2405990128 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fc674017360 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc684016040 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc66affd640 1 -- 172.21.15.122:0/2405990128 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc67409f060 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 10 client.18048 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5622e2afe620 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc66affd640 1 -- 172.21.15.122:0/2405990128 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684016040 con 0x5622e2dbc2e0 2024-09-12T02:05:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc66affd640 1 -- 172.21.15.122:0/2405990128 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc6740ce0f0 con 0x5622e2dbc2e0 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 10 client.18048 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 20 client.18048 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2dd1860 0x5622e2dd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 4 client.18048 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.779+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e2d2ca20 con 0x5622e2dd1860 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.780+0000 7fc68affd640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2dd1860 0x5622e2dd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.780+0000 7fc68affd640 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2dd1860 0x5622e2dd3c50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9907a62c264136e1 server_cookie=bb93c055542426d0 in_seq=0 out_seq=0 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.780+0000 7fc66affd640 10 client.18048 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc66affd640 1 -- 172.21.15.122:0/2405990128 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622e2d2ca20 con 0x5622e2dd1860 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc66affd640 10 client.18048 handle_command_reply: tid=0 2024-09-12T02:05:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 client.18048 shutdown 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 msgr2=0x5622e2dc60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc60d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc674004700 tx=0x7fc674017730 comp rx=0 tx=0).stop 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 shutdown_connections 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2dd1860 0x5622e2dd3c50 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dc1370 0x5622e2dcd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc66affd640 0 client.18048 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dbc2e0 0x5622e2dc60d0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 --2- 172.21.15.122:0/2405990128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dc5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 >> 172.21.15.122:0/2405990128 conn(0x5622e2db7010 msgr2=0x5622e2db8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 shutdown_connections 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 1 -- 172.21.15.122:0/2405990128 wait complete. 2024-09-12T02:05:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.781+0000 7fc69465a740 20 client.18048 trim_cache size 0 max 16384 2024-09-12T02:05:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.795+0000 7fc69465a740 1 Processor -- start 2024-09-12T02:05:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 -- start start 2024-09-12T02:05:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2db8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dd8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1adc50 con 0x5622e2dc1370 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1ab520 con 0x5622e2dbc2e0 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc69465a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1a9350 con 0x5622e2dbb9a0 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dd8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dd8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55528/0 (socket says 172.21.15.122:55528) 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2db8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 -- 172.21.15.122:0/2908334080 learned_addr learned my addr 172.21.15.122:0/2908334080 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc6827fc640 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 -- 172.21.15.122:0/2908334080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 msgr2=0x5622e2db8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2db8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 -- 172.21.15.122:0/2908334080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 msgr2=0x5622e2dcd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 -- 172.21.15.122:0/2908334080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e2dd3fc0 con 0x5622e2dc1370 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dd8500 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc6840077b0 tx=0x7fc684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5da1c3fad84b8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc6817fa640 1 -- 172.21.15.122:0/2908334080 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684015040 con 0x5622e2dc1370 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc6817fa640 1 -- 172.21.15.122:0/2908334080 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6840098f0 con 0x5622e2dc1370 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.796+0000 7fc6817fa640 1 -- 172.21.15.122:0/2908334080 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68401e040 con 0x5622e2dc1370 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 -- 172.21.15.122:0/2908334080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 msgr2=0x5622e2dd8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dd8500 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc6840077b0 tx=0x7fc684004a40 comp rx=0 tx=0).stop 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 -- 172.21.15.122:0/2908334080 shutdown_connections 2024-09-12T02:05:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dd8500 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 --2- 172.21.15.122:0/2908334080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2db8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 -- 172.21.15.122:0/2908334080 >> 172.21.15.122:0/2908334080 conn(0x5622e2db7010 msgr2=0x5622e2db73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 -- 172.21.15.122:0/2908334080 shutdown_connections 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 -- 172.21.15.122:0/2908334080 wait complete. 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.797+0000 7fc69465a740 1 -- start start 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 Processor -- start 2024-09-12T02:05:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dcaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dcc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1a99e0 con 0x5622e2dc1370 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1aae50 con 0x5622e2dbc2e0 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1aa0b0 con 0x5622e2dbb9a0 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc69465a740 10 client.0 _ll_register_callbacks cb 0x7fc6740092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dcc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dcc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55534/0 (socket says 172.21.15.122:55534) 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 -- 172.21.15.122:0/649355197 learned_addr learned my addr 172.21.15.122:0/649355197 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68affd640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dcaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc6827fc640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 -- 172.21.15.122:0/649355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 msgr2=0x5622e2dcaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dcaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 -- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 msgr2=0x5622e2dcb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 -- 172.21.15.122:0/649355197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e2dcc630 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc68b7fe640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dcc020 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc684009720 tx=0x7fc68400acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c68c2d251fb6dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc66affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.798+0000 7fc66affd640 1 -- 172.21.15.122:0/649355197 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68401e030 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc66affd640 1 -- 172.21.15.122:0/649355197 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fc68401a7e0 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6840098f0 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc66affd640 1 -- 172.21.15.122:0/649355197 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc684015070 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 10 client.18054 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5622e2e3c790 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc66affd640 1 -- 172.21.15.122:0/649355197 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6840098f0 con 0x5622e2dc1370 2024-09-12T02:05:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc66affd640 1 -- 172.21.15.122:0/649355197 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fc68402a030 con 0x5622e2dc1370 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 10 client.18054 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 20 client.18054 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2e3d0a0 0x5622e2e3f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 4 client.18054 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622e2d26d70 con 0x5622e2e3d0a0 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.799+0000 7fc68affd640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2e3d0a0 0x5622e2e3f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.800+0000 7fc68affd640 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2e3d0a0 0x5622e2e3f4c0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c072f3e074122cb server_cookie=2330a8aa12606689 in_seq=0 out_seq=0 2024-09-12T02:05:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.800+0000 7fc66affd640 10 client.18054 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc66affd640 1 -- 172.21.15.122:0/649355197 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622e2d26d70 con 0x5622e2e3d0a0 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc66affd640 10 client.18054 handle_command_reply: tid=0 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 client.18054 shutdown 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 msgr2=0x5622e2dcc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dcc020 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc684009720 tx=0x7fc68400acb0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 shutdown_connections 2024-09-12T02:05:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5622e2e3d0a0 0x5622e2e3f4c0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5622e2dc1370 0x5622e2dcc020 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622e2dbc2e0 0x5622e2dcb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc66affd640 0 client.18054 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 --2- 172.21.15.122:0/649355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622e2dbb9a0 0x5622e2dcaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 >> 172.21.15.122:0/649355197 conn(0x5622e2db7010 msgr2=0x5622e2db7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 shutdown_connections 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 1 -- 172.21.15.122:0/649355197 wait complete. 2024-09-12T02:05:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.965+0000 7fc69465a740 20 client.18054 trim_cache size 0 max 16384 2024-09-12T02:05:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:05:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:05:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:05:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6780a2cb0 msgr2=0x7fc6780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6780a2cb0 0x7fc6780a5170 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc68401f000 tx=0x7fc68401b000 comp rx=0 tx=0).stop 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 msgr2=0x7fc68c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 0x7fc68c19a6a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b4a0 tx=0x7fc67400b960 comp rx=0 tx=0).stop 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 shutdown_connections 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc6780a2cb0 0x7fc6780a5170 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc68c103710 0x7fc68c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc68c102d50 0x7fc68c19a6a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 --2- 172.21.15.122:0/112848846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc68c101b60 0x7fc68c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 >> 172.21.15.122:0/112848846 conn(0x7fc68c0fd2e0 msgr2=0x7fc68c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 shutdown_connections 2024-09-12T02:05:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:07.966+0000 7fc6918ba640 1 -- 172.21.15.122:0/112848846 wait complete. 2024-09-12T02:05:08.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.98 MB/sec execute 175 sec latency 2092.067 ms 2024-09-12T02:05:09.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.86 MB/sec execute 176 sec latency 3092.144 ms 2024-09-12T02:05:09.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:08 smithi151 ceph-mon[35229]: pgmap v898: 97 pgs: 97 active+clean; 528 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 646 op/s 2024-09-12T02:05:09.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:08 smithi183 ceph-mon[35127]: pgmap v898: 97 pgs: 97 active+clean; 528 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 646 op/s 2024-09-12T02:05:09.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:08 smithi122 ceph-mon[30925]: pgmap v898: 97 pgs: 97 active+clean; 528 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 646 op/s 2024-09-12T02:05:10.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.73 MB/sec execute 177 sec latency 4092.233 ms 2024-09-12T02:05:11.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.61 MB/sec execute 178 sec latency 5092.314 ms 2024-09-12T02:05:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:10 smithi151 ceph-mon[35229]: pgmap v899: 97 pgs: 97 active+clean; 536 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-09-12T02:05:11.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:10 smithi183 ceph-mon[35127]: pgmap v899: 97 pgs: 97 active+clean; 536 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-09-12T02:05:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:10 smithi122 ceph-mon[30925]: pgmap v899: 97 pgs: 97 active+clean; 536 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 597 op/s 2024-09-12T02:05:12.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.49 MB/sec execute 179 sec latency 6092.416 ms 2024-09-12T02:05:13.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.37 MB/sec execute 180 sec latency 7092.480 ms 2024-09-12T02:05:13.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:12 smithi151 ceph-mon[35229]: pgmap v900: 97 pgs: 97 active+clean; 536 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 502 op/s 2024-09-12T02:05:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:12 smithi183 ceph-mon[35127]: pgmap v900: 97 pgs: 97 active+clean; 536 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 502 op/s 2024-09-12T02:05:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:12 smithi122 ceph-mon[30925]: pgmap v900: 97 pgs: 97 active+clean; 536 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 502 op/s 2024-09-12T02:05:14.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.25 MB/sec execute 181 sec latency 8092.570 ms 2024-09-12T02:05:15.068 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.14 MB/sec execute 182 sec latency 9092.672 ms 2024-09-12T02:05:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:14 smithi151 ceph-mon[35229]: pgmap v901: 97 pgs: 97 active+clean; 531 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 530 op/s 2024-09-12T02:05:15.156 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:14 smithi122 ceph-mon[30925]: pgmap v901: 97 pgs: 97 active+clean; 531 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 530 op/s 2024-09-12T02:05:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:14 smithi183 ceph-mon[35127]: pgmap v901: 97 pgs: 97 active+clean; 531 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 530 op/s 2024-09-12T02:05:16.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 21.02 MB/sec execute 183 sec latency 10092.745 ms 2024-09-12T02:05:17.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.91 MB/sec execute 184 sec latency 11092.823 ms 2024-09-12T02:05:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:16 smithi151 ceph-mon[35229]: pgmap v902: 97 pgs: 97 active+clean; 532 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 355 op/s 2024-09-12T02:05:17.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:16 smithi183 ceph-mon[35127]: pgmap v902: 97 pgs: 97 active+clean; 532 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 355 op/s 2024-09-12T02:05:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:16 smithi122 ceph-mon[30925]: pgmap v902: 97 pgs: 97 active+clean; 532 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 355 op/s 2024-09-12T02:05:18.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.79 MB/sec execute 185 sec latency 12092.922 ms 2024-09-12T02:05:19.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.68 MB/sec execute 186 sec latency 13093.017 ms 2024-09-12T02:05:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:18 smithi151 ceph-mon[35229]: pgmap v903: 97 pgs: 97 active+clean; 530 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 353 op/s 2024-09-12T02:05:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:18 smithi183 ceph-mon[35127]: pgmap v903: 97 pgs: 97 active+clean; 530 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 353 op/s 2024-09-12T02:05:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:18 smithi122 ceph-mon[30925]: pgmap v903: 97 pgs: 97 active+clean; 530 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 353 op/s 2024-09-12T02:05:20.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.57 MB/sec execute 187 sec latency 14093.116 ms 2024-09-12T02:05:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/4062325543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54100cd0 msgr2=0x7f1c54101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/4062325543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54100cd0 0x7f1c54101150 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009900 tx=0x7f1c4402a410 comp rx=0 tx=0).stop 2024-09-12T02:05:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/4062325543 shutdown_connections 2024-09-12T02:05:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/4062325543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c54101690 0x7f1c54108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/4062325543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54100cd0 0x7f1c54101150 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/4062325543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c54102030 0x7f1c54102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/4062325543 >> 172.21.15.122:0/4062325543 conn(0x7f1c540fb970 msgr2=0x7f1c540fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.053+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/4062325543 shutdown_connections 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/4062325543 wait complete. 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 Processor -- start 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 -- start start 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c54100cd0 0x7f1c54196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c54101690 0x7f1c54197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 0x7f1c5419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5410c6c0 con 0x7f1c54102030 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5410c540 con 0x7f1c54101690 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c5b8a4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5410c840 con 0x7f1c54100cd0 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c59e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 0x7f1c5419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c59e41640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 0x7f1c5419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53444/0 (socket says 172.21.15.122:53444) 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.054+0000 7f1c59e41640 1 -- 172.21.15.122:0/3248739772 learned_addr learned my addr 172.21.15.122:0/3248739772 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c59e41640 1 -- 172.21.15.122:0/3248739772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c54100cd0 msgr2=0x7f1c54196cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c59e41640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c54100cd0 0x7f1c54196cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c59e41640 1 -- 172.21.15.122:0/3248739772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c54101690 msgr2=0x7f1c54197200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c59e41640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c54101690 0x7f1c54197200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c59e41640 1 -- 172.21.15.122:0/3248739772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5419e900 con 0x7f1c54102030 2024-09-12T02:05:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c59e41640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 0x7f1c5419e260 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009fd0 tx=0x7f1c4c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c4a7fc640 1 -- 172.21.15.122:0/3248739772 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c017070 con 0x7f1c54102030 2024-09-12T02:05:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c4a7fc640 1 -- 172.21.15.122:0/3248739772 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c4c003e50 con 0x7f1c54102030 2024-09-12T02:05:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5419ebf0 con 0x7f1c54102030 2024-09-12T02:05:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c4a7fc640 1 -- 172.21.15.122:0/3248739772 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0106c0 con 0x7f1c54102030 2024-09-12T02:05:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.055+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5419f140 con 0x7f1c54102030 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.057+0000 7f1c4a7fc640 1 -- 172.21.15.122:0/3248739772 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1c4c003970 con 0x7f1c54102030 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 Processor -- start 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 -- start start 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a59a0 0x5645169a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a62e0 0x5645169aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 0x5645169ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645169092f0 con 0x5645169ab370 2024-09-12T02:05:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645168b1270 con 0x5645169a62e0 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c5e636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645169052b0 con 0x5645169a59a0 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 0x5645169ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c4a7fc640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c400a2cb0 0x7f1c400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 0x5645169ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53454/0 (socket says 172.21.15.122:53454) 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 -- 172.21.15.122:0/1396568956 learned_addr learned my addr 172.21.15.122:0/1396568956 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c59640640 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a62e0 0x5645169aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c59e41640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c400a2cb0 0x7f1c400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a59a0 0x5645169a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c4a7fc640 1 -- 172.21.15.122:0/3248739772 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f1c4c0ceea0 con 0x7f1c54102030 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 -- 172.21.15.122:0/1396568956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a59a0 msgr2=0x5645169a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a59a0 0x5645169a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 -- 172.21.15.122:0/1396568956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a62e0 msgr2=0x5645169aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a62e0 0x5645169aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.058+0000 7f1c58e3f640 1 -- 172.21.15.122:0/1396568956 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645168b0340 con 0x5645169ab370 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 0x5645169ad830 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1c44031610 tx=0x7f1c44031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498f7664ad0ef89a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c59e41640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c400a2cb0 0x7f1c400a5170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005c40 tx=0x7f1c4c06e780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c3f7fe640 1 -- 172.21.15.122:0/1396568956 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44035040 con 0x5645169ab370 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c3f7fe640 1 -- 172.21.15.122:0/1396568956 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c44005030 con 0x5645169ab370 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c3f7fe640 1 -- 172.21.15.122:0/1396568956 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44004920 con 0x5645169ab370 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c5e636740 1 -- 172.21.15.122:0/1396568956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 msgr2=0x5645169ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.059+0000 7f1c5e636740 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 0x5645169ad830 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f1c44031610 tx=0x7f1c44031400 comp rx=0 tx=0).stop 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- 172.21.15.122:0/1396568956 shutdown_connections 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169ab370 0x5645169ad830 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a62e0 0x5645169aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 --2- 172.21.15.122:0/1396568956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a59a0 0x5645169a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- 172.21.15.122:0/1396568956 >> 172.21.15.122:0/1396568956 conn(0x5645169a1010 msgr2=0x5645169a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- 172.21.15.122:0/1396568956 shutdown_connections 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- 172.21.15.122:0/1396568956 wait complete. 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- start start 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 Processor -- start 2024-09-12T02:05:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 0x5645169b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169ab370 0x5645169b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645169ae2e0 con 0x5645169a62e0 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564516892290 con 0x5645169a59a0 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645169ae460 con 0x5645169ab370 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c5e636740 10 client.0 _ll_register_callbacks cb 0x5645168c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c59640640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 0x5645169b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c59640640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 0x5645169b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53462/0 (socket says 172.21.15.122:53462) 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.060+0000 7f1c59640640 1 -- 172.21.15.122:0/3504438519 learned_addr learned my addr 172.21.15.122:0/3504438519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169ab370 0x5645169b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c59640640 1 -- 172.21.15.122:0/3504438519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169ab370 msgr2=0x5645169b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c59640640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169ab370 0x5645169b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c59640640 1 -- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 msgr2=0x5645169afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c59640640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c59640640 1 -- 172.21.15.122:0/3504438519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451661c210 con 0x5645169a62e0 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169ab370 0x5645169b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c59640640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 0x5645169b0100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009550 tx=0x7f1c50009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa393da6ff1b0312 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c3cff9640 1 -- 172.21.15.122:0/3504438519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50005220 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c3cff9640 1 -- 172.21.15.122:0/3504438519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f1c500053c0 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c44005030 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c3cff9640 1 -- 172.21.15.122:0/3504438519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1c500055c0 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c5e636740 10 client.18066 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5645166e8620 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c3cff9640 1 -- 172.21.15.122:0/3504438519 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c44005030 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.061+0000 7f1c3cff9640 1 -- 172.21.15.122:0/3504438519 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1c50017ab0 con 0x5645169a62e0 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c5e636740 10 client.18066 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c5e636740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c5e636740 20 client.18066 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c5e636740 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169bb8b0 0x5645169bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c5e636740 4 client.18066 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564516916a20 con 0x5645169bb8b0 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169bb8b0 0x5645169bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c3cff9640 10 client.18066 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.062+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169bb8b0 0x5645169bdd20 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c902e21a148c0831 server_cookie=9265da84f0459b28 in_seq=0 out_seq=0 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c3cff9640 1 -- 172.21.15.122:0/3504438519 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564516916a20 con 0x5645169bb8b0 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c3cff9640 10 client.18066 handle_command_reply: tid=0 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 client.18066 shutdown 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 msgr2=0x5645169b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 0x5645169b0100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009550 tx=0x7f1c50009a10 comp rx=0 tx=0).stop 2024-09-12T02:05:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 shutdown_connections 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169bb8b0 0x5645169bdd20 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169ab370 0x5645169b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c3cff9640 0 client.18066 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a62e0 0x5645169b0100 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 --2- 172.21.15.122:0/3504438519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 >> 172.21.15.122:0/3504438519 conn(0x5645169a1010 msgr2=0x5645169a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 shutdown_connections 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 1 -- 172.21.15.122:0/3504438519 wait complete. 2024-09-12T02:05:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.063+0000 7f1c5e636740 20 client.18066 trim_cache size 0 max 16384 2024-09-12T02:05:21.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.46 MB/sec execute 188 sec latency 15093.151 ms 2024-09-12T02:05:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 Processor -- start 2024-09-12T02:05:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 -- start start 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a1010 0x5645169b7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 0x5645169a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541a9650 con 0x5645169a59a0 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541a6f20 con 0x5645169a1010 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.077+0000 7f1c5e636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541a4d50 con 0x5645169a6da0 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 0x5645169a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 0x5645169a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53464/0 (socket says 172.21.15.122:53464) 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 -- 172.21.15.122:0/4235788700 learned_addr learned my addr 172.21.15.122:0/4235788700 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 -- 172.21.15.122:0/4235788700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 msgr2=0x5645169a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 -- 172.21.15.122:0/4235788700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a1010 msgr2=0x5645169b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a1010 0x5645169b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 -- 172.21.15.122:0/4235788700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645169a67c0 con 0x5645169a59a0 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 0x5645169a2940 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1c4402a930 tx=0x7f1c44004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83001085667b275c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c3effd640 1 -- 172.21.15.122:0/4235788700 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44055070 con 0x5645169a59a0 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c3effd640 1 -- 172.21.15.122:0/4235788700 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c440046c0 con 0x5645169a59a0 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.078+0000 7f1c3effd640 1 -- 172.21.15.122:0/4235788700 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44035040 con 0x5645169a59a0 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 -- 172.21.15.122:0/4235788700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 msgr2=0x5645169a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 0x5645169a2940 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1c4402a930 tx=0x7f1c44004600 comp rx=0 tx=0).stop 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 -- 172.21.15.122:0/4235788700 shutdown_connections 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a59a0 0x5645169a2940 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a1010 0x5645169b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 --2- 172.21.15.122:0/4235788700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 -- 172.21.15.122:0/4235788700 >> 172.21.15.122:0/4235788700 conn(0x5645169aae90 msgr2=0x5645169a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 -- 172.21.15.122:0/4235788700 shutdown_connections 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 -- 172.21.15.122:0/4235788700 wait complete. 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 -- start start 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.079+0000 7f1c5e636740 1 Processor -- start 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 0x5645169b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541a53e0 con 0x5645169a1010 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541a6850 con 0x5645169a59a0 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541a5ab0 con 0x5645169a6da0 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 10 client.0 _ll_register_callbacks cb 0x7f1c5000d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 0x5645169b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 0x5645169b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53474/0 (socket says 172.21.15.122:53474) 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 -- 172.21.15.122:0/267026030 learned_addr learned my addr 172.21.15.122:0/267026030 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c59640640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 -- 172.21.15.122:0/267026030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 msgr2=0x5645169b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 -- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 msgr2=0x5645169b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 -- 172.21.15.122:0/267026030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645169b6a00 con 0x5645169a1010 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c58e3f640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 0x5645169b4ea0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009900 tx=0x7f1c44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14032f2aea0ef99c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c3cff9640 1 -- 172.21.15.122:0/267026030 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44035070 con 0x5645169a1010 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c440046c0 con 0x5645169a1010 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c3cff9640 1 -- 172.21.15.122:0/267026030 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f1c44004d80 con 0x5645169a1010 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.080+0000 7f1c3cff9640 1 -- 172.21.15.122:0/267026030 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1c44055070 con 0x5645169a1010 2024-09-12T02:05:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 10 client.18072 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564516a26bd0 con 0x5645169a1010 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c3cff9640 1 -- 172.21.15.122:0/267026030 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c440046c0 con 0x5645169a1010 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c3cff9640 1 -- 172.21.15.122:0/267026030 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1c4403a050 con 0x5645169a1010 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 10 client.18072 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 20 client.18072 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169a62e0 0x564516a28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 4 client.18072 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564516910d70 con 0x5645169a62e0 2024-09-12T02:05:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.081+0000 7f1c59640640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169a62e0 0x564516a28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.082+0000 7f1c3cff9640 10 client.18072 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.082+0000 7f1c59640640 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169a62e0 0x564516a28f90 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d05be62d423ea1 server_cookie=dd5beae485c84719 in_seq=0 out_seq=0 2024-09-12T02:05:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:20 smithi151 ceph-mon[35229]: pgmap v904: 97 pgs: 97 active+clean; 526 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-09-12T02:05:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.138+0000 7f1c3cff9640 1 -- 172.21.15.122:0/267026030 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564516910d70 con 0x5645169a62e0 2024-09-12T02:05:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.138+0000 7f1c3cff9640 10 client.18072 handle_command_reply: tid=0 2024-09-12T02:05:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.138+0000 7f1c5e636740 1 client.18072 shutdown 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 msgr2=0x5645169b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 0x5645169b4ea0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009900 tx=0x7f1c44000bc0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 shutdown_connections 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5645169a62e0 0x564516a28f90 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645169a6da0 0x5645169b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645169a59a0 0x5645169b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 --2- 172.21.15.122:0/267026030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645169a1010 0x5645169b4ea0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c3cff9640 0 client.18072 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 >> 172.21.15.122:0/267026030 conn(0x5645169aae90 msgr2=0x5645169a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 shutdown_connections 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 1 -- 172.21.15.122:0/267026030 wait complete. 2024-09-12T02:05:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5e636740 20 client.18072 trim_cache size 0 max 16384 2024-09-12T02:05:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:05:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:05:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:05:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c400a2cb0 msgr2=0x7f1c400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c400a2cb0 0x7f1c400a5170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005c40 tx=0x7f1c4c06e780 comp rx=0 tx=0).stop 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 msgr2=0x7f1c5419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 0x7f1c5419e260 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009fd0 tx=0x7f1c4c00b260 comp rx=0 tx=0).stop 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 shutdown_connections 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1c400a2cb0 0x7f1c400a5170 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c54102030 0x7f1c5419e260 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c54101690 0x7f1c54197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 --2- 172.21.15.122:0/3248739772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c54100cd0 0x7f1c54196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.139+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 >> 172.21.15.122:0/3248739772 conn(0x7f1c540fb970 msgr2=0x7f1c541050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.140+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 shutdown_connections 2024-09-12T02:05:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:21.140+0000 7f1c5b8a4640 1 -- 172.21.15.122:0/3248739772 wait complete. 2024-09-12T02:05:21.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:20 smithi183 ceph-mon[35127]: pgmap v904: 97 pgs: 97 active+clean; 526 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-09-12T02:05:21.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:20 smithi122 ceph-mon[30925]: pgmap v904: 97 pgs: 97 active+clean; 526 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-09-12T02:05:22.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.35 MB/sec execute 189 sec latency 16093.208 ms 2024-09-12T02:05:23.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.25 MB/sec execute 190 sec latency 17093.307 ms 2024-09-12T02:05:23.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:22 smithi151 ceph-mon[35229]: pgmap v905: 97 pgs: 97 active+clean; 523 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 4.2 MiB/s wr, 206 op/s 2024-09-12T02:05:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:22 smithi183 ceph-mon[35127]: pgmap v905: 97 pgs: 97 active+clean; 523 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 4.2 MiB/s wr, 206 op/s 2024-09-12T02:05:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:22 smithi122 ceph-mon[30925]: pgmap v905: 97 pgs: 97 active+clean; 523 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 4.2 MiB/s wr, 206 op/s 2024-09-12T02:05:24.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.14 MB/sec execute 191 sec latency 18093.412 ms 2024-09-12T02:05:24.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:25.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 20.03 MB/sec execute 192 sec latency 19093.477 ms 2024-09-12T02:05:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:24 smithi151 ceph-mon[35229]: pgmap v906: 97 pgs: 97 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-09-12T02:05:25.155 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:24 smithi183 ceph-mon[35127]: pgmap v906: 97 pgs: 97 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-09-12T02:05:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:24 smithi122 ceph-mon[30925]: pgmap v906: 97 pgs: 97 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 5.0 MiB/s wr, 213 op/s 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 -- 172.21.15.122:0/69325066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5638106f50 msgr2=0x7f563810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 --2- 172.21.15.122:0/69325066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5638106f50 0x7f563810eaf0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5630009930 tx=0x7f563002a420 comp rx=0 tx=0).stop 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 -- 172.21.15.122:0/69325066 shutdown_connections 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 --2- 172.21.15.122:0/69325066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5638106f50 0x7f563810eaf0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 --2- 172.21.15.122:0/69325066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5638100ca0 0x7f5638106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 --2- 172.21.15.122:0/69325066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381002f0 0x7f56381006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 -- 172.21.15.122:0/69325066 >> 172.21.15.122:0/69325066 conn(0x7f5638077210 msgr2=0x7f56380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 -- 172.21.15.122:0/69325066 shutdown_connections 2024-09-12T02:05:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 -- 172.21.15.122:0/69325066 wait complete. 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 Processor -- start 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.633+0000 7f563cc98640 1 -- start start 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 0x7f5638198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5638100ca0 0x7f56381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5638106f50 0x7f563819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5638111fa0 con 0x7f56381002f0 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5638111e20 con 0x7f5638100ca0 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5638112120 con 0x7f5638106f50 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 0x7f5638198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 0x7f5638198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53504/0 (socket says 172.21.15.122:53504) 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 -- 172.21.15.122:0/114021364 learned_addr learned my addr 172.21.15.122:0/114021364 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 -- 172.21.15.122:0/114021364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5638106f50 msgr2=0x7f563819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f5636d9d640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5638106f50 0x7f563819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f5635d9b640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5638100ca0 0x7f56381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5638106f50 0x7f563819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 -- 172.21.15.122:0/114021364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5638100ca0 msgr2=0x7f56381989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5638100ca0 0x7f56381989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 -- 172.21.15.122:0/114021364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563819d410 con 0x7f56381002f0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563659c640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 0x7f5638198460 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f562800d950 tx=0x7f562800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628017070 con 0x7f56381002f0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5628009c30 con 0x7f56381002f0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563819d700 con 0x7f56381002f0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.634+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562800f6b0 con 0x7f56381002f0 2024-09-12T02:05:25.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.635+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56381a4fe0 con 0x7f56381002f0 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.636+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56381006d0 con 0x7f56381002f0 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.637+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f56280109d0 con 0x7f56381002f0 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.637+0000 7f56277fe640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56200a2b80 0x7f56200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.637+0000 7f5635d9b640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56200a2b80 0x7f56200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.637+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f56280ced80 con 0x7f56381002f0 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.638+0000 7f5635d9b640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56200a2b80 0x7f56200a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f562c005fd0 tx=0x7f562c008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:25.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.642+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5628009de0 con 0x7f56381002f0 2024-09-12T02:05:25.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.894+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56381997c0 con 0x7f56381002f0 2024-09-12T02:05:25.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.895+0000 7f56277fe640 1 -- 172.21.15.122:0/114021364 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7f562809bce0 con 0x7f56381002f0 2024-09-12T02:05:25.896 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:05:25.897 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:05:25.897 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56200a2b80 msgr2=0x7f56200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56200a2b80 0x7f56200a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f562c005fd0 tx=0x7f562c008230 comp rx=0 tx=0).stop 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 msgr2=0x7f5638198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 0x7f5638198460 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f562800d950 tx=0x7f562800de10 comp rx=0 tx=0).stop 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 shutdown_connections 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f56200a2b80 0x7f56200a5040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5638106f50 0x7f563819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5638100ca0 0x7f56381989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 --2- 172.21.15.122:0/114021364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56381002f0 0x7f5638198460 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 >> 172.21.15.122:0/114021364 conn(0x7f5638077210 msgr2=0x7f5638105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 shutdown_connections 2024-09-12T02:05:25.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:25.897+0000 7f563cc98640 1 -- 172.21.15.122:0/114021364 wait complete. 2024-09-12T02:05:26.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 19.93 MB/sec execute 193 sec latency 20093.550 ms 2024-09-12T02:05:26.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:25 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/114021364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:26.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:25 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/114021364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:26.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:25 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/114021364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:26.673 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 5d8f4fa9 --release --await' 2024-09-12T02:05:26.674 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 5d8f4fa9 --release --await 2024-09-12T02:05:26.917 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:27.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124465 19.83 MB/sec execute 194 sec latency 21093.619 ms 2024-09-12T02:05:27.204 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:26 smithi122 ceph-mon[30925]: pgmap v907: 97 pgs: 97 active+clean; 520 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.3 MiB/s wr, 174 op/s 2024-09-12T02:05:27.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:26 smithi151 ceph-mon[35229]: pgmap v907: 97 pgs: 97 active+clean; 520 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.3 MiB/s wr, 174 op/s 2024-09-12T02:05:27.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:26 smithi183 ceph-mon[35127]: pgmap v907: 97 pgs: 97 active+clean; 520 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.3 MiB/s wr, 174 op/s 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 -- 172.21.15.122:0/4162182123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54101420 msgr2=0x7fae5410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/4162182123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54101420 0x7fae5410d950 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009cd0 tx=0x7fae4c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 -- 172.21.15.122:0/4162182123 shutdown_connections 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/4162182123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54101420 0x7fae5410d950 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/4162182123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae54100a80 0x7fae54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/4162182123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae54106a80 0x7fae54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 -- 172.21.15.122:0/4162182123 >> 172.21.15.122:0/4162182123 conn(0x7fae540fc7a0 msgr2=0x7fae540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 -- 172.21.15.122:0/4162182123 shutdown_connections 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.456+0000 7fae5b4a3640 1 -- 172.21.15.122:0/4162182123 wait complete. 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 Processor -- start 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 -- start start 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 0x7fae5419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae54101420 0x7fae5419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae54106a80 0x7fae5419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae54110e80 con 0x7fae54100a80 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae54110d00 con 0x7fae54106a80 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5b4a3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae54111000 con 0x7fae54101420 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5923f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 0x7fae5419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5923f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 0x7fae5419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53514/0 (socket says 172.21.15.122:53514) 2024-09-12T02:05:27.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae5923f640 1 -- 172.21.15.122:0/35559231 learned_addr learned my addr 172.21.15.122:0/35559231 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.457+0000 7fae58a3e640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae54101420 0x7fae5419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae59a40640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae54106a80 0x7fae5419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5923f640 1 -- 172.21.15.122:0/35559231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae54101420 msgr2=0x7fae5419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5923f640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae54101420 0x7fae5419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5923f640 1 -- 172.21.15.122:0/35559231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae54106a80 msgr2=0x7fae5419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5923f640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae54106a80 0x7fae5419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5923f640 1 -- 172.21.15.122:0/35559231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5419f550 con 0x7fae54100a80 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5923f640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 0x7fae5419a5a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fae5000d770 tx=0x7fae5000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae4a7fc640 1 -- 172.21.15.122:0/35559231 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50017070 con 0x7fae54100a80 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5419f840 con 0x7fae54100a80 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae4a7fc640 1 -- 172.21.15.122:0/35559231 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae50009820 con 0x7fae54100a80 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae4a7fc640 1 -- 172.21.15.122:0/35559231 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50004cd0 con 0x7fae54100a80 2024-09-12T02:05:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.458+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae541a7120 con 0x7fae54100a80 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.460+0000 7fae4a7fc640 1 -- 172.21.15.122:0/35559231 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fae500099d0 con 0x7fae54100a80 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 Processor -- start 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 -- start start 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a060 0x559ceaa4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa54d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa55240 0x559ceaa57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cea9af620 con 0x559ceaa4a980 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559cea956e50 con 0x559ceaa55240 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5d87c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559cea9ab5e0 con 0x559ceaa4a060 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae4a7fc640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fae400a2b80 0x7fae400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5923f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa54d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5923f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa54d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53520/0 (socket says 172.21.15.122:53520) 2024-09-12T02:05:27.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae5923f640 1 -- 172.21.15.122:0/1971872813 learned_addr learned my addr 172.21.15.122:0/1971872813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae59a40640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fae400a2b80 0x7fae400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae59a40640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a060 0x559ceaa4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.461+0000 7fae58a3e640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa55240 0x559ceaa57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae4a7fc640 1 -- 172.21.15.122:0/35559231 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fae500ce960 con 0x7fae54100a80 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae5923f640 1 -- 172.21.15.122:0/1971872813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a060 msgr2=0x559ceaa4a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae5923f640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a060 0x559ceaa4a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae5923f640 1 -- 172.21.15.122:0/1971872813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa55240 msgr2=0x559ceaa57630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae5923f640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa55240 0x559ceaa57630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae5923f640 1 -- 172.21.15.122:0/1971872813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cea947d40 con 0x559ceaa4a980 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae59a40640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a060 0x559ceaa4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae58a3e640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa55240 0x559ceaa57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae5923f640 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa54d00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fae5009b0f0 tx=0x7fae5009abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47371b6e48f88771 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae59a40640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fae400a2b80 0x7fae400a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009430 tx=0x7fae4c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae3f7fe640 1 -- 172.21.15.122:0/1971872813 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50020050 con 0x559ceaa4a980 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae3f7fe640 1 -- 172.21.15.122:0/1971872813 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae500a3e30 con 0x559ceaa4a980 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.462+0000 7fae3f7fe640 1 -- 172.21.15.122:0/1971872813 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500a39a0 con 0x559ceaa4a980 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 -- 172.21.15.122:0/1971872813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 msgr2=0x559ceaa54d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa54d00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fae5009b0f0 tx=0x7fae5009abe0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 -- 172.21.15.122:0/1971872813 shutdown_connections 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa55240 0x559ceaa57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa54d00 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1971872813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a060 0x559ceaa4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 -- 172.21.15.122:0/1971872813 >> 172.21.15.122:0/1971872813 conn(0x559ceaa465f0 msgr2=0x559ceaa469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 -- 172.21.15.122:0/1971872813 shutdown_connections 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 -- 172.21.15.122:0/1971872813 wait complete. 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 -- start start 2024-09-12T02:05:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.463+0000 7fae5d87c740 1 Processor -- start 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa52200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa52740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaa52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559ceaa57e10 con 0x559ceaa4a980 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559cea937f10 con 0x559ceaa4a060 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ceaa57f90 con 0x559ceaa55240 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 10 client.0 _ll_register_callbacks cb 0x559cea95bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa52740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa52740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53532/0 (socket says 172.21.15.122:53532) 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 -- 172.21.15.122:0/1847719996 learned_addr learned my addr 172.21.15.122:0/1847719996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5923f640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa52200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae59a40640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaa52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 -- 172.21.15.122:0/1847719996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 msgr2=0x559ceaa52c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaa52c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 -- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 msgr2=0x559ceaa52200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa52200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 -- 172.21.15.122:0/1847719996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cea6be580 con 0x559ceaa4a980 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae59a40640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaa52c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5923f640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa52200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae58a3e640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa52740 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fae440127a0 tx=0x7fae44012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67e1fbf8ab7985c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae3cff9640 1 -- 172.21.15.122:0/1847719996 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44010dc0 con 0x559ceaa4a980 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae3cff9640 1 -- 172.21.15.122:0/1847719996 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fae440047a0 con 0x559ceaa4a980 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.464+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae500a3e30 con 0x559ceaa4a980 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae3cff9640 1 -- 172.21.15.122:0/1847719996 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae440049e0 con 0x559ceaa4a980 2024-09-12T02:05:27.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 10 client.18090 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559cea788960 con 0x559ceaa4a980 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae3cff9640 1 -- 172.21.15.122:0/1847719996 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae500a3e30 con 0x559ceaa4a980 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae3cff9640 1 -- 172.21.15.122:0/1847719996 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fae4401a550 con 0x559ceaa4a980 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 10 client.18090 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 10 client.18090 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 20 client.18090 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaa67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 4 client.18090 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cea9bcb90 con 0x559ceaa64f20 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.465+0000 7fae5923f640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaa67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.466+0000 7fae3cff9640 10 client.18090 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:27.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.466+0000 7fae5923f640 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaa67350 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37050776fa46c922 server_cookie=8ec1671e3bb1dc2 in_seq=0 out_seq=0 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.466+0000 7fae3cff9640 1 -- 172.21.15.122:0/1847719996 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cea9bcb90 con 0x559ceaa64f20 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.466+0000 7fae3cff9640 10 client.18090 handle_command_reply: tid=0 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.466+0000 7fae5d87c740 1 client.18090 shutdown 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 msgr2=0x559ceaa52740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa52740 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fae440127a0 tx=0x7fae44012c60 comp rx=0 tx=0).stop 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 shutdown_connections 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaa67350 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaa52c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa52740 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae3cff9640 0 client.18090 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 --2- 172.21.15.122:0/1847719996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa52200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 >> 172.21.15.122:0/1847719996 conn(0x559ceaa465f0 msgr2=0x559ceaa4c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 shutdown_connections 2024-09-12T02:05:27.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 1 -- 172.21.15.122:0/1847719996 wait complete. 2024-09-12T02:05:27.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.467+0000 7fae5d87c740 20 client.18090 trim_cache size 0 max 16384 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.484+0000 7fae5d87c740 1 Processor -- start 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.484+0000 7fae5d87c740 1 -- start start 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa4cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a980 0x559ceaa4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 0x559ceaa54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5d87c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae541b2080 con 0x559ceaa55240 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5d87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae541af950 con 0x559ceaa4a060 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5d87c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ad780 con 0x559ceaa4a980 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 0x559ceaa54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 0x559ceaa54170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53538/0 (socket says 172.21.15.122:53538) 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 -- 172.21.15.122:0/843048109 learned_addr learned my addr 172.21.15.122:0/843048109 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:27.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae58a3e640 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a980 0x559ceaa4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae5923f640 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa4cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 -- 172.21.15.122:0/843048109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a980 msgr2=0x559ceaa4d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a980 0x559ceaa4d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 -- 172.21.15.122:0/843048109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 msgr2=0x559ceaa4cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa4cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 -- 172.21.15.122:0/843048109 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ceaa528a0 con 0x559ceaa55240 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae59a40640 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 0x559ceaa54170 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fae4c004ab0 tx=0x7fae4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc765b8c4b496de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae3effd640 1 -- 172.21.15.122:0/843048109 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0041c0 con 0x559ceaa55240 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae3effd640 1 -- 172.21.15.122:0/843048109 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c004840 con 0x559ceaa55240 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.485+0000 7fae3effd640 1 -- 172.21.15.122:0/843048109 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c04a470 con 0x559ceaa55240 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 -- 172.21.15.122:0/843048109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 msgr2=0x559ceaa54170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 0x559ceaa54170 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fae4c004ab0 tx=0x7fae4c004a40 comp rx=0 tx=0).stop 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 -- 172.21.15.122:0/843048109 shutdown_connections 2024-09-12T02:05:27.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa55240 0x559ceaa54170 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa4a980 0x559ceaa4d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 --2- 172.21.15.122:0/843048109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa4cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 -- 172.21.15.122:0/843048109 >> 172.21.15.122:0/843048109 conn(0x559ceaa465f0 msgr2=0x559ceaa53a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 -- 172.21.15.122:0/843048109 shutdown_connections 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.486+0000 7fae5d87c740 1 -- 172.21.15.122:0/843048109 wait complete. 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 -- start start 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 Processor -- start 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa59340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaacebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ade10 con 0x559ceaa4a980 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae541af280 con 0x559ceaa4a060 2024-09-12T02:05:27.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ae4e0 con 0x559ceaa55240 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5d87c740 10 client.0 _ll_register_callbacks cb 0x7fae4400ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae58a3e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae58a3e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa59880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53546/0 (socket says 172.21.15.122:53546) 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae58a3e640 1 -- 172.21.15.122:0/3110380917 learned_addr learned my addr 172.21.15.122:0/3110380917 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae59a40640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaacebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.487+0000 7fae5923f640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa59340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae58a3e640 1 -- 172.21.15.122:0/3110380917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 msgr2=0x559ceaacebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae58a3e640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaacebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae58a3e640 1 -- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 msgr2=0x559ceaa59340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae58a3e640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa59340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae58a3e640 1 -- 172.21.15.122:0/3110380917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ceaacf130 con 0x559ceaa4a980 2024-09-12T02:05:27.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae5923f640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa59340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae58a3e640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa59880 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fae44025020 tx=0x7fae440108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652839fbd34e2534 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae3cff9640 1 -- 172.21.15.122:0/3110380917 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44029040 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae3cff9640 1 -- 172.21.15.122:0/3110380917 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fae44010dc0 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ceaa53010 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae3cff9640 1 -- 172.21.15.122:0/3110380917 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae4401a430 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae5d87c740 10 client.18096 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559ceaad0310 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.488+0000 7fae3cff9640 1 -- 172.21.15.122:0/3110380917 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559ceaa53010 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae3cff9640 1 -- 172.21.15.122:0/3110380917 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fae4402e410 con 0x559ceaa4a980 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5d87c740 10 client.18096 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:27.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5d87c740 10 client.18096 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5d87c740 20 client.18096 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5d87c740 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaad2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5d87c740 4 client.18096 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5d8f4fa9", "release": true, "await": true}] 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5d8f4fa9", "release": true, "await": true}) -- 0x559cea9b70a0 con 0x559ceaa64f20 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5923f640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaad2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae5923f640 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaad2a90 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=742c9db26999ac82 server_cookie=71cb5375707cfd6f in_seq=0 out_seq=0 2024-09-12T02:05:27.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.489+0000 7fae3cff9640 10 client.18096 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae3cff9640 1 -- 172.21.15.122:0/3110380917 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559cea9b70a0 con 0x559ceaa64f20 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae3cff9640 10 client.18096 handle_command_reply: tid=0 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 client.18096 shutdown 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 msgr2=0x559ceaa59880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa59880 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fae44025020 tx=0x7fae440108e0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 shutdown_connections 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559ceaa64f20 0x559ceaad2a90 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ceaa55240 0x559ceaacebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559ceaa4a980 0x559ceaa59880 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 --2- 172.21.15.122:0/3110380917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ceaa4a060 0x559ceaa59340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae3cff9640 0 client.18096 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 >> 172.21.15.122:0/3110380917 conn(0x559ceaa465f0 msgr2=0x559ceaa54280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 shutdown_connections 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 1 -- 172.21.15.122:0/3110380917 wait complete. 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.551+0000 7fae5d87c740 20 client.18096 trim_cache size 0 max 16384 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:05:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 68, 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "5d8f4fa9": { 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 68, 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 141.8, 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:05:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:27.555 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fae400a2b80 msgr2=0x7fae400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fae400a2b80 0x7fae400a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009430 tx=0x7fae4c009360 comp rx=0 tx=0).stop 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 msgr2=0x7fae5419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 0x7fae5419a5a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fae5000d770 tx=0x7fae5000db70 comp rx=0 tx=0).stop 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 shutdown_connections 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fae400a2b80 0x7fae400a5040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae54106a80 0x7fae5419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae54101420 0x7fae5419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 --2- 172.21.15.122:0/35559231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae54100a80 0x7fae5419a5a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.552+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 >> 172.21.15.122:0/35559231 conn(0x7fae540fc7a0 msgr2=0x7fae5406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.553+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 shutdown_connections 2024-09-12T02:05:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:27.553+0000 7fae5b4a3640 1 -- 172.21.15.122:0/35559231 wait complete. 2024-09-12T02:05:28.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124749 19.79 MB/sec execute 195 sec latency 21580.590 ms 2024-09-12T02:05:28.462 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5d8f4fa9', total seconds elapsed: 25.4 2024-09-12T02:05:28.462 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-09-12T02:05:28.462 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:29.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 125197 19.82 MB/sec execute 196 sec latency 217.647 ms 2024-09-12T02:05:29.097 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:28 smithi122 ceph-mon[30925]: pgmap v908: 97 pgs: 97 active+clean; 519 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 5.0 MiB/s wr, 190 op/s 2024-09-12T02:05:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:28 smithi151 ceph-mon[35229]: pgmap v908: 97 pgs: 97 active+clean; 519 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 5.0 MiB/s wr, 190 op/s 2024-09-12T02:05:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:28 smithi183 ceph-mon[35127]: pgmap v908: 97 pgs: 97 active+clean; 519 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 5.0 MiB/s wr, 190 op/s 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.307+0000 7f5212f32640 1 -- 172.21.15.122:0/1867998154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 msgr2=0x7f520c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.307+0000 7f5212f32640 1 --2- 172.21.15.122:0/1867998154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c1029b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009ca0 tx=0x7f51fc02a950 comp rx=0 tx=0).stop 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 -- 172.21.15.122:0/1867998154 shutdown_connections 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 --2- 172.21.15.122:0/1867998154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f520c102ef0 0x7f520c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 --2- 172.21.15.122:0/1867998154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c1029b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 --2- 172.21.15.122:0/1867998154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520c100e20 0x7f520c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 -- 172.21.15.122:0/1867998154 >> 172.21.15.122:0/1867998154 conn(0x7f520c0fc7d0 msgr2=0x7f520c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 -- 172.21.15.122:0/1867998154 shutdown_connections 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 -- 172.21.15.122:0/1867998154 wait complete. 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 Processor -- start 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 -- start start 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f520c100e20 0x7f520c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.308+0000 7f5212f32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520c102ef0 0x7f520c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5212f32640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f520c110f80 con 0x7f520c102570 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5212f32640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f520c110e00 con 0x7f520c100e20 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5212f32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f520c111100 con 0x7f520c102ef0 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5210cce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f520c100e20 0x7f520c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53568/0 (socket says 172.21.15.122:53568) 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 -- 172.21.15.122:0/1457749417 learned_addr learned my addr 172.21.15.122:0/1457749417 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f52114cf640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520c102ef0 0x7f520c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 -- 172.21.15.122:0/1457749417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520c102ef0 msgr2=0x7f520c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520c102ef0 0x7f520c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 -- 172.21.15.122:0/1457749417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f520c100e20 msgr2=0x7f520c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f520c100e20 0x7f520c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 -- 172.21.15.122:0/1457749417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c19c3d0 con 0x7f520c102570 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f520bfff640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c197960 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005cc0 tx=0x7f51fc02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc034040 con 0x7f520c102570 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c19c660 con 0x7f520c102570 2024-09-12T02:05:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc030920 con 0x7f520c102570 2024-09-12T02:05:29.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.309+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0046e0 con 0x7f520c102570 2024-09-12T02:05:29.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.310+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1a3fa0 con 0x7f520c102570 2024-09-12T02:05:29.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.311+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c1021b0 con 0x7f520c102570 2024-09-12T02:05:29.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.312+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f51fc037020 con 0x7f520c102570 2024-09-12T02:05:29.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.313+0000 7f5209ffb640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51f80a2b80 0x7f51f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:29.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.313+0000 7f5210cce640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51f80a2b80 0x7f51f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:29.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.313+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f51fc0effc0 con 0x7f520c102570 2024-09-12T02:05:29.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.313+0000 7f5210cce640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51f80a2b80 0x7f51f80a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52040095f0 tx=0x7f5204005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:29.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.316+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51fc0bcf70 con 0x7f520c102570 2024-09-12T02:05:29.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.554+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f520c108b00 con 0x7f520c102570 2024-09-12T02:05:29.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.555+0000 7f5209ffb640 1 -- 172.21.15.122:0/1457749417 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7f51fc0bc870 con 0x7f520c102570 2024-09-12T02:05:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:05:29.557 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:05:29.557 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51f80a2b80 msgr2=0x7f51f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51f80a2b80 0x7f51f80a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52040095f0 tx=0x7f5204005f90 comp rx=0 tx=0).stop 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 msgr2=0x7f520c197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c197960 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005cc0 tx=0x7f51fc02f420 comp rx=0 tx=0).stop 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 shutdown_connections 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f51f80a2b80 0x7f51f80a5040 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520c102ef0 0x7f520c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f520c102570 0x7f520c197960 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 --2- 172.21.15.122:0/1457749417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f520c100e20 0x7f520c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 >> 172.21.15.122:0/1457749417 conn(0x7f520c0fc7d0 msgr2=0x7f520c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:29.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.557+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 shutdown_connections 2024-09-12T02:05:29.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:29.558+0000 7f5212f32640 1 -- 172.21.15.122:0/1457749417 wait complete. 2024-09-12T02:05:30.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 125769 19.86 MB/sec execute 197 sec latency 118.686 ms 2024-09-12T02:05:30.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:29 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1457749417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:30.179 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:29 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1457749417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:29 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1457749417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:30.211 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-09-12T02:05:30.211 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-09-12T02:05:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:30.868 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:30 smithi122 ceph-mon[30925]: pgmap v909: 97 pgs: 97 active+clean; 516 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 211 op/s 2024-09-12T02:05:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 -- 172.21.15.122:0/705315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 msgr2=0x7f2abc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/705315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc1021d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009900 tx=0x7f2ab802a410 comp rx=0 tx=0).stop 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 -- 172.21.15.122:0/705315124 shutdown_connections 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/705315124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2abc106f20 0x7f2abc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/705315124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2abc1027a0 0x7f2abc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/705315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc1021d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 -- 172.21.15.122:0/705315124 >> 172.21.15.122:0/705315124 conn(0x7f2abc0fd8b0 msgr2=0x7f2abc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 -- 172.21.15.122:0/705315124 shutdown_connections 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 -- 172.21.15.122:0/705315124 wait complete. 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.972+0000 7f2ac3f35640 1 Processor -- start 2024-09-12T02:05:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 -- start start 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2abc1027a0 0x7f2abc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2abc106f20 0x7f2abc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc111f80 con 0x7f2abc101df0 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc111e00 con 0x7f2abc1027a0 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac3f35640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc112100 con 0x7f2abc106f20 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57346/0 (socket says 172.21.15.122:57346) 2024-09-12T02:05:30.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3564308001 learned_addr learned my addr 172.21.15.122:0/3564308001 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2abc1027a0 0x7f2abc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac24d2640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2abc106f20 0x7f2abc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3564308001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2abc106f20 msgr2=0x7f2abc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2abc106f20 0x7f2abc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3564308001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2abc1027a0 msgr2=0x7f2abc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2abc1027a0 0x7f2abc198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.973+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3564308001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc19d400 con 0x7f2abc101df0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.974+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc198450 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006010 tx=0x7f2ab8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.974+0000 7f2ab2ffd640 1 -- 172.21.15.122:0/3564308001 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab803f030 con 0x7f2abc101df0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.974+0000 7f2ab2ffd640 1 -- 172.21.15.122:0/3564308001 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab8004440 con 0x7f2abc101df0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.974+0000 7f2ab2ffd640 1 -- 172.21.15.122:0/3564308001 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8035070 con 0x7f2abc101df0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.974+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc19d690 con 0x7f2abc101df0 2024-09-12T02:05:30.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.974+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc1a4fd0 con 0x7f2abc101df0 2024-09-12T02:05:30.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.977+0000 7f2ab2ffd640 1 -- 172.21.15.122:0/3564308001 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2ab8042030 con 0x7f2abc101df0 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ab2ffd640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2aa80a2c00 0x7f2aa80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 Processor -- start 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2aa80a2c00 0x7f2aa80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ab2ffd640 1 -- 172.21.15.122:0/3564308001 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f2ab80ef3b0 con 0x7f2abc101df0 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 -- start start 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f038b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f043400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f045d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc2ef9e2a0 con 0x55fc2f039080 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc2ef45b10 con 0x55fc2f043940 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.978+0000 7f2ac630e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc2ef9a260 con 0x55fc2f038760 2024-09-12T02:05:30.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f043400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f043400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57360/0 (socket says 172.21.15.122:57360) 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 -- 172.21.15.122:0/742740699 learned_addr learned my addr 172.21.15.122:0/742740699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac24d2640 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f045d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2aa80a2c00 0x7f2aa80a50c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0fdcd0 tx=0x7f2aac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 -- 172.21.15.122:0/742740699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 msgr2=0x55fc2f038b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f038b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 -- 172.21.15.122:0/742740699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 msgr2=0x55fc2f045d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f045d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 -- 172.21.15.122:0/742740699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2ed0fb90 con 0x55fc2f039080 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f043400 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2aac011f90 tx=0x7f2aac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453c82557a30bad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.979+0000 7f2aa7fff640 1 -- 172.21.15.122:0/742740699 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac004230 con 0x55fc2f039080 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2aa7fff640 1 -- 172.21.15.122:0/742740699 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aac0043d0 con 0x55fc2f039080 2024-09-12T02:05:30.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2aa7fff640 1 -- 172.21.15.122:0/742740699 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac01fa90 con 0x55fc2f039080 2024-09-12T02:05:30.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 -- 172.21.15.122:0/742740699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 msgr2=0x55fc2f043400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:30.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f043400 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2aac011f90 tx=0x7f2aac014000 comp rx=0 tx=0).stop 2024-09-12T02:05:30.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 -- 172.21.15.122:0/742740699 shutdown_connections 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f045d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f043400 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 --2- 172.21.15.122:0/742740699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f038b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.980+0000 7f2ac630e740 1 -- 172.21.15.122:0/742740699 >> 172.21.15.122:0/742740699 conn(0x55fc2f034cf0 msgr2=0x55fc2f0350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.981+0000 7f2ac630e740 1 -- 172.21.15.122:0/742740699 shutdown_connections 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.981+0000 7f2ac630e740 1 -- 172.21.15.122:0/742740699 wait complete. 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.981+0000 7f2ac630e740 1 -- start start 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.981+0000 7f2ac630e740 1 Processor -- start 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f0408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f040de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f041320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc2f046510 con 0x55fc2f039080 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc2ef266e0 con 0x55fc2f043940 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc2f046690 con 0x55fc2f038760 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac630e740 10 client.0 _ll_register_callbacks cb 0x55fc2ef4aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac14d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f040de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac14d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f040de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57364/0 (socket says 172.21.15.122:57364) 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac14d0640 1 -- 172.21.15.122:0/1281585360 learned_addr learned my addr 172.21.15.122:0/1281585360 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.982+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f0408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac14d0640 1 -- 172.21.15.122:0/1281585360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 msgr2=0x55fc2f0408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f0408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac14d0640 1 -- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 msgr2=0x55fc2f041320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f041320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac14d0640 1 -- 172.21.15.122:0/1281585360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2f053e90 con 0x55fc2f039080 2024-09-12T02:05:30.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f040de0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2aac01f630 tx=0x7f2aac01f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee471891423f8a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2aa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2aa57fa640 1 -- 172.21.15.122:0/1281585360 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac01fd40 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2aa57fa640 1 -- 172.21.15.122:0/1281585360 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f2aac0035f0 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.983+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aac0043d0 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2aa57fa640 1 -- 172.21.15.122:0/1281585360 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2aac0037f0 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 10 client.18114 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fc2f056820 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2aa57fa640 1 -- 172.21.15.122:0/1281585360 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aac0043d0 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2aa57fa640 1 -- 172.21.15.122:0/1281585360 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f2aac03b050 con 0x55fc2f039080 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 10 client.18114 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 10 client.18114 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 20 client.18114 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:30.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f057c80 0x55fc2f05a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:30.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 4 client.18114 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:30.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc2efab9d0 con 0x55fc2f057c80 2024-09-12T02:05:30.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.984+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f057c80 0x55fc2f05a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:30.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.985+0000 7f2aa57fa640 10 client.18114 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:30.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.985+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f057c80 0x55fc2f05a070 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8956ae150e61ace server_cookie=123229ad6dd428f8 in_seq=0 out_seq=0 2024-09-12T02:05:30.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2aa57fa640 1 -- 172.21.15.122:0/1281585360 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc2efab9d0 con 0x55fc2f057c80 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2aa57fa640 10 client.18114 handle_command_reply: tid=0 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 client.18114 shutdown 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 msgr2=0x55fc2f040de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f040de0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2aac01f630 tx=0x7f2aac01f9c0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 shutdown_connections 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f057c80 0x55fc2f05a070 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f041320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f039080 0x55fc2f040de0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 --2- 172.21.15.122:0/1281585360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f038760 0x55fc2f0408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 >> 172.21.15.122:0/1281585360 conn(0x55fc2f034cf0 msgr2=0x55fc2f03a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.986+0000 7f2aa57fa640 0 client.18114 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:30.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.987+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 shutdown_connections 2024-09-12T02:05:30.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.987+0000 7f2ac630e740 1 -- 172.21.15.122:0/1281585360 wait complete. 2024-09-12T02:05:30.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:30.987+0000 7f2ac630e740 20 client.18114 trim_cache size 0 max 16384 2024-09-12T02:05:31.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 Processor -- start 2024-09-12T02:05:31.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 -- start start 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f038760 0x55fc2f03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f042810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 0x55fc2f03db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1adda0 con 0x55fc2f043940 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1ab670 con 0x55fc2f038760 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac630e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1a94a0 con 0x55fc2f039080 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 0x55fc2f03db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac14d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f042810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 0x55fc2f03db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57372/0 (socket says 172.21.15.122:57372) 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 -- 172.21.15.122:0/2234174269 learned_addr learned my addr 172.21.15.122:0/2234174269 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f038760 0x55fc2f03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 -- 172.21.15.122:0/2234174269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 msgr2=0x55fc2f042810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f042810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 -- 172.21.15.122:0/2234174269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f038760 msgr2=0x55fc2f03b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f038760 0x55fc2f03b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac24d2640 1 -- 172.21.15.122:0/2234174269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2f030260 con 0x55fc2f043940 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.008+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f042810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.009+0000 7f2ac24d2640 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 0x55fc2f03db00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4011440 tx=0x7f2ab4011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbab77a37dcc0d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.009+0000 7f2aa77fe640 1 -- 172.21.15.122:0/2234174269 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4004b20 con 0x55fc2f043940 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.009+0000 7f2aa77fe640 1 -- 172.21.15.122:0/2234174269 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab4004cc0 con 0x55fc2f043940 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.009+0000 7f2aa77fe640 1 -- 172.21.15.122:0/2234174269 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4005020 con 0x55fc2f043940 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 -- 172.21.15.122:0/2234174269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 msgr2=0x55fc2f03db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 0x55fc2f03db00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4011440 tx=0x7f2ab4011840 comp rx=0 tx=0).stop 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 -- 172.21.15.122:0/2234174269 shutdown_connections 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f043940 0x55fc2f03db00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f042810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 --2- 172.21.15.122:0/2234174269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f038760 0x55fc2f03b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 -- 172.21.15.122:0/2234174269 >> 172.21.15.122:0/2234174269 conn(0x55fc2f034cf0 msgr2=0x55fc2f042100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 -- 172.21.15.122:0/2234174269 shutdown_connections 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 -- 172.21.15.122:0/2234174269 wait complete. 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 -- start start 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.010+0000 7f2ac630e740 1 Processor -- start 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 0x55fc2f05ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f05b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f05b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1a9b30 con 0x55fc2f038760 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1aafa0 con 0x55fc2f043940 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1aa200 con 0x55fc2f039080 2024-09-12T02:05:31.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 10 client.0 _ll_register_callbacks cb 0x7f2aac011e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 0x55fc2f05ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 0x55fc2f05ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57376/0 (socket says 172.21.15.122:57376) 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3842113087 learned_addr learned my addr 172.21.15.122:0/3842113087 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac24d2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f05b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f05b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3842113087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 msgr2=0x55fc2f05b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f05b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 msgr2=0x55fc2f05b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f05b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 -- 172.21.15.122:0/3842113087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2f0c1810 con 0x55fc2f038760 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac1cd1640 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 0x55fc2f05ac90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80bb910 tx=0x7f2ab8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dd0eb9d8702e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2aa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:31.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2aa57fa640 1 -- 172.21.15.122:0/3842113087 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8049040 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2aa57fa640 1 -- 172.21.15.122:0/3842113087 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f2ab8003cd0 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.011+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ab4004cc0 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2aa57fa640 1 -- 172.21.15.122:0/3842113087 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2ab80c4880 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 10 client.18120 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fc2f0c2ad0 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2aa57fa640 1 -- 172.21.15.122:0/3842113087 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab4004cc0 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2aa57fa640 1 -- 172.21.15.122:0/3842113087 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f2ab80c43f0 con 0x55fc2f038760 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 10 client.18120 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 10 client.18120 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 20 client.18120 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f0c30d0 0x55fc2f0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:31.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 4 client.18120 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-09-12T02:05:31.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x55fc2efa5d20 con 0x55fc2f0c30d0 2024-09-12T02:05:31.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.012+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f0c30d0 0x55fc2f0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:31.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.013+0000 7f2aa57fa640 10 client.18120 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:31.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.013+0000 7f2ac14d0640 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f0c30d0 0x55fc2f0c54c0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb8fc73d6aab16c server_cookie=154443ca146c4ddb in_seq=0 out_seq=0 2024-09-12T02:05:31.069 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126233 19.90 MB/sec execute 198 sec latency 67.947 ms 2024-09-12T02:05:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:30 smithi151 ceph-mon[35229]: pgmap v909: 97 pgs: 97 active+clean; 516 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 211 op/s 2024-09-12T02:05:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:30 smithi183 ceph-mon[35127]: pgmap v909: 97 pgs: 97 active+clean; 516 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 211 op/s 2024-09-12T02:05:31.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.726+0000 7f2aa57fa640 1 -- 172.21.15.122:0/3842113087 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fc2efa5d20 con 0x55fc2f0c30d0 2024-09-12T02:05:31.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.726+0000 7f2aa57fa640 10 client.18120 handle_command_reply: tid=0 2024-09-12T02:05:31.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.726+0000 7f2ac630e740 1 client.18120 shutdown 2024-09-12T02:05:31.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.726+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 msgr2=0x55fc2f05ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.726+0000 7f2ac630e740 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 0x55fc2f05ac90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80bb910 tx=0x7f2ab8005c90 comp rx=0 tx=0).stop 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 shutdown_connections 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55fc2f0c30d0 0x55fc2f0c54c0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc2f043940 0x55fc2f05b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2f039080 0x55fc2f05b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2aa57fa640 0 client.18120 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 --2- 172.21.15.122:0/3842113087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc2f038760 0x55fc2f05ac90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 >> 172.21.15.122:0/3842113087 conn(0x55fc2f034cf0 msgr2=0x55fc2f042920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 shutdown_connections 2024-09-12T02:05:31.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 1 -- 172.21.15.122:0/3842113087 wait complete. 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.727+0000 7f2ac630e740 20 client.18120 trim_cache size 0 max 16384 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 70, 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "8a53007c": { 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 70, 2024-09-12T02:05:31.730 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 141.8, 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:31.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:31.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:31.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:31.732 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:05:31.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2aa80a2c00 msgr2=0x7f2aa80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2aa80a2c00 0x7f2aa80a50c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0fdcd0 tx=0x7f2aac005f90 comp rx=0 tx=0).stop 2024-09-12T02:05:31.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 msgr2=0x7f2abc198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc198450 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006010 tx=0x7f2ab8004040 comp rx=0 tx=0).stop 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 shutdown_connections 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f2aa80a2c00 0x7f2aa80a50c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2abc106f20 0x7f2abc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2abc1027a0 0x7f2abc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 --2- 172.21.15.122:0/3564308001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2abc101df0 0x7f2abc198450 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 >> 172.21.15.122:0/3564308001 conn(0x7f2abc0fd8b0 msgr2=0x7f2abc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 shutdown_connections 2024-09-12T02:05:31.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:31.728+0000 7f2ac3f35640 1 -- 172.21.15.122:0/3564308001 wait complete. 2024-09-12T02:05:32.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.80 MB/sec execute 199 sec latency 1036.360 ms 2024-09-12T02:05:32.385 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8a53007c', quiesce took 3.9 seconds. Will release after: 17.900000000000002 2024-09-12T02:05:33.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.70 MB/sec execute 200 sec latency 2036.449 ms 2024-09-12T02:05:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:32 smithi151 ceph-mon[35229]: pgmap v910: 97 pgs: 97 active+clean; 514 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 227 op/s 2024-09-12T02:05:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:32 smithi183 ceph-mon[35127]: pgmap v910: 97 pgs: 97 active+clean; 514 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 227 op/s 2024-09-12T02:05:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:32 smithi122 ceph-mon[30925]: pgmap v910: 97 pgs: 97 active+clean; 514 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 227 op/s 2024-09-12T02:05:34.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.61 MB/sec execute 201 sec latency 3036.520 ms 2024-09-12T02:05:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 -- 172.21.15.122:0/2517942137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d01001d0 msgr2=0x7f73d01005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 --2- 172.21.15.122:0/2517942137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d01001d0 0x7f73d01005d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009900 tx=0x7f73cc02a410 comp rx=0 tx=0).stop 2024-09-12T02:05:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 -- 172.21.15.122:0/2517942137 shutdown_connections 2024-09-12T02:05:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 --2- 172.21.15.122:0/2517942137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73d0101d80 0x7f73d0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 --2- 172.21.15.122:0/2517942137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d01013c0 0x7f73d0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 --2- 172.21.15.122:0/2517942137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d01001d0 0x7f73d01005d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 -- 172.21.15.122:0/2517942137 >> 172.21.15.122:0/2517942137 conn(0x7f73d00fb930 msgr2=0x7f73d00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 -- 172.21.15.122:0/2517942137 shutdown_connections 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.225+0000 7f73d8103640 1 -- 172.21.15.122:0/2517942137 wait complete. 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 Processor -- start 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 -- start start 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73d01001d0 0x7f73d01042b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d01013c0 0x7f73d01028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 0x7f73d0102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c720 con 0x7f73d0101d80 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c5a0 con 0x7f73d01001d0 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c8a0 con 0x7f73d01013c0 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 0x7f73d0102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 0x7f73d0102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57392/0 (socket says 172.21.15.122:57392) 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 -- 172.21.15.122:0/253104538 learned_addr learned my addr 172.21.15.122:0/253104538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73d01001d0 0x7f73d01042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d569e640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d01013c0 0x7f73d01028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 -- 172.21.15.122:0/253104538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d01013c0 msgr2=0x7f73d01028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d01013c0 0x7f73d01028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 -- 172.21.15.122:0/253104538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73d01001d0 msgr2=0x7f73d01042b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73d01001d0 0x7f73d01042b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 -- 172.21.15.122:0/253104538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d01036c0 con 0x7f73d0101d80 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d66a0640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 0x7f73d0102e30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f73c800eaa0 tx=0x7f73c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73c6ffd640 1 -- 172.21.15.122:0/253104538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c800cdb0 con 0x7f73d0101d80 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73c6ffd640 1 -- 172.21.15.122:0/253104538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c8005020 con 0x7f73d0101d80 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73c6ffd640 1 -- 172.21.15.122:0/253104538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80116b0 con 0x7f73d0101d80 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.226+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d01a1f20 con 0x7f73d0101d80 2024-09-12T02:05:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.227+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a2380 con 0x7f73d0101d80 2024-09-12T02:05:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73c6ffd640 1 -- 172.21.15.122:0/253104538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f73c8005560 con 0x7f73d0101d80 2024-09-12T02:05:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 Processor -- start 2024-09-12T02:05:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 -- start start 2024-09-12T02:05:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b281697da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b28169cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b28169f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b2815fb2f0 con 0x55b28169d370 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b2815a3270 con 0x55b2816979a0 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.229+0000 7f73dae99740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2815f72b0 con 0x55b2816982e0 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b28169f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b28169f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57398/0 (socket says 172.21.15.122:57398) 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73c6ffd640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73bc0a2cb0 0x7f73bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 -- 172.21.15.122:0/1173277164 learned_addr learned my addr 172.21.15.122:0/1173277164 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b28169cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b281697da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d66a0640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73bc0a2cb0 0x7f73bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73c6ffd640 1 -- 172.21.15.122:0/253104538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f73c80cf0d0 con 0x7f73d0101d80 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 -- 172.21.15.122:0/1173277164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 msgr2=0x55b28169cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b28169cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 -- 172.21.15.122:0/1173277164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 msgr2=0x55b281697da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b281697da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 -- 172.21.15.122:0/1173277164 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2815a2340 con 0x55b28169d370 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d569e640 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b28169f830 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f73c0012840 tx=0x7f73c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a788429ca77c0fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73d66a0640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73bc0a2cb0 0x7f73bc0a5170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f73c800efd0 tx=0x7f73c8051040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73bbfff640 1 -- 172.21.15.122:0/1173277164 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0012dc0 con 0x55b28169d370 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73bbfff640 1 -- 172.21.15.122:0/1173277164 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c0010c50 con 0x55b28169d370 2024-09-12T02:05:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.230+0000 7f73bbfff640 1 -- 172.21.15.122:0/1173277164 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c001fa00 con 0x55b28169d370 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 -- 172.21.15.122:0/1173277164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 msgr2=0x55b28169f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b28169f830 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f73c0012840 tx=0x7f73c0004730 comp rx=0 tx=0).stop 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 -- 172.21.15.122:0/1173277164 shutdown_connections 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b28169f830 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b28169cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 --2- 172.21.15.122:0/1173277164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b281697da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 -- 172.21.15.122:0/1173277164 >> 172.21.15.122:0/1173277164 conn(0x55b281693010 msgr2=0x55b281693410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 -- 172.21.15.122:0/1173277164 shutdown_connections 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 -- 172.21.15.122:0/1173277164 wait complete. 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.231+0000 7f73dae99740 1 -- start start 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 Processor -- start 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b2816a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b2816a9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b2816a0340 con 0x55b28169d370 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b281584290 con 0x55b2816979a0 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2816a04c0 con 0x55b2816982e0 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73dae99740 10 client.0 _ll_register_callbacks cb 0x55b2815b5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b2816a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b2816a9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57410/0 (socket says 172.21.15.122:57410) 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 -- 172.21.15.122:0/561521610 learned_addr learned my addr 172.21.15.122:0/561521610 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b2816a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 -- 172.21.15.122:0/561521610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 msgr2=0x55b2816a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 -- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 msgr2=0x55b2816a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b2816a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 -- 172.21.15.122:0/561521610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b28130e210 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73d569e640 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b2816a9200 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f73c0012810 tx=0x7f73c0021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d7935d88ba3028 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.232+0000 7f73b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73b97fa640 1 -- 172.21.15.122:0/561521610 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0010950 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73b97fa640 1 -- 172.21.15.122:0/561521610 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f73c001f7c0 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c0010c50 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73b97fa640 1 -- 172.21.15.122:0/561521610 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f73c001f9c0 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 10 client.18132 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b2813da620 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73b97fa640 1 -- 172.21.15.122:0/561521610 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c0010c50 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73b97fa640 1 -- 172.21.15.122:0/561521610 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f73c002aba0 con 0x55b28169d370 2024-09-12T02:05:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 10 client.18132 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 10 client.18132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 20 client.18132 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b2816ad630 0x55b2816afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 4 client.18132 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b281608a20 con 0x55b2816ad630 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.233+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b2816ad630 0x55b2816afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.234+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b2816ad630 0x55b2816afa20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba11a6d05179b5d2 server_cookie=5a3cbbcc29a951c6 in_seq=0 out_seq=0 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.234+0000 7f73b97fa640 10 client.18132 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.234+0000 7f73b97fa640 1 -- 172.21.15.122:0/561521610 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b281608a20 con 0x55b2816ad630 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.234+0000 7f73b97fa640 10 client.18132 handle_command_reply: tid=0 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 client.18132 shutdown 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 msgr2=0x55b2816a9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b2816a9200 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f73c0012810 tx=0x7f73c0021040 comp rx=0 tx=0).stop 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 shutdown_connections 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b2816ad630 0x55b2816afa20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b28169d370 0x55b2816a9200 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73b97fa640 0 client.18132 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 --2- 172.21.15.122:0/561521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2816979a0 0x55b2816a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 >> 172.21.15.122:0/561521610 conn(0x55b281693010 msgr2=0x55b281694820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 shutdown_connections 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 1 -- 172.21.15.122:0/561521610 wait complete. 2024-09-12T02:05:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.235+0000 7f73dae99740 20 client.18132 trim_cache size 0 max 16384 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 Processor -- start 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 -- start start 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 0x55b281694b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01adb00 con 0x55b2816979a0 2024-09-12T02:05:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01ab3d0 con 0x55b28169d370 2024-09-12T02:05:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73dae99740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01a9200 con 0x55b2816982e0 2024-09-12T02:05:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 0x55b281694b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 0x55b281694b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57424/0 (socket says 172.21.15.122:57424) 2024-09-12T02:05:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 -- 172.21.15.122:0/3242468090 learned_addr learned my addr 172.21.15.122:0/3242468090 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 -- 172.21.15.122:0/3242468090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 msgr2=0x55b2816a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 -- 172.21.15.122:0/3242468090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 msgr2=0x55b2816b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.249+0000 7f73d569e640 1 -- 172.21.15.122:0/3242468090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b281692e30 con 0x55b2816979a0 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.250+0000 7f73d569e640 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 0x55b281694b10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f73c002d610 tx=0x7f73c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacadc028d3c5123 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.250+0000 7f73bb7fe640 1 -- 172.21.15.122:0/3242468090 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c000bac0 con 0x55b2816979a0 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.250+0000 7f73bb7fe640 1 -- 172.21.15.122:0/3242468090 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c00057b0 con 0x55b2816979a0 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.250+0000 7f73bb7fe640 1 -- 172.21.15.122:0/3242468090 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c002d0d0 con 0x55b2816979a0 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.250+0000 7f73dae99740 1 -- 172.21.15.122:0/3242468090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 msgr2=0x55b281694b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.250+0000 7f73dae99740 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 0x55b281694b10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f73c002d610 tx=0x7f73c0000bc0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- 172.21.15.122:0/3242468090 shutdown_connections 2024-09-12T02:05:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816982e0 0x55b2816a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 --2- 172.21.15.122:0/3242468090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816979a0 0x55b281694b10 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- 172.21.15.122:0/3242468090 >> 172.21.15.122:0/3242468090 conn(0x55b281693010 msgr2=0x55b2816933f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- 172.21.15.122:0/3242468090 shutdown_connections 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- 172.21.15.122:0/3242468090 wait complete. 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- start start 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 Processor -- start 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816979a0 0x55b2816a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 0x55b2816a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01a9890 con 0x55b2816982e0 2024-09-12T02:05:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01aad00 con 0x55b28169d370 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73d01a9f60 con 0x55b2816979a0 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73dae99740 10 client.0 _ll_register_callbacks cb 0x7f73c000beb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73d5e9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 0x55b2816a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73d5e9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 0x55b2816a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57428/0 (socket says 172.21.15.122:57428) 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73d5e9f640 1 -- 172.21.15.122:0/3258772430 learned_addr learned my addr 172.21.15.122:0/3258772430 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.251+0000 7f73d569e640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816979a0 0x55b2816a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d569e640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d5e9f640 1 -- 172.21.15.122:0/3258772430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816979a0 msgr2=0x55b2816a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816979a0 0x55b2816a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d5e9f640 1 -- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 msgr2=0x55b2816a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d5e9f640 1 -- 172.21.15.122:0/3258772430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2816a8400 con 0x55b2816982e0 2024-09-12T02:05:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73d5e9f640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 0x55b2816a6e30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0096c0 tx=0x7f73cc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5a8b6671847e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73b97fa640 1 -- 172.21.15.122:0/3258772430 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc035040 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73b97fa640 1 -- 172.21.15.122:0/3258772430 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f73cc005510 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b281602d70 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73b97fa640 1 -- 172.21.15.122:0/3258772430 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f73cc004770 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 10 client.18138 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b2817182b0 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73b97fa640 1 -- 172.21.15.122:0/3258772430 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b281602d70 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73b97fa640 1 -- 172.21.15.122:0/3258772430 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f73cc031c10 con 0x55b2816982e0 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 10 client.18138 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 10 client.18138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 20 client.18138 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b281718c80 0x55b28171b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.252+0000 7f73dae99740 4 client.18138 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.253+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b28157f900 con 0x55b281718c80 2024-09-12T02:05:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.253+0000 7f73d569e640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b281718c80 0x55b28171b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.253+0000 7f73b97fa640 10 client.18138 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.253+0000 7f73d569e640 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b281718c80 0x55b28171b0a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63b702e1798bf69 server_cookie=c076798ec2766499 in_seq=0 out_seq=0 2024-09-12T02:05:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73b97fa640 1 -- 172.21.15.122:0/3258772430 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b28157f900 con 0x55b281718c80 2024-09-12T02:05:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73b97fa640 10 client.18138 handle_command_reply: tid=0 2024-09-12T02:05:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 client.18138 shutdown 2024-09-12T02:05:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 msgr2=0x55b2816a6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 0x55b2816a6e30 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0096c0 tx=0x7f73cc0045b0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 shutdown_connections 2024-09-12T02:05:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55b281718c80 0x55b28171b0a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28169d370 0x55b2816a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2816982e0 0x55b2816a6e30 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73b97fa640 0 client.18138 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 --2- 172.21.15.122:0/3258772430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2816979a0 0x55b2816a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 >> 172.21.15.122:0/3258772430 conn(0x55b281693010 msgr2=0x55b281693c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 shutdown_connections 2024-09-12T02:05:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 1 -- 172.21.15.122:0/3258772430 wait complete. 2024-09-12T02:05:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.361+0000 7f73dae99740 20 client.18138 trim_cache size 0 max 16384 2024-09-12T02:05:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:05:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:05:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:05:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:05:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73bc0a2cb0 msgr2=0x7f73bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73bc0a2cb0 0x7f73bc0a5170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f73c800efd0 tx=0x7f73c8051040 comp rx=0 tx=0).stop 2024-09-12T02:05:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 msgr2=0x7f73d0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 0x7f73d0102e30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f73c800eaa0 tx=0x7f73c800ef60 comp rx=0 tx=0).stop 2024-09-12T02:05:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 shutdown_connections 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73bc0a2cb0 0x7f73bc0a5170 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d0101d80 0x7f73d0102e30 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d01013c0 0x7f73d01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 --2- 172.21.15.122:0/253104538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73d01001d0 0x7f73d01042b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 >> 172.21.15.122:0/253104538 conn(0x7f73d00fb930 msgr2=0x7f73d00fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 shutdown_connections 2024-09-12T02:05:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:34.362+0000 7f73d8103640 1 -- 172.21.15.122:0/253104538 wait complete. 2024-09-12T02:05:35.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.51 MB/sec execute 202 sec latency 4036.549 ms 2024-09-12T02:05:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:34 smithi151 ceph-mon[35229]: pgmap v911: 97 pgs: 97 active+clean; 519 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.8 MiB/s wr, 347 op/s 2024-09-12T02:05:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:34 smithi183 ceph-mon[35127]: pgmap v911: 97 pgs: 97 active+clean; 519 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.8 MiB/s wr, 347 op/s 2024-09-12T02:05:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:34 smithi122 ceph-mon[30925]: pgmap v911: 97 pgs: 97 active+clean; 519 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.8 MiB/s wr, 347 op/s 2024-09-12T02:05:36.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.41 MB/sec execute 203 sec latency 5036.634 ms 2024-09-12T02:05:37.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.32 MB/sec execute 204 sec latency 6036.708 ms 2024-09-12T02:05:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:36 smithi151 ceph-mon[35229]: pgmap v912: 97 pgs: 97 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 332 op/s 2024-09-12T02:05:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:36 smithi183 ceph-mon[35127]: pgmap v912: 97 pgs: 97 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 332 op/s 2024-09-12T02:05:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:36 smithi122 ceph-mon[30925]: pgmap v912: 97 pgs: 97 active+clean; 518 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.0 MiB/s wr, 332 op/s 2024-09-12T02:05:38.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.22 MB/sec execute 205 sec latency 7036.805 ms 2024-09-12T02:05:39.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.13 MB/sec execute 206 sec latency 8036.933 ms 2024-09-12T02:05:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:38 smithi151 ceph-mon[35229]: pgmap v913: 97 pgs: 97 active+clean; 516 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 353 op/s 2024-09-12T02:05:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:38 smithi183 ceph-mon[35127]: pgmap v913: 97 pgs: 97 active+clean; 516 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 353 op/s 2024-09-12T02:05:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:38 smithi122 ceph-mon[30925]: pgmap v913: 97 pgs: 97 active+clean; 516 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 353 op/s 2024-09-12T02:05:40.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 19.04 MB/sec execute 207 sec latency 9037.039 ms 2024-09-12T02:05:41.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.95 MB/sec execute 208 sec latency 10037.129 ms 2024-09-12T02:05:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:40 smithi151 ceph-mon[35229]: pgmap v914: 97 pgs: 97 active+clean; 488 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-09-12T02:05:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:40 smithi183 ceph-mon[35127]: pgmap v914: 97 pgs: 97 active+clean; 488 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-09-12T02:05:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:40 smithi122 ceph-mon[30925]: pgmap v914: 97 pgs: 97 active+clean; 488 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-09-12T02:05:42.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.85 MB/sec execute 209 sec latency 11037.225 ms 2024-09-12T02:05:43.070 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.77 MB/sec execute 210 sec latency 12037.311 ms 2024-09-12T02:05:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:42 smithi151 ceph-mon[35229]: pgmap v915: 97 pgs: 97 active+clean; 488 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-09-12T02:05:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:42 smithi183 ceph-mon[35127]: pgmap v915: 97 pgs: 97 active+clean; 488 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-09-12T02:05:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:42 smithi122 ceph-mon[30925]: pgmap v915: 97 pgs: 97 active+clean; 488 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-09-12T02:05:44.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.68 MB/sec execute 211 sec latency 13037.399 ms 2024-09-12T02:05:45.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.59 MB/sec execute 212 sec latency 14037.497 ms 2024-09-12T02:05:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:44 smithi151 ceph-mon[35229]: pgmap v916: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-09-12T02:05:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:44 smithi183 ceph-mon[35127]: pgmap v916: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-09-12T02:05:45.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:44 smithi122 ceph-mon[30925]: pgmap v916: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 275 op/s 2024-09-12T02:05:46.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.50 MB/sec execute 213 sec latency 15037.560 ms 2024-09-12T02:05:47.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.41 MB/sec execute 214 sec latency 16037.612 ms 2024-09-12T02:05:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:46 smithi151 ceph-mon[35229]: pgmap v917: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-12T02:05:47.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:46 smithi183 ceph-mon[35127]: pgmap v917: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-12T02:05:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:46 smithi122 ceph-mon[30925]: pgmap v917: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-12T02:05:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 -- 172.21.15.122:0/3382327151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 msgr2=0x7fd3b0101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/3382327151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0101f50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009900 tx=0x7fd3a002a410 comp rx=0 tx=0).stop 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 -- 172.21.15.122:0/3382327151 shutdown_connections 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/3382327151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0103700 0x7fd3b0109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/3382327151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3b0102d40 0x7fd3b01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/3382327151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0101f50 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 -- 172.21.15.122:0/3382327151 >> 172.21.15.122:0/3382327151 conn(0x7fd3b00fd2b0 msgr2=0x7fd3b00ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 -- 172.21.15.122:0/3382327151 shutdown_connections 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 -- 172.21.15.122:0/3382327151 wait complete. 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.457+0000 7fd3b61d6640 1 Processor -- start 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 -- start start 2024-09-12T02:05:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0102d40 0x7fd3b0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3b0103700 0x7fd3b019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b010d7d0 con 0x7fd3b0101b50 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b010d650 con 0x7fd3b0103700 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3b61d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b010d950 con 0x7fd3b0102d40 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57674/0 (socket says 172.21.15.122:57674) 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 -- 172.21.15.122:0/261755273 learned_addr learned my addr 172.21.15.122:0/261755273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0102d40 0x7fd3b0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 -- 172.21.15.122:0/261755273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0102d40 msgr2=0x7fd3b0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0102d40 0x7fd3b0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 -- 172.21.15.122:0/261755273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3b0103700 msgr2=0x7fd3b019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3affff640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3b0103700 0x7fd3b019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3b0103700 0x7fd3b019f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3af7fe640 1 -- 172.21.15.122:0/261755273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b019f9e0 con 0x7fd3b0101b50 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.458+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0102d40 0x7fd3b0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.459+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0197dc0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0031610 tx=0x7fd3a00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.459+0000 7fd3acff9640 1 -- 172.21.15.122:0/261755273 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a003f030 con 0x7fd3b0101b50 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.459+0000 7fd3acff9640 1 -- 172.21.15.122:0/261755273 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a0003ce0 con 0x7fd3b0101b50 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.459+0000 7fd3acff9640 1 -- 172.21.15.122:0/261755273 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0035070 con 0x7fd3b0101b50 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.459+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b019fc70 con 0x7fd3b0101b50 2024-09-12T02:05:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.459+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b01a0110 con 0x7fd3b0101b50 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.461+0000 7fd3acff9640 1 -- 172.21.15.122:0/261755273 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd3a0042030 con 0x7fd3b0101b50 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.461+0000 7fd3b9050740 1 Processor -- start 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.461+0000 7fd3b9050740 1 -- start start 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41b82e0 0x5567b41bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41bd370 0x5567b41bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3b9050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567b411b2f0 con 0x5567b41b79a0 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3b9050740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567b40c3270 con 0x5567b41bd370 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3b9050740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567b41172b0 con 0x5567b41b82e0 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3acff9640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd3980a2cb0 0x7fd3980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.469 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57682/0 (socket says 172.21.15.122:57682) 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 -- 172.21.15.122:0/3183993185 learned_addr learned my addr 172.21.15.122:0/3183993185 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3acff9640 1 -- 172.21.15.122:0/261755273 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fd3a00ef640 con 0x7fd3b0101b50 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41bd370 0x5567b41bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd3980a2cb0 0x7fd3980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41b82e0 0x5567b41bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 -- 172.21.15.122:0/3183993185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41b82e0 msgr2=0x5567b41bcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41b82e0 0x5567b41bcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 -- 172.21.15.122:0/3183993185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41bd370 msgr2=0x5567b41bf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41bd370 0x5567b41bf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 -- 172.21.15.122:0/3183993185 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b40c2340 con 0x5567b41b79a0 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd3affff640 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b7da0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80138c0 tx=0x7fd3a8013d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625f6e3144fa9a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.462+0000 7fd395ffb640 1 -- 172.21.15.122:0/3183993185 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8013250 con 0x5567b41b79a0 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd395ffb640 1 -- 172.21.15.122:0/3183993185 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a8004030 con 0x5567b41b79a0 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd395ffb640 1 -- 172.21.15.122:0/3183993185 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a801d610 con 0x5567b41b79a0 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 -- 172.21.15.122:0/3183993185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 msgr2=0x5567b41b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3affff640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd3980a2cb0 0x7fd3980a5170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8020000 tx=0x7fd3a801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b7da0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80138c0 tx=0x7fd3a8013d80 comp rx=0 tx=0).stop 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 -- 172.21.15.122:0/3183993185 shutdown_connections 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41bd370 0x5567b41bf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41b82e0 0x5567b41bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3183993185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b7da0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 -- 172.21.15.122:0/3183993185 >> 172.21.15.122:0/3183993185 conn(0x5567b41b3010 msgr2=0x5567b41b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 -- 172.21.15.122:0/3183993185 shutdown_connections 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.463+0000 7fd3b9050740 1 -- 172.21.15.122:0/3183993185 wait complete. 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 -- start start 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 Processor -- start 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567b41c0340 con 0x5567b41b82e0 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567b40a4290 con 0x5567b41b79a0 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567b41c04c0 con 0x5567b41bd370 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3b9050740 10 client.0 _ll_register_callbacks cb 0x5567b40d5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57688/0 (socket says 172.21.15.122:57688) 2024-09-12T02:05:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3aeffd640 1 -- 172.21.15.122:0/278051423 learned_addr learned my addr 172.21.15.122:0/278051423 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3affff640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.464+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3aeffd640 1 -- 172.21.15.122:0/278051423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 msgr2=0x5567b41c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3aeffd640 1 -- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 msgr2=0x5567b41c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3aeffd640 1 -- 172.21.15.122:0/278051423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b3e2e210 con 0x5567b41b82e0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3affff640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c20e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009f80 tx=0x7fd3a4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb3446dc7d8e08c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd38effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd38effd640 1 -- 172.21.15.122:0/278051423 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4012920 con 0x5567b41b82e0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd38effd640 1 -- 172.21.15.122:0/278051423 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fd3a4012ac0 con 0x5567b41b82e0 2024-09-12T02:05:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a8004030 con 0x5567b41b82e0 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd38effd640 1 -- 172.21.15.122:0/278051423 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd3a4012cc0 con 0x5567b41b82e0 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3b9050740 10 client.18150 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5567b3efa620 con 0x5567b41b82e0 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.465+0000 7fd38effd640 1 -- 172.21.15.122:0/278051423 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a8004030 con 0x5567b41b82e0 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd38effd640 1 -- 172.21.15.122:0/278051423 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd3a400f040 con 0x5567b41b82e0 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3b9050740 10 client.18150 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3b9050740 10 client.18150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3b9050740 20 client.18150 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3b9050740 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b41cd850 0x5567b41cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3b9050740 4 client.18150 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b4128a20 con 0x5567b41cd850 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b41cd850 0x5567b41cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd38effd640 10 client.18150 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:47.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.466+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b41cd850 0x5567b41cfc40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfef92911343a4a1 server_cookie=86d75f41300221a5 in_seq=0 out_seq=0 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.467+0000 7fd38effd640 1 -- 172.21.15.122:0/278051423 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567b4128a20 con 0x5567b41cd850 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.467+0000 7fd38effd640 10 client.18150 handle_command_reply: tid=0 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.467+0000 7fd3b9050740 1 client.18150 shutdown 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 msgr2=0x5567b41c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c20e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009f80 tx=0x7fd3a4011870 comp rx=0 tx=0).stop 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 shutdown_connections 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b41cd850 0x5567b41cfc40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c20e0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd38effd640 0 client.18150 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 --2- 172.21.15.122:0/278051423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 >> 172.21.15.122:0/278051423 conn(0x5567b41b3010 msgr2=0x5567b41b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 shutdown_connections 2024-09-12T02:05:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 1 -- 172.21.15.122:0/278051423 wait complete. 2024-09-12T02:05:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.468+0000 7fd3b9050740 20 client.18150 trim_cache size 0 max 16384 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 Processor -- start 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 -- start start 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b82e0 0x5567b41c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01ab910 con 0x5567b41b79a0 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01a91e0 con 0x5567b41b82e0 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3b9050740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01a7010 con 0x5567b41bd370 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57694/0 (socket says 172.21.15.122:57694) 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 -- 172.21.15.122:0/3283653988 learned_addr learned my addr 172.21.15.122:0/3283653988 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 -- 172.21.15.122:0/3283653988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 msgr2=0x5567b41b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 -- 172.21.15.122:0/3283653988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b82e0 msgr2=0x5567b41c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b82e0 0x5567b41c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b82e0 0x5567b41c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.482+0000 7fd3af7fe640 1 -- 172.21.15.122:0/3283653988 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b41b2e30 con 0x5567b41b79a0 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b4400 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00b82b0 tx=0x7fd3a00ed010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9858322d2b54a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3957fa640 1 -- 172.21.15.122:0/3283653988 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0049040 con 0x5567b41b79a0 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3957fa640 1 -- 172.21.15.122:0/3283653988 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a00c0250 con 0x5567b41b79a0 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3957fa640 1 -- 172.21.15.122:0/3283653988 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a003b460 con 0x5567b41b79a0 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 -- 172.21.15.122:0/3283653988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 msgr2=0x5567b41b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b4400 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00b82b0 tx=0x7fd3a00ed010 comp rx=0 tx=0).stop 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 -- 172.21.15.122:0/3283653988 shutdown_connections 2024-09-12T02:05:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b82e0 0x5567b41c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 --2- 172.21.15.122:0/3283653988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b79a0 0x5567b41b4400 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 -- 172.21.15.122:0/3283653988 >> 172.21.15.122:0/3283653988 conn(0x5567b41b3010 msgr2=0x5567b41b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 -- 172.21.15.122:0/3283653988 shutdown_connections 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.483+0000 7fd3b9050740 1 -- 172.21.15.122:0/3283653988 wait complete. 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 -- start start 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 Processor -- start 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01a76a0 con 0x5567b41b82e0 2024-09-12T02:05:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01a8b10 con 0x5567b41b79a0 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01a7d70 con 0x5567b41bd370 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3b9050740 10 client.0 _ll_register_callbacks cb 0x7fd3a40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57706/0 (socket says 172.21.15.122:57706) 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3aeffd640 1 -- 172.21.15.122:0/1023361623 learned_addr learned my addr 172.21.15.122:0/1023361623 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.484+0000 7fd3affff640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3aeffd640 1 -- 172.21.15.122:0/1023361623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 msgr2=0x5567b41c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3aeffd640 1 -- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 msgr2=0x5567b41c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3aeffd640 1 -- 172.21.15.122:0/1023361623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b41c8a50 con 0x5567b41b82e0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3affff640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3aeffd640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c7480 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4019480 tx=0x7fd3a4012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5a04866c598582 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd38effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd38effd640 1 -- 172.21.15.122:0/1023361623 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4029040 con 0x5567b41b82e0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd38effd640 1 -- 172.21.15.122:0/1023361623 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fd3a4019cc0 con 0x5567b41b82e0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a00c0250 con 0x5567b41b82e0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd38effd640 1 -- 172.21.15.122:0/1023361623 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd3a4012720 con 0x5567b41b82e0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3b9050740 10 client.18156 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5567b4238ce0 con 0x5567b41b82e0 2024-09-12T02:05:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd38effd640 1 -- 172.21.15.122:0/1023361623 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a00c0250 con 0x5567b41b82e0 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.485+0000 7fd38effd640 1 -- 172.21.15.122:0/1023361623 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fd3a4004680 con 0x5567b41b82e0 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3b9050740 10 client.18156 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3b9050740 10 client.18156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3b9050740 20 client.18156 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3b9050740 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b4239710 0x5567b423bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3b9050740 4 client.18156 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567b4122d70 con 0x5567b4239710 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b4239710 0x5567b423bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b4239710 0x5567b423bb30 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753c64dbcb137c18 server_cookie=646ba2a61b59a53c in_seq=0 out_seq=0 2024-09-12T02:05:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.486+0000 7fd38effd640 10 client.18156 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.518+0000 7fd38effd640 1 -- 172.21.15.122:0/1023361623 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567b4122d70 con 0x5567b4239710 2024-09-12T02:05:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.518+0000 7fd38effd640 10 client.18156 handle_command_reply: tid=0 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 client.18156 shutdown 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 msgr2=0x5567b41c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c7480 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4019480 tx=0x7fd3a4012ee0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 shutdown_connections 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5567b4239710 0x5567b423bb30 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567b41bd370 0x5567b41c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd38effd640 0 client.18156 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567b41b82e0 0x5567b41c7480 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 --2- 172.21.15.122:0/1023361623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567b41b79a0 0x5567b41c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 >> 172.21.15.122:0/1023361623 conn(0x5567b41b3010 msgr2=0x5567b41b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 shutdown_connections 2024-09-12T02:05:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 1 -- 172.21.15.122:0/1023361623 wait complete. 2024-09-12T02:05:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b9050740 20 client.18156 trim_cache size 0 max 16384 2024-09-12T02:05:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:05:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:05:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:05:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd3980a2cb0 msgr2=0x7fd3980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd3980a2cb0 0x7fd3980a5170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8020000 tx=0x7fd3a801e000 comp rx=0 tx=0).stop 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 msgr2=0x7fd3b0197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.519+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0197dc0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0031610 tx=0x7fd3a00043b0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 shutdown_connections 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd3980a2cb0 0x7fd3980a5170 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3b0103700 0x7fd3b019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3b0102d40 0x7fd3b0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 --2- 172.21.15.122:0/261755273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b0101b50 0x7fd3b0197dc0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 >> 172.21.15.122:0/261755273 conn(0x7fd3b00fd2b0 msgr2=0x7fd3b01060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 shutdown_connections 2024-09-12T02:05:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:05:47.520+0000 7fd3b61d6640 1 -- 172.21.15.122:0/261755273 wait complete. 2024-09-12T02:05:48.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.33 MB/sec execute 215 sec latency 17037.676 ms 2024-09-12T02:05:49.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.24 MB/sec execute 216 sec latency 18037.758 ms 2024-09-12T02:05:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:48 smithi151 ceph-mon[35229]: pgmap v918: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-12T02:05:49.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:48 smithi183 ceph-mon[35127]: pgmap v918: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-12T02:05:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:48 smithi122 ceph-mon[30925]: pgmap v918: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-12T02:05:50.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.16 MB/sec execute 217 sec latency 19037.847 ms 2024-09-12T02:05:50.286 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:05:50.545 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:50.918 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:50 smithi122 ceph-mon[30925]: pgmap v919: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-09-12T02:05:51.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 -- 172.21.15.122:0/2834135076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 msgr2=0x7f574c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 --2- 172.21.15.122:0/2834135076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c107b80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5738009930 tx=0x7f573802a420 comp rx=0 tx=0).stop 2024-09-12T02:05:51.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 -- 172.21.15.122:0/2834135076 shutdown_connections 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 --2- 172.21.15.122:0/2834135076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f574c108290 0x7f574c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 --2- 172.21.15.122:0/2834135076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c107b80 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 --2- 172.21.15.122:0/2834135076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c102d90 0x7f574c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 -- 172.21.15.122:0/2834135076 >> 172.21.15.122:0/2834135076 conn(0x7f574c0fc7d0 msgr2=0x7f574c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 -- 172.21.15.122:0/2834135076 shutdown_connections 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.996+0000 7f5752ebc640 1 -- 172.21.15.122:0/2834135076 wait complete. 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 Processor -- start 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 -- start start 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f574c102d90 0x7f574c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c108290 0x7f574c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f574c112090 con 0x7f574c103740 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f574c111f10 con 0x7f574c108290 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5752ebc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f574c112210 con 0x7f574c102d90 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34266/0 (socket says 172.21.15.122:34266) 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 -- 172.21.15.122:0/1576283801 learned_addr learned my addr 172.21.15.122:0/1576283801 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5750c58640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f574c102d90 0x7f574c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f5751459640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c108290 0x7f574c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 -- 172.21.15.122:0/1576283801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f574c102d90 msgr2=0x7f574c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f574c102d90 0x7f574c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 -- 172.21.15.122:0/1576283801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c108290 msgr2=0x7f574c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c108290 0x7f574c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.997+0000 7f574bfff640 1 -- 172.21.15.122:0/1576283801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c1a69e0 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.998+0000 7f574bfff640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c100f30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5738005940 tx=0x7f57380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.998+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573803f030 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.998+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c1a6c70 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.998+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5738003ce0 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.998+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738035070 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.998+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c1a71a0 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:50.999+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c103170 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.001+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5738042030 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.001+0000 7f5749ffb640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f573c0a2b80 0x7f573c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.001+0000 7f5750c58640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f573c0a2b80 0x7f573c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.001+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f57380ef710 con 0x7f574c103740 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.002+0000 7f5750c58640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f573c0a2b80 0x7f573c0a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f574000abe0 tx=0x7f574000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:51.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.004+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57380bc630 con 0x7f574c103740 2024-09-12T02:05:51.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 18.08 MB/sec execute 218 sec latency 20037.922 ms 2024-09-12T02:05:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:50 smithi151 ceph-mon[35229]: pgmap v919: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-09-12T02:05:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:50 smithi183 ceph-mon[35127]: pgmap v919: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-09-12T02:05:51.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.215+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f574c10a2a0 con 0x7f574c103740 2024-09-12T02:05:51.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.215+0000 7f5749ffb640 1 -- 172.21.15.122:0/1576283801 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11931 (secure 0 0 0) 0x7f57380bbf30 con 0x7f574c103740 2024-09-12T02:05:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:05:51.216 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"btime":"2024-09-12T02:02:49:777282+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3451432768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3451432768},{"type":"v1","addr":"172.21.15.151:6833","nonce":3451432768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:02:49.747410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26737":{"gid":26737,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4050812174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4050812174},{"type":"v1","addr":"172.21.15.183:6835","nonce":4050812174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:05:51.216 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.217+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f573c0a2b80 msgr2=0x7f573c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.217+0000 7f5752ebc640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f573c0a2b80 0x7f573c0a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f574000abe0 tx=0x7f574000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 msgr2=0x7f574c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c100f30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5738005940 tx=0x7f57380042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 shutdown_connections 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f573c0a2b80 0x7f573c0a5040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c108290 0x7f574c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f574c103740 0x7f574c100f30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 --2- 172.21.15.122:0/1576283801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f574c102d90 0x7f574c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:51.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 >> 172.21.15.122:0/1576283801 conn(0x7f574c0fc7d0 msgr2=0x7f574c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:51.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 shutdown_connections 2024-09-12T02:05:51.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:51.218+0000 7f5752ebc640 1 -- 172.21.15.122:0/1576283801 wait complete. 2024-09-12T02:05:52.007 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 8a53007c --release --await' 2024-09-12T02:05:52.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 8a53007c --release --await 2024-09-12T02:05:52.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126248 17.99 MB/sec execute 219 sec latency 21037.981 ms 2024-09-12T02:05:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:51 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1576283801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:52.179 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:51 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1576283801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:51 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1576283801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:05:52.266 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.754+0000 7f33eb9f1640 1 -- 172.21.15.122:0/3807300269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 msgr2=0x7f33e4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.754+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/3807300269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e4105920 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009900 tx=0x7f33d402a470 comp rx=0 tx=0).stop 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 -- 172.21.15.122:0/3807300269 shutdown_connections 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/3807300269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4105f40 0x7f33e41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/3807300269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e4105920 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/3807300269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33e4100b30 0x7f33e4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 -- 172.21.15.122:0/3807300269 >> 172.21.15.122:0/3807300269 conn(0x7f33e40fc7a0 msgr2=0x7f33e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 -- 172.21.15.122:0/3807300269 shutdown_connections 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 -- 172.21.15.122:0/3807300269 wait complete. 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 Processor -- start 2024-09-12T02:05:52.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 -- start start 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33e4100b30 0x7f33e41a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e41a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4105f40 0x7f33e419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.755+0000 7f33eb9f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4118540 con 0x7f33e4101450 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33eb9f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41183c0 con 0x7f33e4105f40 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33eb9f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41186c0 con 0x7f33e4100b30 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e41a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e41a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34286/0 (socket says 172.21.15.122:34286) 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 -- 172.21.15.122:0/251044806 learned_addr learned my addr 172.21.15.122:0/251044806 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e978d640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33e4100b30 0x7f33e41a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4105f40 0x7f33e419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 -- 172.21.15.122:0/251044806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33e4100b30 msgr2=0x7f33e41a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33e4100b30 0x7f33e41a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 -- 172.21.15.122:0/251044806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4105f40 msgr2=0x7f33e419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4105f40 0x7f33e419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 -- 172.21.15.122:0/251044806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e419ef70 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33e8f8c640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e41a4b90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005ab0 tx=0x7f33d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33da7fc640 1 -- 172.21.15.122:0/251044806 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d403f030 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33da7fc640 1 -- 172.21.15.122:0/251044806 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d4003ce0 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e419f200 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33da7fc640 1 -- 172.21.15.122:0/251044806 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4035070 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.756+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41ab4b0 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.758+0000 7f33da7fc640 1 -- 172.21.15.122:0/251044806 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f33d4042030 con 0x7f33e4101450 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 Processor -- start 2024-09-12T02:05:52.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 -- start start 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33da7fc640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33d00a2b80 0x7f33d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf97630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55defbeef620 con 0x55defbf8a980 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55defbe96e50 con 0x55defbf95240 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.759+0000 7f33eddca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55defbeeb5e0 con 0x55defbf8a060 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e8f8c640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33d00a2b80 0x7f33d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf94d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34288/0 (socket says 172.21.15.122:34288) 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 -- 172.21.15.122:0/264837823 learned_addr learned my addr 172.21.15.122:0/264837823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:52.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33da7fc640 1 -- 172.21.15.122:0/251044806 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f33d40ef950 con 0x7f33e4101450 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf97630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 -- 172.21.15.122:0/264837823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 msgr2=0x55defbf8a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 -- 172.21.15.122:0/264837823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 msgr2=0x55defbf97630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf97630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 -- 172.21.15.122:0/264837823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55defbe87d40 con 0x55defbf8a980 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e8f8c640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33d00a2b80 0x7f33d00a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f33d40b8610 tx=0x7f33d40b1690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33e978d640 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf94d00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f33e000cc60 tx=0x7f33e000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bda4bb161efc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33cf7fe640 1 -- 172.21.15.122:0/264837823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0014070 con 0x55defbf8a980 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33cf7fe640 1 -- 172.21.15.122:0/264837823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e00047a0 con 0x55defbf8a980 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.760+0000 7f33cf7fe640 1 -- 172.21.15.122:0/264837823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e00107b0 con 0x55defbf8a980 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 -- 172.21.15.122:0/264837823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 msgr2=0x55defbf94d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf94d00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f33e000cc60 tx=0x7f33e000e720 comp rx=0 tx=0).stop 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 -- 172.21.15.122:0/264837823 shutdown_connections 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf97630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf94d00 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 --2- 172.21.15.122:0/264837823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 -- 172.21.15.122:0/264837823 >> 172.21.15.122:0/264837823 conn(0x55defbf865f0 msgr2=0x55defbf869d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 -- 172.21.15.122:0/264837823 shutdown_connections 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.761+0000 7f33eddca740 1 -- 172.21.15.122:0/264837823 wait complete. 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 -- start start 2024-09-12T02:05:52.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 Processor -- start 2024-09-12T02:05:52.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf92610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf92b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55defbf97e10 con 0x55defbf8a980 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55defbe77f10 con 0x55defbf95240 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55defbf97f90 con 0x55defbf8a060 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33eddca740 10 client.0 _ll_register_callbacks cb 0x55defbe9bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33e978d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf92610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33e978d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf92610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34300/0 (socket says 172.21.15.122:34300) 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33e978d640 1 -- 172.21.15.122:0/93875643 learned_addr learned my addr 172.21.15.122:0/93875643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.762+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf92b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e978d640 1 -- 172.21.15.122:0/93875643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 msgr2=0x55defbf920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e978d640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf920d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e978d640 1 -- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 msgr2=0x55defbf92b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e978d640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf92b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e978d640 1 -- 172.21.15.122:0/93875643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55defbbfe580 con 0x55defbf8a980 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf92b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33e978d640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf92610 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f33e000f010 tx=0x7f33e0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81d8be23723174 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33ccff9640 1 -- 172.21.15.122:0/93875643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0027060 con 0x55defbf8a980 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33ccff9640 1 -- 172.21.15.122:0/93875643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f33e0005160 con 0x55defbf8a980 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e00047a0 con 0x55defbf8a980 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33ccff9640 1 -- 172.21.15.122:0/93875643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f33e0014070 con 0x55defbf8a980 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33eddca740 10 client.18174 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.763+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55defbcc8960 con 0x55defbf8a980 2024-09-12T02:05:52.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33ccff9640 1 -- 172.21.15.122:0/93875643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e00047a0 con 0x55defbf8a980 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33ccff9640 1 -- 172.21.15.122:0/93875643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f33e002c070 con 0x55defbf8a980 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33eddca740 10 client.18174 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33eddca740 10 client.18174 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33eddca740 20 client.18174 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33eddca740 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defbfa7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33eddca740 4 client.18174 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55defbefcb90 con 0x55defbfa4d40 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.764+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defbfa7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33ccff9640 10 client.18174 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defbfa7130 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5987c9f8606bc1 server_cookie=89afbca2ba56073f in_seq=0 out_seq=0 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33ccff9640 1 -- 172.21.15.122:0/93875643 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55defbefcb90 con 0x55defbfa4d40 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33ccff9640 10 client.18174 handle_command_reply: tid=0 2024-09-12T02:05:52.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 client.18174 shutdown 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 msgr2=0x55defbf92610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf92610 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f33e000f010 tx=0x7f33e0005d60 comp rx=0 tx=0).stop 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 shutdown_connections 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defbfa7130 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf95240 0x55defbf92b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33ccff9640 0 client.18174 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf92610 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 --2- 172.21.15.122:0/93875643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.765+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 >> 172.21.15.122:0/93875643 conn(0x55defbf865f0 msgr2=0x55defbf8c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.766+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 shutdown_connections 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.766+0000 7f33eddca740 1 -- 172.21.15.122:0/93875643 wait complete. 2024-09-12T02:05:52.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.766+0000 7f33eddca740 20 client.18174 trim_cache size 0 max 16384 2024-09-12T02:05:52.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.782+0000 7f33eddca740 1 Processor -- start 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.782+0000 7f33eddca740 1 -- start start 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33eddca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33eddca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a980 0x55defbf85640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33eddca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 0x55defbfac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33eddca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b63d0 con 0x55defbf95240 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33eddca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b3ca0 con 0x55defbf8a980 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33eddca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b1ad0 con 0x55defbf8a060 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 0x55defbfac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 0x55defbfac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34310/0 (socket says 172.21.15.122:34310) 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 -- 172.21.15.122:0/4121510102 learned_addr learned my addr 172.21.15.122:0/4121510102 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e978d640 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a980 0x55defbf85640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 -- 172.21.15.122:0/4121510102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 msgr2=0x55defbf8cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 -- 172.21.15.122:0/4121510102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a980 msgr2=0x55defbf85640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a980 0x55defbf85640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 -- 172.21.15.122:0/4121510102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55defbf89830 con 0x55defbf95240 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 0x55defbfac700 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00ac30 tx=0x7f33dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3710606ecc1568de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.783+0000 7f33ceffd640 1 -- 172.21.15.122:0/4121510102 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc004d20 con 0x55defbf95240 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33ceffd640 1 -- 172.21.15.122:0/4121510102 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc005020 con 0x55defbf95240 2024-09-12T02:05:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33ceffd640 1 -- 172.21.15.122:0/4121510102 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc01d660 con 0x55defbf95240 2024-09-12T02:05:52.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 -- 172.21.15.122:0/4121510102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 msgr2=0x55defbfac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 0x55defbfac700 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00ac30 tx=0x7f33dc004a40 comp rx=0 tx=0).stop 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 -- 172.21.15.122:0/4121510102 shutdown_connections 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf95240 0x55defbfac700 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a980 0x55defbf85640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 --2- 172.21.15.122:0/4121510102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf8a060 0x55defbf8cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.784+0000 7f33eddca740 1 -- 172.21.15.122:0/4121510102 >> 172.21.15.122:0/4121510102 conn(0x55defbf865f0 msgr2=0x55defbf84f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.785+0000 7f33eddca740 1 -- 172.21.15.122:0/4121510102 shutdown_connections 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.785+0000 7f33eddca740 1 -- 172.21.15.122:0/4121510102 wait complete. 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.785+0000 7f33eddca740 1 -- start start 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.785+0000 7f33eddca740 1 Processor -- start 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.785+0000 7f33eddca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a060 0x55defbf991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.785+0000 7f33eddca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33eddca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf95240 0x55defbf99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33eddca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b2160 con 0x55defbf8a980 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33eddca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b35d0 con 0x55defbf8a060 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33eddca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b2830 con 0x55defbf95240 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33eddca740 10 client.0 _ll_register_callbacks cb 0x7f33e000cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e9f8e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a060 0x55defbf991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34322/0 (socket says 172.21.15.122:34322) 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 -- 172.21.15.122:0/467538182 learned_addr learned my addr 172.21.15.122:0/467538182 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf95240 0x55defbf99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 -- 172.21.15.122:0/467538182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf95240 msgr2=0x55defbf99c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf95240 0x55defbf99c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 -- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a060 msgr2=0x55defbf991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a060 0x55defbf991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 -- 172.21.15.122:0/467538182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55defc00e920 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a060 0x55defbf991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33e978d640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf996f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e000cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a7bc1d17c095d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33ccff9640 1 -- 172.21.15.122:0/467538182 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0032030 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33ccff9640 1 -- 172.21.15.122:0/467538182 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f33e001f040 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.786+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33dc005020 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33ccff9640 1 -- 172.21.15.122:0/467538182 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f33e0022910 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 10 client.18180 fetch_fsmap learned FSMap version 41 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55defc00fc60 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33ccff9640 1 -- 172.21.15.122:0/467538182 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc005020 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33ccff9640 1 -- 172.21.15.122:0/467538182 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f33e002c070 con 0x55defbf8a980 2024-09-12T02:05:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 10 client.18180 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 10 client.18180 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 20 client.18180 populate_metadata read hostname 'smithi122' 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defc012330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 4 client.18180 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8a53007c", "release": true, "await": true}] 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8a53007c", "release": true, "await": true}) -- 0x55defbef70a0 con 0x55defbfa4d40 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.787+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defc012330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.788+0000 7f33e9f8e640 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defc012330 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f352780f8963904 server_cookie=4b69d15df2baf320 in_seq=0 out_seq=0 2024-09-12T02:05:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.788+0000 7f33ccff9640 10 client.18180 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:52.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.851+0000 7f33ccff9640 1 -- 172.21.15.122:0/467538182 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55defbef70a0 con 0x55defbfa4d40 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.851+0000 7f33ccff9640 10 client.18180 handle_command_reply: tid=0 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.851+0000 7f33eddca740 1 client.18180 shutdown 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 msgr2=0x55defbf996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf996f0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e000cfd0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 shutdown_connections 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55defbfa4d40 0x55defc012330 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55defbf95240 0x55defbf99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55defbf8a980 0x55defbf996f0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 --2- 172.21.15.122:0/467538182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55defbf8a060 0x55defbf991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 >> 172.21.15.122:0/467538182 conn(0x55defbf865f0 msgr2=0x55defbf92870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33ccff9640 0 client.18180 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 shutdown_connections 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 1 -- 172.21.15.122:0/467538182 wait complete. 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.852+0000 7f33eddca740 20 client.18180 trim_cache size 0 max 16384 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 41, 2024-09-12T02:05:52.854 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 72, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "8a53007c": { 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 72, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 141.8, 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:05:52.855 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:05:52.856 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:05:52.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33d00a2b80 msgr2=0x7f33d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33d00a2b80 0x7f33d00a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f33d40b8610 tx=0x7f33d40b1690 comp rx=0 tx=0).stop 2024-09-12T02:05:52.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 msgr2=0x7f33e41a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:05:52.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e41a4b90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005ab0 tx=0x7f33d40042f0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 shutdown_connections 2024-09-12T02:05:52.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f33d00a2b80 0x7f33d00a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4105f40 0x7f33e419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e4101450 0x7f33e41a4b90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 --2- 172.21.15.122:0/251044806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33e4100b30 0x7f33e41a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:05:52.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 >> 172.21.15.122:0/251044806 conn(0x7f33e40fc7a0 msgr2=0x7f33e41038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:05:52.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 shutdown_connections 2024-09-12T02:05:52.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:05:52.853+0000 7f33eb9f1640 1 -- 172.21.15.122:0/251044806 wait complete. 2024-09-12T02:05:53.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126367 17.95 MB/sec execute 220 sec latency 21831.350 ms 2024-09-12T02:05:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:52 smithi151 ceph-mon[35229]: pgmap v920: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 853 KiB/s wr, 48 op/s 2024-09-12T02:05:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:52 smithi183 ceph-mon[35127]: pgmap v920: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 853 KiB/s wr, 48 op/s 2024-09-12T02:05:53.231 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:52 smithi122 ceph-mon[30925]: pgmap v920: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 853 KiB/s wr, 48 op/s 2024-09-12T02:05:53.571 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8a53007c', total seconds elapsed: 25.1 2024-09-12T02:05:54.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 127000 18.03 MB/sec execute 221 sec latency 58.292 ms 2024-09-12T02:05:55.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 127439 18.05 MB/sec execute 222 sec latency 62.203 ms 2024-09-12T02:05:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:54 smithi151 ceph-mon[35229]: pgmap v921: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 738 KiB/s wr, 49 op/s 2024-09-12T02:05:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:54 smithi183 ceph-mon[35127]: pgmap v921: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 738 KiB/s wr, 49 op/s 2024-09-12T02:05:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:54 smithi122 ceph-mon[30925]: pgmap v921: 97 pgs: 97 active+clean; 482 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 738 KiB/s wr, 49 op/s 2024-09-12T02:05:56.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 128064 18.09 MB/sec execute 223 sec latency 94.637 ms 2024-09-12T02:05:57.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 128663 18.18 MB/sec execute 224 sec latency 54.548 ms 2024-09-12T02:05:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:56 smithi151 ceph-mon[35229]: pgmap v922: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 511 KiB/s wr, 89 op/s 2024-09-12T02:05:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:56 smithi183 ceph-mon[35127]: pgmap v922: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 511 KiB/s wr, 89 op/s 2024-09-12T02:05:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:56 smithi122 ceph-mon[30925]: pgmap v922: 97 pgs: 97 active+clean; 486 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 511 KiB/s wr, 89 op/s 2024-09-12T02:05:58.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 129388 18.26 MB/sec execute 225 sec latency 54.436 ms 2024-09-12T02:05:58.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:58 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:05:58.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:58 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:05:59.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 129919 18.31 MB/sec execute 226 sec latency 91.732 ms 2024-09-12T02:05:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:58 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:05:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:59 smithi183 ceph-mon[35127]: pgmap v923: 97 pgs: 97 active+clean; 492 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T02:05:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:05:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:05:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:05:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:05:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:59 smithi122 ceph-mon[30925]: pgmap v923: 97 pgs: 97 active+clean; 492 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T02:05:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:05:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:05:59.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:05:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:06:00.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 130438 18.36 MB/sec execute 227 sec latency 141.659 ms 2024-09-12T02:06:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:59 smithi151 ceph-mon[35229]: pgmap v923: 97 pgs: 97 active+clean; 492 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T02:06:00.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:06:00.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:06:00.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:05:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:06:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.614+0000 7ff608c65640 1 -- 172.21.15.122:0/3368104377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff604104f00 msgr2=0x7ff6041072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.614+0000 7ff608c65640 1 --2- 172.21.15.122:0/3368104377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff604104f00 0x7ff6041072f0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f802a410 comp rx=0 tx=0).stop 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 -- 172.21.15.122:0/3368104377 shutdown_connections 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 --2- 172.21.15.122:0/3368104377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff604107830 0x7ff604109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 --2- 172.21.15.122:0/3368104377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff604104f00 0x7ff6041072f0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 --2- 172.21.15.122:0/3368104377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6041025d0 0x7ff6041049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 -- 172.21.15.122:0/3368104377 >> 172.21.15.122:0/3368104377 conn(0x7ff6040fc1b0 msgr2=0x7ff6040fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 -- 172.21.15.122:0/3368104377 shutdown_connections 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 -- 172.21.15.122:0/3368104377 wait complete. 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 Processor -- start 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.615+0000 7ff608c65640 1 -- start start 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 0x7ff60419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff604104f00 0x7ff60419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff604107830 0x7ff6041a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d810 con 0x7ff6041025d0 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d690 con 0x7ff604107830 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d990 con 0x7ff604104f00 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 0x7ff60419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 0x7ff60419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44768/0 (socket says 172.21.15.122:44768) 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 -- 172.21.15.122:0/4037057743 learned_addr learned my addr 172.21.15.122:0/4037057743 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff601d9b640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff604104f00 0x7ff60419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff602d9d640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff604107830 0x7ff6041a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 -- 172.21.15.122:0/4037057743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff604104f00 msgr2=0x7ff60419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff604104f00 0x7ff60419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 -- 172.21.15.122:0/4037057743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff604107830 msgr2=0x7ff6041a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff604107830 0x7ff6041a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 -- 172.21.15.122:0/4037057743 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6041a1c50 con 0x7ff6041025d0 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff602d9d640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff604107830 0x7ff6041a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff60259c640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 0x7ff60419a030 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9f0 tx=0x7ff5f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff5f37fe640 1 -- 172.21.15.122:0/4037057743 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400cd80 con 0x7ff6041025d0 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff5f37fe640 1 -- 172.21.15.122:0/4037057743 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4010910 con 0x7ff6041025d0 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.616+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6041a1f40 con 0x7ff6041025d0 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.617+0000 7ff5f37fe640 1 -- 172.21.15.122:0/4037057743 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4018990 con 0x7ff6041025d0 2024-09-12T02:06:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.617+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6041a23a0 con 0x7ff6041025d0 2024-09-12T02:06:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.619+0000 7ff5f37fe640 1 -- 172.21.15.122:0/4037057743 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff5f4010ac0 con 0x7ff6041025d0 2024-09-12T02:06:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.619+0000 7ff60ba04740 1 Processor -- start 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 -- start start 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae471da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae476da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae479830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558bae3d52f0 con 0x558bae477370 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558bae37d270 con 0x558bae4722e0 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60ba04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bae3d12b0 con 0x558bae4719a0 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae476da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae471da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae476da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:59874/0 (socket says 172.21.15.122:59874) 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff60259c640 1 -- 172.21.15.122:0/774278723 learned_addr learned my addr 172.21.15.122:0/774278723 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff601d9b640 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae479830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff602d9d640 1 -- 172.21.15.122:0/774278723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 msgr2=0x558bae476da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff602d9d640 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae476da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff602d9d640 1 -- 172.21.15.122:0/774278723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 msgr2=0x558bae479830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff602d9d640 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae479830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff602d9d640 1 -- 172.21.15.122:0/774278723 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bae37c340 con 0x558bae4719a0 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff5f37fe640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5ec0a2c30 0x7ff5ec0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.620+0000 7ff601d9b640 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae479830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.621+0000 7ff602d9d640 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae471da0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff60406b380 tx=0x7ff5fc011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d4d520a0963d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.621+0000 7ff60259c640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5ec0a2c30 0x7ff5ec0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.621+0000 7ff5f0ff9640 1 -- 172.21.15.122:0/774278723 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc004d20 con 0x558bae4719a0 2024-09-12T02:06:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.621+0000 7ff5f0ff9640 1 -- 172.21.15.122:0/774278723 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc00f970 con 0x558bae4719a0 2024-09-12T02:06:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.621+0000 7ff5f37fe640 1 -- 172.21.15.122:0/4037057743 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7ff5f40cf330 con 0x7ff6041025d0 2024-09-12T02:06:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.621+0000 7ff5f0ff9640 1 -- 172.21.15.122:0/774278723 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0044d0 con 0x558bae4719a0 2024-09-12T02:06:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 -- 172.21.15.122:0/774278723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 msgr2=0x558bae471da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae471da0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff60406b380 tx=0x7ff5fc011bf0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60259c640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5ec0a2c30 0x7ff5ec0a50f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40966e0 tx=0x7ff5f40d2db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 -- 172.21.15.122:0/774278723 shutdown_connections 2024-09-12T02:06:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae479830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae476da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 --2- 172.21.15.122:0/774278723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae471da0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 -- 172.21.15.122:0/774278723 >> 172.21.15.122:0/774278723 conn(0x558bae46d010 msgr2=0x558bae46d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 -- 172.21.15.122:0/774278723 shutdown_connections 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 -- 172.21.15.122:0/774278723 wait complete. 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 -- start start 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.622+0000 7ff60ba04740 1 Processor -- start 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae4719a0 0x558bae47bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae47c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae477370 0x558bae483160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558bae47a390 con 0x558bae4719a0 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558bae35e290 con 0x558bae4722e0 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bae47a510 con 0x558bae477370 2024-09-12T02:06:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff60ba04740 10 client.0 _ll_register_callbacks cb 0x558bae38f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae4719a0 0x558bae47bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae4719a0 0x558bae47bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44782/0 (socket says 172.21.15.122:44782) 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff602d9d640 1 -- 172.21.15.122:0/821840659 learned_addr learned my addr 172.21.15.122:0/821840659 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae47c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff602d9d640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae477370 0x558bae483160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 -- 172.21.15.122:0/821840659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae477370 msgr2=0x558bae483160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae477370 0x558bae483160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 -- 172.21.15.122:0/821840659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae4719a0 msgr2=0x558bae47bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae4719a0 0x558bae47bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 -- 172.21.15.122:0/821840659 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bae0e8210 con 0x558bae4722e0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff601d9b640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae47c120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009230 tx=0x7ff5f8009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f61efcff01186c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.623+0000 7ff5e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:06:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/821840659 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8035040 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5fc00f970 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/821840659 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7ff5f8004040 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/821840659 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff5f8004240 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff60ba04740 10 client.27212 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558bae1b4620 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.624+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/821840659 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc00f970 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/821840659 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff5f8004960 con 0x558bae4722e0 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff60ba04740 10 client.27212 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff60ba04740 10 client.27212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff60ba04740 20 client.27212 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff60ba04740 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae487800 0x558bae489bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff60ba04740 4 client.27212 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bae3e2a20 con 0x558bae487800 2024-09-12T02:06:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff602d9d640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae487800 0x558bae489bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff5e9ffb640 10 client.27212 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.625+0000 7ff602d9d640 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae487800 0x558bae489bf0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462e9c00234e6696 server_cookie=29fa8e86595cb4bd in_seq=0 out_seq=0 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/821840659 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558bae3e2a20 con 0x558bae487800 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff5e9ffb640 10 client.27212 handle_command_reply: tid=0 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 client.27212 shutdown 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 msgr2=0x558bae47c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae47c120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009230 tx=0x7ff5f8009670 comp rx=0 tx=0).stop 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 shutdown_connections 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae487800 0x558bae489bf0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae477370 0x558bae483160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae47c120 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff5e9ffb640 0 client.27212 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 --2- 172.21.15.122:0/821840659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae4719a0 0x558bae47bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 >> 172.21.15.122:0/821840659 conn(0x558bae46d010 msgr2=0x558bae46e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 shutdown_connections 2024-09-12T02:06:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 1 -- 172.21.15.122:0/821840659 wait complete. 2024-09-12T02:06:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.627+0000 7ff60ba04740 20 client.27212 trim_cache size 0 max 16384 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 Processor -- start 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 -- start start 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae47a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae483120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae46c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041ada60 con 0x558bae477370 2024-09-12T02:06:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041ab330 con 0x558bae4722e0 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff60ba04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a9160 con 0x558bae4719a0 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae47a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae46c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae46c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44788/0 (socket says 172.21.15.122:44788) 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff602d9d640 1 -- 172.21.15.122:0/2391876414 learned_addr learned my addr 172.21.15.122:0/2391876414 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.644+0000 7ff601d9b640 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae483120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff602d9d640 1 -- 172.21.15.122:0/2391876414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 msgr2=0x558bae47a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae47a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff602d9d640 1 -- 172.21.15.122:0/2391876414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 msgr2=0x558bae483120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae483120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff602d9d640 1 -- 172.21.15.122:0/2391876414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bae48a270 con 0x558bae477370 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff601d9b640 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae483120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae46c0b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc011730 tx=0x7ff5fc00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c137439921942be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff5ebfff640 1 -- 172.21.15.122:0/2391876414 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc003c80 con 0x558bae477370 2024-09-12T02:06:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff5ebfff640 1 -- 172.21.15.122:0/2391876414 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc01f040 con 0x558bae477370 2024-09-12T02:06:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.645+0000 7ff5ebfff640 1 -- 172.21.15.122:0/2391876414 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc005a70 con 0x558bae477370 2024-09-12T02:06:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 -- 172.21.15.122:0/2391876414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 msgr2=0x558bae46c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae46c0b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc011730 tx=0x7ff5fc00fbe0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 -- 172.21.15.122:0/2391876414 shutdown_connections 2024-09-12T02:06:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae46c0b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4722e0 0x558bae483120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2391876414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4719a0 0x558bae47a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 -- 172.21.15.122:0/2391876414 >> 172.21.15.122:0/2391876414 conn(0x558bae46d010 msgr2=0x558bae46b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 -- 172.21.15.122:0/2391876414 shutdown_connections 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.646+0000 7ff60ba04740 1 -- 172.21.15.122:0/2391876414 wait complete. 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff60ba04740 1 -- start start 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff60ba04740 1 Processor -- start 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4719a0 0x558bae480a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 0x558bae480f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff60ba04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae481f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff60ba04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a97f0 con 0x558bae477370 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.647+0000 7ff601d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 0x558bae480f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff601d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 0x558bae480f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:52852/0 (socket says 172.21.15.122:52852) 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff601d9b640 1 -- 172.21.15.122:0/2447083701 learned_addr learned my addr 172.21.15.122:0/2447083701 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041aac60 con 0x558bae4719a0 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae481f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a9ec0 con 0x558bae4722e0 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff60ba04740 10 client.0 _ll_register_callbacks cb 0x7ff5f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4719a0 0x558bae480a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 -- 172.21.15.122:0/2447083701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 msgr2=0x558bae480f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 0x558bae480f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 -- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4719a0 msgr2=0x558bae480a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4719a0 0x558bae480a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 -- 172.21.15.122:0/2447083701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bae482560 con 0x558bae477370 2024-09-12T02:06:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff601d9b640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 0x558bae480f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff602d9d640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae481f50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc013010 tx=0x7ff5fc00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a33982f5a01fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff5e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/2447083701 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc01e040 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/2447083701 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7ff5fc0050c0 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.648+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5fc01f040 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/2447083701 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff5fc0052c0 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 10 client.18198 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558bae4faee0 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/2447083701 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc01f040 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/2447083701 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff5fc044050 con 0x558bae477370 2024-09-12T02:06:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 10 client.18198 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 20 client.18198 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae4fb590 0x558bae4fd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 4 client.18198 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bae3dcd70 con 0x558bae4fb590 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.649+0000 7ff601d9b640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae4fb590 0x558bae4fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.650+0000 7ff5e9ffb640 10 client.18198 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.650+0000 7ff601d9b640 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae4fb590 0x558bae4fd980 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a08c45a831dd7d server_cookie=ffba82a15cc63fa2 in_seq=0 out_seq=0 2024-09-12T02:06:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff5e9ffb640 1 -- 172.21.15.122:0/2447083701 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bae3dcd70 con 0x558bae4fb590 2024-09-12T02:06:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff5e9ffb640 10 client.18198 handle_command_reply: tid=0 2024-09-12T02:06:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 client.18198 shutdown 2024-09-12T02:06:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 msgr2=0x558bae481f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae481f50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc013010 tx=0x7ff5fc00ff90 comp rx=0 tx=0).stop 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 shutdown_connections 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558bae4fb590 0x558bae4fd980 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558bae477370 0x558bae481f50 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bae4722e0 0x558bae480f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 --2- 172.21.15.122:0/2447083701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bae4719a0 0x558bae480a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 >> 172.21.15.122:0/2447083701 conn(0x558bae46d010 msgr2=0x558bae46e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.823+0000 7ff5e9ffb640 0 client.18198 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.824+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 shutdown_connections 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.824+0000 7ff60ba04740 1 -- 172.21.15.122:0/2447083701 wait complete. 2024-09-12T02:06:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.824+0000 7ff60ba04740 20 client.18198 trim_cache size 0 max 16384 2024-09-12T02:06:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:06:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:06:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:06:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5ec0a2c30 msgr2=0x7ff5ec0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5ec0a2c30 0x7ff5ec0a50f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40966e0 tx=0x7ff5f40d2db0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 msgr2=0x7ff60419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 0x7ff60419a030 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9f0 tx=0x7ff5f400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 shutdown_connections 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff5ec0a2c30 0x7ff5ec0a50f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff604107830 0x7ff6041a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff604104f00 0x7ff60419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 --2- 172.21.15.122:0/4037057743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6041025d0 0x7ff60419a030 secure :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9f0 tx=0x7ff5f400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:06:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 >> 172.21.15.122:0/4037057743 conn(0x7ff6040fc1b0 msgr2=0x7ff604101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 shutdown_connections 2024-09-12T02:06:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:00.825+0000 7ff608c65640 1 -- 172.21.15.122:0/4037057743 wait complete. 2024-09-12T02:06:01.071 INFO:tasks.workunit.client.0.smithi122.stdout: 10 130984 18.41 MB/sec execute 228 sec latency 187.456 ms 2024-09-12T02:06:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:00 smithi151 ceph-mon[35229]: pgmap v924: 97 pgs: 97 active+clean; 494 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-09-12T02:06:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:00 smithi183 ceph-mon[35127]: pgmap v924: 97 pgs: 97 active+clean; 494 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-09-12T02:06:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:00 smithi122 ceph-mon[30925]: pgmap v924: 97 pgs: 97 active+clean; 494 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-09-12T02:06:02.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 131619 18.46 MB/sec execute 229 sec latency 69.606 ms 2024-09-12T02:06:03.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 132252 18.53 MB/sec execute 230 sec latency 69.990 ms 2024-09-12T02:06:03.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:02 smithi151 ceph-mon[35229]: pgmap v925: 97 pgs: 97 active+clean; 498 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.7 MiB/s wr, 343 op/s 2024-09-12T02:06:03.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:02 smithi183 ceph-mon[35127]: pgmap v925: 97 pgs: 97 active+clean; 498 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.7 MiB/s wr, 343 op/s 2024-09-12T02:06:03.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:02 smithi122 ceph-mon[30925]: pgmap v925: 97 pgs: 97 active+clean; 498 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.7 MiB/s wr, 343 op/s 2024-09-12T02:06:04.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 132885 18.60 MB/sec execute 231 sec latency 73.359 ms 2024-09-12T02:06:05.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 133468 18.68 MB/sec execute 232 sec latency 106.923 ms 2024-09-12T02:06:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:04 smithi151 ceph-mon[35229]: pgmap v926: 97 pgs: 97 active+clean; 506 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-09-12T02:06:05.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:04 smithi183 ceph-mon[35127]: pgmap v926: 97 pgs: 97 active+clean; 506 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-09-12T02:06:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:04 smithi122 ceph-mon[30925]: pgmap v926: 97 pgs: 97 active+clean; 506 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.2 MiB/s wr, 502 op/s 2024-09-12T02:06:06.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 134092 18.70 MB/sec execute 233 sec latency 59.886 ms 2024-09-12T02:06:07.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 134737 18.79 MB/sec execute 234 sec latency 52.062 ms 2024-09-12T02:06:07.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:06 smithi151 ceph-mon[35229]: pgmap v927: 97 pgs: 97 active+clean; 507 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.6 MiB/s wr, 516 op/s 2024-09-12T02:06:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:06 smithi183 ceph-mon[35127]: pgmap v927: 97 pgs: 97 active+clean; 507 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.6 MiB/s wr, 516 op/s 2024-09-12T02:06:07.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:06 smithi122 ceph-mon[30925]: pgmap v927: 97 pgs: 97 active+clean; 507 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.6 MiB/s wr, 516 op/s 2024-09-12T02:06:08.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 135445 18.86 MB/sec execute 235 sec latency 66.837 ms 2024-09-12T02:06:09.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 136089 18.97 MB/sec execute 236 sec latency 80.540 ms 2024-09-12T02:06:09.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:08 smithi151 ceph-mon[35229]: pgmap v928: 97 pgs: 97 active+clean; 511 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-09-12T02:06:09.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:08 smithi183 ceph-mon[35127]: pgmap v928: 97 pgs: 97 active+clean; 511 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-09-12T02:06:09.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:08 smithi122 ceph-mon[30925]: pgmap v928: 97 pgs: 97 active+clean; 511 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-09-12T02:06:10.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 136750 19.01 MB/sec execute 237 sec latency 58.657 ms 2024-09-12T02:06:11.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 137276 19.07 MB/sec execute 238 sec latency 79.897 ms 2024-09-12T02:06:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:10 smithi151 ceph-mon[35229]: pgmap v929: 97 pgs: 97 active+clean; 520 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 574 op/s 2024-09-12T02:06:11.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:10 smithi183 ceph-mon[35127]: pgmap v929: 97 pgs: 97 active+clean; 520 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 574 op/s 2024-09-12T02:06:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:10 smithi122 ceph-mon[30925]: pgmap v929: 97 pgs: 97 active+clean; 520 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 574 op/s 2024-09-12T02:06:12.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 137846 19.12 MB/sec execute 239 sec latency 54.146 ms 2024-09-12T02:06:13.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 138545 19.16 MB/sec execute 240 sec latency 65.015 ms 2024-09-12T02:06:13.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:12 smithi151 ceph-mon[35229]: pgmap v930: 97 pgs: 97 active+clean; 522 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-09-12T02:06:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:12 smithi183 ceph-mon[35127]: pgmap v930: 97 pgs: 97 active+clean; 522 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-09-12T02:06:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:12 smithi122 ceph-mon[30925]: pgmap v930: 97 pgs: 97 active+clean; 522 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 -- 172.21.15.122:0/2967442479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 msgr2=0x7f4c60107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 --2- 172.21.15.122:0/2967442479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c60107370 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009900 tx=0x7f4c5402a410 comp rx=0 tx=0).stop 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 -- 172.21.15.122:0/2967442479 shutdown_connections 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 --2- 172.21.15.122:0/2967442479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c601078b0 0x7f4c60109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 --2- 172.21.15.122:0/2967442479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c60107370 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 --2- 172.21.15.122:0/2967442479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 -- 172.21.15.122:0/2967442479 >> 172.21.15.122:0/2967442479 conn(0x7f4c600fc1b0 msgr2=0x7f4c600fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 -- 172.21.15.122:0/2967442479 shutdown_connections 2024-09-12T02:06:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.931+0000 7f4c65a23640 1 -- 172.21.15.122:0/2967442479 wait complete. 2024-09-12T02:06:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 Processor -- start 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 -- start start 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c601983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c601078b0 0x7f4c6019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6010d800 con 0x7f4c60104f80 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6010d680 con 0x7f4c601078b0 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.932+0000 7f4c65a23640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6010d980 con 0x7f4c60102650 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c601983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c601983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54436/0 (socket says 172.21.15.122:54436) 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/371959538 learned_addr learned my addr 172.21.15.122:0/371959538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33626/0 (socket says 172.21.15.122:33626) 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c601078b0 0x7f4c6019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 -- 172.21.15.122:0/371959538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c601078b0 msgr2=0x7f4c6019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c601078b0 0x7f4c6019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 -- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 msgr2=0x7f4c601983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c601983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 -- 172.21.15.122:0/371959538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6019fa80 con 0x7f4c60102650 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c5effd640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60197e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9f0 tx=0x7f4c5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c64a21640 1 -- 172.21.15.122:0/371959538 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000ccc0 con 0x7f4c60102650 2024-09-12T02:06:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c64a21640 1 -- 172.21.15.122:0/371959538 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50005020 con 0x7f4c60102650 2024-09-12T02:06:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6019fd70 con 0x7f4c60102650 2024-09-12T02:06:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.933+0000 7f4c64a21640 1 -- 172.21.15.122:0/371959538 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500049e0 con 0x7f4c60102650 2024-09-12T02:06:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.934+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a0180 con 0x7f4c60102650 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c64a21640 1 -- 172.21.15.122:0/371959538 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4c50005560 con 0x7f4c60102650 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 Processor -- start 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 -- start start 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41077fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410784da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f410787830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f4106e32f0 con 0x55f410785370 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f41068b270 con 0x55f4107802e0 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c687bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f4106df2b0 con 0x55f41077f9a0 2024-09-12T02:06:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c64a21640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4c480a2c30 0x7f4c480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410784da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.936+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f410787830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410784da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40908/0 (socket says 172.21.15.122:40908) 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f410787830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54440/0 (socket says 172.21.15.122:54440) 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5effd640 1 -- 172.21.15.122:0/3809322153 learned_addr learned my addr 172.21.15.122:0/3809322153 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41077fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4c480a2c30 0x7f4c480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c64a21640 1 -- 172.21.15.122:0/371959538 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4c500cef90 con 0x7f4c60102650 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/3809322153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 msgr2=0x55f41077fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41077fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/3809322153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 msgr2=0x55f410784da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410784da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/3809322153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41068a340 con 0x55f410785370 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5effd640 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410784da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f410787830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005ab0 tx=0x7f4c54005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eba6ac9468cf9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4c480a2c30 0x7f4c480a50f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c6006b380 tx=0x7f4c58009e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c45ffb640 1 -- 172.21.15.122:0/3809322153 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54040070 con 0x55f410785370 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c45ffb640 1 -- 172.21.15.122:0/3809322153 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c540319b0 con 0x55f410785370 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.937+0000 7f4c45ffb640 1 -- 172.21.15.122:0/3809322153 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54035040 con 0x55f410785370 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 -- 172.21.15.122:0/3809322153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 msgr2=0x55f410787830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f410787830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005ab0 tx=0x7f4c54005de0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 -- 172.21.15.122:0/3809322153 shutdown_connections 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f410787830 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410784da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 --2- 172.21.15.122:0/3809322153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41077fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 -- 172.21.15.122:0/3809322153 >> 172.21.15.122:0/3809322153 conn(0x55f41077b010 msgr2=0x55f41077b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 -- 172.21.15.122:0/3809322153 shutdown_connections 2024-09-12T02:06:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.938+0000 7f4c687bb740 1 -- 172.21.15.122:0/3809322153 wait complete. 2024-09-12T02:06:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 -- start start 2024-09-12T02:06:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 Processor -- start 2024-09-12T02:06:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f410789b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f4107910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f4107882e0 con 0x55f4107802e0 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f41066c290 con 0x55f41077f9a0 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f410788460 con 0x55f410785370 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c687bb740 10 client.0 _ll_register_callbacks cb 0x55f41069d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f410789b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f4107910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f410789b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40922/0 (socket says 172.21.15.122:40922) 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54442/0 (socket says 172.21.15.122:54442) 2024-09-12T02:06:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.939+0000 7f4c5effd640 1 -- 172.21.15.122:0/2161113719 learned_addr learned my addr 172.21.15.122:0/2161113719 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5effd640 1 -- 172.21.15.122:0/2161113719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 msgr2=0x55f4107910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f4107910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5effd640 1 -- 172.21.15.122:0/2161113719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 msgr2=0x55f41078a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5effd640 1 -- 172.21.15.122:0/2161113719 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4103f6210 con 0x55f41077f9a0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f4107910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f410789b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000c6a0 tx=0x7f4c5009bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87139ce794f82784 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c3effd640 1 -- 172.21.15.122:0/2161113719 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001c030 con 0x55f41077f9a0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c540319b0 con 0x55f41077f9a0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c3effd640 1 -- 172.21.15.122:0/2161113719 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4c500860c0 con 0x55f41077f9a0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c3effd640 1 -- 172.21.15.122:0/2161113719 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4c5009b7f0 con 0x55f41077f9a0 2024-09-12T02:06:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c687bb740 10 client.27230 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f4104c2620 con 0x55f41077f9a0 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c3effd640 1 -- 172.21.15.122:0/2161113719 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c540319b0 con 0x55f41077f9a0 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.940+0000 7f4c3effd640 1 -- 172.21.15.122:0/2161113719 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4c500cc030 con 0x55f41077f9a0 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c687bb740 10 client.27230 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c687bb740 10 client.27230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c687bb740 20 client.27230 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410797c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c687bb740 4 client.27230 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4106f0a20 con 0x55f410795810 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410797c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410797c00 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c478f734b1a3f7 server_cookie=1b7ccb87ce39cb29 in_seq=0 out_seq=0 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.941+0000 7f4c3effd640 10 client.27230 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c3effd640 1 -- 172.21.15.122:0/2161113719 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4106f0a20 con 0x55f410795810 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c3effd640 10 client.27230 handle_command_reply: tid=0 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 client.27230 shutdown 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 msgr2=0x55f410789b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f410789b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000c6a0 tx=0x7f4c5009bef0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 shutdown_connections 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410797c00 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f4107910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c3effd640 0 client.27230 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2161113719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f410789b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 >> 172.21.15.122:0/2161113719 conn(0x55f41077b010 msgr2=0x55f41077c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 shutdown_connections 2024-09-12T02:06:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.942+0000 7f4c687bb740 1 -- 172.21.15.122:0/2161113719 wait complete. 2024-09-12T02:06:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.943+0000 7f4c687bb740 20 client.27230 trim_cache size 0 max 16384 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.958+0000 7f4c687bb740 1 Processor -- start 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.958+0000 7f4c687bb740 1 -- start start 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41078a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410791080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f41077bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c687bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a9680 con 0x55f410785370 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c687bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a6f50 con 0x55f4107802e0 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c687bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a4d80 con 0x55f41077f9a0 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410791080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f41077bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410791080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40926/0 (socket says 172.21.15.122:40926) 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f41077bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54456/0 (socket says 172.21.15.122:54456) 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/2982958466 learned_addr learned my addr 172.21.15.122:0/2982958466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41078a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 -- 172.21.15.122:0/2982958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 msgr2=0x55f410791080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410791080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 -- 172.21.15.122:0/2982958466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 msgr2=0x55f41077bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f41077bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 -- 172.21.15.122:0/2982958466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f410794180 con 0x55f41077f9a0 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f41077bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.959+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41078a000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c50097c50 tx=0x7f4c50099d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba083bb1fcf02377 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c457fa640 1 -- 172.21.15.122:0/2982958466 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001c070 con 0x55f41077f9a0 2024-09-12T02:06:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c457fa640 1 -- 172.21.15.122:0/2982958466 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5009fba0 con 0x55f41077f9a0 2024-09-12T02:06:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c457fa640 1 -- 172.21.15.122:0/2982958466 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50004e20 con 0x55f41077f9a0 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 -- 172.21.15.122:0/2982958466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 msgr2=0x55f41078a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41078a000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c50097c50 tx=0x7f4c50099d00 comp rx=0 tx=0).stop 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 -- 172.21.15.122:0/2982958466 shutdown_connections 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f410785370 0x55f41077bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f4107802e0 0x55f410791080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2982958466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f41077f9a0 0x55f41078a000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.960+0000 7f4c687bb740 1 -- 172.21.15.122:0/2982958466 >> 172.21.15.122:0/2982958466 conn(0x55f41077b010 msgr2=0x55f4107898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.961+0000 7f4c687bb740 1 -- 172.21.15.122:0/2982958466 shutdown_connections 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.961+0000 7f4c687bb740 1 -- 172.21.15.122:0/2982958466 wait complete. 2024-09-12T02:06:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.961+0000 7f4c687bb740 1 -- start start 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.961+0000 7f4c687bb740 1 Processor -- start 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.961+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f41078e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c687bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f41078f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c687bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a5410 con 0x55f4107802e0 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c687bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a6880 con 0x55f41077f9a0 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c687bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a5ae0 con 0x55f410785370 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c687bb740 10 client.0 _ll_register_callbacks cb 0x7f4c500cc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54460/0 (socket says 172.21.15.122:54460) 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/2759777253 learned_addr learned my addr 172.21.15.122:0/2759777253 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5f7fe640 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f41078f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/2759777253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 msgr2=0x55f41078f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f41078f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 msgr2=0x55f41078e3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f41078e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 -- 172.21.15.122:0/2759777253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41078ff40 con 0x55f4107802e0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c5e7fc640 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078e920 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55f41077be90 tx=0x7f4c540315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7601e2d3161fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.962+0000 7f4c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c3effd640 1 -- 172.21.15.122:0/2759777253 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54040040 con 0x55f4107802e0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c3effd640 1 -- 172.21.15.122:0/2759777253 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f4c54005040 con 0x55f4107802e0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5009fba0 con 0x55f4107802e0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c3effd640 1 -- 172.21.15.122:0/2759777253 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4c54060050 con 0x55f4107802e0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 10 client.18216 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f4107912e0 con 0x55f4107802e0 2024-09-12T02:06:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c3effd640 1 -- 172.21.15.122:0/2759777253 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5009fba0 con 0x55f4107802e0 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c3effd640 1 -- 172.21.15.122:0/2759777253 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4c540035f0 con 0x55f4107802e0 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 10 client.18216 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 20 client.18216 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410802810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 4 client.18216 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4106ead70 con 0x55f410795810 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.963+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410802810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.964+0000 7f4c3effd640 10 client.18216 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:13.964+0000 7f4c5effd640 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410802810 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c405060f3e4c58 server_cookie=68e0c6e156c053dc in_seq=0 out_seq=0 2024-09-12T02:06:14.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 138991 19.18 MB/sec execute 241 sec latency 77.376 ms 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c3effd640 1 -- 172.21.15.122:0/2759777253 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4106ead70 con 0x55f410795810 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c3effd640 10 client.18216 handle_command_reply: tid=0 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 client.18216 shutdown 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 msgr2=0x55f41078e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078e920 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55f41077be90 tx=0x7f4c540315b0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 shutdown_connections 2024-09-12T02:06:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f410795810 0x55f410802810 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f410785370 0x55f41078f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c3effd640 0 client.18216 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4107802e0 0x55f41078e920 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 --2- 172.21.15.122:0/2759777253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f41077f9a0 0x55f41078e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 >> 172.21.15.122:0/2759777253 conn(0x55f41077b010 msgr2=0x55f4107799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 shutdown_connections 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 1 -- 172.21.15.122:0/2759777253 wait complete. 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.212+0000 7f4c687bb740 20 client.18216 trim_cache size 0 max 16384 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:06:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:06:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:06:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4c480a2c30 msgr2=0x7f4c480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4c480a2c30 0x7f4c480a50f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c6006b380 tx=0x7f4c58009e90 comp rx=0 tx=0).stop 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 msgr2=0x7f4c60197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60197e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9f0 tx=0x7f4c5000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 shutdown_connections 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f4c480a2c30 0x7f4c480a50f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c601078b0 0x7f4c6019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c60104f80 0x7f4c601983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 --2- 172.21.15.122:0/371959538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c60102650 0x7f4c60197e60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 >> 172.21.15.122:0/371959538 conn(0x7f4c600fc1b0 msgr2=0x7f4c60102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 shutdown_connections 2024-09-12T02:06:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:14.213+0000 7f4c65a23640 1 -- 172.21.15.122:0/371959538 wait complete. 2024-09-12T02:06:15.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 139763 19.30 MB/sec execute 242 sec latency 164.202 ms 2024-09-12T02:06:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:14 smithi151 ceph-mon[35229]: pgmap v931: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-09-12T02:06:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:14 smithi183 ceph-mon[35127]: pgmap v931: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-09-12T02:06:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:14 smithi122 ceph-mon[30925]: pgmap v931: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-09-12T02:06:16.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 140340 19.34 MB/sec execute 243 sec latency 63.725 ms 2024-09-12T02:06:17.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 141071 19.44 MB/sec execute 244 sec latency 61.745 ms 2024-09-12T02:06:17.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:16 smithi151 ceph-mon[35229]: pgmap v932: 97 pgs: 97 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.1 MiB/s wr, 429 op/s 2024-09-12T02:06:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:16 smithi183 ceph-mon[35127]: pgmap v932: 97 pgs: 97 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.1 MiB/s wr, 429 op/s 2024-09-12T02:06:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:16 smithi122 ceph-mon[30925]: pgmap v932: 97 pgs: 97 active+clean; 529 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.1 MiB/s wr, 429 op/s 2024-09-12T02:06:18.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 141538 19.45 MB/sec execute 245 sec latency 108.390 ms 2024-09-12T02:06:19.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 142183 19.53 MB/sec execute 246 sec latency 59.068 ms 2024-09-12T02:06:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:18 smithi151 ceph-mon[35229]: pgmap v933: 97 pgs: 97 active+clean; 519 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.3 MiB/s wr, 518 op/s 2024-09-12T02:06:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:18 smithi183 ceph-mon[35127]: pgmap v933: 97 pgs: 97 active+clean; 519 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.3 MiB/s wr, 518 op/s 2024-09-12T02:06:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:18 smithi122 ceph-mon[30925]: pgmap v933: 97 pgs: 97 active+clean; 519 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.3 MiB/s wr, 518 op/s 2024-09-12T02:06:20.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 142743 19.54 MB/sec execute 247 sec latency 61.704 ms 2024-09-12T02:06:21.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143435 19.60 MB/sec execute 248 sec latency 68.462 ms 2024-09-12T02:06:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:20 smithi151 ceph-mon[35229]: pgmap v934: 97 pgs: 97 active+clean; 512 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.8 MiB/s wr, 545 op/s 2024-09-12T02:06:21.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:20 smithi183 ceph-mon[35127]: pgmap v934: 97 pgs: 97 active+clean; 512 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.8 MiB/s wr, 545 op/s 2024-09-12T02:06:21.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:20 smithi122 ceph-mon[30925]: pgmap v934: 97 pgs: 97 active+clean; 512 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.8 MiB/s wr, 545 op/s 2024-09-12T02:06:22.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144118 19.68 MB/sec execute 249 sec latency 55.974 ms 2024-09-12T02:06:23.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144876 19.77 MB/sec execute 250 sec latency 81.002 ms 2024-09-12T02:06:23.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:22 smithi151 ceph-mon[35229]: pgmap v935: 97 pgs: 97 active+clean; 517 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-09-12T02:06:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:22 smithi183 ceph-mon[35127]: pgmap v935: 97 pgs: 97 active+clean; 517 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-09-12T02:06:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:22 smithi122 ceph-mon[30925]: pgmap v935: 97 pgs: 97 active+clean; 517 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-09-12T02:06:24.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 145411 19.83 MB/sec execute 251 sec latency 341.413 ms 2024-09-12T02:06:25.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 145924 19.85 MB/sec execute 252 sec latency 137.148 ms 2024-09-12T02:06:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:24 smithi151 ceph-mon[35229]: pgmap v936: 97 pgs: 97 active+clean; 528 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-12T02:06:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:24 smithi183 ceph-mon[35127]: pgmap v936: 97 pgs: 97 active+clean; 528 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-12T02:06:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:24 smithi122 ceph-mon[30925]: pgmap v936: 97 pgs: 97 active+clean; 528 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-12T02:06:26.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 146405 19.88 MB/sec execute 253 sec latency 167.553 ms 2024-09-12T02:06:27.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147064 19.92 MB/sec execute 254 sec latency 52.667 ms 2024-09-12T02:06:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:26 smithi151 ceph-mon[35229]: pgmap v937: 97 pgs: 97 active+clean; 530 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.6 MiB/s wr, 489 op/s 2024-09-12T02:06:27.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:26 smithi183 ceph-mon[35127]: pgmap v937: 97 pgs: 97 active+clean; 530 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.6 MiB/s wr, 489 op/s 2024-09-12T02:06:27.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:26 smithi122 ceph-mon[30925]: pgmap v937: 97 pgs: 97 active+clean; 530 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.6 MiB/s wr, 489 op/s 2024-09-12T02:06:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 -- 172.21.15.122:0/4166586786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 msgr2=0x7fe57c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4166586786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c1020b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe570009900 tx=0x7fe57002a410 comp rx=0 tx=0).stop 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 -- 172.21.15.122:0/4166586786 shutdown_connections 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4166586786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c1025f0 0x7fe57c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4166586786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c1020b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4166586786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe57c100a40 0x7fe57c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 -- 172.21.15.122:0/4166586786 >> 172.21.15.122:0/4166586786 conn(0x7fe57c0fc1e0 msgr2=0x7fe57c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 -- 172.21.15.122:0/4166586786 shutdown_connections 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.312+0000 7fe581bc4640 1 -- 172.21.15.122:0/4166586786 wait complete. 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 Processor -- start 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 -- start start 2024-09-12T02:06:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c100a40 0x7fe57c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe57c1025f0 0x7fe57c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c10c710 con 0x7fe57c101c30 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c10c590 con 0x7fe57c100a40 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe581bc4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c10c890 con 0x7fe57c1025f0 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42092/0 (socket says 172.21.15.122:42092) 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57affd640 1 -- 172.21.15.122:0/4008657640 learned_addr learned my addr 172.21.15.122:0/4008657640 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57bfff640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe57c1025f0 0x7fe57c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c100a40 0x7fe57c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57affd640 1 -- 172.21.15.122:0/4008657640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe57c1025f0 msgr2=0x7fe57c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.313+0000 7fe57affd640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe57c1025f0 0x7fe57c19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe57affd640 1 -- 172.21.15.122:0/4008657640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c100a40 msgr2=0x7fe57c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe57affd640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c100a40 0x7fe57c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe57affd640 1 -- 172.21.15.122:0/4008657640 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c19f9e0 con 0x7fe57c101c30 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c100a40 0x7fe57c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe57affd640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c198300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe570031440 tx=0x7fe570032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe578ff9640 1 -- 172.21.15.122:0/4008657640 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57003f030 con 0x7fe57c101c30 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c19fc70 con 0x7fe57c101c30 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe578ff9640 1 -- 172.21.15.122:0/4008657640 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe570003ce0 con 0x7fe57c101c30 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe578ff9640 1 -- 172.21.15.122:0/4008657640 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570035070 con 0x7fe57c101c30 2024-09-12T02:06:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.314+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c1a01b0 con 0x7fe57c101c30 2024-09-12T02:06:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 Processor -- start 2024-09-12T02:06:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe578ff9640 1 -- 172.21.15.122:0/4008657640 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe570042030 con 0x7fe57c101c30 2024-09-12T02:06:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 -- start start 2024-09-12T02:06:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5870a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f5870f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f58711c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f586692f0 con 0x559f5870f780 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559f58611270 con 0x559f5870a6f0 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe584a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f586652b0 con 0x559f58709db0 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5870a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe57affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f58711c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5870a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:46630/0 (socket says 172.21.15.122:46630) 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe57bfff640 1 -- 172.21.15.122:0/3235251823 learned_addr learned my addr 172.21.15.122:0/3235251823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.317+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f5870f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57bfff640 1 -- 172.21.15.122:0/3235251823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 msgr2=0x559f5870f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57bfff640 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f5870f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57bfff640 1 -- 172.21.15.122:0/3235251823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 msgr2=0x559f58711c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57bfff640 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f58711c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57bfff640 1 -- 172.21.15.122:0/3235251823 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f58610340 con 0x559f58709db0 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57affd640 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f58711c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f5870f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe578ff9640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5640a2c30 0x7fe5640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5640a2c30 0x7fe5640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe578ff9640 1 -- 172.21.15.122:0/4008657640 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe5700ef220 con 0x7fe57c101c30 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe57bfff640 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5870a1b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe574009fd0 tx=0x7fe5740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fad83ced29c5f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.318+0000 7fe561ffb640 1 -- 172.21.15.122:0/3235251823 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574004d20 con 0x559f58709db0 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe561ffb640 1 -- 172.21.15.122:0/3235251823 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe574004050 con 0x559f58709db0 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe561ffb640 1 -- 172.21.15.122:0/3235251823 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574005280 con 0x559f58709db0 2024-09-12T02:06:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5640a2c30 0x7fe5640a50f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559f5870b120 tx=0x7fe56c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 -- 172.21.15.122:0/3235251823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 msgr2=0x559f5870a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5870a1b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe574009fd0 tx=0x7fe5740118d0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 -- 172.21.15.122:0/3235251823 shutdown_connections 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f58711c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f5870f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 --2- 172.21.15.122:0/3235251823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5870a1b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.319+0000 7fe584a3c740 1 -- 172.21.15.122:0/3235251823 >> 172.21.15.122:0/3235251823 conn(0x559f58701010 msgr2=0x559f58701410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 -- 172.21.15.122:0/3235251823 shutdown_connections 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 -- 172.21.15.122:0/3235251823 wait complete. 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 -- start start 2024-09-12T02:06:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 Processor -- start 2024-09-12T02:06:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58714030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 0x559f58714570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.320+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f5871b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe584a3c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f58712750 con 0x559f5870f780 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe584a3c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559f585f2290 con 0x559f58709db0 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe584a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f587128d0 con 0x559f5870a6f0 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe584a3c740 10 client.0 _ll_register_callbacks cb 0x559f58623140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f5871b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 0x559f58714570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58714030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 0x559f58714570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:46646/0 (socket says 172.21.15.122:46646) 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f5871b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42104/0 (socket says 172.21.15.122:42104) 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58714030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:54892/0 (socket says 172.21.15.122:54892) 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57affd640 1 -- 172.21.15.122:0/1644765850 learned_addr learned my addr 172.21.15.122:0/1644765850 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 -- 172.21.15.122:0/1644765850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 msgr2=0x559f58714570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 0x559f58714570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 -- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 msgr2=0x559f58714030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58714030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 -- 172.21.15.122:0/1644765850 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5837c210 con 0x559f5870f780 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57affd640 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 0x559f58714570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f5871b5b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe57401b040 tx=0x7fe574004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6272e69d416c2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe55affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe55affd640 1 -- 172.21.15.122:0/1644765850 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574029020 con 0x559f5870f780 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe55affd640 1 -- 172.21.15.122:0/1644765850 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe57400f4e0 con 0x559f5870f780 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.321+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe574004050 con 0x559f5870f780 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe55affd640 1 -- 172.21.15.122:0/1644765850 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe57400f6e0 con 0x559f5870f780 2024-09-12T02:06:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 10 client.18228 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559f58448620 con 0x559f5870f780 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe55affd640 1 -- 172.21.15.122:0/1644765850 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe574004050 con 0x559f5870f780 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe55affd640 1 -- 172.21.15.122:0/1644765850 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe574025740 con 0x559f5870f780 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 10 client.18228 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 10 client.18228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 20 client.18228 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5871fac0 0x559f58721eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 4 client.18228 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.322+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f58676a20 con 0x559f5871fac0 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.323+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5871fac0 0x559f58721eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.323+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5871fac0 0x559f58721eb0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce315c5edc8ec1b server_cookie=f80904f75029e69a in_seq=0 out_seq=0 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.323+0000 7fe55affd640 10 client.18228 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe55affd640 1 -- 172.21.15.122:0/1644765850 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f58676a20 con 0x559f5871fac0 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe55affd640 10 client.18228 handle_command_reply: tid=0 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 client.18228 shutdown 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 msgr2=0x559f5871b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f5871b5b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe57401b040 tx=0x7fe574004430 comp rx=0 tx=0).stop 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 shutdown_connections 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5871fac0 0x559f58721eb0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870f780 0x559f5871b5b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe55affd640 0 client.18228 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.324+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f5870a6f0 0x559f58714570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.325+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1644765850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58714030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.325+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 >> 172.21.15.122:0/1644765850 conn(0x559f58701010 msgr2=0x559f58702820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.325+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 shutdown_connections 2024-09-12T02:06:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.325+0000 7fe584a3c740 1 -- 172.21.15.122:0/1644765850 wait complete. 2024-09-12T02:06:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.325+0000 7fe584a3c740 20 client.18228 trim_cache size 0 max 16384 2024-09-12T02:06:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 Processor -- start 2024-09-12T02:06:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 -- start start 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 0x559f587000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5871b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f587267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1a96e0 con 0x559f58701010 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1a6fb0 con 0x559f5870a6f0 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe584a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1a4de0 con 0x559f58709db0 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 0x559f587000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 0x559f587000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42118/0 (socket says 172.21.15.122:42118) 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 -- 172.21.15.122:0/1365020040 learned_addr learned my addr 172.21.15.122:0/1365020040 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57affd640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5871b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f587267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 -- 172.21.15.122:0/1365020040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 msgr2=0x559f5871b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5871b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 -- 172.21.15.122:0/1365020040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 msgr2=0x559f587267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f587267b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57b7fe640 1 -- 172.21.15.122:0/1365020040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f58722220 con 0x559f58701010 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57affd640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5871b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.341+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f587267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.342+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 0x559f587000b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe57c101a90 tx=0x7fe56c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6d69821e9eb779 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.342+0000 7fe5617fa640 1 -- 172.21.15.122:0/1365020040 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c003af0 con 0x559f58701010 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.342+0000 7fe5617fa640 1 -- 172.21.15.122:0/1365020040 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c003c90 con 0x559f58701010 2024-09-12T02:06:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.342+0000 7fe5617fa640 1 -- 172.21.15.122:0/1365020040 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c021670 con 0x559f58701010 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 -- 172.21.15.122:0/1365020040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 msgr2=0x559f587000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 0x559f587000b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe57c101a90 tx=0x7fe56c017000 comp rx=0 tx=0).stop 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 -- 172.21.15.122:0/1365020040 shutdown_connections 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f5870a6f0 0x559f587267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58709db0 0x559f5871b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1365020040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f58701010 0x559f587000b0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 -- 172.21.15.122:0/1365020040 >> 172.21.15.122:0/1365020040 conn(0x559f5870f2a0 msgr2=0x559f586ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 -- 172.21.15.122:0/1365020040 shutdown_connections 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 -- 172.21.15.122:0/1365020040 wait complete. 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 -- start start 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.343+0000 7fe584a3c740 1 Processor -- start 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58701010 0x559f58718950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58718e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 0x559f58719ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1a5470 con 0x559f5870a6f0 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1a68e0 con 0x559f58709db0 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1a5b40 con 0x559f58701010 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 10 client.0 _ll_register_callbacks cb 0x7fe5700f2b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 0x559f58719ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 0x559f58719ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42128/0 (socket says 172.21.15.122:42128) 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 -- 172.21.15.122:0/1225835043 learned_addr learned my addr 172.21.15.122:0/1225835043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57affd640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58718e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58701010 0x559f58718950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 -- 172.21.15.122:0/1225835043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58701010 msgr2=0x559f58718950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58701010 0x559f58718950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 -- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 msgr2=0x559f58718e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58718e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 -- 172.21.15.122:0/1225835043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5871a4b0 con 0x559f5870a6f0 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58701010 0x559f58718950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe57bfff640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 0x559f58719ea0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe574002230 tx=0x7fe574004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27e310b8b95c3b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe55affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe55affd640 1 -- 172.21.15.122:0/1225835043 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574029040 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe55affd640 1 -- 172.21.15.122:0/1225835043 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7fe574009b30 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56c003c90 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe55affd640 1 -- 172.21.15.122:0/1225835043 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe574025470 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.344+0000 7fe584a3c740 10 client.18234 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559f5871b7c0 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe55affd640 1 -- 172.21.15.122:0/1225835043 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c003c90 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe55affd640 1 -- 172.21.15.122:0/1225835043 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe574041030 con 0x559f5870a6f0 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 10 client.18234 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 20 client.18234 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5870f780 0x559f5878cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 4 client.18234 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f58670d70 con 0x559f5870f780 2024-09-12T02:06:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5870f780 0x559f5878cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe57b7fe640 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5870f780 0x559f5878cee0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de42fe8f87a4ba1a server_cookie=fb44e60263a19819 in_seq=0 out_seq=0 2024-09-12T02:06:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.345+0000 7fe55affd640 10 client.18234 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe55affd640 1 -- 172.21.15.122:0/1225835043 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f58670d70 con 0x559f5870f780 2024-09-12T02:06:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe55affd640 10 client.18234 handle_command_reply: tid=0 2024-09-12T02:06:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe584a3c740 1 client.18234 shutdown 2024-09-12T02:06:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 msgr2=0x559f58719ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 0x559f58719ea0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe574002230 tx=0x7fe574004180 comp rx=0 tx=0).stop 2024-09-12T02:06:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 shutdown_connections 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.575+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x559f5870f780 0x559f5878cee0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f5870a6f0 0x559f58719ea0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe55affd640 0 client.18234 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f58709db0 0x559f58718e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 1 --2- 172.21.15.122:0/1225835043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f58701010 0x559f58718950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 >> 172.21.15.122:0/1225835043 conn(0x559f5870f2a0 msgr2=0x559f587001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 shutdown_connections 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 1 -- 172.21.15.122:0/1225835043 wait complete. 2024-09-12T02:06:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.576+0000 7fe584a3c740 20 client.18234 trim_cache size 0 max 16384 2024-09-12T02:06:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:06:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:06:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:06:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5640a2c30 msgr2=0x7fe5640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5640a2c30 0x7fe5640a50f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559f5870b120 tx=0x7fe56c005f90 comp rx=0 tx=0).stop 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 msgr2=0x7fe57c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c198300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe570031440 tx=0x7fe570032ce0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 shutdown_connections 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fe5640a2c30 0x7fe5640a50f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe57c1025f0 0x7fe57c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe57c101c30 0x7fe57c198300 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 --2- 172.21.15.122:0/4008657640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe57c100a40 0x7fe57c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 >> 172.21.15.122:0/4008657640 conn(0x7fe57c0fc1e0 msgr2=0x7fe57c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 shutdown_connections 2024-09-12T02:06:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:27.577+0000 7fe581bc4640 1 -- 172.21.15.122:0/4008657640 wait complete. 2024-09-12T02:06:28.072 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147688 19.96 MB/sec execute 255 sec latency 151.916 ms 2024-09-12T02:06:29.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 148329 20.05 MB/sec execute 256 sec latency 76.386 ms 2024-09-12T02:06:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:28 smithi151 ceph-mon[35229]: pgmap v938: 97 pgs: 97 active+clean; 528 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.7 MiB/s wr, 486 op/s 2024-09-12T02:06:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:28 smithi183 ceph-mon[35127]: pgmap v938: 97 pgs: 97 active+clean; 528 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.7 MiB/s wr, 486 op/s 2024-09-12T02:06:29.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:28 smithi122 ceph-mon[30925]: pgmap v938: 97 pgs: 97 active+clean; 528 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.7 MiB/s wr, 486 op/s 2024-09-12T02:06:30.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 149140 20.15 MB/sec execute 257 sec latency 47.135 ms 2024-09-12T02:06:31.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 149453 20.13 MB/sec execute 258 sec latency 281.518 ms 2024-09-12T02:06:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:30 smithi151 ceph-mon[35229]: pgmap v939: 97 pgs: 97 active+clean; 536 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.7 MiB/s wr, 577 op/s 2024-09-12T02:06:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:30 smithi183 ceph-mon[35127]: pgmap v939: 97 pgs: 97 active+clean; 536 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.7 MiB/s wr, 577 op/s 2024-09-12T02:06:31.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:30 smithi122 ceph-mon[30925]: pgmap v939: 97 pgs: 97 active+clean; 536 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.7 MiB/s wr, 577 op/s 2024-09-12T02:06:32.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 150051 20.22 MB/sec execute 259 sec latency 107.339 ms 2024-09-12T02:06:33.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 150467 20.19 MB/sec execute 260 sec latency 114.038 ms 2024-09-12T02:06:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:32 smithi151 ceph-mon[35229]: pgmap v940: 97 pgs: 97 active+clean; 540 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-09-12T02:06:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:32 smithi183 ceph-mon[35127]: pgmap v940: 97 pgs: 97 active+clean; 540 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-09-12T02:06:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:32 smithi122 ceph-mon[30925]: pgmap v940: 97 pgs: 97 active+clean; 540 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-09-12T02:06:34.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 150897 20.19 MB/sec execute 261 sec latency 106.150 ms 2024-09-12T02:06:35.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 151445 20.23 MB/sec execute 262 sec latency 86.846 ms 2024-09-12T02:06:35.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:34 smithi151 ceph-mon[35229]: pgmap v941: 97 pgs: 97 active+clean; 546 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.8 MiB/s wr, 638 op/s 2024-09-12T02:06:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:34 smithi183 ceph-mon[35127]: pgmap v941: 97 pgs: 97 active+clean; 546 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.8 MiB/s wr, 638 op/s 2024-09-12T02:06:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:34 smithi122 ceph-mon[30925]: pgmap v941: 97 pgs: 97 active+clean; 546 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.8 MiB/s wr, 638 op/s 2024-09-12T02:06:36.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 151743 20.17 MB/sec execute 263 sec latency 326.183 ms 2024-09-12T02:06:37.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 152403 20.22 MB/sec execute 264 sec latency 122.123 ms 2024-09-12T02:06:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:36 smithi151 ceph-mon[35229]: pgmap v942: 97 pgs: 97 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 577 op/s 2024-09-12T02:06:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:36 smithi183 ceph-mon[35127]: pgmap v942: 97 pgs: 97 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 577 op/s 2024-09-12T02:06:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:36 smithi122 ceph-mon[30925]: pgmap v942: 97 pgs: 97 active+clean; 550 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 577 op/s 2024-09-12T02:06:38.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 153150 20.33 MB/sec execute 265 sec latency 53.339 ms 2024-09-12T02:06:39.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 153640 20.37 MB/sec execute 266 sec latency 117.734 ms 2024-09-12T02:06:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:38 smithi151 ceph-mon[35229]: pgmap v943: 97 pgs: 97 active+clean; 545 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 609 op/s 2024-09-12T02:06:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:38 smithi183 ceph-mon[35127]: pgmap v943: 97 pgs: 97 active+clean; 545 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 609 op/s 2024-09-12T02:06:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:38 smithi122 ceph-mon[30925]: pgmap v943: 97 pgs: 97 active+clean; 545 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 609 op/s 2024-09-12T02:06:40.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 154239 20.44 MB/sec execute 267 sec latency 82.329 ms 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 -- 172.21.15.122:0/4190306295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 msgr2=0x7f8b0c101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 --2- 172.21.15.122:0/4190306295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c101f50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009900 tx=0x7f8b0002a410 comp rx=0 tx=0).stop 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 -- 172.21.15.122:0/4190306295 shutdown_connections 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 --2- 172.21.15.122:0/4190306295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b0c103700 0x7f8b0c109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 --2- 172.21.15.122:0/4190306295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b0c102d40 0x7f8b0c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 --2- 172.21.15.122:0/4190306295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c101f50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 -- 172.21.15.122:0/4190306295 >> 172.21.15.122:0/4190306295 conn(0x7f8b0c0fd2b0 msgr2=0x7f8b0c0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 -- 172.21.15.122:0/4190306295 shutdown_connections 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.675+0000 7f8b130e6640 1 -- 172.21.15.122:0/4190306295 wait complete. 2024-09-12T02:06:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 Processor -- start 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 -- start start 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b0c102d40 0x7f8b0c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b0c103700 0x7f8b0c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c10d7d0 con 0x7f8b0c101b50 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c10d650 con 0x7f8b0c103700 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b130e6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c10d950 con 0x7f8b0c102d40 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45020/0 (socket says 172.21.15.122:45020) 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 -- 172.21.15.122:0/1707831721 learned_addr learned my addr 172.21.15.122:0/1707831721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b0c102d40 0x7f8b0c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 -- 172.21.15.122:0/1707831721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b0c102d40 msgr2=0x7f8b0c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b11683640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b0c103700 0x7f8b0c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b0c102d40 0x7f8b0c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 -- 172.21.15.122:0/1707831721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b0c103700 msgr2=0x7f8b0c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b0c103700 0x7f8b0c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.676+0000 7f8b10e82640 1 -- 172.21.15.122:0/1707831721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c0753b0 con 0x7f8b0c101b50 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.677+0000 7f8b10e82640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c075fa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8b00031610 tx=0x7f8b00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.677+0000 7f8b09ffb640 1 -- 172.21.15.122:0/1707831721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0003f030 con 0x7f8b0c101b50 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.677+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c1a1e80 con 0x7f8b0c101b50 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.677+0000 7f8b09ffb640 1 -- 172.21.15.122:0/1707831721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b00003ad0 con 0x7f8b0c101b50 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.677+0000 7f8b09ffb640 1 -- 172.21.15.122:0/1707831721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00035070 con 0x7f8b0c101b50 2024-09-12T02:06:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.677+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a22e0 con 0x7f8b0c101b50 2024-09-12T02:06:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.680+0000 7f8b09ffb640 1 -- 172.21.15.122:0/1707831721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8b00042030 con 0x7f8b0c101b50 2024-09-12T02:06:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.682+0000 7f8b09ffb640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8afc0a2c30 0x7f8afc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.682+0000 7f8b09ffb640 1 -- 172.21.15.122:0/1707831721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f8b000ef2e0 con 0x7f8b0c101b50 2024-09-12T02:06:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.682+0000 7f8b15e7f740 1 Processor -- start 2024-09-12T02:06:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.682+0000 7f8b15e7f740 1 -- start start 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f176d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176daf0 0x5620f17725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1772b80 0x5620f1775040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5620f16c82f0 con 0x5620f176d1b0 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5620f1670270 con 0x5620f176daf0 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5620f16c42b0 con 0x5620f1772b80 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8afc0a2c30 0x7f8afc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b11683640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1772b80 0x5620f1775040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b11683640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1772b80 0x5620f1775040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:44058/0 (socket says 172.21.15.122:44058) 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b11683640 1 -- 172.21.15.122:0/333747093 learned_addr learned my addr 172.21.15.122:0/333747093 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176daf0 0x5620f17725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.683+0000 7f8b10e82640 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f176d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8afc0a2c30 0x7f8afc0a50f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c075d10 tx=0x7f8af8011920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 -- 172.21.15.122:0/333747093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1772b80 msgr2=0x5620f1775040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1772b80 0x5620f1775040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 -- 172.21.15.122:0/333747093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 msgr2=0x5620f176d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f176d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 -- 172.21.15.122:0/333747093 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f166f340 con 0x5620f176daf0 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176daf0 0x5620f17725b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8af8004340 tx=0x7f8af801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56f8983059f96c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8af6ffd640 1 -- 172.21.15.122:0/333747093 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af801a040 con 0x5620f176daf0 2024-09-12T02:06:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8af6ffd640 1 -- 172.21.15.122:0/333747093 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af8026070 con 0x5620f176daf0 2024-09-12T02:06:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.684+0000 7f8af6ffd640 1 -- 172.21.15.122:0/333747093 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8020810 con 0x5620f176daf0 2024-09-12T02:06:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 -- 172.21.15.122:0/333747093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176daf0 msgr2=0x5620f17725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176daf0 0x5620f17725b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8af8004340 tx=0x7f8af801d000 comp rx=0 tx=0).stop 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 -- 172.21.15.122:0/333747093 shutdown_connections 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1772b80 0x5620f1775040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176daf0 0x5620f17725b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/333747093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f176d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 -- 172.21.15.122:0/333747093 >> 172.21.15.122:0/333747093 conn(0x5620f1768820 msgr2=0x5620f1768c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 -- 172.21.15.122:0/333747093 shutdown_connections 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 -- 172.21.15.122:0/333747093 wait complete. 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 -- start start 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.686+0000 7f8b15e7f740 1 Processor -- start 2024-09-12T02:06:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176d1b0 0x5620f1777400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f1777940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1772b80 0x5620f177e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5620f1775af0 con 0x5620f176daf0 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5620f1651290 con 0x5620f1772b80 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5620f1775c70 con 0x5620f176d1b0 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b15e7f740 10 client.0 _ll_register_callbacks cb 0x5620f1682140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f1777940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f1777940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45032/0 (socket says 172.21.15.122:45032) 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b0bfff640 1 -- 172.21.15.122:0/486856184 learned_addr learned my addr 172.21.15.122:0/486856184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b10e82640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176d1b0 0x5620f1777400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.687+0000 7f8b11683640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1772b80 0x5620f177e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b10e82640 1 -- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1772b80 msgr2=0x5620f177e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b10e82640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1772b80 0x5620f177e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b10e82640 1 -- 172.21.15.122:0/486856184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 msgr2=0x5620f1777940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b10e82640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f1777940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b10e82640 1 -- 172.21.15.122:0/486856184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f13db210 con 0x5620f176d1b0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f1777940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b10e82640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176d1b0 0x5620f1777400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b000314e0 tx=0x7f8b0003be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16e3eb6b3fb19a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8aebfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8aebfff640 1 -- 172.21.15.122:0/486856184 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000d4020 con 0x5620f176d1b0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8aebfff640 1 -- 172.21.15.122:0/486856184 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f8b0003b460 con 0x5620f176d1b0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af8026070 con 0x5620f176d1b0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8aebfff640 1 -- 172.21.15.122:0/486856184 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8b00049040 con 0x5620f176d1b0 2024-09-12T02:06:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b15e7f740 10 client.27232 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.688+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5620f14a7620 con 0x5620f176d1b0 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8aebfff640 1 -- 172.21.15.122:0/486856184 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af8026070 con 0x5620f176d1b0 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8aebfff640 1 -- 172.21.15.122:0/486856184 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8b0004c060 con 0x5620f176d1b0 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b15e7f740 10 client.27232 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b15e7f740 10 client.27232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b15e7f740 20 client.27232 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1783060 0x5620f1785490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b15e7f740 4 client.27232 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f16d5a20 con 0x5620f1783060 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.689+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1783060 0x5620f1785490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.690+0000 7f8aebfff640 10 client.27232 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.690+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1783060 0x5620f1785490 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b938df07249419da server_cookie=7588991b5fd71541 in_seq=0 out_seq=0 2024-09-12T02:06:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8aebfff640 1 -- 172.21.15.122:0/486856184 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620f16d5a20 con 0x5620f1783060 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8aebfff640 10 client.27232 handle_command_reply: tid=0 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 client.27232 shutdown 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176d1b0 msgr2=0x5620f1777400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176d1b0 0x5620f1777400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b000314e0 tx=0x7f8b0003be40 comp rx=0 tx=0).stop 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 shutdown_connections 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1783060 0x5620f1785490 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8aebfff640 0 client.27232 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1772b80 0x5620f177e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f1777940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/486856184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176d1b0 0x5620f1777400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 >> 172.21.15.122:0/486856184 conn(0x5620f1768820 msgr2=0x5620f176a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 shutdown_connections 2024-09-12T02:06:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 1 -- 172.21.15.122:0/486856184 wait complete. 2024-09-12T02:06:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.691+0000 7f8b15e7f740 20 client.27232 trim_cache size 0 max 16384 2024-09-12T02:06:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 Processor -- start 2024-09-12T02:06:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 -- start start 2024-09-12T02:06:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1768820 0x5620f175f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f177e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176daf0 0x5620f1789d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1a9650 con 0x5620f176d1b0 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1a6f20 con 0x5620f1768820 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1a4d50 con 0x5620f176daf0 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f177e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f177e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45042/0 (socket says 172.21.15.122:45042) 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b0bfff640 1 -- 172.21.15.122:0/1040943581 learned_addr learned my addr 172.21.15.122:0/1040943581 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.706+0000 7f8b10e82640 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1768820 0x5620f175f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b11683640 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176daf0 0x5620f1789d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b0bfff640 1 -- 172.21.15.122:0/1040943581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176daf0 msgr2=0x5620f1789d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176daf0 0x5620f1789d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b0bfff640 1 -- 172.21.15.122:0/1040943581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1768820 msgr2=0x5620f175f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1768820 0x5620f175f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b0bfff640 1 -- 172.21.15.122:0/1040943581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f1768640 con 0x5620f176d1b0 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b11683640 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176daf0 0x5620f1789d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f177e910 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8af8004340 tx=0x7f8af8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473168055c83f011 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8af67fc640 1 -- 172.21.15.122:0/1040943581 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af801a040 con 0x5620f176d1b0 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8af67fc640 1 -- 172.21.15.122:0/1040943581 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af8026070 con 0x5620f176d1b0 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.707+0000 7f8af67fc640 1 -- 172.21.15.122:0/1040943581 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8003e20 con 0x5620f176d1b0 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 -- 172.21.15.122:0/1040943581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 msgr2=0x5620f177e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f177e910 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8af8004340 tx=0x7f8af8005d50 comp rx=0 tx=0).stop 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 -- 172.21.15.122:0/1040943581 shutdown_connections 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f176daf0 0x5620f1789d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176d1b0 0x5620f177e910 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/1040943581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f1768820 0x5620f175f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 -- 172.21.15.122:0/1040943581 >> 172.21.15.122:0/1040943581 conn(0x5620f17726d0 msgr2=0x5620f175e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 -- 172.21.15.122:0/1040943581 shutdown_connections 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 -- 172.21.15.122:0/1040943581 wait complete. 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 -- start start 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.708+0000 7f8b15e7f740 1 Processor -- start 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 0x5620f177be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176d1b0 0x5620f177c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f177d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1a53e0 con 0x5620f176daf0 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1a6850 con 0x5620f176d1b0 2024-09-12T02:06:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1a5ab0 con 0x5620f1768820 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b15e7f740 10 client.0 _ll_register_callbacks cb 0x7f8b000bb680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f177d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b10e82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 0x5620f177be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b10e82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 0x5620f177be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:44102/0 (socket says 172.21.15.122:44102) 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b10e82640 1 -- 172.21.15.122:0/2952198895 learned_addr learned my addr 172.21.15.122:0/2952198895 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f177d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45054/0 (socket says 172.21.15.122:45054) 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b0bfff640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176d1b0 0x5620f177c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 -- 172.21.15.122:0/2952198895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 msgr2=0x5620f177be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 0x5620f177be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 -- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176d1b0 msgr2=0x5620f177c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176d1b0 0x5620f177c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 -- 172.21.15.122:0/2952198895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f177d960 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b10e82640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 0x5620f177be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8b11683640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f177d350 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005bd0 tx=0x7f8b04013940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb28e9ce697c048b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8aebfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.709+0000 7f8aebfff640 1 -- 172.21.15.122:0/2952198895 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04011710 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af8026070 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8aebfff640 1 -- 172.21.15.122:0/2952198895 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8240+0+0 (secure 0 0 0) 0x7f8b040118b0 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8aebfff640 1 -- 172.21.15.122:0/2952198895 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8b04011ab0 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 10 client.18252 fetch_fsmap learned FSMap version 41 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5620f177ecb0 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8aebfff640 1 -- 172.21.15.122:0/2952198895 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af8026070 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8aebfff640 1 -- 172.21.15.122:0/2952198895 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f8b04027400 con 0x5620f176daf0 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 10 client.18252 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:06:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 20 client.18252 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1772b80 0x5620f17f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 4 client.18252 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620f16cfd70 con 0x5620f1772b80 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.710+0000 7f8b10e82640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1772b80 0x5620f17f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.711+0000 7f8b10e82640 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1772b80 0x5620f17f0590 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3626d735875c15d server_cookie=34eb0d4593bd202b in_seq=0 out_seq=0 2024-09-12T02:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.711+0000 7f8aebfff640 10 client.18252 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8aebfff640 1 -- 172.21.15.122:0/2952198895 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620f16cfd70 con 0x5620f1772b80 2024-09-12T02:06:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8aebfff640 10 client.18252 handle_command_reply: tid=0 2024-09-12T02:06:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 client.18252 shutdown 2024-09-12T02:06:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 msgr2=0x5620f177d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f177d350 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005bd0 tx=0x7f8b04013940 comp rx=0 tx=0).stop 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 shutdown_connections 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5620f1772b80 0x5620f17f0590 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620f176daf0 0x5620f177d350 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5620f176d1b0 0x5620f177c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 --2- 172.21.15.122:0/2952198895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620f1768820 0x5620f177be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8aebfff640 0 client.18252 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.969+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 >> 172.21.15.122:0/2952198895 conn(0x5620f17726d0 msgr2=0x5620f175f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 shutdown_connections 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b15e7f740 1 -- 172.21.15.122:0/2952198895 wait complete. 2024-09-12T02:06:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b15e7f740 20 client.18252 trim_cache size 0 max 16384 2024-09-12T02:06:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:06:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:06:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:06:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8afc0a2c30 msgr2=0x7f8afc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b130e6640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8afc0a2c30 0x7f8afc0a50f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c075d10 tx=0x7f8af8011920 comp rx=0 tx=0).stop 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 msgr2=0x7f8b0c075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.970+0000 7f8b130e6640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c075fa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8b00031610 tx=0x7f8b00004270 comp rx=0 tx=0).stop 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 shutdown_connections 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8afc0a2c30 0x7f8afc0a50f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b0c103700 0x7f8b0c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b0c102d40 0x7f8b0c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 --2- 172.21.15.122:0/1707831721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b0c101b50 0x7f8b0c075fa0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 >> 172.21.15.122:0/1707831721 conn(0x7f8b0c0fd2b0 msgr2=0x7f8b0c1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 shutdown_connections 2024-09-12T02:06:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:40.971+0000 7f8b130e6640 1 -- 172.21.15.122:0/1707831721 wait complete. 2024-09-12T02:06:41.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 154452 20.42 MB/sec execute 268 sec latency 291.413 ms 2024-09-12T02:06:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:40 smithi151 ceph-mon[35229]: pgmap v944: 97 pgs: 97 active+clean; 535 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.9 MiB/s wr, 633 op/s 2024-09-12T02:06:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:40 smithi183 ceph-mon[35127]: pgmap v944: 97 pgs: 97 active+clean; 535 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.9 MiB/s wr, 633 op/s 2024-09-12T02:06:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:40 smithi122 ceph-mon[30925]: pgmap v944: 97 pgs: 97 active+clean; 535 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.9 MiB/s wr, 633 op/s 2024-09-12T02:06:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:41 smithi183 ceph-mon[35127]: daemon mds.f restarted 2024-09-12T02:06:41.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:06:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setuser ceph since I am not root 2024-09-12T02:06:41.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:06:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: ignoring --setgroup ceph since I am not root 2024-09-12T02:06:41.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:06:41 smithi183 ceph-mds[64439]: -- 172.21.15.183:0/245777244 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648542b3860 con 0x564854291000 2024-09-12T02:06:41.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:06:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: starting mds.f at 2024-09-12T02:06:42.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 154854 20.39 MB/sec execute 269 sec latency 124.267 ms 2024-09-12T02:06:42.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:41 smithi151 ceph-mon[35229]: daemon mds.f restarted 2024-09-12T02:06:42.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:41 smithi122 ceph-mon[30925]: daemon mds.f restarted 2024-09-12T02:06:43.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 155436 20.42 MB/sec execute 270 sec latency 94.539 ms 2024-09-12T02:06:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:42 smithi151 ceph-mon[35229]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T02:06:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:42 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6834/1657619890,v1:172.21.15.183:6835/1657619890] up:boot 2024-09-12T02:06:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:42 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:06:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:42 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:06:43.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:42 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:06:43.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:42 smithi151 ceph-mon[35229]: pgmap v946: 97 pgs: 97 active+clean; 537 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.4 MiB/s wr, 597 op/s 2024-09-12T02:06:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:42 smithi183 ceph-mon[35127]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T02:06:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:42 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6834/1657619890,v1:172.21.15.183:6835/1657619890] up:boot 2024-09-12T02:06:43.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:42 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:06:43.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:42 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:06:43.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:42 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:06:43.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:42 smithi183 ceph-mon[35127]: pgmap v946: 97 pgs: 97 active+clean; 537 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.4 MiB/s wr, 597 op/s 2024-09-12T02:06:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:42 smithi122 ceph-mon[30925]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T02:06:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:42 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6834/1657619890,v1:172.21.15.183:6835/1657619890] up:boot 2024-09-12T02:06:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:42 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:06:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:42 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:06:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:42 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:06:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:42 smithi122 ceph-mon[30925]: pgmap v946: 97 pgs: 97 active+clean; 537 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.4 MiB/s wr, 597 op/s 2024-09-12T02:06:44.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 156043 20.44 MB/sec execute 271 sec latency 105.569 ms 2024-09-12T02:06:45.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 156818 20.53 MB/sec execute 272 sec latency 47.716 ms 2024-09-12T02:06:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:44 smithi151 ceph-mon[35229]: pgmap v947: 97 pgs: 97 active+clean; 533 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.1 MiB/s wr, 636 op/s 2024-09-12T02:06:45.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:44 smithi183 ceph-mon[35127]: pgmap v947: 97 pgs: 97 active+clean; 533 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.1 MiB/s wr, 636 op/s 2024-09-12T02:06:45.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:44 smithi122 ceph-mon[30925]: pgmap v947: 97 pgs: 97 active+clean; 533 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.1 MiB/s wr, 636 op/s 2024-09-12T02:06:46.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 157396 20.58 MB/sec execute 273 sec latency 189.790 ms 2024-09-12T02:06:47.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 157847 20.61 MB/sec execute 274 sec latency 82.998 ms 2024-09-12T02:06:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:46 smithi151 ceph-mon[35229]: pgmap v948: 97 pgs: 97 active+clean; 535 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 652 op/s 2024-09-12T02:06:47.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:46 smithi183 ceph-mon[35127]: pgmap v948: 97 pgs: 97 active+clean; 535 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 652 op/s 2024-09-12T02:06:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:46 smithi122 ceph-mon[30925]: pgmap v948: 97 pgs: 97 active+clean; 535 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 652 op/s 2024-09-12T02:06:48.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 158516 20.69 MB/sec execute 275 sec latency 72.074 ms 2024-09-12T02:06:49.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 158897 20.68 MB/sec execute 276 sec latency 107.230 ms 2024-09-12T02:06:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:48 smithi151 ceph-mon[35229]: pgmap v949: 97 pgs: 97 active+clean; 537 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.9 MiB/s wr, 691 op/s 2024-09-12T02:06:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:48 smithi183 ceph-mon[35127]: pgmap v949: 97 pgs: 97 active+clean; 537 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.9 MiB/s wr, 691 op/s 2024-09-12T02:06:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:48 smithi122 ceph-mon[30925]: pgmap v949: 97 pgs: 97 active+clean; 537 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.9 MiB/s wr, 691 op/s 2024-09-12T02:06:50.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 159516 20.69 MB/sec execute 277 sec latency 66.813 ms 2024-09-12T02:06:51.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 159919 20.69 MB/sec execute 278 sec latency 229.505 ms 2024-09-12T02:06:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:50 smithi151 ceph-mon[35229]: pgmap v950: 97 pgs: 97 active+clean; 539 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 711 op/s 2024-09-12T02:06:51.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:50 smithi183 ceph-mon[35127]: pgmap v950: 97 pgs: 97 active+clean; 539 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 711 op/s 2024-09-12T02:06:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:50 smithi122 ceph-mon[30925]: pgmap v950: 97 pgs: 97 active+clean; 539 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 711 op/s 2024-09-12T02:06:52.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 160573 20.72 MB/sec execute 279 sec latency 136.159 ms 2024-09-12T02:06:53.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 161421 20.82 MB/sec execute 280 sec latency 57.809 ms 2024-09-12T02:06:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:52 smithi151 ceph-mon[35229]: pgmap v951: 97 pgs: 97 active+clean; 539 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-09-12T02:06:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:52 smithi183 ceph-mon[35127]: pgmap v951: 97 pgs: 97 active+clean; 539 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-09-12T02:06:53.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:52 smithi122 ceph-mon[30925]: pgmap v951: 97 pgs: 97 active+clean; 539 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 -- 172.21.15.122:0/2697434469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60101b60 msgr2=0x7fac60101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 --2- 172.21.15.122:0/2697434469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60101b60 0x7fac60101f60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fac50009900 tx=0x7fac5002a410 comp rx=0 tx=0).stop 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 -- 172.21.15.122:0/2697434469 shutdown_connections 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 --2- 172.21.15.122:0/2697434469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac60103710 0x7fac60109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 --2- 172.21.15.122:0/2697434469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60102d50 0x7fac601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 --2- 172.21.15.122:0/2697434469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60101b60 0x7fac60101f60 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.066+0000 7fac669c6640 1 -- 172.21.15.122:0/2697434469 >> 172.21.15.122:0/2697434469 conn(0x7fac600fd2e0 msgr2=0x7fac600ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.067+0000 7fac669c6640 1 -- 172.21.15.122:0/2697434469 shutdown_connections 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.067+0000 7fac669c6640 1 -- 172.21.15.122:0/2697434469 wait complete. 2024-09-12T02:06:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 Processor -- start 2024-09-12T02:06:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 -- start start 2024-09-12T02:06:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 0x7fac60197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac60102d50 0x7fac601983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60103710 0x7fac6019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010d860 con 0x7fac60103710 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010d6e0 con 0x7fac60102d50 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac669c6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010d9e0 con 0x7fac60101b60 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 0x7fac60197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60103710 0x7fac6019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 0x7fac60197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53866/0 (socket says 172.21.15.122:53866) 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60103710 0x7fac6019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52690/0 (socket says 172.21.15.122:52690) 2024-09-12T02:06:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.068+0000 7fac5ffff640 1 -- 172.21.15.122:0/1534706782 learned_addr learned my addr 172.21.15.122:0/1534706782 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5ffff640 1 -- 172.21.15.122:0/1534706782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac60102d50 msgr2=0x7fac601983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5ffff640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac60102d50 0x7fac601983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5ffff640 1 -- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60103710 msgr2=0x7fac6019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5ffff640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60103710 0x7fac6019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5ffff640 1 -- 172.21.15.122:0/1534706782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6019faa0 con 0x7fac60101b60 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5ffff640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 0x7fac60197e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac50031610 tx=0x7fac50004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5d7fa640 1 -- 172.21.15.122:0/1534706782 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5003f030 con 0x7fac60101b60 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac5d7fa640 1 -- 172.21.15.122:0/1534706782 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac50003ad0 con 0x7fac60101b60 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.069+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6019fd30 con 0x7fac60101b60 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.070+0000 7fac5d7fa640 1 -- 172.21.15.122:0/1534706782 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50035070 con 0x7fac60101b60 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.070+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac601a0160 con 0x7fac60101b60 2024-09-12T02:06:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.073+0000 7fac5d7fa640 1 -- 172.21.15.122:0/1534706782 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fac50042030 con 0x7fac60101b60 2024-09-12T02:06:54.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 161970 20.87 MB/sec execute 281 sec latency 86.203 ms 2024-09-12T02:06:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 Processor -- start 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 -- start start 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 0x55636993ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993f260 0x556369941d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556369942330 0x55636994f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5563698a22f0 con 0x55636993e920 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55636984a270 con 0x556369942330 2024-09-12T02:06:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac69836740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55636989e2b0 con 0x55636993f260 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.074+0000 7fac5d7fa640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fac480a2c30 0x7fac480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 0x55636993ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 0x55636993ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52692/0 (socket says 172.21.15.122:52692) 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 -- 172.21.15.122:0/3164967979 learned_addr learned my addr 172.21.15.122:0/3164967979 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac5d7fa640 1 -- 172.21.15.122:0/1534706782 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fac500ef9e0 con 0x7fac60101b60 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fac480a2c30 0x7fac480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 -- 172.21.15.122:0/3164967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993f260 msgr2=0x556369941d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:06:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993f260 0x556369941d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 -- 172.21.15.122:0/3164967979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556369942330 msgr2=0x55636994f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556369942330 0x55636994f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 -- 172.21.15.122:0/3164967979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556369849340 con 0x55636993e920 2024-09-12T02:06:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.075+0000 7fac64f63640 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 0x55636993ed20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fac5800a9c0 tx=0x7fac5800ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed328df06f42de76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.076+0000 7fac64f63640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fac480a2c30 0x7fac480a50f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac5801f000 tx=0x7fac58018740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.076+0000 7fac467fc640 1 -- 172.21.15.122:0/3164967979 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5801d070 con 0x55636993e920 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.076+0000 7fac467fc640 1 -- 172.21.15.122:0/3164967979 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac58022930 con 0x55636993e920 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.077+0000 7fac467fc640 1 -- 172.21.15.122:0/3164967979 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58019960 con 0x55636993e920 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 -- 172.21.15.122:0/3164967979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 msgr2=0x55636993ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 0x55636993ed20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fac5800a9c0 tx=0x7fac5800ae80 comp rx=0 tx=0).stop 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 -- 172.21.15.122:0/3164967979 shutdown_connections 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556369942330 0x55636994f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993f260 0x556369941d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 --2- 172.21.15.122:0/3164967979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993e920 0x55636993ed20 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 -- 172.21.15.122:0/3164967979 >> 172.21.15.122:0/3164967979 conn(0x55636993a010 msgr2=0x55636993a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 -- 172.21.15.122:0/3164967979 shutdown_connections 2024-09-12T02:06:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.078+0000 7fac69836740 1 -- 172.21.15.122:0/3164967979 wait complete. 2024-09-12T02:06:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 -- start start 2024-09-12T02:06:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 Processor -- start 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993f260 0x5563699519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 0x556369958a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55636994fb70 con 0x556369942330 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55636982b290 con 0x55636993f260 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55636994fcf0 con 0x55636993e920 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.079+0000 7fac69836740 10 client.0 _ll_register_callbacks cb 0x55636985c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 0x556369958a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 0x556369958a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52708/0 (socket says 172.21.15.122:52708) 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 -- 172.21.15.122:0/1232398929 learned_addr learned my addr 172.21.15.122:0/1232398929 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac5f7fe640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993f260 0x5563699519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac5ffff640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 -- 172.21.15.122:0/1232398929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 msgr2=0x5563699514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 -- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993f260 msgr2=0x5563699519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993f260 0x5563699519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.080+0000 7fac64f63640 1 -- 172.21.15.122:0/1232398929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563695b5210 con 0x556369942330 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac64f63640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 0x556369958a30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fac5800aec0 tx=0x7fac58009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c162704d46131af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac3f7fe640 1 -- 172.21.15.122:0/1232398929 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58030020 con 0x556369942330 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac3f7fe640 1 -- 172.21.15.122:0/1232398929 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fac58022ae0 con 0x556369942330 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac58022930 con 0x556369942330 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac3f7fe640 1 -- 172.21.15.122:0/1232398929 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fac5801d040 con 0x556369942330 2024-09-12T02:06:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac69836740 10 client.18270 fetch_fsmap learned FSMap version 43 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556369681620 con 0x556369942330 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac3f7fe640 1 -- 172.21.15.122:0/1232398929 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac58022930 con 0x556369942330 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac3f7fe640 1 -- 172.21.15.122:0/1232398929 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fac5802c510 con 0x556369942330 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac69836740 10 client.18270 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac69836740 10 client.18270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.081+0000 7fac69836740 20 client.18270 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.082+0000 7fac69836740 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55636995cfc0 0x55636995f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.082+0000 7fac69836740 4 client.18270 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.082+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563698afa20 con 0x55636995cfc0 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.082+0000 7fac5ffff640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55636995cfc0 0x55636995f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.083+0000 7fac3f7fe640 10 client.18270 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.082+0000 7fac5ffff640 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55636995cfc0 0x55636995f400 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923a8950b93285f4 server_cookie=9d8b5835586d4d1c in_seq=0 out_seq=0 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.085+0000 7fac3f7fe640 1 -- 172.21.15.122:0/1232398929 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563698afa20 con 0x55636995cfc0 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.085+0000 7fac3f7fe640 10 client.18270 handle_command_reply: tid=0 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.085+0000 7fac69836740 1 client.18270 shutdown 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 msgr2=0x556369958a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 0x556369958a30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fac5800aec0 tx=0x7fac58009650 comp rx=0 tx=0).stop 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 shutdown_connections 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55636995cfc0 0x55636995f400 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556369942330 0x556369958a30 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993f260 0x5563699519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 --2- 172.21.15.122:0/1232398929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 >> 172.21.15.122:0/1232398929 conn(0x55636993a010 msgr2=0x55636993b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac3f7fe640 0 client.18270 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 shutdown_connections 2024-09-12T02:06:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 1 -- 172.21.15.122:0/1232398929 wait complete. 2024-09-12T02:06:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.086+0000 7fac69836740 20 client.18270 trim_cache size 0 max 16384 2024-09-12T02:06:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.104+0000 7fac69836740 1 Processor -- start 2024-09-12T02:06:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.104+0000 7fac69836740 1 -- start start 2024-09-12T02:06:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac69836740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993a010 0x55636993a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac69836740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac69836740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x55636993aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac69836740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c010d70 con 0x55636993f260 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac69836740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c00e640 con 0x55636993a010 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac69836740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c00c470 con 0x55636993e920 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x55636993aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac64f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x55636993aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52714/0 (socket says 172.21.15.122:52714) 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac64f63640 1 -- 172.21.15.122:0/662817041 learned_addr learned my addr 172.21.15.122:0/662817041 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5f7fe640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993a010 0x55636993a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 -- 172.21.15.122:0/662817041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 msgr2=0x5563699589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 -- 172.21.15.122:0/662817041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 msgr2=0x55636993aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x55636993aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 -- 172.21.15.122:0/662817041 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563699352d0 con 0x55636993a010 2024-09-12T02:06:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5f7fe640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac64f63640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x55636993aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:06:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.105+0000 7fac5ffff640 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993a010 0x55636993a620 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55636993fc90 tx=0x7fac500dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acace1b7e17d2580 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac45ffb640 1 -- 172.21.15.122:0/662817041 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50049040 con 0x55636993a010 2024-09-12T02:06:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac45ffb640 1 -- 172.21.15.122:0/662817041 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac500c05f0 con 0x55636993a010 2024-09-12T02:06:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac45ffb640 1 -- 172.21.15.122:0/662817041 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500b8a40 con 0x55636993a010 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac69836740 1 -- 172.21.15.122:0/662817041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993a010 msgr2=0x55636993a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac69836740 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993a010 0x55636993a620 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55636993fc90 tx=0x7fac500dc040 comp rx=0 tx=0).stop 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac69836740 1 -- 172.21.15.122:0/662817041 shutdown_connections 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac69836740 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x55636993aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac69836740 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993e920 0x5563699589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.106+0000 7fac69836740 1 --2- 172.21.15.122:0/662817041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993a010 0x55636993a620 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 -- 172.21.15.122:0/662817041 >> 172.21.15.122:0/662817041 conn(0x55636994f400 msgr2=0x55636994ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 -- 172.21.15.122:0/662817041 shutdown_connections 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 -- 172.21.15.122:0/662817041 wait complete. 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 -- start start 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 Processor -- start 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 0x556369955e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.107+0000 7fac69836740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993e920 0x5563699563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x5563699573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c00cb00 con 0x55636993f260 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c00df70 con 0x55636993e920 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c00d1d0 con 0x55636993a010 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 0x556369955e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 0x556369955e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:53898/0 (socket says 172.21.15.122:53898) 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993e920 0x5563699563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 -- 172.21.15.122:0/617469455 learned_addr learned my addr 172.21.15.122:0/617469455 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 10 client.0 _ll_register_callbacks cb 0x7fac5800a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac64f63640 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x5563699573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 -- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993e920 msgr2=0x5563699563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993e920 0x5563699563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 -- 172.21.15.122:0/617469455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 msgr2=0x5563699573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x5563699573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 -- 172.21.15.122:0/617469455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563699579d0 con 0x55636993a010 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac5ffff640 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 0x556369955e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac50031440 tx=0x7fac500b8a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f81837f61baa353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac3f7fe640 1 -- 172.21.15.122:0/617469455 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500d4070 con 0x55636993a010 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac500c05f0 con 0x55636993a010 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac3f7fe640 1 -- 172.21.15.122:0/617469455 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fac5003b660 con 0x55636993a010 2024-09-12T02:06:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac3f7fe640 1 -- 172.21.15.122:0/617469455 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fac50049040 con 0x55636993a010 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 10 client.27256 fetch_fsmap learned FSMap version 43 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.108+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556369958d40 con 0x55636993a010 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac3f7fe640 1 -- 172.21.15.122:0/617469455 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac500c05f0 con 0x55636993a010 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac3f7fe640 1 -- 172.21.15.122:0/617469455 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fac500dc140 con 0x55636993a010 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac69836740 10 client.27256 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac69836740 10 client.27256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac69836740 20 client.27256 populate_metadata read hostname 'smithi122' 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac69836740 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556369942330 0x5563699ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac69836740 4 client.27256 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563698a9d70 con 0x556369942330 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac5f7fe640 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556369942330 0x5563699ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac3f7fe640 10 client.27256 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.109+0000 7fac5f7fe640 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556369942330 0x5563699ca470 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34635abbfc272761 server_cookie=122da8729e659ce5 in_seq=0 out_seq=0 2024-09-12T02:06:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac3f7fe640 1 -- 172.21.15.122:0/617469455 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563698a9d70 con 0x556369942330 2024-09-12T02:06:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac3f7fe640 10 client.27256 handle_command_reply: tid=0 2024-09-12T02:06:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 client.27256 shutdown 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 msgr2=0x556369955e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 0x556369955e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac50031440 tx=0x7fac500b8a60 comp rx=0 tx=0).stop 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 shutdown_connections 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556369942330 0x5563699ca470 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55636993f260 0x5563699573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac3f7fe640 0 client.27256 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55636993e920 0x5563699563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 --2- 172.21.15.122:0/617469455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55636993a010 0x556369955e70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 >> 172.21.15.122:0/617469455 conn(0x55636994f400 msgr2=0x5563699389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 shutdown_connections 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.356+0000 7fac69836740 1 -- 172.21.15.122:0/617469455 wait complete. 2024-09-12T02:06:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac69836740 20 client.27256 trim_cache size 0 max 16384 2024-09-12T02:06:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:06:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:06:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:06:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fac480a2c30 msgr2=0x7fac480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fac480a2c30 0x7fac480a50f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac5801f000 tx=0x7fac58018740 comp rx=0 tx=0).stop 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 msgr2=0x7fac60197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 0x7fac60197e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac50031610 tx=0x7fac50004270 comp rx=0 tx=0).stop 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 shutdown_connections 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fac480a2c30 0x7fac480a50f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac60103710 0x7fac6019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac60102d50 0x7fac601983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 --2- 172.21.15.122:0/1534706782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac60101b60 0x7fac60197e80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:06:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.357+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 >> 172.21.15.122:0/1534706782 conn(0x7fac600fd2e0 msgr2=0x7fac60106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:06:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.358+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 shutdown_connections 2024-09-12T02:06:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:06:54.358+0000 7fac669c6640 1 -- 172.21.15.122:0/1534706782 wait complete. 2024-09-12T02:06:54.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:54 smithi151 ceph-mon[35229]: daemon mds.k restarted 2024-09-12T02:06:54.866 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:06:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: ignoring --setuser ceph since I am not root 2024-09-12T02:06:54.866 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:06:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: ignoring --setgroup ceph since I am not root 2024-09-12T02:06:54.866 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:06:54 smithi151 ceph-mds[64109]: -- 172.21.15.151:0/3910587792 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570b6d35860 con 0x5570b6d13400 2024-09-12T02:06:54.866 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:06:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: starting mds.k at 2024-09-12T02:06:54.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:54 smithi183 ceph-mon[35127]: daemon mds.k restarted 2024-09-12T02:06:54.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:54 smithi122 ceph-mon[30925]: daemon mds.k restarted 2024-09-12T02:06:55.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 162472 20.92 MB/sec execute 282 sec latency 185.914 ms 2024-09-12T02:06:55.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:55 smithi151 ceph-mon[35229]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T02:06:55.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:55 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6832/503509846,v1:172.21.15.151:6833/503509846] up:boot 2024-09-12T02:06:55.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:55 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:06:55.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:55 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:06:55.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:55 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:06:55.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:55 smithi151 ceph-mon[35229]: pgmap v953: 97 pgs: 97 active+clean; 542 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 612 op/s 2024-09-12T02:06:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:55 smithi183 ceph-mon[35127]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T02:06:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:55 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6832/503509846,v1:172.21.15.151:6833/503509846] up:boot 2024-09-12T02:06:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:55 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:06:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:55 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:06:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:55 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:06:55.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:55 smithi183 ceph-mon[35127]: pgmap v953: 97 pgs: 97 active+clean; 542 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 612 op/s 2024-09-12T02:06:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:55 smithi122 ceph-mon[30925]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T02:06:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:55 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6832/503509846,v1:172.21.15.151:6833/503509846] up:boot 2024-09-12T02:06:55.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:55 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T02:06:55.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:55 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:06:55.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:55 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:06:55.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:55 smithi122 ceph-mon[30925]: pgmap v953: 97 pgs: 97 active+clean; 542 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 612 op/s 2024-09-12T02:06:56.073 INFO:tasks.workunit.client.0.smithi122.stdout: 10 162810 20.92 MB/sec execute 283 sec latency 354.432 ms 2024-09-12T02:06:57.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 163178 20.92 MB/sec execute 284 sec latency 315.150 ms 2024-09-12T02:06:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:56 smithi151 ceph-mon[35229]: pgmap v954: 97 pgs: 97 active+clean; 544 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.5 MiB/s wr, 577 op/s 2024-09-12T02:06:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:56 smithi183 ceph-mon[35127]: pgmap v954: 97 pgs: 97 active+clean; 544 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.5 MiB/s wr, 577 op/s 2024-09-12T02:06:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:56 smithi122 ceph-mon[30925]: pgmap v954: 97 pgs: 97 active+clean; 544 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.5 MiB/s wr, 577 op/s 2024-09-12T02:06:58.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 163759 20.92 MB/sec execute 285 sec latency 161.445 ms 2024-09-12T02:06:59.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 164149 20.91 MB/sec execute 286 sec latency 167.141 ms 2024-09-12T02:06:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:58 smithi151 ceph-mon[35229]: pgmap v955: 97 pgs: 97 active+clean; 530 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.2 MiB/s wr, 666 op/s 2024-09-12T02:06:59.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:58 smithi183 ceph-mon[35127]: pgmap v955: 97 pgs: 97 active+clean; 530 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.2 MiB/s wr, 666 op/s 2024-09-12T02:06:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:58 smithi122 ceph-mon[30925]: pgmap v955: 97 pgs: 97 active+clean; 530 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.2 MiB/s wr, 666 op/s 2024-09-12T02:07:00.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 164618 20.90 MB/sec execute 287 sec latency 135.011 ms 2024-09-12T02:07:00.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:06:59 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:07:00.116 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:06:59 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:07:00.240 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:06:59 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:07:01.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 165204 20.99 MB/sec execute 288 sec latency 253.381 ms 2024-09-12T02:07:02.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 165745 20.98 MB/sec execute 289 sec latency 290.593 ms 2024-09-12T02:07:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:01 smithi151 ceph-mon[35229]: pgmap v956: 97 pgs: 97 active+clean; 522 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.1 MiB/s wr, 619 op/s 2024-09-12T02:07:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:01 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:01 smithi183 ceph-mon[35127]: pgmap v956: 97 pgs: 97 active+clean; 522 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.1 MiB/s wr, 619 op/s 2024-09-12T02:07:02.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:01 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:01 smithi122 ceph-mon[30925]: pgmap v956: 97 pgs: 97 active+clean; 522 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.1 MiB/s wr, 619 op/s 2024-09-12T02:07:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:02.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:01 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:03.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 166221 21.00 MB/sec execute 290 sec latency 92.831 ms 2024-09-12T02:07:03.104 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:02 smithi183 ceph-mon[35127]: pgmap v957: 97 pgs: 97 active+clean; 522 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.4 MiB/s wr, 681 op/s 2024-09-12T02:07:03.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:02 smithi151 ceph-mon[35229]: pgmap v957: 97 pgs: 97 active+clean; 522 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.4 MiB/s wr, 681 op/s 2024-09-12T02:07:03.140 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:02 smithi122 ceph-mon[30925]: pgmap v957: 97 pgs: 97 active+clean; 522 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 8.4 MiB/s wr, 681 op/s 2024-09-12T02:07:04.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 166826 21.08 MB/sec execute 291 sec latency 97.696 ms 2024-09-12T02:07:04.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:04 smithi122 ceph-mon[30925]: pgmap v958: 97 pgs: 97 active+clean; 526 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.0 MiB/s wr, 690 op/s 2024-09-12T02:07:05.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 167164 21.08 MB/sec execute 292 sec latency 187.534 ms 2024-09-12T02:07:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:04 smithi151 ceph-mon[35229]: pgmap v958: 97 pgs: 97 active+clean; 526 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.0 MiB/s wr, 690 op/s 2024-09-12T02:07:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:04 smithi183 ceph-mon[35127]: pgmap v958: 97 pgs: 97 active+clean; 526 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.0 MiB/s wr, 690 op/s 2024-09-12T02:07:06.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 167407 21.10 MB/sec execute 293 sec latency 175.160 ms 2024-09-12T02:07:07.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 167675 21.07 MB/sec execute 294 sec latency 358.344 ms 2024-09-12T02:07:07.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:06 smithi151 ceph-mon[35229]: pgmap v959: 97 pgs: 97 active+clean; 527 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 561 op/s 2024-09-12T02:07:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:06 smithi183 ceph-mon[35127]: pgmap v959: 97 pgs: 97 active+clean; 527 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 561 op/s 2024-09-12T02:07:07.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:06 smithi122 ceph-mon[30925]: pgmap v959: 97 pgs: 97 active+clean; 527 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 561 op/s 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 -- 172.21.15.122:0/4028944047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778101c30 msgr2=0x7f07781020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 --2- 172.21.15.122:0/4028944047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778101c30 0x7f07781020b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f076c009900 tx=0x7f076c02a410 comp rx=0 tx=0).stop 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 -- 172.21.15.122:0/4028944047 shutdown_connections 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 --2- 172.21.15.122:0/4028944047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07781025f0 0x7f0778108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 --2- 172.21.15.122:0/4028944047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778101c30 0x7f07781020b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 --2- 172.21.15.122:0/4028944047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0778100a40 0x7f0778100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 -- 172.21.15.122:0/4028944047 >> 172.21.15.122:0/4028944047 conn(0x7f07780fc1e0 msgr2=0x7f07780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 -- 172.21.15.122:0/4028944047 shutdown_connections 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 -- 172.21.15.122:0/4028944047 wait complete. 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 Processor -- start 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.468+0000 7f077cc04640 1 -- start start 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077cc04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 0x7f07781049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077cc04640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101c30 0x7f0778103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077cc04640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07781025f0 0x7f0778103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077cc04640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c6c0 con 0x7f0778100a40 2024-09-12T02:07:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077cc04640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c540 con 0x7f0778101c30 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077cc04640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c840 con 0x7f07781025f0 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 0x7f07781049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 0x7f07781049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60498/0 (socket says 172.21.15.122:60498) 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 -- 172.21.15.122:0/3017464203 learned_addr learned my addr 172.21.15.122:0/3017464203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f0776d9d640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07781025f0 0x7f0778103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f0775d9b640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101c30 0x7f0778103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 -- 172.21.15.122:0/3017464203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07781025f0 msgr2=0x7f0778103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07781025f0 0x7f0778103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 -- 172.21.15.122:0/3017464203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101c30 msgr2=0x7f0778103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101c30 0x7f0778103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 -- 172.21.15.122:0/3017464203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0778103de0 con 0x7f0778100a40 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f077659c640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 0x7f07781049d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f076800e9c0 tx=0x7f076800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f07677fe640 1 -- 172.21.15.122:0/3017464203 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076800cd80 con 0x7f0778100a40 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.469+0000 7f07677fe640 1 -- 172.21.15.122:0/3017464203 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07680035f0 con 0x7f0778100a40 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.470+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781a1f50 con 0x7f0778100a40 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.470+0000 7f07677fe640 1 -- 172.21.15.122:0/3017464203 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768010690 con 0x7f0778100a40 2024-09-12T02:07:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.470+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a2350 con 0x7f0778100a40 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.472+0000 7f07677fe640 1 -- 172.21.15.122:0/3017464203 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f07680037a0 con 0x7f0778100a40 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.472+0000 7f077fa78740 1 Processor -- start 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 -- start start 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a79da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a81830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555d309dd2f0 con 0x555d30a7a2e0 2024-09-12T02:07:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d30985270 con 0x555d30a799a0 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077fa78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d309d92b0 con 0x555d30a7f370 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f0776d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a79da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f07677fe640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07600a2cb0 0x7f07600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f0776d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a79da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:47540/0 (socket says 172.21.15.122:47540) 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f0776d9d640 1 -- 172.21.15.122:0/117121348 learned_addr learned my addr 172.21.15.122:0/117121348 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f0775d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a81830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f0776d9d640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07600a2cb0 0x7f07600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077659c640 1 -- 172.21.15.122:0/117121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 msgr2=0x555d30a81830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077659c640 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a81830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077659c640 1 -- 172.21.15.122:0/117121348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 msgr2=0x555d30a79da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077659c640 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a79da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f077659c640 1 -- 172.21.15.122:0/117121348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d30984340 con 0x555d30a7a2e0 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f0776d9d640 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a79da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.473+0000 7f07677fe640 1 -- 172.21.15.122:0/3017464203 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f07680cf310 con 0x7f0778100a40 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.474+0000 7f077659c640 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a7eda0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0768098540 tx=0x7f0768097810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86bb2daabab32ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.474+0000 7f0776d9d640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07600a2cb0 0x7f07600a5170 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0770009fd0 tx=0x7f0770004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.474+0000 7f0764ff9640 1 -- 172.21.15.122:0/117121348 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076809c270 con 0x555d30a7a2e0 2024-09-12T02:07:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.474+0000 7f0764ff9640 1 -- 172.21.15.122:0/117121348 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076809bb70 con 0x555d30a7a2e0 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.474+0000 7f0764ff9640 1 -- 172.21.15.122:0/117121348 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076804d1f0 con 0x555d30a7a2e0 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 -- 172.21.15.122:0/117121348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 msgr2=0x555d30a7eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a7eda0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0768098540 tx=0x7f0768097810 comp rx=0 tx=0).stop 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 -- 172.21.15.122:0/117121348 shutdown_connections 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a81830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a7eda0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 --2- 172.21.15.122:0/117121348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a79da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 -- 172.21.15.122:0/117121348 >> 172.21.15.122:0/117121348 conn(0x555d30a75010 msgr2=0x555d30a75410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 -- 172.21.15.122:0/117121348 shutdown_connections 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 -- 172.21.15.122:0/117121348 wait complete. 2024-09-12T02:07:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 -- start start 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.475+0000 7f077fa78740 1 Processor -- start 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 0x555d30a83ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7f370 0x555d30a8b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555d30a822e0 con 0x555d30a7a2e0 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d30966290 con 0x555d30a7f370 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d30a82460 con 0x555d30a799a0 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077fa78740 10 client.0 _ll_register_callbacks cb 0x555d30997140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f0775d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 0x555d30a83ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 0x555d30a83ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:59848/0 (socket says 172.21.15.122:59848) 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 -- 172.21.15.122:0/816571995 learned_addr learned my addr 172.21.15.122:0/816571995 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f0776d9d640 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7f370 0x555d30a8b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 -- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7f370 msgr2=0x555d30a8b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7f370 0x555d30a8b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 -- 172.21.15.122:0/816571995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 msgr2=0x555d30a84020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a84020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 -- 172.21.15.122:0/816571995 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d306f0210 con 0x555d30a799a0 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f077659c640 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 0x555d30a83ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07680058c0 tx=0x7f07680976f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b19a51414e4a9e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.476+0000 7f075dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:07:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f075dffb640 1 -- 172.21.15.122:0/816571995 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076809c270 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f075dffb640 1 -- 172.21.15.122:0/816571995 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f07680a4350 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f076809bb70 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f075dffb640 1 -- 172.21.15.122:0/816571995 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f076804b110 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 10 client.27274 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555d307bc620 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f075dffb640 1 -- 172.21.15.122:0/816571995 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076809bb70 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f075dffb640 1 -- 172.21.15.122:0/816571995 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f07680978c0 con 0x555d30a799a0 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 10 client.27274 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 10 client.27274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 20 client.27274 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30a91750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 4 client.27274 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.477+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d309eaa20 con 0x555d30a8f360 2024-09-12T02:07:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.478+0000 7f0775d9b640 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30a91750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.478+0000 7f075dffb640 10 client.27274 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.478+0000 7f0775d9b640 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30a91750 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa79364e6ddac51 server_cookie=724c1069f664ba20 in_seq=0 out_seq=0 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f075dffb640 1 -- 172.21.15.122:0/816571995 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d309eaa20 con 0x555d30a8f360 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f075dffb640 10 client.27274 handle_command_reply: tid=0 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 client.27274 shutdown 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 msgr2=0x555d30a83ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 0x555d30a83ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07680058c0 tx=0x7f07680976f0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 shutdown_connections 2024-09-12T02:07:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30a91750 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7f370 0x555d30a8b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f075dffb640 0 client.27274 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a84020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 --2- 172.21.15.122:0/816571995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a799a0 0x555d30a83ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 >> 172.21.15.122:0/816571995 conn(0x555d30a75010 msgr2=0x555d30a76820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 shutdown_connections 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 1 -- 172.21.15.122:0/816571995 wait complete. 2024-09-12T02:07:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.481+0000 7f077fa78740 20 client.27274 trim_cache size 0 max 16384 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 Processor -- start 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 -- start start 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 0x555d30a840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7a2e0 0x555d30a8b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07781adb80 con 0x555d30a799a0 2024-09-12T02:07:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07781ab450 con 0x555d30a7a2e0 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077fa78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07781a9280 con 0x555d30a7f370 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.497+0000 7f077659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 0x555d30a840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f077659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 0x555d30a840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60520/0 (socket says 172.21.15.122:60520) 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f077659c640 1 -- 172.21.15.122:0/2296148941 learned_addr learned my addr 172.21.15.122:0/2296148941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0775d9b640 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7a2e0 0x555d30a8b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 -- 172.21.15.122:0/2296148941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7a2e0 msgr2=0x555d30a8b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7a2e0 0x555d30a8b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 -- 172.21.15.122:0/2296148941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 msgr2=0x555d30a840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 0x555d30a840b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 -- 172.21.15.122:0/2296148941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d30a74e30 con 0x555d30a7f370 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f077659c640 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 0x555d30a840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f0776d9d640 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a740b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0770012040 tx=0x7f0770009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00ae7d0d7c9651f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f075ffff640 1 -- 172.21.15.122:0/2296148941 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770013e50 con 0x555d30a7f370 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f075ffff640 1 -- 172.21.15.122:0/2296148941 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0770026070 con 0x555d30a7f370 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.498+0000 7f075ffff640 1 -- 172.21.15.122:0/2296148941 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770022af0 con 0x555d30a7f370 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 -- 172.21.15.122:0/2296148941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 msgr2=0x555d30a740b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a740b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0770012040 tx=0x7f0770009be0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 -- 172.21.15.122:0/2296148941 shutdown_connections 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a740b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a7a2e0 0x555d30a8b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 --2- 172.21.15.122:0/2296148941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a799a0 0x555d30a840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 -- 172.21.15.122:0/2296148941 >> 172.21.15.122:0/2296148941 conn(0x555d30a75010 msgr2=0x555d30a739a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 -- 172.21.15.122:0/2296148941 shutdown_connections 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 -- 172.21.15.122:0/2296148941 wait complete. 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.499+0000 7f077fa78740 1 -- start start 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 Processor -- start 2024-09-12T02:07:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a88570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a88ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a89ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07781a9910 con 0x555d30a7a2e0 2024-09-12T02:07:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07781aad80 con 0x555d30a799a0 2024-09-12T02:07:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07781a9fe0 con 0x555d30a7f370 2024-09-12T02:07:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f077fa78740 10 client.0 _ll_register_callbacks cb 0x7f0770009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a88ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a88ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60524/0 (socket says 172.21.15.122:60524) 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 -- 172.21.15.122:0/3899907877 learned_addr learned my addr 172.21.15.122:0/3899907877 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0776d9d640 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a89ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 -- 172.21.15.122:0/3899907877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 msgr2=0x555d30a89ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a89ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 -- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 msgr2=0x555d30a88570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a88570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 -- 172.21.15.122:0/3899907877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d30a8a0d0 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f0775d9b640 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a88ab0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f076c0314e0 tx=0x7f076c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c470e7a690c906 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f075dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.500+0000 7f075dffb640 1 -- 172.21.15.122:0/3899907877 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035040 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f075dffb640 1 -- 172.21.15.122:0/3899907877 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f076c005240 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0770026070 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f075dffb640 1 -- 172.21.15.122:0/3899907877 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f076c055050 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 10 client.18300 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555d30af9fd0 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f075dffb640 1 -- 172.21.15.122:0/3899907877 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0770026070 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f075dffb640 1 -- 172.21.15.122:0/3899907877 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f076c0316b0 con 0x555d30a7a2e0 2024-09-12T02:07:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 10 client.18300 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 10 client.18300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 20 client.18300 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30afc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 4 client.18300 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d309e4d70 con 0x555d30a8f360 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.501+0000 7f077659c640 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30afc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.502+0000 7f075dffb640 10 client.18300 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.502+0000 7f077659c640 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30afc990 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e987af27efe54b1 server_cookie=c16f615be9074dda in_seq=0 out_seq=0 2024-09-12T02:07:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f075dffb640 1 -- 172.21.15.122:0/3899907877 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d309e4d70 con 0x555d30a8f360 2024-09-12T02:07:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f075dffb640 10 client.18300 handle_command_reply: tid=0 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f077fa78740 1 client.18300 shutdown 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 msgr2=0x555d30a88ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f077fa78740 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a88ab0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f076c0314e0 tx=0x7f076c0041a0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 shutdown_connections 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.759+0000 7f077fa78740 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555d30a8f360 0x555d30afc990 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d30a7f370 0x555d30a89ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f075dffb640 0 client.18300 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555d30a7a2e0 0x555d30a88ab0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 1 --2- 172.21.15.122:0/3899907877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d30a799a0 0x555d30a88570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 >> 172.21.15.122:0/3899907877 conn(0x555d30a75010 msgr2=0x555d30a741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 shutdown_connections 2024-09-12T02:07:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 1 -- 172.21.15.122:0/3899907877 wait complete. 2024-09-12T02:07:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077fa78740 20 client.18300 trim_cache size 0 max 16384 2024-09-12T02:07:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:07:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:07:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:07:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:07:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07600a2cb0 msgr2=0x7f07600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077cc04640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07600a2cb0 0x7f07600a5170 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0770009fd0 tx=0x7f0770004ab0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 msgr2=0x7f07781049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.760+0000 7f077cc04640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 0x7f07781049d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f076800e9c0 tx=0x7f076800ee80 comp rx=0 tx=0).stop 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 shutdown_connections 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f07600a2cb0 0x7f07600a5170 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07781025f0 0x7f0778103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101c30 0x7f0778103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 --2- 172.21.15.122:0/3017464203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778100a40 0x7f07781049d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 >> 172.21.15.122:0/3017464203 conn(0x7f07780fc1e0 msgr2=0x7f07780fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 shutdown_connections 2024-09-12T02:07:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:07.761+0000 7f077cc04640 1 -- 172.21.15.122:0/3017464203 wait complete. 2024-09-12T02:07:08.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 168089 21.06 MB/sec execute 295 sec latency 226.531 ms 2024-09-12T02:07:09.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 168637 21.08 MB/sec execute 296 sec latency 78.630 ms 2024-09-12T02:07:09.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:07:09.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:07:09.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:09 smithi151 ceph-mon[35229]: pgmap v960: 97 pgs: 97 active+clean; 522 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-09-12T02:07:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:07:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:07:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:09 smithi183 ceph-mon[35127]: pgmap v960: 97 pgs: 97 active+clean; 522 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-09-12T02:07:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:07:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:07:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:07:09.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:09 smithi122 ceph-mon[30925]: pgmap v960: 97 pgs: 97 active+clean; 522 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 590 op/s 2024-09-12T02:07:10.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 169367 21.14 MB/sec execute 297 sec latency 88.097 ms 2024-09-12T02:07:11.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170026 21.19 MB/sec execute 298 sec latency 70.498 ms 2024-09-12T02:07:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:10 smithi151 ceph-mon[35229]: pgmap v961: 97 pgs: 97 active+clean; 517 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-09-12T02:07:11.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:10 smithi183 ceph-mon[35127]: pgmap v961: 97 pgs: 97 active+clean; 517 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-09-12T02:07:11.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:10 smithi122 ceph-mon[30925]: pgmap v961: 97 pgs: 97 active+clean; 517 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-09-12T02:07:12.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170547 21.21 MB/sec execute 299 sec latency 271.131 ms 2024-09-12T02:07:13.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 171129 21.28 MB/sec execute 300 sec latency 91.978 ms 2024-09-12T02:07:13.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:12 smithi151 ceph-mon[35229]: pgmap v962: 97 pgs: 97 active+clean; 515 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 586 op/s 2024-09-12T02:07:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:12 smithi183 ceph-mon[35127]: pgmap v962: 97 pgs: 97 active+clean; 515 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 586 op/s 2024-09-12T02:07:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:12 smithi122 ceph-mon[30925]: pgmap v962: 97 pgs: 97 active+clean; 515 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 586 op/s 2024-09-12T02:07:14.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 171702 21.32 MB/sec execute 301 sec latency 58.428 ms 2024-09-12T02:07:14.543 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.9 2024-09-12T02:07:14.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:07:14.843 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:14.855 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:14 smithi122 ceph-mon[30925]: pgmap v963: 97 pgs: 97 active+clean; 518 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 580 op/s 2024-09-12T02:07:15.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 172227 21.33 MB/sec execute 302 sec latency 105.817 ms 2024-09-12T02:07:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:14 smithi151 ceph-mon[35229]: pgmap v963: 97 pgs: 97 active+clean; 518 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 580 op/s 2024-09-12T02:07:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:14 smithi183 ceph-mon[35127]: pgmap v963: 97 pgs: 97 active+clean; 518 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 580 op/s 2024-09-12T02:07:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 -- 172.21.15.122:0/2649816154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 msgr2=0x7f57341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 --2- 172.21.15.122:0/2649816154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f57341029b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5724009900 tx=0x7f572402a480 comp rx=0 tx=0).stop 2024-09-12T02:07:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 -- 172.21.15.122:0/2649816154 shutdown_connections 2024-09-12T02:07:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 --2- 172.21.15.122:0/2649816154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 0x7f573410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 --2- 172.21.15.122:0/2649816154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f57341029b0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 --2- 172.21.15.122:0/2649816154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 0x7f5734101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 -- 172.21.15.122:0/2649816154 >> 172.21.15.122:0/2649816154 conn(0x7f57340fc7d0 msgr2=0x7f57340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 -- 172.21.15.122:0/2649816154 shutdown_connections 2024-09-12T02:07:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.661+0000 7f573addf640 1 -- 172.21.15.122:0/2649816154 wait complete. 2024-09-12T02:07:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.662+0000 7f573addf640 1 Processor -- start 2024-09-12T02:07:15.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.662+0000 7f573addf640 1 -- start start 2024-09-12T02:07:15.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.662+0000 7f573addf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 0x7f573419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.662+0000 7f573addf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f573419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.662+0000 7f573addf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 0x7f57341962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.662+0000 7f572bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f573419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f572bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f573419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47108/0 (socket says 172.21.15.122:47108) 2024-09-12T02:07:15.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f5738b7b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 0x7f573419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f572bfff640 1 -- 172.21.15.122:0/3674534861 learned_addr learned my addr 172.21.15.122:0/3674534861 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f573410fe80 con 0x7f5734102570 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f573410fd00 con 0x7f5734102ef0 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5734110000 con 0x7f5734100e20 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f573937c640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 0x7f57341962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f5738b7b640 1 -- 172.21.15.122:0/3674534861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 msgr2=0x7f57341962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f5738b7b640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 0x7f57341962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f5738b7b640 1 -- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 msgr2=0x7f573419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f5738b7b640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f573419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f5738b7b640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5734196b90 con 0x7f5734100e20 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f573937c640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 0x7f57341962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:15.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.663+0000 7f572bfff640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f573419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.664+0000 7f5738b7b640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 0x7f573419d060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f573001fa00 tx=0x7f573001fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.664+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573000ccc0 con 0x7f5734100e20 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.664+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5734196e80 con 0x7f5734100e20 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.664+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5730005020 con 0x7f5734100e20 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.664+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730004a00 con 0x7f5734100e20 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.664+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573419ae20 con 0x7f5734100e20 2024-09-12T02:07:15.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.665+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57341021b0 con 0x7f5734100e20 2024-09-12T02:07:15.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.667+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5730005560 con 0x7f5734100e20 2024-09-12T02:07:15.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.668+0000 7f5729ffb640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f571c0a2c00 0x7f571c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:15.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.668+0000 7f572bfff640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f571c0a2c00 0x7f571c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:15.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.668+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f57300dff20 con 0x7f5734100e20 2024-09-12T02:07:15.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.669+0000 7f572bfff640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f571c0a2c00 0x7f571c0a50c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57341978b0 tx=0x7f5724005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:15.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.673+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57300051d0 con 0x7f5734100e20 2024-09-12T02:07:15.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.975+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5734101200 con 0x7f5734100e20 2024-09-12T02:07:15.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.976+0000 7f5729ffb640 1 -- 172.21.15.122:0/3674534861 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7f57300ace80 con 0x7f5734100e20 2024-09-12T02:07:15.976 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:07:15.977 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:07:15.977 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:07:15.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f571c0a2c00 msgr2=0x7f571c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f571c0a2c00 0x7f571c0a50c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57341978b0 tx=0x7f5724005a20 comp rx=0 tx=0).stop 2024-09-12T02:07:15.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 msgr2=0x7f573419d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:15.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 0x7f573419d060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f573001fa00 tx=0x7f573001fec0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 shutdown_connections 2024-09-12T02:07:15.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f571c0a2c00 0x7f571c0a50c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5734102ef0 0x7f57341962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5734102570 0x7f573419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 --2- 172.21.15.122:0/3674534861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5734100e20 0x7f573419d060 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:15.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 >> 172.21.15.122:0/3674534861 conn(0x7f57340fc7d0 msgr2=0x7f57340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:15.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 shutdown_connections 2024-09-12T02:07:15.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:15.978+0000 7f573addf640 1 -- 172.21.15.122:0/3674534861 wait complete. 2024-09-12T02:07:16.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 172704 21.32 MB/sec execute 303 sec latency 94.171 ms 2024-09-12T02:07:16.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:16 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3674534861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:16.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:16 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3674534861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:16.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3674534861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:17.067 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.9 --await-for 45.0' 2024-09-12T02:07:17.067 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.9 --await-for 45.0 2024-09-12T02:07:17.074 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173073 21.31 MB/sec execute 304 sec latency 322.321 ms 2024-09-12T02:07:17.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:17 smithi151 ceph-mon[35229]: pgmap v964: 97 pgs: 97 active+clean; 522 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.3 MiB/s wr, 562 op/s 2024-09-12T02:07:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:17.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:17 smithi183 ceph-mon[35127]: pgmap v964: 97 pgs: 97 active+clean; 522 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.3 MiB/s wr, 562 op/s 2024-09-12T02:07:17.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:17 smithi122 ceph-mon[30925]: pgmap v964: 97 pgs: 97 active+clean; 522 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.3 MiB/s wr, 562 op/s 2024-09-12T02:07:17.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/4050938874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 msgr2=0x7f8ed01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/4050938874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed01029b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009900 tx=0x7f8ebc02a480 comp rx=0 tx=0).stop 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/4050938874 shutdown_connections 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/4050938874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ed0102ef0 0x7f8ed010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/4050938874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed01029b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/4050938874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 0x7f8ed0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/4050938874 >> 172.21.15.122:0/4050938874 conn(0x7f8ed00fc7d0 msgr2=0x7f8ed00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/4050938874 shutdown_connections 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.973+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/4050938874 wait complete. 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 Processor -- start 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 -- start start 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 0x7f8ed0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ed0102ef0 0x7f8ed019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0110f80 con 0x7f8ed0102570 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0110e00 con 0x7f8ed0100e20 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed6b5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0111100 con 0x7f8ed0102ef0 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed48fa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 0x7f8ed0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed0197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47132/0 (socket says 172.21.15.122:47132) 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ecffff640 1 -- 172.21.15.122:0/29831231 learned_addr learned my addr 172.21.15.122:0/29831231 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.974+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ed0102ef0 0x7f8ed019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed48fa640 1 -- 172.21.15.122:0/29831231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ed0102ef0 msgr2=0x7f8ed019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ed0102ef0 0x7f8ed019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed48fa640 1 -- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 msgr2=0x7f8ed0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed0197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed48fa640 1 -- 172.21.15.122:0/29831231 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed019c2d0 con 0x7f8ed0100e20 2024-09-12T02:07:17.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ecffff640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:17.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 0x7f8ed0197320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e9f0 tx=0x7f8ec400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ecdffb640 1 -- 172.21.15.122:0/29831231 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400ccc0 con 0x7f8ed0100e20 2024-09-12T02:07:17.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed019c5c0 con 0x7f8ed0100e20 2024-09-12T02:07:17.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed01a3ea0 con 0x7f8ed0100e20 2024-09-12T02:07:17.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ecdffb640 1 -- 172.21.15.122:0/29831231 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4010910 con 0x7f8ed0100e20 2024-09-12T02:07:17.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.975+0000 7f8ecdffb640 1 -- 172.21.15.122:0/29831231 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4018880 con 0x7f8ed0100e20 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ecdffb640 1 -- 172.21.15.122:0/29831231 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8ec4010430 con 0x7f8ed0100e20 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 Processor -- start 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 -- start start 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090df3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfe940 0x556090e00d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556090d592a0 con 0x556090dfe940 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556090d00b10 con 0x556090df3760 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556090d55260 con 0x556090df4080 2024-09-12T02:07:17.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed48fa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed50fb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090df3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfe940 0x556090e00d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed50fb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090df3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:58426/0 (socket says 172.21.15.122:58426) 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.978+0000 7f8ed50fb640 1 -- 172.21.15.122:0/2237782540 learned_addr learned my addr 172.21.15.122:0/2237782540 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecffff640 1 -- 172.21.15.122:0/2237782540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 msgr2=0x556090dfe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecffff640 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecffff640 1 -- 172.21.15.122:0/2237782540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 msgr2=0x556090df3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecffff640 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090df3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecffff640 1 -- 172.21.15.122:0/2237782540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556090acab90 con 0x556090dfe940 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090df3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecdffb640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ec00a2b80 0x7f8ec00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecffff640 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfe940 0x556090e00d30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009230 tx=0x7f8ebc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4928b3af79b810df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ec00a2b80 0x7f8ec00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ecdffb640 1 -- 172.21.15.122:0/29831231 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8ec40cf2a0 con 0x7f8ed0100e20 2024-09-12T02:07:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.979+0000 7f8ebaffd640 1 -- 172.21.15.122:0/2237782540 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc040070 con 0x556090dfe940 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.980+0000 7f8ebaffd640 1 -- 172.21.15.122:0/2237782540 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc003e10 con 0x556090dfe940 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.980+0000 7f8ebaffd640 1 -- 172.21.15.122:0/2237782540 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc035040 con 0x556090dfe940 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.980+0000 7f8ed8f37740 1 -- 172.21.15.122:0/2237782540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfe940 msgr2=0x556090e00d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.980+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfe940 0x556090e00d30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009230 tx=0x7f8ebc005e80 comp rx=0 tx=0).stop 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.980+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ec00a2b80 0x7f8ec00a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8010680 tx=0x7f8ec8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 -- 172.21.15.122:0/2237782540 shutdown_connections 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfe940 0x556090e00d30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/2237782540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090df3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 -- 172.21.15.122:0/2237782540 >> 172.21.15.122:0/2237782540 conn(0x556090defcf0 msgr2=0x556090df00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 -- 172.21.15.122:0/2237782540 shutdown_connections 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 -- 172.21.15.122:0/2237782540 wait complete. 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 -- start start 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.981+0000 7f8ed8f37740 1 Processor -- start 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090df3760 0x556090dfba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 0x556090dfbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090dfc490 0x556090e0a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556090e01510 con 0x556090df3760 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556090ce16e0 con 0x556090df4080 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556090e01690 con 0x556090dfc490 2024-09-12T02:07:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed8f37740 10 client.0 _ll_register_callbacks cb 0x556090d05a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 0x556090dfbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed48fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090df3760 0x556090dfba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 0x556090dfbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:58434/0 (socket says 172.21.15.122:58434) 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 -- 172.21.15.122:0/1728810241 learned_addr learned my addr 172.21.15.122:0/1728810241 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090dfc490 0x556090e0a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 -- 172.21.15.122:0/1728810241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090dfc490 msgr2=0x556090e0a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090dfc490 0x556090e0a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 -- 172.21.15.122:0/1728810241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090df3760 msgr2=0x556090dfba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090df3760 0x556090dfba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ecffff640 1 -- 172.21.15.122:0/1728810241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556090e0b020 con 0x556090df4080 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.982+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090df3760 0x556090dfba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8ecffff640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 0x556090dfbf50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005770 tx=0x7f8ebc031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ff5be8f5dbe129 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8eaffff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:07:17.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8eaffff640 1 -- 172.21.15.122:0/1728810241 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc04c020 con 0x556090df4080 2024-09-12T02:07:17.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8eaffff640 1 -- 172.21.15.122:0/1728810241 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8ebc003b40 con 0x556090df4080 2024-09-12T02:07:17.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc003e10 con 0x556090df4080 2024-09-12T02:07:17.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8eaffff640 1 -- 172.21.15.122:0/1728810241 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8ebc035070 con 0x556090df4080 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8ed8f37740 10 client.27320 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556090e0d9b0 con 0x556090df4080 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8eaffff640 1 -- 172.21.15.122:0/1728810241 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc003e10 con 0x556090df4080 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.983+0000 7f8eaffff640 1 -- 172.21.15.122:0/1728810241 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f8ebc03c440 con 0x556090df4080 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed8f37740 10 client.27320 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed8f37740 10 client.27320 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed8f37740 20 client.27320 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed8f37740 4 client.27320 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556090d669d0 con 0x556090e0ed00 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8eaffff640 10 client.27320 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.984+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e110f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cbf953a58e7fca server_cookie=aafafc435ba63296 in_seq=0 out_seq=0 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8eaffff640 1 -- 172.21.15.122:0/1728810241 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556090d669d0 con 0x556090e0ed00 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8eaffff640 10 client.27320 handle_command_reply: tid=0 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 client.27320 shutdown 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 msgr2=0x556090dfbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 0x556090dfbf50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005770 tx=0x7f8ebc031770 comp rx=0 tx=0).stop 2024-09-12T02:07:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 shutdown_connections 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e110f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090dfc490 0x556090e0a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8eaffff640 0 client.27320 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df4080 0x556090dfbf50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/1728810241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090df3760 0x556090dfba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.985+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 >> 172.21.15.122:0/1728810241 conn(0x556090defcf0 msgr2=0x556090df5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.986+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 shutdown_connections 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.986+0000 7f8ed8f37740 1 -- 172.21.15.122:0/1728810241 wait complete. 2024-09-12T02:07:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:17.986+0000 7f8ed8f37740 20 client.27320 trim_cache size 0 max 16384 2024-09-12T02:07:18.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 Processor -- start 2024-09-12T02:07:18.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 -- start start 2024-09-12T02:07:18.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e0a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090df8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4010d60 con 0x556090dfc490 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb400e630 con 0x556090df3760 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb400c460 con 0x556090df4080 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed50fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090df8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed50fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090df8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47174/0 (socket says 172.21.15.122:47174) 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed50fb640 1 -- 172.21.15.122:0/3979758569 learned_addr learned my addr 172.21.15.122:0/3979758569 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e0a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ecffff640 1 -- 172.21.15.122:0/3979758569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 msgr2=0x556090e0a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ecffff640 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e0a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ecffff640 1 -- 172.21.15.122:0/3979758569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 msgr2=0x556090df8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ecffff640 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090df8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.006+0000 7f8ecffff640 1 -- 172.21.15.122:0/3979758569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556090df7300 con 0x556090df4080 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.007+0000 7f8ecffff640 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfd980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009850 tx=0x7f8ebc005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=879b427ca5e94421 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.007+0000 7f8eba7fc640 1 -- 172.21.15.122:0/3979758569 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc04c040 con 0x556090df4080 2024-09-12T02:07:18.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.007+0000 7f8eba7fc640 1 -- 172.21.15.122:0/3979758569 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc03c440 con 0x556090df4080 2024-09-12T02:07:18.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.007+0000 7f8eba7fc640 1 -- 172.21.15.122:0/3979758569 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc040040 con 0x556090df4080 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 -- 172.21.15.122:0/3979758569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 msgr2=0x556090dfd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfd980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009850 tx=0x7f8ebc005e80 comp rx=0 tx=0).stop 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 -- 172.21.15.122:0/3979758569 shutdown_connections 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090df8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090dfd980 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/3979758569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e0a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 -- 172.21.15.122:0/3979758569 >> 172.21.15.122:0/3979758569 conn(0x556090defcf0 msgr2=0x556090dfd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 -- 172.21.15.122:0/3979758569 shutdown_connections 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 -- 172.21.15.122:0/3979758569 wait complete. 2024-09-12T02:07:18.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.008+0000 7f8ed8f37740 1 -- start start 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 Processor -- start 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e12170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090e126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090e12bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb400caf0 con 0x556090dfc490 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb400df60 con 0x556090df3760 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb400d1c0 con 0x556090df4080 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed8f37740 10 client.0 _ll_register_callbacks cb 0x7f8ebc0043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed50fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090e12bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090e126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed48fa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e12170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed50fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090e12bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47184/0 (socket says 172.21.15.122:47184) 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090e126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:34394/0 (socket says 172.21.15.122:34394) 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.009+0000 7f8ed50fb640 1 -- 172.21.15.122:0/661432201 learned_addr learned my addr 172.21.15.122:0/661432201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed50fb640 1 -- 172.21.15.122:0/661432201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 msgr2=0x556090e126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090e126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed50fb640 1 -- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 msgr2=0x556090e12170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e12170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed50fb640 1 -- 172.21.15.122:0/661432201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556090df6710 con 0x556090dfc490 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e12170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:18.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ecffff640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090e126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed50fb640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090e12bf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800fe70 tx=0x7f8ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c50e17092d9ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8eaffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8eaffff640 1 -- 172.21.15.122:0/661432201 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec802d060 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc03c440 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8eaffff640 1 -- 172.21.15.122:0/661432201 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8ec8003970 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8eaffff640 1 -- 172.21.15.122:0/661432201 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8003bb0 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed8f37740 10 client.18324 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556090e79a70 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.010+0000 7f8eaffff640 1 -- 172.21.15.122:0/661432201 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc03c440 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8eaffff640 1 -- 172.21.15.122:0/661432201 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec8032070 con 0x556090dfc490 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed8f37740 10 client.18324 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:18.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed8f37740 10 client.18324 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed8f37740 20 client.18324 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed8f37740 4 client.18324 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.9, "await_for": 45.0}] 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.9, "await_for": 45.0}) -- 0x556090d60d20 con 0x556090e0ed00 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8eaffff640 10 client.18324 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:18.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:18.011+0000 7f8ed48fa640 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e7c200 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2042e4838f983c server_cookie=388df9887fbbb844 in_seq=0 out_seq=0 2024-09-12T02:07:18.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173623 21.34 MB/sec execute 305 sec latency 104.440 ms 2024-09-12T02:07:19.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173634 21.27 MB/sec execute 306 sec latency 1080.797 ms 2024-09-12T02:07:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:18 smithi151 ceph-mon[35229]: pgmap v965: 97 pgs: 97 active+clean; 522 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.7 MiB/s wr, 580 op/s 2024-09-12T02:07:19.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.124+0000 7f8eaffff640 1 -- 172.21.15.122:0/661432201 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556090d60d20 con 0x556090e0ed00 2024-09-12T02:07:19.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.124+0000 7f8eaffff640 10 client.18324 handle_command_reply: tid=0 2024-09-12T02:07:19.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.124+0000 7f8ed8f37740 1 client.18324 shutdown 2024-09-12T02:07:19.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.124+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 msgr2=0x556090e12bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:19.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.124+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090e12bf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800fe70 tx=0x7f8ec8004a40 comp rx=0 tx=0).stop 2024-09-12T02:07:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 shutdown_connections 2024-09-12T02:07:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x556090e0ed00 0x556090e7c200 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.126 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:18 smithi122 ceph-mon[30925]: pgmap v965: 97 pgs: 97 active+clean; 522 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.7 MiB/s wr, 580 op/s 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556090dfc490 0x556090e12bf0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8eaffff640 0 client.18324 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556090df4080 0x556090e126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 --2- 172.21.15.122:0/661432201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556090df3760 0x556090e12170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 >> 172.21.15.122:0/661432201 conn(0x556090defcf0 msgr2=0x556090dee650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 shutdown_connections 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 1 -- 172.21.15.122:0/661432201 wait complete. 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.125+0000 7f8ed8f37740 20 client.18324 trim_cache size 0 max 16384 2024-09-12T02:07:19.127 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 74, 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "4b6e1ae5": { 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 74, 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:07:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 139.9, 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:19.129 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:07:19.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ec00a2b80 msgr2=0x7f8ec00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:19.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ec00a2b80 0x7f8ec00a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8010680 tx=0x7f8ec8007520 comp rx=0 tx=0).stop 2024-09-12T02:07:19.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 msgr2=0x7f8ed0197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:19.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 0x7f8ed0197320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e9f0 tx=0x7f8ec400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 shutdown_connections 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f8ec00a2b80 0x7f8ec00a5040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ed0102ef0 0x7f8ed019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ed0102570 0x7f8ed0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 --2- 172.21.15.122:0/29831231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed0100e20 0x7f8ed0197320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 >> 172.21.15.122:0/29831231 conn(0x7f8ed00fc7d0 msgr2=0x7f8ed00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 shutdown_connections 2024-09-12T02:07:19.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:19.126+0000 7f8ed6b5e640 1 -- 172.21.15.122:0/29831231 wait complete. 2024-09-12T02:07:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:18 smithi183 ceph-mon[35127]: pgmap v965: 97 pgs: 97 active+clean; 522 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.7 MiB/s wr, 580 op/s 2024-09-12T02:07:20.041 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4b6e1ae5', quiesce took 5.5 seconds. Will release after: 14.399999999999999 2024-09-12T02:07:20.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 21.20 MB/sec execute 307 sec latency 2080.867 ms 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 -- 172.21.15.122:0/2651340534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 msgr2=0x7fd320109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 --2- 172.21.15.122:0/2651340534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd320109c90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd318009ca0 tx=0x7fd31802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 -- 172.21.15.122:0/2651340534 shutdown_connections 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 --2- 172.21.15.122:0/2651340534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd320109c90 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 --2- 172.21.15.122:0/2651340534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd3201072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 --2- 172.21.15.122:0/2651340534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3201025c0 0x7fd3201049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 -- 172.21.15.122:0/2651340534 >> 172.21.15.122:0/2651340534 conn(0x7fd3200fc1e0 msgr2=0x7fd3200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 -- 172.21.15.122:0/2651340534 shutdown_connections 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 -- 172.21.15.122:0/2651340534 wait complete. 2024-09-12T02:07:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 Processor -- start 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.860+0000 7fd328766640 1 -- start start 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd328766640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3201025c0 0x7fd320197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd328766640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd320198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd328766640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd32019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd328766640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d800 con 0x7fd320107820 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd328766640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d680 con 0x7fd320104ef0 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd328766640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d980 con 0x7fd3201025c0 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd32019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd325d01640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd320198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd32019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44386/0 (socket says 172.21.15.122:44386) 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd325d01640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd320198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36650/0 (socket says 172.21.15.122:36650) 2024-09-12T02:07:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 -- 172.21.15.122:0/1854023434 learned_addr learned my addr 172.21.15.122:0/1854023434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326502640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3201025c0 0x7fd320197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 -- 172.21.15.122:0/1854023434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3201025c0 msgr2=0x7fd320197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3201025c0 0x7fd320197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 -- 172.21.15.122:0/1854023434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 msgr2=0x7fd320198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd320198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd326d03640 1 -- 172.21.15.122:0/1854023434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32019fa70 con 0x7fd320107820 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.861+0000 7fd325d01640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd320198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.862+0000 7fd326d03640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd32019f400 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd318006010 tx=0x7fd318004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.862+0000 7fd3177fe640 1 -- 172.21.15.122:0/1854023434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318003b70 con 0x7fd320107820 2024-09-12T02:07:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.862+0000 7fd3177fe640 1 -- 172.21.15.122:0/1854023434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd318003d10 con 0x7fd320107820 2024-09-12T02:07:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.862+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32019fd00 con 0x7fd320107820 2024-09-12T02:07:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.862+0000 7fd3177fe640 1 -- 172.21.15.122:0/1854023434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31802fc40 con 0x7fd320107820 2024-09-12T02:07:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.862+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3201a01f0 con 0x7fd320107820 2024-09-12T02:07:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.864+0000 7fd32b5de740 1 Processor -- start 2024-09-12T02:07:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.865+0000 7fd3177fe640 1 -- 172.21.15.122:0/1854023434 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd318004050 con 0x7fd320107820 2024-09-12T02:07:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.865+0000 7fd32b5de740 1 -- start start 2024-09-12T02:07:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.865+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.865+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.865+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f47370 0x562cc6f49830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd32b5de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cc6ea52f0 con 0x562cc6f419a0 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd32b5de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cc6e4d270 con 0x562cc6f422e0 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd32b5de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cc6ea12b0 con 0x562cc6f47370 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f41da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44398/0 (socket says 172.21.15.122:44398) 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 -- 172.21.15.122:0/530473262 learned_addr learned my addr 172.21.15.122:0/530473262 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd326502640 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f47370 0x562cc6f49830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd3177fe640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd30c0a2c30 0x7fd30c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 -- 172.21.15.122:0/530473262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f47370 msgr2=0x562cc6f49830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f47370 0x562cc6f49830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 -- 172.21.15.122:0/530473262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 msgr2=0x562cc6f46da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f46da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 -- 172.21.15.122:0/530473262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc6e4c340 con 0x562cc6f419a0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd326d03640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd30c0a2c30 0x7fd30c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd325d01640 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f41da0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd310013bf0 tx=0x7fd310009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8741a623b12d25e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.866+0000 7fd3177fe640 1 -- 172.21.15.122:0/1854023434 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd3180f0210 con 0x7fd320107820 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd314ff9640 1 -- 172.21.15.122:0/530473262 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310013130 con 0x562cc6f419a0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd314ff9640 1 -- 172.21.15.122:0/530473262 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31000fc50 con 0x562cc6f419a0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd314ff9640 1 -- 172.21.15.122:0/530473262 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310005440 con 0x562cc6f419a0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 -- 172.21.15.122:0/530473262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 msgr2=0x562cc6f41da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd326d03640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd30c0a2c30 0x7fd30c0a50f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3180bc770 tx=0x7fd318033040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f41da0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd310013bf0 tx=0x7fd310009c00 comp rx=0 tx=0).stop 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 -- 172.21.15.122:0/530473262 shutdown_connections 2024-09-12T02:07:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f47370 0x562cc6f49830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f46da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 --2- 172.21.15.122:0/530473262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f41da0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.867+0000 7fd32b5de740 1 -- 172.21.15.122:0/530473262 >> 172.21.15.122:0/530473262 conn(0x562cc6f3d010 msgr2=0x562cc6f3d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 -- 172.21.15.122:0/530473262 shutdown_connections 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 -- 172.21.15.122:0/530473262 wait complete. 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 -- start start 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 Processor -- start 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f422e0 0x562cc6f4c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f47370 0x562cc6f530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cc6f4a2e0 con 0x562cc6f419a0 2024-09-12T02:07:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cc6e2e290 con 0x562cc6f47370 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cc6f4a460 con 0x562cc6f422e0 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd32b5de740 10 client.0 _ll_register_callbacks cb 0x562cc6e5f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f4bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44402/0 (socket says 172.21.15.122:44402) 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.868+0000 7fd325d01640 1 -- 172.21.15.122:0/1186249331 learned_addr learned my addr 172.21.15.122:0/1186249331 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd326502640 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f422e0 0x562cc6f4c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd325d01640 1 -- 172.21.15.122:0/1186249331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f422e0 msgr2=0x562cc6f4c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd325d01640 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f422e0 0x562cc6f4c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd325d01640 1 -- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f47370 msgr2=0x562cc6f530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd325d01640 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f47370 0x562cc6f530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd325d01640 1 -- 172.21.15.122:0/1186249331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc6bb8210 con 0x562cc6f419a0 2024-09-12T02:07:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd325d01640 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f4bb40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd31000a010 tx=0x7fd310013990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa2f8ea0a96bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd309ffb640 1 -- 172.21.15.122:0/1186249331 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310004230 con 0x562cc6f419a0 2024-09-12T02:07:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd309ffb640 1 -- 172.21.15.122:0/1186249331 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd3100043d0 con 0x562cc6f419a0 2024-09-12T02:07:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31000fc50 con 0x562cc6f419a0 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd309ffb640 1 -- 172.21.15.122:0/1186249331 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd31000fe00 con 0x562cc6f419a0 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd32b5de740 10 client.18336 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562cc6c84620 con 0x562cc6f419a0 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd309ffb640 1 -- 172.21.15.122:0/1186249331 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31000fc50 con 0x562cc6f419a0 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.869+0000 7fd309ffb640 1 -- 172.21.15.122:0/1186249331 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fd310003a80 con 0x562cc6f419a0 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd32b5de740 10 client.18336 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd32b5de740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd32b5de740 20 client.18336 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd32b5de740 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6f57540 0x562cc6f59930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd32b5de740 4 client.18336 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cc6eb2a20 con 0x562cc6f57540 2024-09-12T02:07:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd326502640 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6f57540 0x562cc6f59930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd309ffb640 10 client.18336 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.870+0000 7fd326502640 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6f57540 0x562cc6f59930 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b7a20eb3049a72 server_cookie=b0070a08fc2e435a in_seq=0 out_seq=0 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd309ffb640 1 -- 172.21.15.122:0/1186249331 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cc6eb2a20 con 0x562cc6f57540 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd309ffb640 10 client.18336 handle_command_reply: tid=0 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 client.18336 shutdown 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 msgr2=0x562cc6f4bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f4bb40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd31000a010 tx=0x7fd310013990 comp rx=0 tx=0).stop 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 shutdown_connections 2024-09-12T02:07:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6f57540 0x562cc6f59930 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f47370 0x562cc6f530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f422e0 0x562cc6f4c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd309ffb640 0 client.18336 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 --2- 172.21.15.122:0/1186249331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f419a0 0x562cc6f4bb40 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 >> 172.21.15.122:0/1186249331 conn(0x562cc6f3d010 msgr2=0x562cc6f3e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 shutdown_connections 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 1 -- 172.21.15.122:0/1186249331 wait complete. 2024-09-12T02:07:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.871+0000 7fd32b5de740 20 client.18336 trim_cache size 0 max 16384 2024-09-12T02:07:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.885+0000 7fd32b5de740 1 Processor -- start 2024-09-12T02:07:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.885+0000 7fd32b5de740 1 -- start start 2024-09-12T02:07:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f4bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd32b5de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201ab970 con 0x562cc6f47370 2024-09-12T02:07:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd32b5de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201a9240 con 0x562cc6f422e0 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd32b5de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201a7070 con 0x562cc6f419a0 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f4bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f4bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44404/0 (socket says 172.21.15.122:44404) 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 -- 172.21.15.122:0/2428865058 learned_addr learned my addr 172.21.15.122:0/2428865058 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd326502640 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 -- 172.21.15.122:0/2428865058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 msgr2=0x562cc6f3c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f3c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 -- 172.21.15.122:0/2428865058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 msgr2=0x562cc6f53080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f53080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 -- 172.21.15.122:0/2428865058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc6f46760 con 0x562cc6f47370 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd325d01640 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f4bf60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd310009730 tx=0x7fd3100096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b31af0c28f217ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd30bfff640 1 -- 172.21.15.122:0/2428865058 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310024070 con 0x562cc6f47370 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd30bfff640 1 -- 172.21.15.122:0/2428865058 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31000fc20 con 0x562cc6f47370 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.886+0000 7fd30bfff640 1 -- 172.21.15.122:0/2428865058 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310004ca0 con 0x562cc6f47370 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 -- 172.21.15.122:0/2428865058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 msgr2=0x562cc6f4bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f4bf60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd310009730 tx=0x7fd3100096c0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 -- 172.21.15.122:0/2428865058 shutdown_connections 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f4bf60 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f53080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 --2- 172.21.15.122:0/2428865058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 -- 172.21.15.122:0/2428865058 >> 172.21.15.122:0/2428865058 conn(0x562cc6f3d010 msgr2=0x562cc6f3e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 -- 172.21.15.122:0/2428865058 shutdown_connections 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 -- 172.21.15.122:0/2428865058 wait complete. 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 -- start start 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.887+0000 7fd32b5de740 1 Processor -- start 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f50a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f50f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f51f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201a7700 con 0x562cc6f47370 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201a8b70 con 0x562cc6f422e0 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201a7dd0 con 0x562cc6f419a0 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 10 client.0 _ll_register_callbacks cb 0x7fd31c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f51f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f51f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44418/0 (socket says 172.21.15.122:44418) 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 -- 172.21.15.122:0/3117141463 learned_addr learned my addr 172.21.15.122:0/3117141463 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd326502640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f50f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f50a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 -- 172.21.15.122:0/3117141463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 msgr2=0x562cc6f50a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f50a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 -- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 msgr2=0x562cc6f50f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f50f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 -- 172.21.15.122:0/3117141463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc6f52590 con 0x562cc6f47370 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd325d01640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f51f80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd31000a010 tx=0x7fd310005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bb49a03cd33493 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd309ffb640 1 -- 172.21.15.122:0/3117141463 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310024070 con 0x562cc6f47370 2024-09-12T02:07:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd309ffb640 1 -- 172.21.15.122:0/3117141463 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd310004e30 con 0x562cc6f47370 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.888+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31000fc20 con 0x562cc6f47370 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd309ffb640 1 -- 172.21.15.122:0/3117141463 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd310004000 con 0x562cc6f47370 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 10 client.18342 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562cc6fc23a0 con 0x562cc6f47370 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd309ffb640 1 -- 172.21.15.122:0/3117141463 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31000fc20 con 0x562cc6f47370 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd309ffb640 1 -- 172.21.15.122:0/3117141463 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fd3100035f0 con 0x562cc6f47370 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 10 client.18342 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 10 client.18342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 20 client.18342 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6fc2cd0 0x562cc6fc50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 4 client.18342 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cc6eacd70 con 0x562cc6fc2cd0 2024-09-12T02:07:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.889+0000 7fd326502640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6fc2cd0 0x562cc6fc50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.890+0000 7fd326502640 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6fc2cd0 0x562cc6fc50f0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d621850b0ee73 server_cookie=8505204f93065978 in_seq=0 out_seq=0 2024-09-12T02:07:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:20.890+0000 7fd309ffb640 10 client.18342 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.060+0000 7fd309ffb640 1 -- 172.21.15.122:0/3117141463 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cc6eacd70 con 0x562cc6fc2cd0 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.060+0000 7fd309ffb640 10 client.18342 handle_command_reply: tid=0 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.060+0000 7fd32b5de740 1 client.18342 shutdown 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.060+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 msgr2=0x562cc6f51f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.060+0000 7fd32b5de740 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f51f80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd31000a010 tx=0x7fd310005f90 comp rx=0 tx=0).stop 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 shutdown_connections 2024-09-12T02:07:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x562cc6fc2cd0 0x562cc6fc50f0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cc6f47370 0x562cc6f51f80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd309ffb640 0 client.18342 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cc6f422e0 0x562cc6f50f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 --2- 172.21.15.122:0/3117141463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc6f419a0 0x562cc6f50a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 >> 172.21.15.122:0/3117141463 conn(0x562cc6f3d010 msgr2=0x562cc6f3b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 shutdown_connections 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 1 -- 172.21.15.122:0/3117141463 wait complete. 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd32b5de740 20 client.18342 trim_cache size 0 max 16384 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:07:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:07:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:07:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:07:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd30c0a2c30 msgr2=0x7fd30c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd328766640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd30c0a2c30 0x7fd30c0a50f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3180bc770 tx=0x7fd318033040 comp rx=0 tx=0).stop 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 msgr2=0x7fd32019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.061+0000 7fd328766640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd32019f400 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd318006010 tx=0x7fd318004ab0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 shutdown_connections 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd30c0a2c30 0x7fd30c0a50f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320107820 0x7fd32019f400 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd320104ef0 0x7fd320198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 --2- 172.21.15.122:0/1854023434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3201025c0 0x7fd320197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 >> 172.21.15.122:0/1854023434 conn(0x7fd3200fc1e0 msgr2=0x7fd320102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 shutdown_connections 2024-09-12T02:07:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:21.062+0000 7fd328766640 1 -- 172.21.15.122:0/1854023434 wait complete. 2024-09-12T02:07:21.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 21.13 MB/sec execute 308 sec latency 3080.919 ms 2024-09-12T02:07:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:20 smithi151 ceph-mon[35229]: pgmap v966: 97 pgs: 97 active+clean; 526 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 8.3 MiB/s wr, 611 op/s 2024-09-12T02:07:21.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:20 smithi183 ceph-mon[35127]: pgmap v966: 97 pgs: 97 active+clean; 526 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 8.3 MiB/s wr, 611 op/s 2024-09-12T02:07:21.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:20 smithi122 ceph-mon[30925]: pgmap v966: 97 pgs: 97 active+clean; 526 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 8.3 MiB/s wr, 611 op/s 2024-09-12T02:07:22.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 21.07 MB/sec execute 309 sec latency 4081.037 ms 2024-09-12T02:07:23.075 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 21.00 MB/sec execute 310 sec latency 5081.136 ms 2024-09-12T02:07:23.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:22 smithi151 ceph-mon[35229]: pgmap v967: 97 pgs: 97 active+clean; 528 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 556 op/s 2024-09-12T02:07:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:22 smithi183 ceph-mon[35127]: pgmap v967: 97 pgs: 97 active+clean; 528 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 556 op/s 2024-09-12T02:07:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:22 smithi122 ceph-mon[30925]: pgmap v967: 97 pgs: 97 active+clean; 528 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 556 op/s 2024-09-12T02:07:24.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.93 MB/sec execute 311 sec latency 6081.232 ms 2024-09-12T02:07:25.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.86 MB/sec execute 312 sec latency 7081.329 ms 2024-09-12T02:07:25.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:24 smithi151 ceph-mon[35229]: pgmap v968: 97 pgs: 97 active+clean; 530 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 538 op/s 2024-09-12T02:07:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:24 smithi183 ceph-mon[35127]: pgmap v968: 97 pgs: 97 active+clean; 530 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 538 op/s 2024-09-12T02:07:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:24 smithi122 ceph-mon[30925]: pgmap v968: 97 pgs: 97 active+clean; 530 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 538 op/s 2024-09-12T02:07:26.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.80 MB/sec execute 313 sec latency 8081.430 ms 2024-09-12T02:07:27.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.73 MB/sec execute 314 sec latency 9081.547 ms 2024-09-12T02:07:27.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:26 smithi151 ceph-mon[35229]: pgmap v969: 97 pgs: 97 active+clean; 525 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 459 op/s 2024-09-12T02:07:27.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:26 smithi183 ceph-mon[35127]: pgmap v969: 97 pgs: 97 active+clean; 525 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 459 op/s 2024-09-12T02:07:27.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:26 smithi122 ceph-mon[30925]: pgmap v969: 97 pgs: 97 active+clean; 525 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.3 MiB/s wr, 459 op/s 2024-09-12T02:07:28.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.66 MB/sec execute 315 sec latency 10081.636 ms 2024-09-12T02:07:29.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.60 MB/sec execute 316 sec latency 11081.731 ms 2024-09-12T02:07:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:28 smithi151 ceph-mon[35229]: pgmap v970: 97 pgs: 97 active+clean; 516 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 372 op/s 2024-09-12T02:07:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:28 smithi183 ceph-mon[35127]: pgmap v970: 97 pgs: 97 active+clean; 516 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 372 op/s 2024-09-12T02:07:29.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:28 smithi122 ceph-mon[30925]: pgmap v970: 97 pgs: 97 active+clean; 516 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 372 op/s 2024-09-12T02:07:30.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.53 MB/sec execute 317 sec latency 12081.784 ms 2024-09-12T02:07:31.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.47 MB/sec execute 318 sec latency 13081.828 ms 2024-09-12T02:07:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:30 smithi151 ceph-mon[35229]: pgmap v971: 97 pgs: 97 active+clean; 502 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2024-09-12T02:07:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:30 smithi183 ceph-mon[35127]: pgmap v971: 97 pgs: 97 active+clean; 502 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2024-09-12T02:07:31.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:30 smithi122 ceph-mon[30925]: pgmap v971: 97 pgs: 97 active+clean; 502 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2024-09-12T02:07:32.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.40 MB/sec execute 319 sec latency 14081.882 ms 2024-09-12T02:07:33.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.34 MB/sec execute 320 sec latency 15081.928 ms 2024-09-12T02:07:33.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:32 smithi151 ceph-mon[35229]: pgmap v972: 97 pgs: 97 active+clean; 502 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-09-12T02:07:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:32 smithi183 ceph-mon[35127]: pgmap v972: 97 pgs: 97 active+clean; 502 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-09-12T02:07:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:32 smithi122 ceph-mon[30925]: pgmap v972: 97 pgs: 97 active+clean; 502 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 4.8 MiB/s wr, 246 op/s 2024-09-12T02:07:34.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.28 MB/sec execute 321 sec latency 16081.961 ms 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 -- 172.21.15.122:0/2294530798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041003b0 msgr2=0x7ff5041027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 --2- 172.21.15.122:0/2294530798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041003b0 0x7ff5041027c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff50002a410 comp rx=0 tx=0).stop 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 -- 172.21.15.122:0/2294530798 shutdown_connections 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 --2- 172.21.15.122:0/2294530798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5041056b0 0x7ff504105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 --2- 172.21.15.122:0/2294530798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff504102d00 0x7ff504105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 --2- 172.21.15.122:0/2294530798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041003b0 0x7ff5041027c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 -- 172.21.15.122:0/2294530798 >> 172.21.15.122:0/2294530798 conn(0x7ff5040fc1e0 msgr2=0x7ff5040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 -- 172.21.15.122:0/2294530798 shutdown_connections 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.150+0000 7ff50b31f640 1 -- 172.21.15.122:0/2294530798 wait complete. 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 Processor -- start 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 -- start start 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5041003b0 0x7ff50419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504102d00 0x7ff50419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 0x7ff5041a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff504113dc0 con 0x7ff5041056b0 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff504113c40 con 0x7ff504102d00 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff50b31f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff504113f40 con 0x7ff5041003b0 2024-09-12T02:07:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 0x7ff5041a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 0x7ff5041a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35428/0 (socket says 172.21.15.122:35428) 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 -- 172.21.15.122:0/3478879386 learned_addr learned my addr 172.21.15.122:0/3478879386 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5090bb640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5041003b0 0x7ff50419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5088ba640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504102d00 0x7ff50419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 -- 172.21.15.122:0/3478879386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5041003b0 msgr2=0x7ff50419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5041003b0 0x7ff50419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 -- 172.21.15.122:0/3478879386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504102d00 msgr2=0x7ff50419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504102d00 0x7ff50419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5098bc640 1 -- 172.21.15.122:0/3478879386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5041a5f20 con 0x7ff5041056b0 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5090bb640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5041003b0 0x7ff50419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.151+0000 7ff5088ba640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504102d00 0x7ff50419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.152+0000 7ff5098bc640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 0x7ff5041a58b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00eaa0 tx=0x7ff4fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.152+0000 7ff4fa7fc640 1 -- 172.21.15.122:0/3478879386 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00cdb0 con 0x7ff5041056b0 2024-09-12T02:07:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.152+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5041a6210 con 0x7ff5041056b0 2024-09-12T02:07:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.152+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5041a6760 con 0x7ff5041056b0 2024-09-12T02:07:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.152+0000 7ff4fa7fc640 1 -- 172.21.15.122:0/3478879386 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4fc005020 con 0x7ff5041056b0 2024-09-12T02:07:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.152+0000 7ff4fa7fc640 1 -- 172.21.15.122:0/3478879386 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc01f060 con 0x7ff5041056b0 2024-09-12T02:07:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.154+0000 7ff50e0b4740 1 Processor -- start 2024-09-12T02:07:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.154+0000 7ff4fa7fc640 1 -- 172.21.15.122:0/3478879386 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff4fc004a40 con 0x7ff5041056b0 2024-09-12T02:07:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.154+0000 7ff50e0b4740 1 -- start start 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.154+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e1eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e23da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e26830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b90d822f0 con 0x555b90e1f2e0 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b90d2a270 con 0x555b90e24370 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555b90d7e2b0 con 0x555b90e1e9a0 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e23da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e23da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35436/0 (socket says 172.21.15.122:35436) 2024-09-12T02:07:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 -- 172.21.15.122:0/2917232066 learned_addr learned my addr 172.21.15.122:0/2917232066 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5088ba640 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e1eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5088ba640 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e26830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 -- 172.21.15.122:0/2917232066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 msgr2=0x555b90e1eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff4fa7fc640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4f00a2c30 0x7ff4f00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e1eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 -- 172.21.15.122:0/2917232066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 msgr2=0x555b90e26830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e26830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 -- 172.21.15.122:0/2917232066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b90d29340 con 0x555b90e1f2e0 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5098bc640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4f00a2c30 0x7ff4f00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff4fa7fc640 1 -- 172.21.15.122:0/3478879386 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff4fc0cf2d0 con 0x7ff5041056b0 2024-09-12T02:07:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.155+0000 7ff5090bb640 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e23da0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff500006010 tx=0x7ff500032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fecbf756068dadd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff4ef7fe640 1 -- 172.21.15.122:0/2917232066 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50003f070 con 0x555b90e1f2e0 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff5098bc640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4f00a2c30 0x7ff4f00a50f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0ccdd0 tx=0x7ff4fc04b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff4ef7fe640 1 -- 172.21.15.122:0/2917232066 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5000318b0 con 0x555b90e1f2e0 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff4ef7fe640 1 -- 172.21.15.122:0/2917232066 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500035070 con 0x555b90e1f2e0 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 -- 172.21.15.122:0/2917232066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 msgr2=0x555b90e23da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e23da0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff500006010 tx=0x7ff500032d60 comp rx=0 tx=0).stop 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 -- 172.21.15.122:0/2917232066 shutdown_connections 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e26830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e23da0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/2917232066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e1eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 -- 172.21.15.122:0/2917232066 >> 172.21.15.122:0/2917232066 conn(0x555b90e1a010 msgr2=0x555b90e1a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 -- 172.21.15.122:0/2917232066 shutdown_connections 2024-09-12T02:07:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.156+0000 7ff50e0b4740 1 -- 172.21.15.122:0/2917232066 wait complete. 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 -- start start 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 Processor -- start 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e30080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b90e272e0 con 0x555b90e1f2e0 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b90d0b290 con 0x555b90e24370 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555b90e27460 con 0x555b90e1e9a0 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff50e0b4740 10 client.0 _ll_register_callbacks cb 0x555b90d3c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e29040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35442/0 (socket says 172.21.15.122:35442) 2024-09-12T02:07:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 -- 172.21.15.122:0/1366662448 learned_addr learned my addr 172.21.15.122:0/1366662448 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e30080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 -- 172.21.15.122:0/1366662448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 msgr2=0x555b90e28b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e28b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 -- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 msgr2=0x555b90e30080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e30080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.157+0000 7ff5090bb640 1 -- 172.21.15.122:0/1366662448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b90a95210 con 0x555b90e1f2e0 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e28b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff5090bb640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e29040 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff5000314e0 tx=0x7ff500031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c18fba2a80b7e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff4ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1366662448 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50003f070 con 0x555b90e1f2e0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1366662448 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff5000035f0 con 0x555b90e1f2e0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5000318b0 con 0x555b90e1f2e0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1366662448 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff500035040 con 0x555b90e1f2e0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 10 client.18354 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555b90b61620 con 0x555b90e1f2e0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1366662448 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5000318b0 con 0x555b90e1f2e0 2024-09-12T02:07:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1366662448 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff50003b8b0 con 0x555b90e1f2e0 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 10 client.18354 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 10 client.18354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 20 client.18354 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e343d0 0x555b90e367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 4 client.18354 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.158+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b90d8fa20 con 0x555b90e343d0 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.159+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e343d0 0x555b90e367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.159+0000 7ff4ecff9640 10 client.18354 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.159+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e343d0 0x555b90e367c0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40218b7f34159817 server_cookie=d2e7e3882079c623 in_seq=0 out_seq=0 2024-09-12T02:07:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1366662448 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b90d8fa20 con 0x555b90e343d0 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff4ecff9640 10 client.18354 handle_command_reply: tid=0 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 client.18354 shutdown 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 msgr2=0x555b90e29040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e29040 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff5000314e0 tx=0x7ff500031730 comp rx=0 tx=0).stop 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 shutdown_connections 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e343d0 0x555b90e367c0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e30080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff4ecff9640 0 client.18354 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1f2e0 0x555b90e29040 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1366662448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1e9a0 0x555b90e28b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 >> 172.21.15.122:0/1366662448 conn(0x555b90e1a010 msgr2=0x555b90e1b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 shutdown_connections 2024-09-12T02:07:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1366662448 wait complete. 2024-09-12T02:07:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.160+0000 7ff50e0b4740 20 client.18354 trim_cache size 0 max 16384 2024-09-12T02:07:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 Processor -- start 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 -- start start 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e1b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8010d90 con 0x555b90e1e9a0 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800e660 con 0x555b90e24370 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800c490 con 0x555b90e1f2e0 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff5088ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff5088ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e190b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35452/0 (socket says 172.21.15.122:35452) 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.174+0000 7ff5088ba640 1 -- 172.21.15.122:0/357645604 learned_addr learned my addr 172.21.15.122:0/357645604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e1b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 -- 172.21.15.122:0/357645604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 msgr2=0x555b90e30040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 -- 172.21.15.122:0/357645604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 msgr2=0x555b90e1b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e1b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 -- 172.21.15.122:0/357645604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b90e36df0 con 0x555b90e1e9a0 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff5088ba640 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e190b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4005fd0 tx=0x7ff4f4005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a119c778c2aca9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff4eeffd640 1 -- 172.21.15.122:0/357645604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f401d040 con 0x555b90e1e9a0 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff4eeffd640 1 -- 172.21.15.122:0/357645604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4005500 con 0x555b90e1e9a0 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.175+0000 7ff4eeffd640 1 -- 172.21.15.122:0/357645604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004d00 con 0x555b90e1e9a0 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- 172.21.15.122:0/357645604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 msgr2=0x555b90e190b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e190b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4005fd0 tx=0x7ff4f4005c30 comp rx=0 tx=0).stop 2024-09-12T02:07:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- 172.21.15.122:0/357645604 shutdown_connections 2024-09-12T02:07:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e1b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e30040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/357645604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e190b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- 172.21.15.122:0/357645604 >> 172.21.15.122:0/357645604 conn(0x555b90e1a010 msgr2=0x555b90e189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- 172.21.15.122:0/357645604 shutdown_connections 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- 172.21.15.122:0/357645604 wait complete. 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- start start 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 Processor -- start 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e2d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e2ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800cb20 con 0x555b90e1e9a0 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800df90 con 0x555b90e24370 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800d1f0 con 0x555b90e1f2e0 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff50e0b4740 10 client.0 _ll_register_callbacks cb 0x7ff4f400d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff5088ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e2d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff5088ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e2d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35456/0 (socket says 172.21.15.122:35456) 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.176+0000 7ff5088ba640 1 -- 172.21.15.122:0/1614586605 learned_addr learned my addr 172.21.15.122:0/1614586605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5090bb640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e2ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 -- 172.21.15.122:0/1614586605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 msgr2=0x555b90e2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e2dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 -- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 msgr2=0x555b90e2ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e2ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 -- 172.21.15.122:0/1614586605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b90e2f2e0 con 0x555b90e1e9a0 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff5088ba640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e2d780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4005fa0 tx=0x7ff4f40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da1187c80192988 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff4ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1614586605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4012ae0 con 0x555b90e1e9a0 2024-09-12T02:07:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f4005500 con 0x555b90e1e9a0 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1614586605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff4f4012c80 con 0x555b90e1e9a0 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1614586605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff4f403a050 con 0x555b90e1e9a0 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff50e0b4740 10 client.18360 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555b90e9f0b0 con 0x555b90e1e9a0 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.177+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1614586605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4005500 con 0x555b90e1e9a0 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1614586605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ff4f402a740 con 0x555b90e1e9a0 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff50e0b4740 10 client.18360 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff50e0b4740 10 client.18360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff50e0b4740 20 client.18360 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e9f9d0 0x555b90ea1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff50e0b4740 4 client.18360 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b90d89d70 con 0x555b90e9f9d0 2024-09-12T02:07:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff5090bb640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e9f9d0 0x555b90ea1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff4ecff9640 10 client.18360 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.178+0000 7ff5090bb640 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e9f9d0 0x555b90ea1df0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ad84bc0f16f76b server_cookie=cfc5749a19c1e820 in_seq=0 out_seq=0 2024-09-12T02:07:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff4ecff9640 1 -- 172.21.15.122:0/1614586605 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b90d89d70 con 0x555b90e9f9d0 2024-09-12T02:07:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff4ecff9640 10 client.18360 handle_command_reply: tid=0 2024-09-12T02:07:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 client.18360 shutdown 2024-09-12T02:07:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 msgr2=0x555b90e2d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e2d780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4005fa0 tx=0x7ff4f40047a0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 shutdown_connections 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x555b90e9f9d0 0x555b90ea1df0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b90e24370 0x555b90e2ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff4ecff9640 0 client.18360 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555b90e1f2e0 0x555b90e2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 --2- 172.21.15.122:0/1614586605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b90e1e9a0 0x555b90e2d780 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 >> 172.21.15.122:0/1614586605 conn(0x555b90e1a010 msgr2=0x555b90e191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 shutdown_connections 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 1 -- 172.21.15.122:0/1614586605 wait complete. 2024-09-12T02:07:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.242+0000 7ff50e0b4740 20 client.18360 trim_cache size 0 max 16384 2024-09-12T02:07:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:07:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:07:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:07:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4f00a2c30 msgr2=0x7ff4f00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4f00a2c30 0x7ff4f00a50f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0ccdd0 tx=0x7ff4fc04b040 comp rx=0 tx=0).stop 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 msgr2=0x7ff5041a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 0x7ff5041a58b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00eaa0 tx=0x7ff4fc00ef60 comp rx=0 tx=0).stop 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 shutdown_connections 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff4f00a2c30 0x7ff4f00a50f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5041056b0 0x7ff5041a58b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504102d00 0x7ff50419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 --2- 172.21.15.122:0/3478879386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5041003b0 0x7ff50419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 >> 172.21.15.122:0/3478879386 conn(0x7ff5040fc1e0 msgr2=0x7ff504103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 shutdown_connections 2024-09-12T02:07:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:34.243+0000 7ff50b31f640 1 -- 172.21.15.122:0/3478879386 wait complete. 2024-09-12T02:07:34.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:07:34.689 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:35.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.21 MB/sec execute 322 sec latency 17082.029 ms 2024-09-12T02:07:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:34 smithi151 ceph-mon[35229]: pgmap v973: 97 pgs: 97 active+clean; 501 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 4.8 MiB/s wr, 226 op/s 2024-09-12T02:07:35.123 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:34 smithi122 ceph-mon[30925]: pgmap v973: 97 pgs: 97 active+clean; 501 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 4.8 MiB/s wr, 226 op/s 2024-09-12T02:07:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:34 smithi183 ceph-mon[35127]: pgmap v973: 97 pgs: 97 active+clean; 501 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 4.8 MiB/s wr, 226 op/s 2024-09-12T02:07:35.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 -- 172.21.15.122:0/861644574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c102790 msgr2=0x7f998c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 --2- 172.21.15.122:0/861644574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c102790 0x7f998c1067e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402a410 comp rx=0 tx=0).stop 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 -- 172.21.15.122:0/861644574 shutdown_connections 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 --2- 172.21.15.122:0/861644574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c106f10 0x7f998c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 --2- 172.21.15.122:0/861644574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c102790 0x7f998c1067e0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 --2- 172.21.15.122:0/861644574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c101de0 0x7f998c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 -- 172.21.15.122:0/861644574 >> 172.21.15.122:0/861644574 conn(0x7f998c0fd8c0 msgr2=0x7f998c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 -- 172.21.15.122:0/861644574 shutdown_connections 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.202+0000 7f9990eb0640 1 -- 172.21.15.122:0/861644574 wait complete. 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 Processor -- start 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 -- start start 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c101de0 0x7f998c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:35.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c102790 0x7f998c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 0x7f998c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f998c111f70 con 0x7f998c106f10 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f998c111df0 con 0x7f998c102790 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9990eb0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1120f0 con 0x7f998c101de0 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 0x7f998c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 0x7f998c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35480/0 (socket says 172.21.15.122:35480) 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 -- 172.21.15.122:0/427922260 learned_addr learned my addr 172.21.15.122:0/427922260 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998a59c640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c101de0 0x7f998c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9983fff640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c102790 0x7f998c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 -- 172.21.15.122:0/427922260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c101de0 msgr2=0x7f998c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c101de0 0x7f998c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 -- 172.21.15.122:0/427922260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c102790 msgr2=0x7f998c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c102790 0x7f998c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f998ad9d640 1 -- 172.21.15.122:0/427922260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c1a6900 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.203+0000 7f9983fff640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c102790 0x7f998c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.204+0000 7f998ad9d640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 0x7f998c074f70 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f998400eaa0 tx=0x7f998400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.204+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998400cdb0 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.204+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c1a6bf0 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.204+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9984005020 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.204+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840116d0 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.204+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1a7180 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.205+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c1021c0 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.206+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f99840054d0 con 0x7f998c106f10 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.207+0000 7f99837fe640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f99780a2b80 0x7f99780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.207+0000 7f998a59c640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f99780a2b80 0x7f99780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:35.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.207+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f99840cf550 con 0x7f998c106f10 2024-09-12T02:07:35.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.207+0000 7f998a59c640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f99780a2b80 0x7f99780a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f997c005d00 tx=0x7f997c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:35.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.210+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9984016030 con 0x7f998c106f10 2024-09-12T02:07:35.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.457+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f998c02d7c0 con 0x7f998c106f10 2024-09-12T02:07:35.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.458+0000 7f99837fe640 1 -- 172.21.15.122:0/427922260 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7f998409c4b0 con 0x7f998c106f10 2024-09-12T02:07:35.459 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:07:35.459 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:07:35.459 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:07:35.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.460+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f99780a2b80 msgr2=0x7f99780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:35.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.460+0000 7f9990eb0640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f99780a2b80 0x7f99780a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f997c005d00 tx=0x7f997c00b410 comp rx=0 tx=0).stop 2024-09-12T02:07:35.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.460+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 msgr2=0x7f998c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:35.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.460+0000 7f9990eb0640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 0x7f998c074f70 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f998400eaa0 tx=0x7f998400ef60 comp rx=0 tx=0).stop 2024-09-12T02:07:35.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.460+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 shutdown_connections 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f99780a2b80 0x7f99780a5040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f998c106f10 0x7f998c074f70 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f998c102790 0x7f998c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 --2- 172.21.15.122:0/427922260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f998c101de0 0x7f998c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 >> 172.21.15.122:0/427922260 conn(0x7f998c0fd8c0 msgr2=0x7f998c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 shutdown_connections 2024-09-12T02:07:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:35.461+0000 7f9990eb0640 1 -- 172.21.15.122:0/427922260 wait complete. 2024-09-12T02:07:36.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.15 MB/sec execute 323 sec latency 18082.118 ms 2024-09-12T02:07:36.541 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:35 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/427922260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:36.542 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:35 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/427922260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:36.542 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:35 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/427922260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:07:36.543 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 4b6e1ae5 --release --await' 2024-09-12T02:07:36.543 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 4b6e1ae5 --release --await 2024-09-12T02:07:36.862 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:07:37.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173637 20.09 MB/sec execute 324 sec latency 19082.197 ms 2024-09-12T02:07:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:36 smithi151 ceph-mon[35229]: pgmap v974: 97 pgs: 97 active+clean; 500 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.0 MiB/s wr, 157 op/s 2024-09-12T02:07:37.119 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:36 smithi122 ceph-mon[30925]: pgmap v974: 97 pgs: 97 active+clean; 500 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.0 MiB/s wr, 157 op/s 2024-09-12T02:07:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:36 smithi183 ceph-mon[35127]: pgmap v974: 97 pgs: 97 active+clean; 500 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.0 MiB/s wr, 157 op/s 2024-09-12T02:07:37.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 -- 172.21.15.122:0/2498713099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c106f70 msgr2=0x7f5c7c10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2498713099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c106f70 0x7f5c7c10eab0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009ca0 tx=0x7f5c7402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 -- 172.21.15.122:0/2498713099 shutdown_connections 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2498713099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c106f70 0x7f5c7c10eab0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2498713099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2498713099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c100500 0x7f5c7c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.361+0000 7f5c83aac640 1 -- 172.21.15.122:0/2498713099 >> 172.21.15.122:0/2498713099 conn(0x7f5c7c0fc5c0 msgr2=0x7f5c7c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 -- 172.21.15.122:0/2498713099 shutdown_connections 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 -- 172.21.15.122:0/2498713099 wait complete. 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 Processor -- start 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 -- start start 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c100500 0x7f5c7c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c106f70 0x7f5c7c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c111f90 con 0x7f5c7c100eb0 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c111e10 con 0x7f5c7c100500 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.362+0000 7f5c83aac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c112110 con 0x7f5c7c106f70 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35494/0 (socket says 172.21.15.122:35494) 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 -- 172.21.15.122:0/2190112096 learned_addr learned my addr 172.21.15.122:0/2190112096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c82049640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c106f70 0x7f5c7c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81848640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c100500 0x7f5c7c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 -- 172.21.15.122:0/2190112096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c106f70 msgr2=0x7f5c7c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c106f70 0x7f5c7c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 -- 172.21.15.122:0/2190112096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c100500 msgr2=0x7f5c7c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c100500 0x7f5c7c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 -- 172.21.15.122:0/2190112096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c19d3c0 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c81047640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c198950 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c970 tx=0x7f5c6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c72ffd640 1 -- 172.21.15.122:0/2190112096 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004220 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c19d6b0 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c72ffd640 1 -- 172.21.15.122:0/2190112096 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c0043c0 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c72ffd640 1 -- 172.21.15.122:0/2190112096 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004a00 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.363+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1a4f90 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.365+0000 7f5c72ffd640 1 -- 172.21.15.122:0/2190112096 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5c6c0079b0 con 0x7f5c7c100eb0 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 Processor -- start 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 -- start start 2024-09-12T02:07:37.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa509340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa518110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa51aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5643aa46e620 con 0x5643aa508f60 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643aa415e50 con 0x5643aa509880 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c85e85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643aa46a5e0 con 0x5643aa518650 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c72ffd640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5c680a2b80 0x7f5c680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c81848640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa518110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c82049640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa509340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c81848640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa518110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37868/0 (socket says 172.21.15.122:37868) 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c81848640 1 -- 172.21.15.122:0/2698540539 learned_addr learned my addr 172.21.15.122:0/2698540539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:37.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c82049640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5c680a2b80 0x7f5c680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.366+0000 7f5c81047640 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa51aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c72ffd640 1 -- 172.21.15.122:0/2190112096 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5c6c0ceca0 con 0x7f5c7c100eb0 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c81848640 1 -- 172.21.15.122:0/2698540539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 msgr2=0x5643aa51aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c81848640 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa51aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c81848640 1 -- 172.21.15.122:0/2698540539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 msgr2=0x5643aa509340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c81848640 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa509340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c81848640 1 -- 172.21.15.122:0/2698540539 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643aa406d40 con 0x5643aa509880 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c82049640 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa509340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c81848640 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa518110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800bc90 tx=0x7f5c7800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdcdd55e31b2df4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c82049640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5c680a2b80 0x7f5c680a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5c74004af0 tx=0x7f5c74004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c67fff640 1 -- 172.21.15.122:0/2698540539 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78015070 con 0x5643aa509880 2024-09-12T02:07:37.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c67fff640 1 -- 172.21.15.122:0/2698540539 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c780035f0 con 0x5643aa509880 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.367+0000 7f5c67fff640 1 -- 172.21.15.122:0/2698540539 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780116a0 con 0x5643aa509880 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 -- 172.21.15.122:0/2698540539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 msgr2=0x5643aa518110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa518110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800bc90 tx=0x7f5c7800f730 comp rx=0 tx=0).stop 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 -- 172.21.15.122:0/2698540539 shutdown_connections 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa51aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa518110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2698540539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa509340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 -- 172.21.15.122:0/2698540539 >> 172.21.15.122:0/2698540539 conn(0x5643aa5055f0 msgr2=0x5643aa5059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 -- 172.21.15.122:0/2698540539 shutdown_connections 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 -- 172.21.15.122:0/2698540539 wait complete. 2024-09-12T02:07:37.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 -- start start 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.368+0000 7f5c85e85740 1 Processor -- start 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa515660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa509880 0x5643aa515ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 0x5643aa5160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5643aa51b280 con 0x5643aa508f60 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643aa3f6f10 con 0x5643aa518650 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643aa51b400 con 0x5643aa509880 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c85e85740 10 client.0 _ll_register_callbacks cb 0x5643aa41adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 0x5643aa5160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 0x5643aa5160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37874/0 (socket says 172.21.15.122:37874) 2024-09-12T02:07:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c81848640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa515660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 -- 172.21.15.122:0/4018801785 learned_addr learned my addr 172.21.15.122:0/4018801785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c81047640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa509880 0x5643aa515ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 -- 172.21.15.122:0/4018801785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa509880 msgr2=0x5643aa515ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa509880 0x5643aa515ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 -- 172.21.15.122:0/4018801785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 msgr2=0x5643aa515660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa515660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c82049640 1 -- 172.21.15.122:0/4018801785 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643aa17d580 con 0x5643aa518650 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.369+0000 7f5c81848640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa515660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c82049640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 0x5643aa5160e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c74035000 tx=0x7f5c740095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19553ae5efc7d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c657fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c657fa640 1 -- 172.21.15.122:0/4018801785 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7404f070 con 0x5643aa518650 2024-09-12T02:07:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c657fa640 1 -- 172.21.15.122:0/4018801785 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5c74003b70 con 0x5643aa518650 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c780035f0 con 0x5643aa518650 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c657fa640 1 -- 172.21.15.122:0/4018801785 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5c74003d70 con 0x5643aa518650 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c85e85740 10 client.27371 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5643aa247960 con 0x5643aa518650 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c657fa640 1 -- 172.21.15.122:0/4018801785 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c780035f0 con 0x5643aa518650 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.370+0000 7f5c657fa640 1 -- 172.21.15.122:0/4018801785 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f5c74048040 con 0x5643aa518650 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c85e85740 10 client.27371 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c85e85740 10 client.27371 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c85e85740 20 client.27371 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c85e85740 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5282b0 0x5643aa52a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c85e85740 4 client.27371 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643aa47bb90 con 0x5643aa5282b0 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c81848640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5282b0 0x5643aa52a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c81848640 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5282b0 0x5643aa52a730 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57aaea2bcd6ab76 server_cookie=a361eb18d59e335c in_seq=0 out_seq=0 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.371+0000 7f5c657fa640 10 client.27371 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c657fa640 1 -- 172.21.15.122:0/4018801785 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643aa47bb90 con 0x5643aa5282b0 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c657fa640 10 client.27371 handle_command_reply: tid=0 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 client.27371 shutdown 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 msgr2=0x5643aa5160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 0x5643aa5160e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c74035000 tx=0x7f5c740095c0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 shutdown_connections 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5282b0 0x5643aa52a730 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa518650 0x5643aa5160e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa509880 0x5643aa515ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 --2- 172.21.15.122:0/4018801785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa508f60 0x5643aa515660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c657fa640 0 client.27371 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 >> 172.21.15.122:0/4018801785 conn(0x5643aa5055f0 msgr2=0x5643aa50b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 shutdown_connections 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 1 -- 172.21.15.122:0/4018801785 wait complete. 2024-09-12T02:07:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.372+0000 7f5c85e85740 20 client.27371 trim_cache size 0 max 16384 2024-09-12T02:07:37.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 Processor -- start 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 -- start start 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa508f60 0x5643aa530040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 0x5643aa530580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa5128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1adc70 con 0x5643aa509880 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1ab540 con 0x5643aa508f60 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c85e85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1a9370 con 0x5643aa518650 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 0x5643aa530580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 0x5643aa530580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35520/0 (socket says 172.21.15.122:35520) 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 -- 172.21.15.122:0/3228128560 learned_addr learned my addr 172.21.15.122:0/3228128560 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c82049640 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa5128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81848640 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa508f60 0x5643aa530040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 -- 172.21.15.122:0/3228128560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 msgr2=0x5643aa5128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa5128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 -- 172.21.15.122:0/3228128560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa508f60 msgr2=0x5643aa530040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa508f60 0x5643aa530040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.393+0000 7f5c81047640 1 -- 172.21.15.122:0/3228128560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643aa517fd0 con 0x5643aa509880 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c81047640 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 0x5643aa530580 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c007590 tx=0x7f5c6c01cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aaab5d2f066652 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c677fe640 1 -- 172.21.15.122:0/3228128560 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c09bc00 con 0x5643aa509880 2024-09-12T02:07:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c677fe640 1 -- 172.21.15.122:0/3228128560 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c0b3070 con 0x5643aa509880 2024-09-12T02:07:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c677fe640 1 -- 172.21.15.122:0/3228128560 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c097b70 con 0x5643aa509880 2024-09-12T02:07:37.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 -- 172.21.15.122:0/3228128560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 msgr2=0x5643aa530580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 0x5643aa530580 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c007590 tx=0x7f5c6c01cb40 comp rx=0 tx=0).stop 2024-09-12T02:07:37.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 -- 172.21.15.122:0/3228128560 shutdown_connections 2024-09-12T02:07:37.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa518650 0x5643aa5128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa509880 0x5643aa530580 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 --2- 172.21.15.122:0/3228128560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa508f60 0x5643aa530040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 -- 172.21.15.122:0/3228128560 >> 172.21.15.122:0/3228128560 conn(0x5643aa5055f0 msgr2=0x5643aa5176e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 -- 172.21.15.122:0/3228128560 shutdown_connections 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.394+0000 7f5c85e85740 1 -- 172.21.15.122:0/3228128560 wait complete. 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 -- start start 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 Processor -- start 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa508f60 0x5643aa520b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa51be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa518650 0x5643aa51c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1a9a00 con 0x5643aa518650 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1aae70 con 0x5643aa509880 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1aa0d0 con 0x5643aa508f60 2024-09-12T02:07:37.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c85e85740 10 client.0 _ll_register_callbacks cb 0x7f5c74031a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c81047640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa51be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c81848640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa508f60 0x5643aa520b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c82049640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa518650 0x5643aa51c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c81047640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa51be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37898/0 (socket says 172.21.15.122:37898) 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c81848640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa508f60 0x5643aa520b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:56142/0 (socket says 172.21.15.122:56142) 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.395+0000 7f5c81047640 1 -- 172.21.15.122:0/2907972275 learned_addr learned my addr 172.21.15.122:0/2907972275 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c82049640 1 -- 172.21.15.122:0/2907972275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa508f60 msgr2=0x5643aa520b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c82049640 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa508f60 0x5643aa520b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c82049640 1 -- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 msgr2=0x5643aa51be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c82049640 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa51be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c82049640 1 -- 172.21.15.122:0/2907972275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643aa51c990 con 0x5643aa518650 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c81047640 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa51be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:37.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c82049640 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa518650 0x5643aa51c3d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5c74031620 tx=0x7f5c74005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6328c6740758677d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c657fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c657fa640 1 -- 172.21.15.122:0/2907972275 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7404f070 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c657fa640 1 -- 172.21.15.122:0/2907972275 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5c74003f10 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c6c0b3070 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c657fa640 1 -- 172.21.15.122:0/2907972275 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5c74004110 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c85e85740 10 client.18384 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5643aa592dc0 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c657fa640 1 -- 172.21.15.122:0/2907972275 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c0b3070 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.396+0000 7f5c657fa640 1 -- 172.21.15.122:0/2907972275 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f5c74048070 con 0x5643aa518650 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c85e85740 10 client.18384 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c85e85740 10 client.18384 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:07:37.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c85e85740 20 client.18384 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:37.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5934e0 0x5643aa595900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:37.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c85e85740 4 client.18384 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4b6e1ae5", "release": true, "await": true}] 2024-09-12T02:07:37.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4b6e1ae5", "release": true, "await": true}) -- 0x5643aa4760a0 con 0x5643aa5934e0 2024-09-12T02:07:37.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c81848640 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5934e0 0x5643aa595900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:37.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c657fa640 10 client.18384 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:37.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.397+0000 7f5c81848640 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5934e0 0x5643aa595900 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f895f0d09923f8 server_cookie=84cad4192d2c6dfb in_seq=0 out_seq=0 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c657fa640 1 -- 172.21.15.122:0/2907972275 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5643aa4760a0 con 0x5643aa5934e0 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c657fa640 10 client.18384 handle_command_reply: tid=0 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 client.18384 shutdown 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa518650 msgr2=0x5643aa51c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa518650 0x5643aa51c3d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5c74031620 tx=0x7f5c74005cf0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 shutdown_connections 2024-09-12T02:07:37.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643aa5934e0 0x5643aa595900 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643aa518650 0x5643aa51c3d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643aa509880 0x5643aa51be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 --2- 172.21.15.122:0/2907972275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643aa508f60 0x5643aa520b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c657fa640 0 client.18384 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 >> 172.21.15.122:0/2907972275 conn(0x5643aa5055f0 msgr2=0x5643aa50aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 shutdown_connections 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 1 -- 172.21.15.122:0/2907972275 wait complete. 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.456+0000 7f5c85e85740 20 client.18384 trim_cache size 0 max 16384 2024-09-12T02:07:37.458 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 76, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "4b6e1ae5": { 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 76, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:07:37.459 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 139.9, 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:07:37.460 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5c680a2b80 msgr2=0x7f5c680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5c680a2b80 0x7f5c680a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5c74004af0 tx=0x7f5c74004a80 comp rx=0 tx=0).stop 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 msgr2=0x7f5c7c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c198950 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c970 tx=0x7f5c6c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 shutdown_connections 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5c680a2b80 0x7f5c680a5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c106f70 0x7f5c7c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c7c100eb0 0x7f5c7c198950 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 --2- 172.21.15.122:0/2190112096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c100500 0x7f5c7c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:37.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 >> 172.21.15.122:0/2190112096 conn(0x7f5c7c0fc5c0 msgr2=0x7f5c7c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:37.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 shutdown_connections 2024-09-12T02:07:37.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:07:37.457+0000 7f5c83aac640 1 -- 172.21.15.122:0/2190112096 wait complete. 2024-09-12T02:07:38.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173916 20.07 MB/sec execute 325 sec latency 19486.397 ms 2024-09-12T02:07:38.187 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4b6e1ae5', total seconds elapsed: 23.6 2024-09-12T02:07:39.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 174611 20.12 MB/sec execute 326 sec latency 69.310 ms 2024-09-12T02:07:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:38 smithi151 ceph-mon[35229]: pgmap v975: 97 pgs: 97 active+clean; 509 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.1 MiB/s wr, 174 op/s 2024-09-12T02:07:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:38 smithi183 ceph-mon[35127]: pgmap v975: 97 pgs: 97 active+clean; 509 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.1 MiB/s wr, 174 op/s 2024-09-12T02:07:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:38 smithi122 ceph-mon[30925]: pgmap v975: 97 pgs: 97 active+clean; 509 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.1 MiB/s wr, 174 op/s 2024-09-12T02:07:40.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 175214 20.16 MB/sec execute 327 sec latency 50.313 ms 2024-09-12T02:07:41.076 INFO:tasks.workunit.client.0.smithi122.stdout: 10 175860 20.23 MB/sec execute 328 sec latency 72.031 ms 2024-09-12T02:07:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:40 smithi151 ceph-mon[35229]: pgmap v976: 97 pgs: 97 active+clean; 514 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 204 op/s 2024-09-12T02:07:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:40 smithi183 ceph-mon[35127]: pgmap v976: 97 pgs: 97 active+clean; 514 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 204 op/s 2024-09-12T02:07:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:40 smithi122 ceph-mon[30925]: pgmap v976: 97 pgs: 97 active+clean; 514 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 204 op/s 2024-09-12T02:07:42.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 176245 20.25 MB/sec execute 329 sec latency 296.166 ms 2024-09-12T02:07:43.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 176629 20.24 MB/sec execute 330 sec latency 319.001 ms 2024-09-12T02:07:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:42 smithi151 ceph-mon[35229]: pgmap v977: 97 pgs: 97 active+clean; 517 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 200 op/s 2024-09-12T02:07:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:42 smithi183 ceph-mon[35127]: pgmap v977: 97 pgs: 97 active+clean; 517 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 200 op/s 2024-09-12T02:07:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:42 smithi122 ceph-mon[30925]: pgmap v977: 97 pgs: 97 active+clean; 517 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 200 op/s 2024-09-12T02:07:44.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 177232 20.27 MB/sec execute 331 sec latency 52.467 ms 2024-09-12T02:07:45.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 177741 20.27 MB/sec execute 332 sec latency 76.549 ms 2024-09-12T02:07:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:44 smithi151 ceph-mon[35229]: pgmap v978: 97 pgs: 97 active+clean; 519 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 387 op/s 2024-09-12T02:07:45.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:44 smithi183 ceph-mon[35127]: pgmap v978: 97 pgs: 97 active+clean; 519 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 387 op/s 2024-09-12T02:07:45.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:44 smithi122 ceph-mon[30925]: pgmap v978: 97 pgs: 97 active+clean; 519 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.3 MiB/s wr, 387 op/s 2024-09-12T02:07:46.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 178478 20.33 MB/sec execute 333 sec latency 78.313 ms 2024-09-12T02:07:47.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 179065 20.36 MB/sec execute 334 sec latency 154.888 ms 2024-09-12T02:07:47.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:46 smithi151 ceph-mon[35229]: pgmap v979: 97 pgs: 97 active+clean; 519 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.3 MiB/s wr, 391 op/s 2024-09-12T02:07:47.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:46 smithi183 ceph-mon[35127]: pgmap v979: 97 pgs: 97 active+clean; 519 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.3 MiB/s wr, 391 op/s 2024-09-12T02:07:47.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:46 smithi122 ceph-mon[30925]: pgmap v979: 97 pgs: 97 active+clean; 519 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 7.3 MiB/s wr, 391 op/s 2024-09-12T02:07:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/1351087901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 msgr2=0x7f6d04100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/1351087901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d04100e50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009900 tx=0x7f6cf802a410 comp rx=0 tx=0).stop 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/1351087901 shutdown_connections 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/1351087901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102600 0x7f6d04108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/1351087901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04101c40 0x7f6d041020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/1351087901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d04100e50 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/1351087901 >> 172.21.15.122:0/1351087901 conn(0x7f6d040fc1b0 msgr2=0x7f6d040fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/1351087901 shutdown_connections 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.343+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/1351087901 wait complete. 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 Processor -- start 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 -- start start 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d041049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04101c40 0x7f6d04103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04102600 0x7f6d04103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410c6d0 con 0x7f6d04100a50 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410c550 con 0x7f6d04102600 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0aaaf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410c850 con 0x7f6d04101c40 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d041049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d041049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50396/0 (socket says 172.21.15.122:50396) 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 -- 172.21.15.122:0/2657952359 learned_addr learned my addr 172.21.15.122:0/2657952359 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d03fff640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04101c40 0x7f6d04103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0904c640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04102600 0x7f6d04103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 -- 172.21.15.122:0/2657952359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04101c40 msgr2=0x7f6d04103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04101c40 0x7f6d04103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 -- 172.21.15.122:0/2657952359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04102600 msgr2=0x7f6d04103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04102600 0x7f6d04103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.344+0000 7f6d0884b640 1 -- 172.21.15.122:0/2657952359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d04103df0 con 0x7f6d04100a50 2024-09-12T02:07:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.345+0000 7f6d0884b640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d041049e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8031610 tx=0x7f6cf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.345+0000 7f6d01ffb640 1 -- 172.21.15.122:0/2657952359 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf803f030 con 0x7f6d04100a50 2024-09-12T02:07:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.345+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d041a1e70 con 0x7f6d04100a50 2024-09-12T02:07:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.345+0000 7f6d01ffb640 1 -- 172.21.15.122:0/2657952359 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf8003ad0 con 0x7f6d04100a50 2024-09-12T02:07:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.345+0000 7f6d01ffb640 1 -- 172.21.15.122:0/2657952359 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8035070 con 0x7f6d04100a50 2024-09-12T02:07:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.345+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041a2280 con 0x7f6d04100a50 2024-09-12T02:07:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.347+0000 7f6d01ffb640 1 -- 172.21.15.122:0/2657952359 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6cf8042030 con 0x7f6d04100a50 2024-09-12T02:07:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 Processor -- start 2024-09-12T02:07:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 -- start start 2024-09-12T02:07:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d47a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639d47042f0 con 0x5639d47a6370 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5639d46ac270 con 0x5639d47a09a0 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639d47002b0 con 0x5639d47a12e0 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d01ffb640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6cf40a2c30 0x7f6cf40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d47a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d47a8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50406/0 (socket says 172.21.15.122:50406) 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d03fff640 1 -- 172.21.15.122:0/2402421632 learned_addr learned my addr 172.21.15.122:0/2402421632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.348+0000 7f6d0904c640 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d0904c640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6cf40a2c30 0x7f6cf40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d03fff640 1 -- 172.21.15.122:0/2402421632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 msgr2=0x5639d47a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d03fff640 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d03fff640 1 -- 172.21.15.122:0/2402421632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 msgr2=0x5639d47a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d03fff640 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d01ffb640 1 -- 172.21.15.122:0/2657952359 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6cf80ef570 con 0x7f6d04100a50 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d03fff640 1 -- 172.21.15.122:0/2402421632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d46ab340 con 0x5639d47a6370 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d03fff640 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d47a8830 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009c90 tx=0x7f6cf0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1f2866c8b0cd6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6d0904c640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6cf40a2c30 0x7f6cf40a50f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005c40 tx=0x7f6cfc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6ceeffd640 1 -- 172.21.15.122:0/2402421632 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf001e070 con 0x5639d47a6370 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6ceeffd640 1 -- 172.21.15.122:0/2402421632 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0009e40 con 0x5639d47a6370 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.349+0000 7f6ceeffd640 1 -- 172.21.15.122:0/2402421632 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf000e9d0 con 0x5639d47a6370 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 -- 172.21.15.122:0/2402421632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 msgr2=0x5639d47a8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d47a8830 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009c90 tx=0x7f6cf0011810 comp rx=0 tx=0).stop 2024-09-12T02:07:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 -- 172.21.15.122:0/2402421632 shutdown_connections 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d47a8830 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/2402421632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 -- 172.21.15.122:0/2402421632 >> 172.21.15.122:0/2402421632 conn(0x5639d479c010 msgr2=0x5639d479c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 -- 172.21.15.122:0/2402421632 shutdown_connections 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 -- 172.21.15.122:0/2402421632 wait complete. 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 -- start start 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.350+0000 7f6d0d83d740 1 Processor -- start 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 0x5639d47aab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a6370 0x5639d47b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639d47a9340 con 0x5639d47a09a0 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5639d468d290 con 0x5639d47a6370 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639d47a94c0 con 0x5639d47a12e0 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 10 client.0 _ll_register_callbacks cb 0x5639d46be140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 0x5639d47aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 0x5639d47aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50420/0 (socket says 172.21.15.122:50420) 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 -- 172.21.15.122:0/3832113497 learned_addr learned my addr 172.21.15.122:0/3832113497 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d03fff640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0904c640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a6370 0x5639d47b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 -- 172.21.15.122:0/3832113497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 msgr2=0x5639d47ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 -- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a6370 msgr2=0x5639d47b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a6370 0x5639d47b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 -- 172.21.15.122:0/3832113497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d4417210 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d03fff640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0884b640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 0x5639d47aab90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80ed070 tx=0x7f6cf8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81caff306547f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6ce3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6ce3fff640 1 -- 172.21.15.122:0/3832113497 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80bf030 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6ce3fff640 1 -- 172.21.15.122:0/3832113497 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6cf80ed180 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.351+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639d4711a20 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6ce3fff640 1 -- 172.21.15.122:0/3832113497 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6cf8049070 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 10 client.18396 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5639d44e3620 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6ce3fff640 1 -- 172.21.15.122:0/3832113497 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5639d4711a20 con 0x5639d47a09a0 2024-09-12T02:07:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6ce3fff640 1 -- 172.21.15.122:0/3832113497 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6cf80b84f0 con 0x5639d47a09a0 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 10 client.18396 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 10 client.18396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 20 client.18396 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d47b67f0 0x5639d47b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 4 client.18396 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639d470bd70 con 0x5639d47b67f0 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.352+0000 7f6d03fff640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d47b67f0 0x5639d47b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.353+0000 7f6ce3fff640 10 client.18396 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.353+0000 7f6d03fff640 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d47b67f0 0x5639d47b8c40 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b338639fa94a2fa1 server_cookie=8b271e873df0424d in_seq=0 out_seq=0 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.353+0000 7f6ce3fff640 1 -- 172.21.15.122:0/3832113497 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639d470bd70 con 0x5639d47b67f0 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.353+0000 7f6ce3fff640 10 client.18396 handle_command_reply: tid=0 2024-09-12T02:07:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.353+0000 7f6d0d83d740 1 client.18396 shutdown 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 msgr2=0x5639d47aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 0x5639d47aab90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80ed070 tx=0x7f6cf8009620 comp rx=0 tx=0).stop 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 shutdown_connections 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d47b67f0 0x5639d47b8c40 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a6370 0x5639d47b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6ce3fff640 0 client.18396 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/3832113497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a09a0 0x5639d47aab90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 >> 172.21.15.122:0/3832113497 conn(0x5639d479c010 msgr2=0x5639d479d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 shutdown_connections 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 1 -- 172.21.15.122:0/3832113497 wait complete. 2024-09-12T02:07:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.354+0000 7f6d0d83d740 20 client.18396 trim_cache size 0 max 16384 2024-09-12T02:07:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.370+0000 7f6d0d83d740 1 Processor -- start 2024-09-12T02:07:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 -- start start 2024-09-12T02:07:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d479b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041ad9b0 con 0x5639d47a6370 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041ab280 con 0x5639d47a09a0 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041a90b0 con 0x5639d47a12e0 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37850/0 (socket says 172.21.15.122:37850) 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 -- 172.21.15.122:0/691069966 learned_addr learned my addr 172.21.15.122:0/691069966 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d03fff640 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0904c640 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d479b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 -- 172.21.15.122:0/691069966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 msgr2=0x5639d47b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 -- 172.21.15.122:0/691069966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 msgr2=0x5639d479b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d479b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d0884b640 1 -- 172.21.15.122:0/691069966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639d47aa6d0 con 0x5639d47a09a0 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.371+0000 7f6d03fff640 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.372+0000 7f6d0884b640 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47ab160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8031440 tx=0x7f6cf8005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba86eeaef75605 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.372+0000 7f6cee7fc640 1 -- 172.21.15.122:0/691069966 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80bf030 con 0x5639d47a09a0 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.372+0000 7f6cee7fc640 1 -- 172.21.15.122:0/691069966 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf803be40 con 0x5639d47a09a0 2024-09-12T02:07:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.372+0000 7f6cee7fc640 1 -- 172.21.15.122:0/691069966 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80ed0f0 con 0x5639d47a09a0 2024-09-12T02:07:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 -- 172.21.15.122:0/691069966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 msgr2=0x5639d47ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47ab160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8031440 tx=0x7f6cf8005b50 comp rx=0 tx=0).stop 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 -- 172.21.15.122:0/691069966 shutdown_connections 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a6370 0x5639d479b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a12e0 0x5639d47b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/691069966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47ab160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 -- 172.21.15.122:0/691069966 >> 172.21.15.122:0/691069966 conn(0x5639d479c010 msgr2=0x5639d479a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 -- 172.21.15.122:0/691069966 shutdown_connections 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 -- 172.21.15.122:0/691069966 wait complete. 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 -- start start 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.373+0000 7f6d0d83d740 1 Processor -- start 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 0x5639d47af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a6370 0x5639d47b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041a9740 con 0x5639d47a12e0 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041aabb0 con 0x5639d47a09a0 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041a9e10 con 0x5639d47a6370 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0d83d740 10 client.0 _ll_register_callbacks cb 0x7f6cf8031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 0x5639d47af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 0x5639d47af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50444/0 (socket says 172.21.15.122:50444) 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0884b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:37858/0 (socket says 172.21.15.122:37858) 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 -- 172.21.15.122:0/1209323930 learned_addr learned my addr 172.21.15.122:0/1209323930 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d0904c640 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a6370 0x5639d47b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 -- 172.21.15.122:0/1209323930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a6370 msgr2=0x5639d47b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a6370 0x5639d47b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 -- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 msgr2=0x5639d47af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.374+0000 7f6d03fff640 1 -- 172.21.15.122:0/1209323930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639d47b0e30 con 0x5639d47a12e0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6d03fff640 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 0x5639d47af810 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009be0 tx=0x7f6cf00096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfdfa7318954858 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6ce3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6ce3fff640 1 -- 172.21.15.122:0/1209323930 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf001e040 con 0x5639d47a12e0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6ce3fff640 1 -- 172.21.15.122:0/1209323930 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6cf0005590 con 0x5639d47a12e0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639d4688900 con 0x5639d47a12e0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6ce3fff640 1 -- 172.21.15.122:0/1209323930 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6cf003c050 con 0x5639d47a12e0 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6d0d83d740 10 client.18402 fetch_fsmap learned FSMap version 45 2024-09-12T02:07:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5639d47b21b0 con 0x5639d47a12e0 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6ce3fff640 1 -- 172.21.15.122:0/1209323930 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5639d4688900 con 0x5639d47a12e0 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.375+0000 7f6ce3fff640 1 -- 172.21.15.122:0/1209323930 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6cf0019550 con 0x5639d47a12e0 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0d83d740 10 client.18402 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0d83d740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0d83d740 20 client.18402 populate_metadata read hostname 'smithi122' 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d48218b0 0x5639d4823cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0d83d740 4 client.18402 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639d46d5820 con 0x5639d48218b0 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0884b640 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d48218b0 0x5639d4823cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6d0884b640 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d48218b0 0x5639d4823cd0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fec282d32d45f0e server_cookie=aad4762019df41ab in_seq=0 out_seq=0 2024-09-12T02:07:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.376+0000 7f6ce3fff640 10 client.18402 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.637+0000 7f6ce3fff640 1 -- 172.21.15.122:0/1209323930 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639d46d5820 con 0x5639d48218b0 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.637+0000 7f6ce3fff640 10 client.18402 handle_command_reply: tid=0 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.637+0000 7f6d0d83d740 1 client.18402 shutdown 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.637+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 msgr2=0x5639d47af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.637+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 0x5639d47af810 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009be0 tx=0x7f6cf00096d0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 shutdown_connections 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5639d48218b0 0x5639d4823cd0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639d47a6370 0x5639d47b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6ce3fff640 0 client.18402 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639d47a12e0 0x5639d47af810 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 --2- 172.21.15.122:0/1209323930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639d47a09a0 0x5639d47af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 >> 172.21.15.122:0/1209323930 conn(0x5639d479c010 msgr2=0x5639d479cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 shutdown_connections 2024-09-12T02:07:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 1 -- 172.21.15.122:0/1209323930 wait complete. 2024-09-12T02:07:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.638+0000 7f6d0d83d740 20 client.18402 trim_cache size 0 max 16384 2024-09-12T02:07:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:07:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:07:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:07:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6cf40a2c30 msgr2=0x7f6cf40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6cf40a2c30 0x7f6cf40a50f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005c40 tx=0x7f6cfc009500 comp rx=0 tx=0).stop 2024-09-12T02:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 msgr2=0x7f6d041049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d041049e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8031610 tx=0x7f6cf8004270 comp rx=0 tx=0).stop 2024-09-12T02:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 shutdown_connections 2024-09-12T02:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6cf40a2c30 0x7f6cf40a50f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04102600 0x7f6d04103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04101c40 0x7f6d04103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 --2- 172.21.15.122:0/2657952359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d04100a50 0x7f6d041049e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 >> 172.21.15.122:0/2657952359 conn(0x7f6d040fc1b0 msgr2=0x7f6d040fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 shutdown_connections 2024-09-12T02:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:07:47.639+0000 7f6d0aaaf640 1 -- 172.21.15.122:0/2657952359 wait complete. 2024-09-12T02:07:48.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 179538 20.38 MB/sec execute 335 sec latency 200.147 ms 2024-09-12T02:07:49.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 180237 20.45 MB/sec execute 336 sec latency 57.452 ms 2024-09-12T02:07:49.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:48 smithi151 ceph-mon[35229]: pgmap v980: 97 pgs: 97 active+clean; 515 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 451 op/s 2024-09-12T02:07:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:48 smithi183 ceph-mon[35127]: pgmap v980: 97 pgs: 97 active+clean; 515 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 451 op/s 2024-09-12T02:07:49.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:48 smithi122 ceph-mon[30925]: pgmap v980: 97 pgs: 97 active+clean; 515 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.9 MiB/s wr, 451 op/s 2024-09-12T02:07:50.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 180717 20.48 MB/sec execute 337 sec latency 98.250 ms 2024-09-12T02:07:51.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 181226 20.50 MB/sec execute 338 sec latency 186.675 ms 2024-09-12T02:07:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:50 smithi151 ceph-mon[35229]: pgmap v981: 97 pgs: 97 active+clean; 525 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 612 op/s 2024-09-12T02:07:51.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:50 smithi183 ceph-mon[35127]: pgmap v981: 97 pgs: 97 active+clean; 525 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 612 op/s 2024-09-12T02:07:51.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:50 smithi122 ceph-mon[30925]: pgmap v981: 97 pgs: 97 active+clean; 525 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.2 MiB/s wr, 612 op/s 2024-09-12T02:07:52.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 181648 20.49 MB/sec execute 339 sec latency 70.965 ms 2024-09-12T02:07:53.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 182179 20.49 MB/sec execute 340 sec latency 88.138 ms 2024-09-12T02:07:53.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:52 smithi151 ceph-mon[35229]: pgmap v982: 97 pgs: 97 active+clean; 524 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 594 op/s 2024-09-12T02:07:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:52 smithi183 ceph-mon[35127]: pgmap v982: 97 pgs: 97 active+clean; 524 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 594 op/s 2024-09-12T02:07:53.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:52 smithi122 ceph-mon[30925]: pgmap v982: 97 pgs: 97 active+clean; 524 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 594 op/s 2024-09-12T02:07:54.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 182772 20.53 MB/sec execute 341 sec latency 65.347 ms 2024-09-12T02:07:55.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 183407 20.57 MB/sec execute 342 sec latency 85.257 ms 2024-09-12T02:07:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:54 smithi151 ceph-mon[35229]: pgmap v983: 97 pgs: 97 active+clean; 533 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 714 op/s 2024-09-12T02:07:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:54 smithi183 ceph-mon[35127]: pgmap v983: 97 pgs: 97 active+clean; 533 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 714 op/s 2024-09-12T02:07:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:54 smithi122 ceph-mon[30925]: pgmap v983: 97 pgs: 97 active+clean; 533 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.1 MiB/s wr, 714 op/s 2024-09-12T02:07:56.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 184101 20.63 MB/sec execute 343 sec latency 72.034 ms 2024-09-12T02:07:57.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 184528 20.64 MB/sec execute 344 sec latency 194.339 ms 2024-09-12T02:07:57.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:56 smithi151 ceph-mon[35229]: pgmap v984: 97 pgs: 97 active+clean; 534 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 527 op/s 2024-09-12T02:07:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:56 smithi183 ceph-mon[35127]: pgmap v984: 97 pgs: 97 active+clean; 534 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 527 op/s 2024-09-12T02:07:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:56 smithi122 ceph-mon[30925]: pgmap v984: 97 pgs: 97 active+clean; 534 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 527 op/s 2024-09-12T02:07:58.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 185106 20.69 MB/sec execute 345 sec latency 84.096 ms 2024-09-12T02:07:59.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 185515 20.69 MB/sec execute 346 sec latency 83.164 ms 2024-09-12T02:07:59.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:07:58 smithi151 ceph-mon[35229]: pgmap v985: 97 pgs: 97 active+clean; 532 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-09-12T02:07:59.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:07:58 smithi183 ceph-mon[35127]: pgmap v985: 97 pgs: 97 active+clean; 532 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-09-12T02:07:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:07:58 smithi122 ceph-mon[30925]: pgmap v985: 97 pgs: 97 active+clean; 532 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-09-12T02:08:00.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 186031 20.69 MB/sec execute 347 sec latency 71.831 ms 2024-09-12T02:08:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.744+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/3090065943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 msgr2=0x7fbcd41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.744+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/3090065943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd41020c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009900 tx=0x7fbcbc02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.744+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/3090065943 shutdown_connections 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/3090065943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 0x7fbcd4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/3090065943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd41020c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/3090065943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/3090065943 >> 172.21.15.122:0/3090065943 conn(0x7fbcd40fc1b0 msgr2=0x7fbcd40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/3090065943 shutdown_connections 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/3090065943 wait complete. 2024-09-12T02:08:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.745+0000 7fbcd9dd6640 1 Processor -- start 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 -- start start 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 0x7fbcd41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd410c720 con 0x7fbcd4101c40 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd410c5a0 con 0x7fbcd4100a50 2024-09-12T02:08:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd9dd6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd410c8a0 con 0x7fbcd4102600 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 0x7fbcd41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:33548/0 (socket says 172.21.15.122:33548) 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.746+0000 7fbcd37fe640 1 -- 172.21.15.122:0/4050024604 learned_addr learned my addr 172.21.15.122:0/4050024604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd3fff640 1 -- 172.21.15.122:0/4050024604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 msgr2=0x7fbcd4199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd3fff640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd3fff640 1 -- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 msgr2=0x7fbcd419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd3fff640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd3fff640 1 -- 172.21.15.122:0/4050024604 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd41a1bd0 con 0x7fbcd4102600 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd3fff640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 0x7fbcd41a1560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009fd0 tx=0x7fbccc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd0ff9640 1 -- 172.21.15.122:0/4050024604 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00ce20 con 0x7fbcd4102600 2024-09-12T02:08:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd0ff9640 1 -- 172.21.15.122:0/4050024604 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbccc005020 con 0x7fbcd4102600 2024-09-12T02:08:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd41a1ec0 con 0x7fbcd4102600 2024-09-12T02:08:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd0ff9640 1 -- 172.21.15.122:0/4050024604 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc010720 con 0x7fbcd4102600 2024-09-12T02:08:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.747+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41a2380 con 0x7fbcd4102600 2024-09-12T02:08:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.749+0000 7fbcd0ff9640 1 -- 172.21.15.122:0/4050024604 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbccc005540 con 0x7fbcd4102600 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.749+0000 7fbcdcc4f740 1 Processor -- start 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.749+0000 7fbcdcc4f740 1 -- start start 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005a80da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a812e0 0x558005a85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a88830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5580059e42f0 con 0x558005a86370 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55800598c270 con 0x558005a812e0 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580059e02b0 con 0x558005a809a0 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005a80da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005a80da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:52318/0 (socket says 172.21.15.122:52318) 2024-09-12T02:08:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/3656895697 learned_addr learned my addr 172.21.15.122:0/3656895697 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a812e0 0x558005a85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a88830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd37fe640 1 -- 172.21.15.122:0/3656895697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 msgr2=0x558005a80da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005a80da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd37fe640 1 -- 172.21.15.122:0/3656895697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 msgr2=0x558005a88830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a88830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd37fe640 1 -- 172.21.15.122:0/3656895697 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800598b340 con 0x558005a812e0 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.750+0000 7fbcd0ff9640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbcc00a2cb0 0x7fbcc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.751+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a812e0 0x558005a85da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40114c0 tx=0x7fbcc4011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bac90fe6e15dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.751+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbcc00a2cb0 0x7fbcc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.751+0000 7fbcd0ff9640 1 -- 172.21.15.122:0/4050024604 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbccc0cf690 con 0x7fbcd4102600 2024-09-12T02:08:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.751+0000 7fbcc9ffb640 1 -- 172.21.15.122:0/3656895697 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4011310 con 0x558005a812e0 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.751+0000 7fbcc9ffb640 1 -- 172.21.15.122:0/3656895697 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc4005500 con 0x558005a812e0 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.751+0000 7fbcc9ffb640 1 -- 172.21.15.122:0/3656895697 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400f820 con 0x558005a812e0 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3656895697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a812e0 msgr2=0x558005a85da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbcc00a2cb0 0x7fbcc00a5170 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0057c0 tx=0x7fbcbc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a812e0 0x558005a85da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40114c0 tx=0x7fbcc4011980 comp rx=0 tx=0).stop 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3656895697 shutdown_connections 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a88830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a812e0 0x558005a85da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3656895697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005a80da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3656895697 >> 172.21.15.122:0/3656895697 conn(0x558005a7c010 msgr2=0x558005a7c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3656895697 shutdown_connections 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3656895697 wait complete. 2024-09-12T02:08:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.752+0000 7fbcdcc4f740 1 -- start start 2024-09-12T02:08:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 Processor -- start 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a809a0 0x558005a93460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005a9a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558005a89340 con 0x558005a809a0 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55800596d290 con 0x558005a86370 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558005a894c0 con 0x558005a812e0 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcdcc4f740 10 client.0 _ll_register_callbacks cb 0x55800599e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:52324/0 (socket says 172.21.15.122:52324) 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/2835694235 learned_addr learned my addr 172.21.15.122:0/2835694235 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd3fff640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005a9a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a809a0 0x558005a93460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 msgr2=0x558005a9a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005a9a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/2835694235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a809a0 msgr2=0x558005a93460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a809a0 0x558005a93460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.753+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/2835694235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580056f7210 con 0x558005a812e0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a939a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02a930 tx=0x7fbcbc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9dd78240e82f8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/2835694235 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc005220 con 0x558005a812e0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/2835694235 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbcbc0053c0 con 0x558005a812e0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc4005500 con 0x558005a812e0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/2835694235 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc0055c0 con 0x558005a812e0 2024-09-12T02:08:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcdcc4f740 10 client.27373 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5580057c3620 con 0x558005a812e0 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/2835694235 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc4005500 con 0x558005a812e0 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.754+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/2835694235 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fbcbc003ac0 con 0x558005a812e0 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcdcc4f740 10 client.27373 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcdcc4f740 10 client.27373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcdcc4f740 20 client.27373 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005aa1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcdcc4f740 4 client.27373 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580059f1a20 con 0x558005a9f010 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005aa1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005aa1480 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be5a6a034a6ba93 server_cookie=4248cf8deb42b1fd in_seq=0 out_seq=0 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.755+0000 7fbcb2ffd640 10 client.27373 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.756+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/2835694235 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580059f1a20 con 0x558005a9f010 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.756+0000 7fbcb2ffd640 10 client.27373 handle_command_reply: tid=0 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.756+0000 7fbcdcc4f740 1 client.27373 shutdown 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 msgr2=0x558005a939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a939a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02a930 tx=0x7fbcbc02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 shutdown_connections 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005aa1480 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005a9a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcb2ffd640 0 client.27373 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a939a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/2835694235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a809a0 0x558005a93460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 >> 172.21.15.122:0/2835694235 conn(0x558005a7c010 msgr2=0x558005a7d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 shutdown_connections 2024-09-12T02:08:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/2835694235 wait complete. 2024-09-12T02:08:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.757+0000 7fbcdcc4f740 20 client.27373 trim_cache size 0 max 16384 2024-09-12T02:08:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 Processor -- start 2024-09-12T02:08:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 -- start start 2024-09-12T02:08:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005aa5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 0x558005a9a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005aa6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41a9670 con 0x558005a812e0 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41a6f40 con 0x558005a86370 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41a4d90 con 0x558005a809a0 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 0x558005a9a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 0x558005a9a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50912/0 (socket says 172.21.15.122:50912) 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/3328903173 learned_addr learned my addr 172.21.15.122:0/3328903173 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd3fff640 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005aa6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005aa5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/3328903173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 msgr2=0x558005aa5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005aa5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/3328903173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 msgr2=0x558005aa6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005aa6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 -- 172.21.15.122:0/3328903173 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558005a7cca0 con 0x558005a812e0 2024-09-12T02:08:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.775+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 0x558005a9a930 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc003ee0 tx=0x7fbcbc032a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab615244573d50b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcc97fa640 1 -- 172.21.15.122:0/3328903173 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc054750 con 0x558005a812e0 2024-09-12T02:08:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcc97fa640 1 -- 172.21.15.122:0/3328903173 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc0036b0 con 0x558005a812e0 2024-09-12T02:08:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcc97fa640 1 -- 172.21.15.122:0/3328903173 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc03d7b0 con 0x558005a812e0 2024-09-12T02:08:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3328903173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 msgr2=0x558005a9a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 0x558005a9a930 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc003ee0 tx=0x7fbcbc032a60 comp rx=0 tx=0).stop 2024-09-12T02:08:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3328903173 shutdown_connections 2024-09-12T02:08:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a86370 0x558005aa6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a812e0 0x558005a9a930 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/3328903173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a809a0 0x558005aa5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3328903173 >> 172.21.15.122:0/3328903173 conn(0x558005a7c010 msgr2=0x558005a7b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.776+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3328903173 shutdown_connections 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/3328903173 wait complete. 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 -- start start 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 Processor -- start 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 0x558005a98770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a98cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a99cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41a5400 con 0x558005a86370 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41a6870 con 0x558005a809a0 2024-09-12T02:08:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41a5ad0 con 0x558005a812e0 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.777+0000 7fbcdcc4f740 10 client.0 _ll_register_callbacks cb 0x7fbcbc032b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a99cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 0x558005a98770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a98cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 0x558005a98770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:33556/0 (socket says 172.21.15.122:33556) 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 -- 172.21.15.122:0/122624543 learned_addr learned my addr 172.21.15.122:0/122624543 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a99cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50924/0 (socket says 172.21.15.122:50924) 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 -- 172.21.15.122:0/122624543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 msgr2=0x558005a98cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a98cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 -- 172.21.15.122:0/122624543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 msgr2=0x558005a99cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a99cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 -- 172.21.15.122:0/122624543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558005a9a2d0 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd3fff640 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a99cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcd37fe640 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 0x558005a98770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009230 tx=0x7fbcc4005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19209f1174d44afc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/122624543 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc401d040 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/122624543 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbcc4004220 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcbc0036b0 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/122624543 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbcc403a050 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcdcc4f740 10 client.27413 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.778+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558005b0a410 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/122624543 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc0036b0 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/122624543 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fbcc4004920 con 0x558005a809a0 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcdcc4f740 10 client.27413 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcdcc4f740 10 client.27413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcdcc4f740 20 client.27413 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005b0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcdcc4f740 4 client.27413 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580059ebd70 con 0x558005a9f010 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005b0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.779+0000 7fbcb2ffd640 10 client.27413 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:00.780+0000 7fbcd2ffd640 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005b0cc70 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7500f66f5327d1 server_cookie=fcd693e2e3b7be1a in_seq=0 out_seq=0 2024-09-12T02:08:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.051+0000 7fbcb2ffd640 1 -- 172.21.15.122:0/122624543 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580059ebd70 con 0x558005a9f010 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.051+0000 7fbcb2ffd640 10 client.27413 handle_command_reply: tid=0 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.051+0000 7fbcdcc4f740 1 client.27413 shutdown 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.051+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 msgr2=0x558005a98770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.051+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 0x558005a98770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009230 tx=0x7fbcc4005d40 comp rx=0 tx=0).stop 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 shutdown_connections 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x558005a9f010 0x558005b0cc70 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558005a86370 0x558005a99cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcb2ffd640 0 client.27413 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558005a812e0 0x558005a98cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 --2- 172.21.15.122:0/122624543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558005a809a0 0x558005a98770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 >> 172.21.15.122:0/122624543 conn(0x558005a7c010 msgr2=0x558005a7cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 shutdown_connections 2024-09-12T02:08:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 1 -- 172.21.15.122:0/122624543 wait complete. 2024-09-12T02:08:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcdcc4f740 20 client.27413 trim_cache size 0 max 16384 2024-09-12T02:08:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:08:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:08:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:08:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:08:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbcc00a2cb0 msgr2=0x7fbcc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbcc00a2cb0 0x7fbcc00a5170 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0057c0 tx=0x7fbcbc0315b0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 msgr2=0x7fbcd41a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 0x7fbcd41a1560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009fd0 tx=0x7fbccc00ef10 comp rx=0 tx=0).stop 2024-09-12T02:08:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 shutdown_connections 2024-09-12T02:08:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fbcc00a2cb0 0x7fbcc00a5170 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbcd4102600 0x7fbcd41a1560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcd4101c40 0x7fbcd419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 --2- 172.21.15.122:0/4050024604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcd4100a50 0x7fbcd4199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.052+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 >> 172.21.15.122:0/4050024604 conn(0x7fbcd40fc1b0 msgr2=0x7fbcd40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.053+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 shutdown_connections 2024-09-12T02:08:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:01.053+0000 7fbcd9dd6640 1 -- 172.21.15.122:0/4050024604 wait complete. 2024-09-12T02:08:01.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 186509 20.71 MB/sec execute 348 sec latency 202.189 ms 2024-09-12T02:08:01.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:00 smithi151 ceph-mon[35229]: pgmap v986: 97 pgs: 97 active+clean; 515 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 528 op/s 2024-09-12T02:08:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:00 smithi183 ceph-mon[35127]: pgmap v986: 97 pgs: 97 active+clean; 515 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 528 op/s 2024-09-12T02:08:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:00 smithi122 ceph-mon[30925]: pgmap v986: 97 pgs: 97 active+clean; 515 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 528 op/s 2024-09-12T02:08:02.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 187149 20.73 MB/sec execute 349 sec latency 60.579 ms 2024-09-12T02:08:03.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 187883 20.80 MB/sec execute 350 sec latency 49.463 ms 2024-09-12T02:08:03.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:02 smithi151 ceph-mon[35229]: pgmap v987: 97 pgs: 97 active+clean; 516 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-12T02:08:03.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:02 smithi183 ceph-mon[35127]: pgmap v987: 97 pgs: 97 active+clean; 516 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-12T02:08:03.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:02 smithi122 ceph-mon[30925]: pgmap v987: 97 pgs: 97 active+clean; 516 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-12T02:08:04.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 188553 20.84 MB/sec execute 351 sec latency 61.540 ms 2024-09-12T02:08:05.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 189086 20.87 MB/sec execute 352 sec latency 83.615 ms 2024-09-12T02:08:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:04 smithi151 ceph-mon[35229]: pgmap v988: 97 pgs: 97 active+clean; 520 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 531 op/s 2024-09-12T02:08:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:04 smithi183 ceph-mon[35127]: pgmap v988: 97 pgs: 97 active+clean; 520 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 531 op/s 2024-09-12T02:08:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:04 smithi122 ceph-mon[30925]: pgmap v988: 97 pgs: 97 active+clean; 520 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 531 op/s 2024-09-12T02:08:06.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 189532 20.89 MB/sec execute 353 sec latency 102.216 ms 2024-09-12T02:08:07.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 190017 20.89 MB/sec execute 354 sec latency 78.471 ms 2024-09-12T02:08:07.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:06 smithi151 ceph-mon[35229]: pgmap v989: 97 pgs: 97 active+clean; 525 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 442 op/s 2024-09-12T02:08:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:06 smithi183 ceph-mon[35127]: pgmap v989: 97 pgs: 97 active+clean; 525 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 442 op/s 2024-09-12T02:08:07.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:06 smithi122 ceph-mon[30925]: pgmap v989: 97 pgs: 97 active+clean; 525 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 442 op/s 2024-09-12T02:08:08.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 190531 20.91 MB/sec execute 355 sec latency 61.440 ms 2024-09-12T02:08:08.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:08 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:08:08.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:08 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:08:09.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 191161 20.93 MB/sec execute 356 sec latency 104.148 ms 2024-09-12T02:08:09.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:08 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: pgmap v990: 97 pgs: 97 active+clean; 529 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 488 op/s 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: pgmap v990: 97 pgs: 97 active+clean; 529 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 488 op/s 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:09.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 191786 20.97 MB/sec execute 357 sec latency 54.892 ms 2024-09-12T02:08:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: pgmap v990: 97 pgs: 97 active+clean; 529 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 488 op/s 2024-09-12T02:08:10.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:08:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:10 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:08:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:10 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:11.077 INFO:tasks.workunit.client.0.smithi122.stdout: 10 192362 21.00 MB/sec execute 358 sec latency 64.407 ms 2024-09-12T02:08:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:11.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:11.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:08:11.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:10 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:08:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:11 smithi183 ceph-mon[35127]: pgmap v991: 97 pgs: 97 active+clean; 536 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.5 MiB/s wr, 513 op/s 2024-09-12T02:08:11.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:11 smithi122 ceph-mon[30925]: pgmap v991: 97 pgs: 97 active+clean; 536 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.5 MiB/s wr, 513 op/s 2024-09-12T02:08:12.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 193078 21.07 MB/sec execute 359 sec latency 86.261 ms 2024-09-12T02:08:12.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:11 smithi151 ceph-mon[35229]: pgmap v991: 97 pgs: 97 active+clean; 536 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.5 MiB/s wr, 513 op/s 2024-09-12T02:08:13.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 193427 21.07 MB/sec execute 360 sec latency 300.604 ms 2024-09-12T02:08:13.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:12 smithi151 ceph-mon[35229]: pgmap v992: 97 pgs: 97 active+clean; 539 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 473 op/s 2024-09-12T02:08:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:12 smithi183 ceph-mon[35127]: pgmap v992: 97 pgs: 97 active+clean; 539 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 473 op/s 2024-09-12T02:08:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:12 smithi122 ceph-mon[30925]: pgmap v992: 97 pgs: 97 active+clean; 539 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 473 op/s 2024-09-12T02:08:14.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 193846 21.07 MB/sec execute 361 sec latency 92.725 ms 2024-09-12T02:08:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 -- 172.21.15.122:0/422880468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0120104f70 msgr2=0x7f0120107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 --2- 172.21.15.122:0/422880468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0120104f70 0x7f0120107360 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f011400dd10 tx=0x7f011402a160 comp rx=0 tx=0).stop 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 -- 172.21.15.122:0/422880468 shutdown_connections 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 --2- 172.21.15.122:0/422880468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01201078a0 0x7f0120109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 --2- 172.21.15.122:0/422880468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0120104f70 0x7f0120107360 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 --2- 172.21.15.122:0/422880468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0120102640 0x7f0120104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 -- 172.21.15.122:0/422880468 >> 172.21.15.122:0/422880468 conn(0x7f01200fc1e0 msgr2=0x7f01200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 -- 172.21.15.122:0/422880468 shutdown_connections 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 -- 172.21.15.122:0/422880468 wait complete. 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.150+0000 7f0125e53640 1 Processor -- start 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 -- start start 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0120102640 0x7f0120195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0120104f70 0x7f0120196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 0x7f012019d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f012010b620 con 0x7f01201078a0 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f012010b4a0 con 0x7f0120102640 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f0125e53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f012010b7a0 con 0x7f0120104f70 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 0x7f012019d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0120102640 0x7f0120195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0120104f70 0x7f0120196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 0x7f012019d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38958/0 (socket says 172.21.15.122:38958) 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 -- 172.21.15.122:0/3162553086 learned_addr learned my addr 172.21.15.122:0/3162553086 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 -- 172.21.15.122:0/3162553086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0120104f70 msgr2=0x7f0120196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0120104f70 0x7f0120196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 -- 172.21.15.122:0/3162553086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0120102640 msgr2=0x7f0120195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0120102640 0x7f0120195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011ffff640 1 -- 172.21.15.122:0/3162553086 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012019d860 con 0x7f01201078a0 2024-09-12T02:08:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.151+0000 7f011effd640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0120104f70 0x7f0120196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.152+0000 7f011ffff640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 0x7f012019d1c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f01180163c0 tx=0x7f0118016880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.152+0000 7f011cff9640 1 -- 172.21.15.122:0/3162553086 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011800fbe0 con 0x7f01201078a0 2024-09-12T02:08:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.152+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012019daf0 con 0x7f01201078a0 2024-09-12T02:08:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.152+0000 7f011cff9640 1 -- 172.21.15.122:0/3162553086 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011800fd80 con 0x7f01201078a0 2024-09-12T02:08:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.152+0000 7f011cff9640 1 -- 172.21.15.122:0/3162553086 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011801c6e0 con 0x7f01201078a0 2024-09-12T02:08:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.152+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012019dff0 con 0x7f01201078a0 2024-09-12T02:08:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.154+0000 7f011cff9640 1 -- 172.21.15.122:0/3162553086 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0118023020 con 0x7f01201078a0 2024-09-12T02:08:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.154+0000 7f0128bef740 1 Processor -- start 2024-09-12T02:08:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.154+0000 7f0128bef740 1 -- start start 2024-09-12T02:08:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f0128bef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aad4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f0128bef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aad9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f0128bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aada370 0x564e3aadc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f0128bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564e3aa382f0 con 0x564e3aada370 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f0128bef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564e3a9e0270 con 0x564e3aad52e0 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f0128bef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e3aa342b0 con 0x564e3aad49a0 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aada370 0x564e3aadc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aad4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aad9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aad9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36394/0 (socket says 172.21.15.122:36394) 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011f7fe640 1 -- 172.21.15.122:0/2372630252 learned_addr learned my addr 172.21.15.122:0/2372630252 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011cff9640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01080a2c30 0x7f01080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 -- 172.21.15.122:0/2372630252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 msgr2=0x564e3aad9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aad9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 -- 172.21.15.122:0/2372630252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aada370 msgr2=0x564e3aadc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aada370 0x564e3aadc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 -- 172.21.15.122:0/2372630252 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e3a9df340 con 0x564e3aad49a0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011cff9640 1 -- 172.21.15.122:0/3162553086 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f01180d71d0 con 0x7f01201078a0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011ffff640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01080a2c30 0x7f01080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.155+0000 7f011effd640 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aad4da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0114000c00 tx=0x7f011400d670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6ee3d22aa510d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.156+0000 7f0105ffb640 1 -- 172.21.15.122:0/2372630252 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114047070 con 0x564e3aad49a0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.156+0000 7f0105ffb640 1 -- 172.21.15.122:0/2372630252 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400d2e0 con 0x564e3aad49a0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.156+0000 7f0105ffb640 1 -- 172.21.15.122:0/2372630252 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011400b5d0 con 0x564e3aad49a0 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.156+0000 7f0128bef740 1 -- 172.21.15.122:0/2372630252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 msgr2=0x564e3aad4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.156+0000 7f011ffff640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01080a2c30 0x7f01080a50f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f01180a39c0 tx=0x7f0118016fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.156+0000 7f0128bef740 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aad4da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0114000c00 tx=0x7f011400d670 comp rx=0 tx=0).stop 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- 172.21.15.122:0/2372630252 shutdown_connections 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aada370 0x564e3aadc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aad9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 --2- 172.21.15.122:0/2372630252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aad4da0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- 172.21.15.122:0/2372630252 >> 172.21.15.122:0/2372630252 conn(0x564e3aad0010 msgr2=0x564e3aad0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- 172.21.15.122:0/2372630252 shutdown_connections 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- 172.21.15.122:0/2372630252 wait complete. 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- start start 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 Processor -- start 2024-09-12T02:08:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aadec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aadf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aada370 0x564e3aae6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564e3aadd2e0 con 0x564e3aad52e0 2024-09-12T02:08:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564e3a9c1290 con 0x564e3aada370 2024-09-12T02:08:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e3aadd460 con 0x564e3aad49a0 2024-09-12T02:08:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.157+0000 7f0128bef740 10 client.0 _ll_register_callbacks cb 0x564e3a9f2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aadec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aadf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aadec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:54434/0 (socket says 172.21.15.122:54434) 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 -- 172.21.15.122:0/979445086 learned_addr learned my addr 172.21.15.122:0/979445086 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aada370 0x564e3aae6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 -- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aada370 msgr2=0x564e3aae6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aada370 0x564e3aae6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 -- 172.21.15.122:0/979445086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 msgr2=0x564e3aadf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aadf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 -- 172.21.15.122:0/979445086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e3a74b210 con 0x564e3aad49a0 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f011effd640 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aadec50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f011402a680 tx=0x7f0114009000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a179d2fe0431aa3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f00feffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f00feffd640 1 -- 172.21.15.122:0/979445086 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114047040 con 0x564e3aad49a0 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f00feffd640 1 -- 172.21.15.122:0/979445086 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0114004940 con 0x564e3aad49a0 2024-09-12T02:08:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.158+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011400d2e0 con 0x564e3aad49a0 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f00feffd640 1 -- 172.21.15.122:0/979445086 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0114004b40 con 0x564e3aad49a0 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 10 client.27391 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564e3a817620 con 0x564e3aad49a0 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f00feffd640 1 -- 172.21.15.122:0/979445086 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400d2e0 con 0x564e3aad49a0 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f00feffd640 1 -- 172.21.15.122:0/979445086 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f011404d040 con 0x564e3aad49a0 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 10 client.27391 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 10 client.27391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 20 client.27391 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3aaea460 0x564e3aaec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 4 client.27391 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e3aa45a20 con 0x564e3aaea460 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.159+0000 7f011f7fe640 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3aaea460 0x564e3aaec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.160+0000 7f011f7fe640 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3aaea460 0x564e3aaec850 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87f872fd70f9c6 server_cookie=e782d531deedb46a in_seq=0 out_seq=0 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.160+0000 7f00feffd640 10 client.27391 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f00feffd640 1 -- 172.21.15.122:0/979445086 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e3aa45a20 con 0x564e3aaea460 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f00feffd640 10 client.27391 handle_command_reply: tid=0 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 client.27391 shutdown 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 msgr2=0x564e3aadec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aadec50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f011402a680 tx=0x7f0114009000 comp rx=0 tx=0).stop 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 shutdown_connections 2024-09-12T02:08:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3aaea460 0x564e3aaec850 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aada370 0x564e3aae6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aadf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f00feffd640 0 client.27391 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 --2- 172.21.15.122:0/979445086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aad49a0 0x564e3aadec50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 >> 172.21.15.122:0/979445086 conn(0x564e3aad0010 msgr2=0x564e3aad1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 shutdown_connections 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 1 -- 172.21.15.122:0/979445086 wait complete. 2024-09-12T02:08:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.162+0000 7f0128bef740 20 client.27391 trim_cache size 0 max 16384 2024-09-12T02:08:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 Processor -- start 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 -- start start 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 0x564e3aadf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aae6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3aacf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01201a9650 con 0x564e3aad49a0 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01201a6f20 con 0x564e3aad52e0 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f0128bef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01201a4d50 con 0x564e3aada370 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f011effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 0x564e3aadf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f011effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 0x564e3aadf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38972/0 (socket says 172.21.15.122:38972) 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aae6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f011effd640 1 -- 172.21.15.122:0/2675206485 learned_addr learned my addr 172.21.15.122:0/2675206485 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.177+0000 7f011effd640 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3aacf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f011effd640 1 -- 172.21.15.122:0/2675206485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 msgr2=0x564e3aacf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f011effd640 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3aacf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f011effd640 1 -- 172.21.15.122:0/2675206485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 msgr2=0x564e3aae6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f011effd640 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aae6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f011effd640 1 -- 172.21.15.122:0/2675206485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e3aacfe30 con 0x564e3aad49a0 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f011effd640 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 0x564e3aadf220 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f011403d600 tx=0x7f011402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b79d60909fafa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f01057fa640 1 -- 172.21.15.122:0/2675206485 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114047070 con 0x564e3aad49a0 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f01057fa640 1 -- 172.21.15.122:0/2675206485 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400d2e0 con 0x564e3aad49a0 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.178+0000 7f01057fa640 1 -- 172.21.15.122:0/2675206485 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011400d930 con 0x564e3aad49a0 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 -- 172.21.15.122:0/2675206485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 msgr2=0x564e3aadf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 0x564e3aadf220 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f011403d600 tx=0x7f011402a160 comp rx=0 tx=0).stop 2024-09-12T02:08:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 -- 172.21.15.122:0/2675206485 shutdown_connections 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3aacf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad52e0 0x564e3aae6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 --2- 172.21.15.122:0/2675206485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad49a0 0x564e3aadf220 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 -- 172.21.15.122:0/2675206485 >> 172.21.15.122:0/2675206485 conn(0x564e3aad0010 msgr2=0x564e3aace9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 -- 172.21.15.122:0/2675206485 shutdown_connections 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 -- 172.21.15.122:0/2675206485 wait complete. 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 -- start start 2024-09-12T02:08:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.179+0000 7f0128bef740 1 Processor -- start 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad49a0 0x564e3aae2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aae3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3ab52fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01201a53e0 con 0x564e3aad52e0 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01201a6850 con 0x564e3aad49a0 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01201a5ab0 con 0x564e3aada370 2024-09-12T02:08:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f0128bef740 10 client.0 _ll_register_callbacks cb 0x7f0114045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aae3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aae3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38984/0 (socket says 172.21.15.122:38984) 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 -- 172.21.15.122:0/1653243214 learned_addr learned my addr 172.21.15.122:0/1653243214 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011effd640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad49a0 0x564e3aae2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011effd640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3ab52fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 -- 172.21.15.122:0/1653243214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 msgr2=0x564e3ab52fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3ab52fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 -- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad49a0 msgr2=0x564e3aae2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad49a0 0x564e3aae2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 -- 172.21.15.122:0/1653243214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e3aae6330 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011effd640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3ab52fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f011f7fe640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aae3240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f01100099b0 tx=0x7f0110004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edd892c893d7863 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.180+0000 7f00feffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f00feffd640 1 -- 172.21.15.122:0/1653243214 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110005020 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011400d2e0 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f00feffd640 1 -- 172.21.15.122:0/1653243214 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f01100051c0 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f00feffd640 1 -- 172.21.15.122:0/1653243214 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f01100053c0 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f0128bef740 10 client.18438 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564e3ab54620 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f00feffd640 1 -- 172.21.15.122:0/1653243214 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400d2e0 con 0x564e3aad52e0 2024-09-12T02:08:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.181+0000 7f00feffd640 1 -- 172.21.15.122:0/1653243214 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f0110020530 con 0x564e3aad52e0 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f0128bef740 10 client.18438 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f0128bef740 10 client.18438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f0128bef740 20 client.18438 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f0128bef740 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3ab54f70 0x564e3ab57390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f0128bef740 4 client.18438 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e3aaddfd0 con 0x564e3ab54f70 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f011effd640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3ab54f70 0x564e3ab57390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f00feffd640 10 client.18438 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.182+0000 7f011effd640 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3ab54f70 0x564e3ab57390 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321d4220754c7599 server_cookie=9e6705320172847a in_seq=0 out_seq=0 2024-09-12T02:08:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f00feffd640 1 -- 172.21.15.122:0/1653243214 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e3aaddfd0 con 0x564e3ab54f70 2024-09-12T02:08:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f00feffd640 10 client.18438 handle_command_reply: tid=0 2024-09-12T02:08:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 client.18438 shutdown 2024-09-12T02:08:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 msgr2=0x564e3aae3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aae3240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f01100099b0 tx=0x7f0110004260 comp rx=0 tx=0).stop 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 shutdown_connections 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564e3ab54f70 0x564e3ab57390 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3aada370 0x564e3ab52fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f00feffd640 0 client.18438 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e3aad52e0 0x564e3aae3240 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 --2- 172.21.15.122:0/1653243214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e3aad49a0 0x564e3aae2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 >> 172.21.15.122:0/1653243214 conn(0x564e3aad0010 msgr2=0x564e3aad16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 shutdown_connections 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.433+0000 7f0128bef740 1 -- 172.21.15.122:0/1653243214 wait complete. 2024-09-12T02:08:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0128bef740 20 client.18438 trim_cache size 0 max 16384 2024-09-12T02:08:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:08:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:08:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:08:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:08:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01080a2c30 msgr2=0x7f01080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01080a2c30 0x7f01080a50f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f01180a39c0 tx=0x7f0118016fd0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 msgr2=0x7f012019d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 0x7f012019d1c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f01180163c0 tx=0x7f0118016880 comp rx=0 tx=0).stop 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 shutdown_connections 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f01080a2c30 0x7f01080a50f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01201078a0 0x7f012019d1c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0120104f70 0x7f0120196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 --2- 172.21.15.122:0/3162553086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0120102640 0x7f0120195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 >> 172.21.15.122:0/3162553086 conn(0x7f01200fc1e0 msgr2=0x7f0120102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 shutdown_connections 2024-09-12T02:08:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:14.434+0000 7f0125e53640 1 -- 172.21.15.122:0/3162553086 wait complete. 2024-09-12T02:08:15.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 194367 21.08 MB/sec execute 362 sec latency 177.356 ms 2024-09-12T02:08:15.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:14 smithi151 ceph-mon[35229]: pgmap v993: 97 pgs: 97 active+clean; 555 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2024-09-12T02:08:15.145 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:08:15.145 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T02:08:15.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:15.168 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:14 smithi122 ceph-mon[30925]: pgmap v993: 97 pgs: 97 active+clean; 555 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2024-09-12T02:08:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:14 smithi183 ceph-mon[35127]: pgmap v993: 97 pgs: 97 active+clean; 555 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.7 MiB/s wr, 608 op/s 2024-09-12T02:08:15.424 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:16.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 194886 21.09 MB/sec execute 363 sec latency 89.739 ms 2024-09-12T02:08:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 -- 172.21.15.122:0/2376757098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c102d90 msgr2=0x7f1a2c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 --2- 172.21.15.122:0/2376757098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c102d90 0x7f1a2c103170 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009930 tx=0x7f1a2802a420 comp rx=0 tx=0).stop 2024-09-12T02:08:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 -- 172.21.15.122:0/2376757098 shutdown_connections 2024-09-12T02:08:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 --2- 172.21.15.122:0/2376757098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c108290 0x7f1a2c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 --2- 172.21.15.122:0/2376757098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c103740 0x7f1a2c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 --2- 172.21.15.122:0/2376757098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c102d90 0x7f1a2c103170 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 -- 172.21.15.122:0/2376757098 >> 172.21.15.122:0/2376757098 conn(0x7f1a2c0fc7d0 msgr2=0x7f1a2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 -- 172.21.15.122:0/2376757098 shutdown_connections 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.189+0000 7f1a3404b640 1 -- 172.21.15.122:0/2376757098 wait complete. 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 Processor -- start 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 -- start start 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 0x7f1a2c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c103740 0x7f1a2c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c108290 0x7f1a2c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c10fe80 con 0x7f1a2c108290 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c10fd00 con 0x7f1a2c103740 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a3404b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c110000 con 0x7f1a2c102d90 2024-09-12T02:08:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a31de7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 0x7f1a2c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a31de7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 0x7f1a2c1963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:54484/0 (socket says 172.21.15.122:54484) 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a31de7640 1 -- 172.21.15.122:0/3225442320 learned_addr learned my addr 172.21.15.122:0/3225442320 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a325e8640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c108290 0x7f1a2c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.190+0000 7f1a315e6640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c103740 0x7f1a2c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a325e8640 1 -- 172.21.15.122:0/3225442320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 msgr2=0x7f1a2c1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a325e8640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 0x7f1a2c1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a325e8640 1 -- 172.21.15.122:0/3225442320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c103740 msgr2=0x7f1a2c196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a325e8640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c103740 0x7f1a2c196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a325e8640 1 -- 172.21.15.122:0/3225442320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c19b370 con 0x7f1a2c108290 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a315e6640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c103740 0x7f1a2c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a31de7640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 0x7f1a2c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a325e8640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c108290 0x7f1a2c19ac90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400da10 tx=0x7f1a2400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:16.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400bd70 con 0x7f1a2c108290 2024-09-12T02:08:16.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c19b660 con 0x7f1a2c108290 2024-09-12T02:08:16.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.191+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c1a2f40 con 0x7f1a2c108290 2024-09-12T02:08:16.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.192+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24003e50 con 0x7f1a2c108290 2024-09-12T02:08:16.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.192+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a240106c0 con 0x7f1a2c108290 2024-09-12T02:08:16.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.193+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a10005290 con 0x7f1a2c108290 2024-09-12T02:08:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.195+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1a24004b40 con 0x7f1a2c108290 2024-09-12T02:08:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.196+0000 7f1a22ffd640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a180a2c00 0x7f1a180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.196+0000 7f1a31de7640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a180a2c00 0x7f1a180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.196+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1a240cefa0 con 0x7f1a2c108290 2024-09-12T02:08:16.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.196+0000 7f1a31de7640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a180a2c00 0x7f1a180a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a28009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.200+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a24021070 con 0x7f1a2c108290 2024-09-12T02:08:16.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.465+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a10005580 con 0x7f1a2c108290 2024-09-12T02:08:16.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.466+0000 7f1a22ffd640 1 -- 172.21.15.122:0/3225442320 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7f1a2409bf80 con 0x7f1a2c108290 2024-09-12T02:08:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:08:16.466 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:08:16.467 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:08:16.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a180a2c00 msgr2=0x7f1a180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:16.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a180a2c00 0x7f1a180a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a28009560 comp rx=0 tx=0).stop 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c108290 msgr2=0x7f1a2c19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c108290 0x7f1a2c19ac90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400da10 tx=0x7f1a2400ded0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 shutdown_connections 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1a180a2c00 0x7f1a180a50c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a2c108290 0x7f1a2c19ac90 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a2c103740 0x7f1a2c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 --2- 172.21.15.122:0/3225442320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a2c102d90 0x7f1a2c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 >> 172.21.15.122:0/3225442320 conn(0x7f1a2c0fc7d0 msgr2=0x7f1a2c105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 shutdown_connections 2024-09-12T02:08:16.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:16.468+0000 7f1a3404b640 1 -- 172.21.15.122:0/3225442320 wait complete. 2024-09-12T02:08:16.805 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:16 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3225442320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:16 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3225442320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3225442320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:17.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 195616 21.13 MB/sec execute 364 sec latency 59.293 ms 2024-09-12T02:08:17.260 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T02:08:17.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T02:08:17.639 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:17.804 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:17 smithi122 ceph-mon[30925]: pgmap v994: 97 pgs: 97 active+clean; 555 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.4 MiB/s wr, 457 op/s 2024-09-12T02:08:17.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:17 smithi151 ceph-mon[35229]: pgmap v994: 97 pgs: 97 active+clean; 555 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.4 MiB/s wr, 457 op/s 2024-09-12T02:08:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:17 smithi183 ceph-mon[35127]: pgmap v994: 97 pgs: 97 active+clean; 555 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.4 MiB/s wr, 457 op/s 2024-09-12T02:08:18.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196077 21.15 MB/sec execute 365 sec latency 207.169 ms 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 -- 172.21.15.122:0/636315580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404106f20 msgr2=0x7f240410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 --2- 172.21.15.122:0/636315580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404106f20 0x7f240410eaa0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009ca0 tx=0x7f23fc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 -- 172.21.15.122:0/636315580 shutdown_connections 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 --2- 172.21.15.122:0/636315580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404106f20 0x7f240410eaa0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 --2- 172.21.15.122:0/636315580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24041027a0 0x7f24041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 --2- 172.21.15.122:0/636315580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2404101df0 0x7f24041021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 -- 172.21.15.122:0/636315580 >> 172.21.15.122:0/636315580 conn(0x7f24040fd8f0 msgr2=0x7f24040ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 -- 172.21.15.122:0/636315580 shutdown_connections 2024-09-12T02:08:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 -- 172.21.15.122:0/636315580 wait complete. 2024-09-12T02:08:18.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.249+0000 7f2409999640 1 Processor -- start 2024-09-12T02:08:18.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 -- start start 2024-09-12T02:08:18.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 0x7f240419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24041027a0 0x7f240419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2404106f20 0x7f240419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2404075d20 con 0x7f2404101df0 2024-09-12T02:08:18.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2404075ba0 con 0x7f2404106f20 2024-09-12T02:08:18.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2409999640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2404075ea0 con 0x7f24041027a0 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 0x7f240419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 0x7f240419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39016/0 (socket says 172.21.15.122:39016) 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 -- 172.21.15.122:0/92710189 learned_addr learned my addr 172.21.15.122:0/92710189 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f24027fc640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24041027a0 0x7f240419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 -- 172.21.15.122:0/92710189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24041027a0 msgr2=0x7f240419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24041027a0 0x7f240419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 -- 172.21.15.122:0/92710189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2404106f20 msgr2=0x7f240419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.250+0000 7f2402ffd640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2404106f20 0x7f240419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2402ffd640 1 -- 172.21.15.122:0/92710189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240419f580 con 0x7f2404101df0 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2402ffd640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 0x7f240419a5d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e9f0 tx=0x7f23f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2408997640 1 -- 172.21.15.122:0/92710189 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400ccc0 con 0x7f2404101df0 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240419f870 con 0x7f2404101df0 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2408997640 1 -- 172.21.15.122:0/92710189 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23f4010910 con 0x7f2404101df0 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2408997640 1 -- 172.21.15.122:0/92710189 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40188a0 con 0x7f2404101df0 2024-09-12T02:08:18.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.251+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a7150 con 0x7f2404101df0 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.253+0000 7f2408997640 1 -- 172.21.15.122:0/92710189 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f23f4010ac0 con 0x7f2404101df0 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 Processor -- start 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 -- start start 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007ab98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007aba3940 0x56007aba5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56007aafe2a0 con 0x56007ab98760 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56007aaa5b10 con 0x56007ab99080 2024-09-12T02:08:18.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f240bd72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56007aafa260 con 0x56007aba3940 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f24037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007ab98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f24037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007ab98b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39028/0 (socket says 172.21.15.122:39028) 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f24037fe640 1 -- 172.21.15.122:0/2497075721 learned_addr learned my addr 172.21.15.122:0/2497075721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f2402ffd640 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f24037fe640 1 -- 172.21.15.122:0/2497075721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007aba3940 msgr2=0x56007aba5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.254+0000 7f24037fe640 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007aba3940 0x56007aba5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f24037fe640 1 -- 172.21.15.122:0/2497075721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 msgr2=0x56007aba3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f24037fe640 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f24037fe640 1 -- 172.21.15.122:0/2497075721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56007a86fb90 con 0x56007ab98760 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f2402ffd640 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f2408997640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23ec0a2b80 0x7f23ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f24037fe640 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007ab98b40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0093d0 tx=0x7f23fc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7442e3f60effc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f2402ffd640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23ec0a2b80 0x7f23ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f2408997640 1 -- 172.21.15.122:0/92710189 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f23f40cf4e0 con 0x7f2404101df0 2024-09-12T02:08:18.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f23e9ffb640 1 -- 172.21.15.122:0/2497075721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc031bb0 con 0x56007ab98760 2024-09-12T02:08:18.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f23e9ffb640 1 -- 172.21.15.122:0/2497075721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23fc036040 con 0x56007ab98760 2024-09-12T02:08:18.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.255+0000 7f23e9ffb640 1 -- 172.21.15.122:0/2497075721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc02fce0 con 0x56007ab98760 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 -- 172.21.15.122:0/2497075721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 msgr2=0x56007ab98b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f2402ffd640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23ec0a2b80 0x7f23ec0a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f23f40ccfe0 tx=0x7f23f40097c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007ab98b40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0093d0 tx=0x7f23fc005ef0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 -- 172.21.15.122:0/2497075721 shutdown_connections 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007aba3940 0x56007aba5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 --2- 172.21.15.122:0/2497075721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007ab98b40 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 -- 172.21.15.122:0/2497075721 >> 172.21.15.122:0/2497075721 conn(0x56007ab94cf0 msgr2=0x56007ab950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 -- 172.21.15.122:0/2497075721 shutdown_connections 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.256+0000 7f240bd72740 1 -- 172.21.15.122:0/2497075721 wait complete. 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.257+0000 7f240bd72740 1 -- start start 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.257+0000 7f240bd72740 1 Processor -- start 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.257+0000 7f240bd72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007aba0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.257+0000 7f240bd72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 0x56007aba0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f240bd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007aba1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f240bd72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56007aba6510 con 0x56007ab98760 2024-09-12T02:08:18.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f240bd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56007aa866e0 con 0x56007aba3940 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f240bd72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56007aba6690 con 0x56007ab99080 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 0x56007aba0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 0x56007aba0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:54522/0 (socket says 172.21.15.122:54522) 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 -- 172.21.15.122:0/1343747134 learned_addr learned my addr 172.21.15.122:0/1343747134 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24037fe640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007aba1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f240bd72740 10 client.0 _ll_register_callbacks cb 0x56007aaaaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24037fe640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007aba0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 -- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 msgr2=0x56007aba1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007aba1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 -- 172.21.15.122:0/1343747134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 msgr2=0x56007aba0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007aba0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 -- 172.21.15.122:0/1343747134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007abafa50 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24037fe640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007aba1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc036040 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f24027fc640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 0x56007aba0dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23f800f6c0 tx=0x7f23f800fb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6433eae688378928 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.258+0000 7f23e2ffd640 10 client.27406 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1343747134 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f801d060 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1343747134 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f23f800daa0 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1343747134 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f23f800dca0 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 10 client.27406 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=45}) -- 0x56007abb2390 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1343747134 <== mon.1 v2:172.21.15.183:3300/0 4 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f23f8018440 con 0x56007ab99080 2024-09-12T02:08:18.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 10 client.27406 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 10 client.27406 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 20 client.27406 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007abb36e0 0x56007abb5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 4 client.27406 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.259+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56007ab0b9d0 con 0x56007abb36e0 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.260+0000 7f24037fe640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007abb36e0 0x56007abb5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.260+0000 7f23e2ffd640 10 client.27406 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.260+0000 7f24037fe640 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007abb36e0 0x56007abb5ad0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf7d8912979dc8f server_cookie=deaba4f13308b734 in_seq=0 out_seq=0 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1343747134 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56007ab0b9d0 con 0x56007abb36e0 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f23e2ffd640 10 client.27406 handle_command_reply: tid=0 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 client.27406 shutdown 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 msgr2=0x56007aba0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 0x56007aba0dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23f800f6c0 tx=0x7f23f800fb80 comp rx=0 tx=0).stop 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 shutdown_connections 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007abb36e0 0x56007abb5ad0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007aba1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab99080 0x56007aba0dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 --2- 172.21.15.122:0/1343747134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab98760 0x56007aba0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 >> 172.21.15.122:0/1343747134 conn(0x56007ab94cf0 msgr2=0x56007ab9aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f23e2ffd640 0 client.27406 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 shutdown_connections 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.261+0000 7f240bd72740 1 -- 172.21.15.122:0/1343747134 wait complete. 2024-09-12T02:08:18.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.262+0000 7f240bd72740 20 client.27406 trim_cache size 0 max 16384 2024-09-12T02:08:18.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 Processor -- start 2024-09-12T02:08:18.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 -- start start 2024-09-12T02:08:18.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007ab9f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab99080 0x56007abbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007abbb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24041b2110 con 0x56007ab99080 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24041af9e0 con 0x56007aba3940 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f240bd72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ad810 con 0x56007ab98760 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f24027fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab99080 0x56007abbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f24037fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007ab9f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f24027fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab99080 0x56007abbb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39036/0 (socket says 172.21.15.122:39036) 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.281+0000 7f24027fc640 1 -- 172.21.15.122:0/3786868922 learned_addr learned my addr 172.21.15.122:0/3786868922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007abbb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 -- 172.21.15.122:0/3786868922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 msgr2=0x56007abbb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007abbb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 -- 172.21.15.122:0/3786868922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab99080 msgr2=0x56007abbb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab99080 0x56007abbb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 -- 172.21.15.122:0/3786868922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56007ab98590 con 0x56007ab98760 2024-09-12T02:08:18.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f24037fe640 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007ab9f150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23fc005f60 tx=0x7f23fc03d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c139450c28e67028 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f23e97fa640 1 -- 172.21.15.122:0/3786868922 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc05a030 con 0x56007ab98760 2024-09-12T02:08:18.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f23e97fa640 1 -- 172.21.15.122:0/3786868922 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23fc031450 con 0x56007ab98760 2024-09-12T02:08:18.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.282+0000 7f23e97fa640 1 -- 172.21.15.122:0/3786868922 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc030740 con 0x56007ab98760 2024-09-12T02:08:18.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 -- 172.21.15.122:0/3786868922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 msgr2=0x56007ab9f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007ab9f150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23fc005f60 tx=0x7f23fc03d040 comp rx=0 tx=0).stop 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 -- 172.21.15.122:0/3786868922 shutdown_connections 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007aba3940 0x56007abbb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007ab99080 0x56007abbb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 --2- 172.21.15.122:0/3786868922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007ab9f150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 -- 172.21.15.122:0/3786868922 >> 172.21.15.122:0/3786868922 conn(0x56007ab94cf0 msgr2=0x56007aba2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 -- 172.21.15.122:0/3786868922 shutdown_connections 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 -- 172.21.15.122:0/3786868922 wait complete. 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.283+0000 7f240bd72740 1 -- start start 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 Processor -- start 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007aba7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 0x56007ac1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24041adea0 con 0x56007aba3940 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24041af310 con 0x56007ab99080 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ae570 con 0x56007ab98760 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f240bd72740 10 client.0 _ll_register_callbacks cb 0x7f23f800abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 0x56007ac1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24027fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24027fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:36506/0 (socket says 172.21.15.122:36506) 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24027fc640 1 -- 172.21.15.122:0/1350618815 learned_addr learned my addr 172.21.15.122:0/1350618815 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 0x56007ac1d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39040/0 (socket says 172.21.15.122:39040) 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007aba7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 -- 172.21.15.122:0/1350618815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 msgr2=0x56007aba7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007aba7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 -- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 msgr2=0x56007aba8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24037fe640 1 -- 172.21.15.122:0/1350618815 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007ab90260 con 0x56007aba3940 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.284+0000 7f24027fc640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f24037fe640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 0x56007ac1d320 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009a60 tx=0x7f23fc02f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59322b4c74d6ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f23e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1350618815 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc05a070 con 0x56007aba3940 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1350618815 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f23fc031840 con 0x56007aba3940 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc031450 con 0x56007aba3940 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1350618815 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f23fc02fab0 con 0x56007aba3940 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f240bd72740 10 client.18462 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56007ac1e870 con 0x56007aba3940 2024-09-12T02:08:18.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.285+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1350618815 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23fc031450 con 0x56007aba3940 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1350618815 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f23fc053030 con 0x56007aba3940 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f240bd72740 10 client.18462 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f240bd72740 10 client.18462 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f240bd72740 20 client.18462 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f240bd72740 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007ac1ea90 0x56007ac20e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f240bd72740 4 client.18462 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x56007ab05d20 con 0x56007ac1ea90 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f24027fc640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007ac1ea90 0x56007ac20e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f23e2ffd640 10 client.18462 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:18.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:18.286+0000 7f24027fc640 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007ac1ea90 0x56007ac20e80 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f383591e9ddf4a server_cookie=d6ca34bc0f72df27 in_seq=0 out_seq=0 2024-09-12T02:08:19.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196189 21.11 MB/sec execute 366 sec latency 791.370 ms 2024-09-12T02:08:19.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:18 smithi151 ceph-mon[35229]: pgmap v995: 97 pgs: 97 active+clean; 555 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.6 MiB/s wr, 578 op/s 2024-09-12T02:08:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:18 smithi183 ceph-mon[35127]: pgmap v995: 97 pgs: 97 active+clean; 555 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.6 MiB/s wr, 578 op/s 2024-09-12T02:08:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:18 smithi122 ceph-mon[30925]: pgmap v995: 97 pgs: 97 active+clean; 555 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.6 MiB/s wr, 578 op/s 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.465+0000 7f23e2ffd640 1 -- 172.21.15.122:0/1350618815 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56007ab05d20 con 0x56007ac1ea90 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.465+0000 7f23e2ffd640 10 client.18462 handle_command_reply: tid=0 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.465+0000 7f240bd72740 1 client.18462 shutdown 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.465+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 msgr2=0x56007ac1d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.465+0000 7f240bd72740 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 0x56007ac1d320 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009a60 tx=0x7f23fc02f7b0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 shutdown_connections 2024-09-12T02:08:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56007ac1ea90 0x56007ac20e80 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007aba3940 0x56007ac1d320 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f23e2ffd640 0 client.18462 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56007ab99080 0x56007aba8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 --2- 172.21.15.122:0/1350618815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56007ab98760 0x56007aba7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 >> 172.21.15.122:0/1350618815 conn(0x56007ab94cf0 msgr2=0x56007ab93650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 shutdown_connections 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 1 -- 172.21.15.122:0/1350618815 wait complete. 2024-09-12T02:08:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f240bd72740 20 client.18462 trim_cache size 0 max 16384 2024-09-12T02:08:19.469 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:08:19.469 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:08:19.469 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:08:19.469 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 78, 2024-09-12T02:08:19.469 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "dea7d3f4": { 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 78, 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:08:19.470 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:08:19.471 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:08:19.471 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:08:19.471 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:08:19.471 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:08:19.471 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:08:19.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:08:19.472 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:08:19.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:19.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:19.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:19.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:19.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:19.473 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:08:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23ec0a2b80 msgr2=0x7f23ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23ec0a2b80 0x7f23ec0a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f23f40ccfe0 tx=0x7f23f40097c0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 msgr2=0x7f240419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 0x7f240419a5d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e9f0 tx=0x7f23f400edf0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 shutdown_connections 2024-09-12T02:08:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f23ec0a2b80 0x7f23ec0a5040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2404106f20 0x7f240419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24041027a0 0x7f240419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 --2- 172.21.15.122:0/92710189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2404101df0 0x7f240419a5d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:19.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.466+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 >> 172.21.15.122:0/92710189 conn(0x7f24040fd8f0 msgr2=0x7f24040fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:19.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.467+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 shutdown_connections 2024-09-12T02:08:19.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:19.467+0000 7f2409999640 1 -- 172.21.15.122:0/92710189 wait complete. 2024-09-12T02:08:20.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 21.05 MB/sec execute 367 sec latency 1782.909 ms 2024-09-12T02:08:20.527 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dea7d3f4', quiesce took 5.4 seconds. Will release after: 24.6 2024-09-12T02:08:21.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.99 MB/sec execute 368 sec latency 2782.998 ms 2024-09-12T02:08:21.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:20 smithi151 ceph-mon[35229]: pgmap v996: 97 pgs: 97 active+clean; 545 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 574 op/s 2024-09-12T02:08:21.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:20 smithi183 ceph-mon[35127]: pgmap v996: 97 pgs: 97 active+clean; 545 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 574 op/s 2024-09-12T02:08:21.242 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:20 smithi122 ceph-mon[30925]: pgmap v996: 97 pgs: 97 active+clean; 545 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 574 op/s 2024-09-12T02:08:22.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.93 MB/sec execute 369 sec latency 3783.104 ms 2024-09-12T02:08:23.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.88 MB/sec execute 370 sec latency 4783.203 ms 2024-09-12T02:08:23.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:22 smithi183 ceph-mon[35127]: pgmap v997: 97 pgs: 97 active+clean; 541 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-09-12T02:08:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:22 smithi122 ceph-mon[30925]: pgmap v997: 97 pgs: 97 active+clean; 541 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-09-12T02:08:23.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:22 smithi151 ceph-mon[35229]: pgmap v997: 97 pgs: 97 active+clean; 541 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-09-12T02:08:24.078 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.82 MB/sec execute 371 sec latency 5783.277 ms 2024-09-12T02:08:25.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.77 MB/sec execute 372 sec latency 6783.377 ms 2024-09-12T02:08:25.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:24 smithi151 ceph-mon[35229]: pgmap v998: 97 pgs: 97 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.6 MiB/s wr, 566 op/s 2024-09-12T02:08:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:24 smithi183 ceph-mon[35127]: pgmap v998: 97 pgs: 97 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.6 MiB/s wr, 566 op/s 2024-09-12T02:08:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:24 smithi122 ceph-mon[30925]: pgmap v998: 97 pgs: 97 active+clean; 532 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.6 MiB/s wr, 566 op/s 2024-09-12T02:08:26.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.71 MB/sec execute 373 sec latency 7783.477 ms 2024-09-12T02:08:27.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.66 MB/sec execute 374 sec latency 8783.577 ms 2024-09-12T02:08:27.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:26 smithi151 ceph-mon[35229]: pgmap v999: 97 pgs: 97 active+clean; 535 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 414 op/s 2024-09-12T02:08:27.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:26 smithi183 ceph-mon[35127]: pgmap v999: 97 pgs: 97 active+clean; 535 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 414 op/s 2024-09-12T02:08:27.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:26 smithi122 ceph-mon[30925]: pgmap v999: 97 pgs: 97 active+clean; 535 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 414 op/s 2024-09-12T02:08:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 -- 172.21.15.122:0/668146909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8103710 msgr2=0x7fa6e8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 --2- 172.21.15.122:0/668146909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8103710 0x7fa6e8109fa0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009ca0 tx=0x7fa6e002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 -- 172.21.15.122:0/668146909 shutdown_connections 2024-09-12T02:08:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 --2- 172.21.15.122:0/668146909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8103710 0x7fa6e8109fa0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 --2- 172.21.15.122:0/668146909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e8102d50 0x7fa6e81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 --2- 172.21.15.122:0/668146909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6e8101b60 0x7fa6e8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 -- 172.21.15.122:0/668146909 >> 172.21.15.122:0/668146909 conn(0x7fa6e80fd2e0 msgr2=0x7fa6e80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 -- 172.21.15.122:0/668146909 shutdown_connections 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 -- 172.21.15.122:0/668146909 wait complete. 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 Processor -- start 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.528+0000 7fa6f020e640 1 -- start start 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 0x7fa6e8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6e8102d50 0x7fa6e8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e8103710 0x7fa6e819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810d7e0 con 0x7fa6e8101b60 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810d660 con 0x7fa6e8103710 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810d960 con 0x7fa6e8102d50 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 0x7fa6e8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 0x7fa6e8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40484/0 (socket says 172.21.15.122:40484) 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 -- 172.21.15.122:0/342144171 learned_addr learned my addr 172.21.15.122:0/342144171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6e8102d50 0x7fa6e8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e8103710 0x7fa6e819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 -- 172.21.15.122:0/342144171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6e8102d50 msgr2=0x7fa6e8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6e8102d50 0x7fa6e8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 -- 172.21.15.122:0/342144171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e8103710 msgr2=0x7fa6e819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e8103710 0x7fa6e819f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 -- 172.21.15.122:0/342144171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e819f9d0 con 0x7fa6e8101b60 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 0x7fa6e8197dc0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d950 tx=0x7fa6e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6deffd640 1 -- 172.21.15.122:0/342144171 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4017070 con 0x7fa6e8101b60 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6deffd640 1 -- 172.21.15.122:0/342144171 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e4009c30 con 0x7fa6e8101b60 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e819fcc0 con 0x7fa6e8101b60 2024-09-12T02:08:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6deffd640 1 -- 172.21.15.122:0/342144171 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400f600 con 0x7fa6e8101b60 2024-09-12T02:08:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.529+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e81a01c0 con 0x7fa6e8101b60 2024-09-12T02:08:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6deffd640 1 -- 172.21.15.122:0/342144171 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa6e401a070 con 0x7fa6e8101b60 2024-09-12T02:08:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 Processor -- start 2024-09-12T02:08:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 -- start start 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ed8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086eddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ee0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e086e3c2f0 con 0x55e086ede370 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e086de4270 con 0x55e086ed89a0 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6f3087740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e086e382b0 con 0x55e086ed92e0 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6ed7a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ee0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6ed7a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ee0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40498/0 (socket says 172.21.15.122:40498) 2024-09-12T02:08:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6ed7a9640 1 -- 172.21.15.122:0/1401099089 learned_addr learned my addr 172.21.15.122:0/1401099089 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.532+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ed8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086eddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6deffd640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6d40a2c30 0x7fa6d40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ed7a9640 1 -- 172.21.15.122:0/1401099089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 msgr2=0x55e086eddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086eddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ed7a9640 1 -- 172.21.15.122:0/1401099089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 msgr2=0x55e086ed8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ed8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ed7a9640 1 -- 172.21.15.122:0/1401099089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e086de3340 con 0x55e086ede370 2024-09-12T02:08:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6d40a2c30 0x7fa6d40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6deffd640 1 -- 172.21.15.122:0/342144171 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa6e40cec10 con 0x7fa6e8101b60 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ee0830 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800bcb0 tx=0x7fa6d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca176e9868793ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6d3fff640 1 -- 172.21.15.122:0/1401099089 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8015070 con 0x55e086ede370 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6d3fff640 1 -- 172.21.15.122:0/1401099089 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d801a940 con 0x55e086ede370 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.533+0000 7fa6d3fff640 1 -- 172.21.15.122:0/1401099089 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011a30 con 0x55e086ede370 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 -- 172.21.15.122:0/1401099089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 msgr2=0x55e086ee0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6d40a2c30 0x7fa6d40a50f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0005e30 tx=0x7fa6e00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ee0830 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800bcb0 tx=0x7fa6d800f730 comp rx=0 tx=0).stop 2024-09-12T02:08:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 -- 172.21.15.122:0/1401099089 shutdown_connections 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ee0830 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086eddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 --2- 172.21.15.122:0/1401099089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ed8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 -- 172.21.15.122:0/1401099089 >> 172.21.15.122:0/1401099089 conn(0x55e086ed4010 msgr2=0x55e086ed4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 -- 172.21.15.122:0/1401099089 shutdown_connections 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 -- 172.21.15.122:0/1401099089 wait complete. 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 -- start start 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.534+0000 7fa6f3087740 1 Processor -- start 2024-09-12T02:08:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086eea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e086ee1340 con 0x55e086ede370 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e086dc5290 con 0x55e086ed92e0 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e086ee14c0 con 0x55e086ed89a0 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6f3087740 10 client.0 _ll_register_callbacks cb 0x55e086df6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086eea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086eea140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40512/0 (socket says 172.21.15.122:40512) 2024-09-12T02:08:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2275907231 learned_addr learned my addr 172.21.15.122:0/2275907231 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2275907231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 msgr2=0x55e086ee2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 msgr2=0x55e086ee3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2275907231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e086b4f210 con 0x55e086ede370 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086eea140 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009ca0 tx=0x7fa6e002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d717a75247790278 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.535+0000 7fa6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2275907231 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0004450 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2275907231 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa6e00045f0 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d801a940 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2275907231 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa6e00047f0 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 10 client.18474 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e086c1b620 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2275907231 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d801a940 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2275907231 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa6e0030780 con 0x55e086ede370 2024-09-12T02:08:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 10 client.18474 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 20 client.18474 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086eee960 0x55e086ef0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 4 client.18474 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e086e49a20 con 0x55e086eee960 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.536+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086eee960 0x55e086ef0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.537+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086eee960 0x55e086ef0dc0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a87970f4d805ee9 server_cookie=251b424d2ccf3a9e in_seq=0 out_seq=0 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.537+0000 7fa6d17fa640 10 client.18474 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.537+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2275907231 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e086e49a20 con 0x55e086eee960 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.537+0000 7fa6d17fa640 10 client.18474 handle_command_reply: tid=0 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.537+0000 7fa6f3087740 1 client.18474 shutdown 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 msgr2=0x55e086eea140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086eea140 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009ca0 tx=0x7fa6e002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 shutdown_connections 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086eee960 0x55e086ef0dc0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086eea140 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2275907231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6d17fa640 0 client.18474 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 >> 172.21.15.122:0/2275907231 conn(0x55e086ed4010 msgr2=0x55e086ed5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 shutdown_connections 2024-09-12T02:08:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 1 -- 172.21.15.122:0/2275907231 wait complete. 2024-09-12T02:08:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.538+0000 7fa6f3087740 20 client.18474 trim_cache size 0 max 16384 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 Processor -- start 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 -- start start 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ed4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81ab880 con 0x55e086ede370 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81a9150 con 0x55e086ed89a0 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6f3087740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81a6f80 con 0x55e086ed92e0 2024-09-12T02:08:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6ee7ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ed4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6ee7ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ed4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40518/0 (socket says 172.21.15.122:40518) 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/117145825 learned_addr learned my addr 172.21.15.122:0/117145825 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.552+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/117145825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 msgr2=0x55e086ee3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/117145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 msgr2=0x55e086ee2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/117145825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e086ed3e30 con 0x55e086ede370 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ed4ec0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00049d0 tx=0x7fa6e0031f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c20f0b2fcb3986f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6d37fe640 1 -- 172.21.15.122:0/117145825 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0030e60 con 0x55e086ede370 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6d37fe640 1 -- 172.21.15.122:0/117145825 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e004c030 con 0x55e086ede370 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6d37fe640 1 -- 172.21.15.122:0/117145825 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e002fad0 con 0x55e086ede370 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 -- 172.21.15.122:0/117145825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 msgr2=0x55e086ed4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ed4ec0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00049d0 tx=0x7fa6e0031f30 comp rx=0 tx=0).stop 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 -- 172.21.15.122:0/117145825 shutdown_connections 2024-09-12T02:08:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086ed4ec0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed92e0 0x55e086ee3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 --2- 172.21.15.122:0/117145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed89a0 0x55e086ee2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.553+0000 7fa6f3087740 1 -- 172.21.15.122:0/117145825 >> 172.21.15.122:0/117145825 conn(0x55e086ed4010 msgr2=0x55e086eda480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 -- 172.21.15.122:0/117145825 shutdown_connections 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 -- 172.21.15.122:0/117145825 wait complete. 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 -- start start 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 Processor -- start 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086f547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086f54d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086f5bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81a7610 con 0x55e086ede370 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81a8a80 con 0x55e086ed92e0 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81a7ce0 con 0x55e086ed89a0 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6f3087740 10 client.0 _ll_register_callbacks cb 0x7fa6e002f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6ee7ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086f5bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6ee7ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086f5bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40520/0 (socket says 172.21.15.122:40520) 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2190559580 learned_addr learned my addr 172.21.15.122:0/2190559580 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6ed7a9640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086f54d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.554+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086f547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2190559580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 msgr2=0x55e086f547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086f547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 msgr2=0x55e086f54d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086f54d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6ee7ab640 1 -- 172.21.15.122:0/2190559580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e086f5c350 con 0x55e086ede370 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086f547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6ee7ab640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086f5bd40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0031ae0 tx=0x7fa6e0005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d63406c48a884a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2190559580 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00307c0 con 0x55e086ede370 2024-09-12T02:08:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2190559580 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa6e0030960 con 0x55e086ede370 2024-09-12T02:08:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e004c030 con 0x55e086ede370 2024-09-12T02:08:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2190559580 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa6e002f850 con 0x55e086ede370 2024-09-12T02:08:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 10 client.18480 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e086f5d5d0 con 0x55e086ede370 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2190559580 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e004c030 con 0x55e086ede370 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2190559580 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa6e005e070 con 0x55e086ede370 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 10 client.18480 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 10 client.18480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 20 client.18480 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086f5dea0 0x55e086f602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 4 client.18480 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.555+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e086e43d70 con 0x55e086f5dea0 2024-09-12T02:08:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.556+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086f5dea0 0x55e086f602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.556+0000 7fa6d17fa640 10 client.18480 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.556+0000 7fa6edfaa640 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086f5dea0 0x55e086f602c0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e2aadf5ef61614 server_cookie=c6734ce6ddb18a2d in_seq=0 out_seq=0 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6d17fa640 1 -- 172.21.15.122:0/2190559580 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e086e43d70 con 0x55e086f5dea0 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6d17fa640 10 client.18480 handle_command_reply: tid=0 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 client.18480 shutdown 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 msgr2=0x55e086f5bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086f5bd40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0031ae0 tx=0x7fa6e0005c30 comp rx=0 tx=0).stop 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 shutdown_connections 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e086f5dea0 0x55e086f602c0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e086ede370 0x55e086f5bd40 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e086ed92e0 0x55e086f54d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6d17fa640 0 client.18480 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 --2- 172.21.15.122:0/2190559580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e086ed89a0 0x55e086f547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 >> 172.21.15.122:0/2190559580 conn(0x55e086ed4010 msgr2=0x55e086eda480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 shutdown_connections 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 1 -- 172.21.15.122:0/2190559580 wait complete. 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.661+0000 7fa6f3087740 20 client.18480 trim_cache size 0 max 16384 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:08:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6d40a2c30 msgr2=0x7fa6d40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6d40a2c30 0x7fa6d40a50f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0005e30 tx=0x7fa6e00095c0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 msgr2=0x7fa6e8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 0x7fa6e8197dc0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d950 tx=0x7fa6e400de10 comp rx=0 tx=0).stop 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 shutdown_connections 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa6d40a2c30 0x7fa6d40a50f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e8103710 0x7fa6e819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6e8102d50 0x7fa6e8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 --2- 172.21.15.122:0/342144171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e8101b60 0x7fa6e8197dc0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 >> 172.21.15.122:0/342144171 conn(0x7fa6e80fd2e0 msgr2=0x7fa6e8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 shutdown_connections 2024-09-12T02:08:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:27.662+0000 7fa6f020e640 1 -- 172.21.15.122:0/342144171 wait complete. 2024-09-12T02:08:28.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.60 MB/sec execute 375 sec latency 9783.616 ms 2024-09-12T02:08:29.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.55 MB/sec execute 376 sec latency 10783.684 ms 2024-09-12T02:08:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:28 smithi151 ceph-mon[35229]: pgmap v1000: 97 pgs: 97 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 391 op/s 2024-09-12T02:08:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:28 smithi183 ceph-mon[35127]: pgmap v1000: 97 pgs: 97 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 391 op/s 2024-09-12T02:08:29.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:28 smithi122 ceph-mon[30925]: pgmap v1000: 97 pgs: 97 active+clean; 528 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 391 op/s 2024-09-12T02:08:30.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.49 MB/sec execute 377 sec latency 11783.743 ms 2024-09-12T02:08:31.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.44 MB/sec execute 378 sec latency 12783.833 ms 2024-09-12T02:08:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:30 smithi151 ceph-mon[35229]: pgmap v1001: 97 pgs: 97 active+clean; 525 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 259 op/s 2024-09-12T02:08:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:30 smithi183 ceph-mon[35127]: pgmap v1001: 97 pgs: 97 active+clean; 525 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 259 op/s 2024-09-12T02:08:31.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:30 smithi122 ceph-mon[30925]: pgmap v1001: 97 pgs: 97 active+clean; 525 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 259 op/s 2024-09-12T02:08:32.101 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.38 MB/sec execute 379 sec latency 13783.864 ms 2024-09-12T02:08:33.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.33 MB/sec execute 380 sec latency 14783.903 ms 2024-09-12T02:08:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:32 smithi151 ceph-mon[35229]: pgmap v1002: 97 pgs: 97 active+clean; 524 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-09-12T02:08:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:32 smithi183 ceph-mon[35127]: pgmap v1002: 97 pgs: 97 active+clean; 524 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-09-12T02:08:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:32 smithi122 ceph-mon[30925]: pgmap v1002: 97 pgs: 97 active+clean; 524 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 4.4 MiB/s wr, 226 op/s 2024-09-12T02:08:34.079 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.28 MB/sec execute 381 sec latency 15783.965 ms 2024-09-12T02:08:35.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.22 MB/sec execute 382 sec latency 16784.048 ms 2024-09-12T02:08:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:34 smithi151 ceph-mon[35229]: pgmap v1003: 97 pgs: 97 active+clean; 520 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 5.4 MiB/s wr, 254 op/s 2024-09-12T02:08:35.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:34 smithi183 ceph-mon[35127]: pgmap v1003: 97 pgs: 97 active+clean; 520 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 5.4 MiB/s wr, 254 op/s 2024-09-12T02:08:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:34 smithi122 ceph-mon[30925]: pgmap v1003: 97 pgs: 97 active+clean; 520 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 5.4 MiB/s wr, 254 op/s 2024-09-12T02:08:36.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.17 MB/sec execute 383 sec latency 17784.147 ms 2024-09-12T02:08:37.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.12 MB/sec execute 384 sec latency 18784.247 ms 2024-09-12T02:08:37.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:36 smithi151 ceph-mon[35229]: pgmap v1004: 97 pgs: 97 active+clean; 520 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.6 MiB/s wr, 197 op/s 2024-09-12T02:08:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:36 smithi183 ceph-mon[35127]: pgmap v1004: 97 pgs: 97 active+clean; 520 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.6 MiB/s wr, 197 op/s 2024-09-12T02:08:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:36 smithi122 ceph-mon[30925]: pgmap v1004: 97 pgs: 97 active+clean; 520 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.6 MiB/s wr, 197 op/s 2024-09-12T02:08:38.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.06 MB/sec execute 385 sec latency 19784.322 ms 2024-09-12T02:08:39.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 20.01 MB/sec execute 386 sec latency 20784.411 ms 2024-09-12T02:08:39.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:38 smithi151 ceph-mon[35229]: pgmap v1005: 97 pgs: 97 active+clean; 517 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.7 MiB/s wr, 212 op/s 2024-09-12T02:08:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:38 smithi183 ceph-mon[35127]: pgmap v1005: 97 pgs: 97 active+clean; 517 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.7 MiB/s wr, 212 op/s 2024-09-12T02:08:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:38 smithi122 ceph-mon[30925]: pgmap v1005: 97 pgs: 97 active+clean; 517 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.7 MiB/s wr, 212 op/s 2024-09-12T02:08:40.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.96 MB/sec execute 387 sec latency 21784.528 ms 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 -- 172.21.15.122:0/899116544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c101b60 msgr2=0x7f6f0c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 --2- 172.21.15.122:0/899116544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c101b60 0x7f6f0c101f60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009900 tx=0x7f6f0802a410 comp rx=0 tx=0).stop 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 -- 172.21.15.122:0/899116544 shutdown_connections 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 --2- 172.21.15.122:0/899116544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f0c103710 0x7f6f0c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 --2- 172.21.15.122:0/899116544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f0c102d50 0x7f6f0c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 --2- 172.21.15.122:0/899116544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c101b60 0x7f6f0c101f60 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 -- 172.21.15.122:0/899116544 >> 172.21.15.122:0/899116544 conn(0x7f6f0c0fd2e0 msgr2=0x7f6f0c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 -- 172.21.15.122:0/899116544 shutdown_connections 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.759+0000 7f6f1445c640 1 -- 172.21.15.122:0/899116544 wait complete. 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 Processor -- start 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 -- start start 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f0c101b60 0x7f6f0c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f0c102d50 0x7f6f0c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 0x7f6f0c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c10d7e0 con 0x7f6f0c103710 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c10d660 con 0x7f6f0c102d50 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f1445c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c10d960 con 0x7f6f0c101b60 2024-09-12T02:08:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 0x7f6f0c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 0x7f6f0c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42140/0 (socket says 172.21.15.122:42140) 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 -- 172.21.15.122:0/4146328617 learned_addr learned my addr 172.21.15.122:0/4146328617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f121f8640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f0c101b60 0x7f6f0c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f119f7640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f0c102d50 0x7f6f0c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 -- 172.21.15.122:0/4146328617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f0c101b60 msgr2=0x7f6f0c075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f0c101b60 0x7f6f0c075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 -- 172.21.15.122:0/4146328617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f0c102d50 msgr2=0x7f6f0c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f0c102d50 0x7f6f0c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 -- 172.21.15.122:0/4146328617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c0753b0 con 0x7f6f0c103710 2024-09-12T02:08:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.760+0000 7f6f129f9640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 0x7f6f0c074b20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400eb10 tx=0x7f6f0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.761+0000 7f6f037fe640 1 -- 172.21.15.122:0/4146328617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04004b20 con 0x7f6f0c103710 2024-09-12T02:08:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.761+0000 7f6f037fe640 1 -- 172.21.15.122:0/4146328617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f04004cc0 con 0x7f6f0c103710 2024-09-12T02:08:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.761+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c1a1ef0 con 0x7f6f0c103710 2024-09-12T02:08:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.761+0000 7f6f037fe640 1 -- 172.21.15.122:0/4146328617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04010760 con 0x7f6f0c103710 2024-09-12T02:08:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.761+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c1a23b0 con 0x7f6f0c103710 2024-09-12T02:08:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f037fe640 1 -- 172.21.15.122:0/4146328617 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6f04005020 con 0x7f6f0c103710 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 Processor -- start 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 -- start start 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b73830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564445acf2f0 con 0x564445b71370 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564445a77270 con 0x564445b6b9a0 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.763+0000 7f6f171f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564445acb2b0 con 0x564445b6c2e0 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b73830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b73830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42146/0 (socket says 172.21.15.122:42146) 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 -- 172.21.15.122:0/2867171942 learned_addr learned my addr 172.21.15.122:0/2867171942 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f121f8640 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f037fe640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6ef80a2cb0 0x7f6ef80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 -- 172.21.15.122:0/2867171942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 msgr2=0x564445b70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b70da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 -- 172.21.15.122:0/2867171942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 msgr2=0x564445b6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b6bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 -- 172.21.15.122:0/2867171942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564445a76340 con 0x564445b71370 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f121f8640 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f129f9640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6ef80a2cb0 0x7f6ef80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f119f7640 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b73830 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0149c0 tx=0x7f6efc00c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3469b82fbfd086f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f037fe640 1 -- 172.21.15.122:0/4146328617 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6f040cf6f0 con 0x7f6f0c103710 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f00bfe640 1 -- 172.21.15.122:0/2867171942 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0142d0 con 0x564445b71370 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f00bfe640 1 -- 172.21.15.122:0/2867171942 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc00fc40 con 0x564445b71370 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.764+0000 7f6f00bfe640 1 -- 172.21.15.122:0/2867171942 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc012b60 con 0x564445b71370 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 -- 172.21.15.122:0/2867171942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 msgr2=0x564445b73830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f129f9640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6ef80a2cb0 0x7f6ef80a5170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6f0409bc50 tx=0x7f6f04096980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b73830 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0149c0 tx=0x7f6efc00c4d0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 -- 172.21.15.122:0/2867171942 shutdown_connections 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b73830 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 --2- 172.21.15.122:0/2867171942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 -- 172.21.15.122:0/2867171942 >> 172.21.15.122:0/2867171942 conn(0x564445b67010 msgr2=0x564445b67410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 -- 172.21.15.122:0/2867171942 shutdown_connections 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 -- 172.21.15.122:0/2867171942 wait complete. 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.765+0000 7f6f171f4740 1 -- start start 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 Processor -- start 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 0x564445b75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b71370 0x564445b7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564445b74340 con 0x564445b6b9a0 2024-09-12T02:08:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564445a58290 con 0x564445b6c2e0 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564445b744c0 con 0x564445b71370 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 10 client.0 _ll_register_callbacks cb 0x564445a89140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 0x564445b75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 0x564445b75b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42158/0 (socket says 172.21.15.122:42158) 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 -- 172.21.15.122:0/3103102301 learned_addr learned my addr 172.21.15.122:0/3103102301 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b71370 0x564445b7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 -- 172.21.15.122:0/3103102301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b71370 msgr2=0x564445b7d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b71370 0x564445b7d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 -- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 msgr2=0x564445b760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 -- 172.21.15.122:0/3103102301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644457e2210 con 0x564445b6b9a0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 0x564445b75b70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005780 tx=0x7f6efc005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72118f19f54f68c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6ef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/3103102301 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0142d0 con 0x564445b6b9a0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/3103102301 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6efc00f730 con 0x564445b6b9a0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.766+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc00fc40 con 0x564445b6b9a0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/3103102301 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6efc00f930 con 0x564445b6b9a0 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 10 client.18492 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5644458ae620 con 0x564445b6b9a0 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/3103102301 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc00fc40 con 0x564445b6b9a0 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/3103102301 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6efc02d060 con 0x564445b6b9a0 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 10 client.18492 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 10 client.18492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 20 client.18492 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445b815e0 0x564445b839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 4 client.18492 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564445adca20 con 0x564445b815e0 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.767+0000 7f6f121f8640 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445b815e0 0x564445b839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6ef5ffb640 10 client.18492 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f121f8640 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445b815e0 0x564445b839d0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf926f5caf89636 server_cookie=51b62c21d73f823e in_seq=0 out_seq=0 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/3103102301 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564445adca20 con 0x564445b815e0 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6ef5ffb640 10 client.18492 handle_command_reply: tid=0 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 client.18492 shutdown 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 msgr2=0x564445b75b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 0x564445b75b70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005780 tx=0x7f6efc005f80 comp rx=0 tx=0).stop 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 shutdown_connections 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445b815e0 0x564445b839d0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b71370 0x564445b7d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6ef5ffb640 0 client.18492 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3103102301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b6b9a0 0x564445b75b70 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 >> 172.21.15.122:0/3103102301 conn(0x564445b67010 msgr2=0x564445b68820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 shutdown_connections 2024-09-12T02:08:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.768+0000 7f6f171f4740 1 -- 172.21.15.122:0/3103102301 wait complete. 2024-09-12T02:08:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.769+0000 7f6f171f4740 20 client.18492 trim_cache size 0 max 16384 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 Processor -- start 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 -- start start 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6b9a0 0x564445b659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b7d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1adb60 con 0x564445b71370 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1ab430 con 0x564445b6c2e0 2024-09-12T02:08:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.782+0000 7f6f171f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1a9260 con 0x564445b6b9a0 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b65ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42174/0 (socket says 172.21.15.122:42174) 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 -- 172.21.15.122:0/3044320165 learned_addr learned my addr 172.21.15.122:0/3044320165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f121f8640 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b7d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6b9a0 0x564445b659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 -- 172.21.15.122:0/3044320165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6b9a0 msgr2=0x564445b659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6b9a0 0x564445b659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 -- 172.21.15.122:0/3044320165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 msgr2=0x564445b7d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b7d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 -- 172.21.15.122:0/3044320165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564445b756b0 con 0x564445b71370 2024-09-12T02:08:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6f119f7640 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b65ee0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00cf00 tx=0x7f6efc012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33b065663519213 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6ef7fff640 1 -- 172.21.15.122:0/3044320165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0037c0 con 0x564445b71370 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6ef7fff640 1 -- 172.21.15.122:0/3044320165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc005510 con 0x564445b71370 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.783+0000 7f6ef7fff640 1 -- 172.21.15.122:0/3044320165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc014070 con 0x564445b71370 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- 172.21.15.122:0/3044320165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 msgr2=0x564445b65ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b65ee0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00cf00 tx=0x7f6efc012620 comp rx=0 tx=0).stop 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- 172.21.15.122:0/3044320165 shutdown_connections 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b65ee0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6c2e0 0x564445b7d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- 172.21.15.122:0/3044320165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6b9a0 0x564445b659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- 172.21.15.122:0/3044320165 >> 172.21.15.122:0/3044320165 conn(0x564445b67010 msgr2=0x564445b75920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- 172.21.15.122:0/3044320165 shutdown_connections 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- 172.21.15.122:0/3044320165 wait complete. 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- start start 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 Processor -- start 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b7d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b79ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b7a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1a98f0 con 0x564445b71370 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1aad60 con 0x564445b6b9a0 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1a9fc0 con 0x564445b6c2e0 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f171f4740 10 client.0 _ll_register_callbacks cb 0x7f6efc00cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b7a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f119f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b7a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42178/0 (socket says 172.21.15.122:42178) 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.784+0000 7f6f119f7640 1 -- 172.21.15.122:0/1301439050 learned_addr learned my addr 172.21.15.122:0/1301439050 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b7d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f121f8640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b79ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 -- 172.21.15.122:0/1301439050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 msgr2=0x564445b79ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b79ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 -- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 msgr2=0x564445b7d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b7d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 -- 172.21.15.122:0/1301439050 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564445bea260 con 0x564445b71370 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f119f7640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b7a060 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6efc012860 tx=0x7f6efc024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636cf704ffe8a4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6ef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/1301439050 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc003870 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/1301439050 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6efc00fad0 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc005510 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/1301439050 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6efc0140e0 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 10 client.18498 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564445beb550 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/1301439050 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc005510 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/1301439050 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f6efc046040 con 0x564445b71370 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 10 client.18498 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 20 client.18498 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445bebc90 0x564445bee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 4 client.18498 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.785+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564445ad6d70 con 0x564445bebc90 2024-09-12T02:08:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.786+0000 7f6f121f8640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445bebc90 0x564445bee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.786+0000 7f6ef5ffb640 10 client.18498 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.786+0000 7f6f121f8640 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445bebc90 0x564445bee0b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac1b90f3db48535 server_cookie=b48d7c64979028f1 in_seq=0 out_seq=0 2024-09-12T02:08:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6ef5ffb640 1 -- 172.21.15.122:0/1301439050 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564445ad6d70 con 0x564445bebc90 2024-09-12T02:08:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6ef5ffb640 10 client.18498 handle_command_reply: tid=0 2024-09-12T02:08:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 client.18498 shutdown 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 msgr2=0x564445b7a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b7a060 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6efc012860 tx=0x7f6efc024040 comp rx=0 tx=0).stop 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 shutdown_connections 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x564445bebc90 0x564445bee0b0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564445b71370 0x564445b7a060 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564445b6c2e0 0x564445b79ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 --2- 172.21.15.122:0/1301439050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564445b6b9a0 0x564445b7d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6ef5ffb640 0 client.18498 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 >> 172.21.15.122:0/1301439050 conn(0x564445b67010 msgr2=0x564445b67f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 shutdown_connections 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 1 -- 172.21.15.122:0/1301439050 wait complete. 2024-09-12T02:08:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f171f4740 20 client.18498 trim_cache size 0 max 16384 2024-09-12T02:08:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:08:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:08:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:08:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:08:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6ef80a2cb0 msgr2=0x7f6ef80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6ef80a2cb0 0x7f6ef80a5170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6f0409bc50 tx=0x7f6f04096980 comp rx=0 tx=0).stop 2024-09-12T02:08:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 msgr2=0x7f6f0c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 0x7f6f0c074b20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400eb10 tx=0x7f6f0400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 shutdown_connections 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6ef80a2cb0 0x7f6ef80a5170 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f0c103710 0x7f6f0c074b20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f0c102d50 0x7f6f0c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 --2- 172.21.15.122:0/4146328617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f0c101b60 0x7f6f0c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 >> 172.21.15.122:0/4146328617 conn(0x7f6f0c0fd2e0 msgr2=0x7f6f0c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 shutdown_connections 2024-09-12T02:08:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:40.848+0000 7f6f1445c640 1 -- 172.21.15.122:0/4146328617 wait complete. 2024-09-12T02:08:41.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.91 MB/sec execute 388 sec latency 22784.575 ms 2024-09-12T02:08:41.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:40 smithi151 ceph-mon[35229]: pgmap v1006: 97 pgs: 97 active+clean; 516 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 4.7 MiB/s wr, 201 op/s 2024-09-12T02:08:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:40 smithi183 ceph-mon[35127]: pgmap v1006: 97 pgs: 97 active+clean; 516 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 4.7 MiB/s wr, 201 op/s 2024-09-12T02:08:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:40 smithi122 ceph-mon[30925]: pgmap v1006: 97 pgs: 97 active+clean; 516 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 4.7 MiB/s wr, 201 op/s 2024-09-12T02:08:42.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.86 MB/sec execute 389 sec latency 23784.649 ms 2024-09-12T02:08:43.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.81 MB/sec execute 390 sec latency 24784.750 ms 2024-09-12T02:08:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:42 smithi183 ceph-mon[35127]: pgmap v1007: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.3 MiB/s wr, 174 op/s 2024-09-12T02:08:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:42 smithi122 ceph-mon[30925]: pgmap v1007: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.3 MiB/s wr, 174 op/s 2024-09-12T02:08:43.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:42 smithi151 ceph-mon[35229]: pgmap v1007: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.3 MiB/s wr, 174 op/s 2024-09-12T02:08:44.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.76 MB/sec execute 391 sec latency 25784.836 ms 2024-09-12T02:08:45.080 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.71 MB/sec execute 392 sec latency 26784.919 ms 2024-09-12T02:08:45.128 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:45.153 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:44 smithi122 ceph-mon[30925]: pgmap v1008: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-09-12T02:08:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:44 smithi183 ceph-mon[35127]: pgmap v1008: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-09-12T02:08:45.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:44 smithi151 ceph-mon[35229]: pgmap v1008: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 5.0 MiB/s wr, 196 op/s 2024-09-12T02:08:45.376 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:46.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.66 MB/sec execute 393 sec latency 27784.985 ms 2024-09-12T02:08:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.013+0000 7f5873db3640 1 -- 172.21.15.122:0/2515042186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c102780 msgr2=0x7f586c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.013+0000 7f5873db3640 1 --2- 172.21.15.122:0/2515042186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c102780 0x7f586c1067d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f585c009900 tx=0x7f585c02a410 comp rx=0 tx=0).stop 2024-09-12T02:08:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- 172.21.15.122:0/2515042186 shutdown_connections 2024-09-12T02:08:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 --2- 172.21.15.122:0/2515042186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c106f00 0x7f586c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 --2- 172.21.15.122:0/2515042186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c102780 0x7f586c1067d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 --2- 172.21.15.122:0/2515042186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c101dd0 0x7f586c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- 172.21.15.122:0/2515042186 >> 172.21.15.122:0/2515042186 conn(0x7f586c0fd8b0 msgr2=0x7f586c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- 172.21.15.122:0/2515042186 shutdown_connections 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- 172.21.15.122:0/2515042186 wait complete. 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 Processor -- start 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- start start 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c101dd0 0x7f586c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c102780 0x7f586c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 0x7f586c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f586c111f20 con 0x7f586c106f00 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f586c111da0 con 0x7f586c101dd0 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.014+0000 7f5873db3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1120a0 con 0x7f586c102780 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 0x7f586c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 0x7f586c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42192/0 (socket says 172.21.15.122:42192) 2024-09-12T02:08:46.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 -- 172.21.15.122:0/4241220393 learned_addr learned my addr 172.21.15.122:0/4241220393 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:46.339 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:08:46.339 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:08:46.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f587134e640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c102780 0x7f586c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5871b4f640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c101dd0 0x7f586c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 -- 172.21.15.122:0/4241220393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c102780 msgr2=0x7f586c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c102780 0x7f586c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 -- 172.21.15.122:0/4241220393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c101dd0 msgr2=0x7f586c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c101dd0 0x7f586c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 -- 172.21.15.122:0/4241220393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c19f610 con 0x7f586c106f00 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f587134e640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c102780 0x7f586c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5871b4f640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c101dd0 0x7f586c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5872350640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 0x7f586c19ef30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f586400cad0 tx=0x7f586400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586400ae20 con 0x7f586c106f00 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c19f900 con 0x7f586c106f00 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5864005020 con 0x7f586c106f00 2024-09-12T02:08:46.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586400f6f0 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.015+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1a71e0 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.016+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c1021b0 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.017+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f58640051d0 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.018+0000 7f5862ffd640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f58580a2b80 0x7f58580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.018+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f58640cf4b0 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.018+0000 7f5871b4f640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f58580a2b80 0x7f58580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.018+0000 7f5871b4f640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f58580a2b80 0x7f58580a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5868005d00 tx=0x7f5868009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.021+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5864014030 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.222+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f586c1a74e0 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.223+0000 7f5862ffd640 1 -- 172.21.15.122:0/4241220393 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7f586409c410 con 0x7f586c106f00 2024-09-12T02:08:46.342 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f58580a2b80 msgr2=0x7f58580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f58580a2b80 0x7f58580a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5868005d00 tx=0x7f5868009550 comp rx=0 tx=0).stop 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 msgr2=0x7f586c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 0x7f586c19ef30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f586400cad0 tx=0x7f586400cf90 comp rx=0 tx=0).stop 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 shutdown_connections 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f58580a2b80 0x7f58580a5040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f586c106f00 0x7f586c19ef30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f586c102780 0x7f586c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 --2- 172.21.15.122:0/4241220393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f586c101dd0 0x7f586c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:46.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 >> 172.21.15.122:0/4241220393 conn(0x7f586c0fd8b0 msgr2=0x7f586c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:46.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 shutdown_connections 2024-09-12T02:08:46.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:46.225+0000 7f5873db3640 1 -- 172.21.15.122:0/4241220393 wait complete. 2024-09-12T02:08:46.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:46 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4241220393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:46.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:46 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4241220393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:46.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:46 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4241220393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:47.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196198 19.61 MB/sec execute 394 sec latency 28785.265 ms 2024-09-12T02:08:47.107 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id dea7d3f4 --release --await' 2024-09-12T02:08:47.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id dea7d3f4 --release --await 2024-09-12T02:08:47.395 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:47.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:47 smithi122 ceph-mon[30925]: pgmap v1009: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 4.3 MiB/s wr, 171 op/s 2024-09-12T02:08:47.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:47 smithi151 ceph-mon[35229]: pgmap v1009: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 4.3 MiB/s wr, 171 op/s 2024-09-12T02:08:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:47 smithi183 ceph-mon[35127]: pgmap v1009: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 4.3 MiB/s wr, 171 op/s 2024-09-12T02:08:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 -- 172.21.15.122:0/3478046797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750106a80 msgr2=0x7f4750106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 --2- 172.21.15.122:0/3478046797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750106a80 0x7f4750106e60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4740009930 tx=0x7f474002a420 comp rx=0 tx=0).stop 2024-09-12T02:08:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 -- 172.21.15.122:0/3478046797 shutdown_connections 2024-09-12T02:08:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 --2- 172.21.15.122:0/3478046797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750101420 0x7f475010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 --2- 172.21.15.122:0/3478046797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750100a80 0x7f4750100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 --2- 172.21.15.122:0/3478046797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750106a80 0x7f4750106e60 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 -- 172.21.15.122:0/3478046797 >> 172.21.15.122:0/3478046797 conn(0x7f47500fc7a0 msgr2=0x7f47500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 -- 172.21.15.122:0/3478046797 shutdown_connections 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.927+0000 7f4756272640 1 -- 172.21.15.122:0/3478046797 wait complete. 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 Processor -- start 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 -- start start 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750100a80 0x7f4750198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 0x7f47501989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750106a80 0x7f475019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4750110ea0 con 0x7f4750101420 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4750110d20 con 0x7f4750106a80 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4756272640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4750111020 con 0x7f4750100a80 2024-09-12T02:08:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 0x7f47501989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 0x7f47501989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42206/0 (socket says 172.21.15.122:42206) 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 -- 172.21.15.122:0/2459679849 learned_addr learned my addr 172.21.15.122:0/2459679849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f475480f640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750106a80 0x7f475019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f474ffff640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750100a80 0x7f4750198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 -- 172.21.15.122:0/2459679849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750100a80 msgr2=0x7f4750198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750100a80 0x7f4750198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 -- 172.21.15.122:0/2459679849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750106a80 msgr2=0x7f475019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750106a80 0x7f475019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.928+0000 7f4747fff640 1 -- 172.21.15.122:0/2459679849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475019d440 con 0x7f4750101420 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.929+0000 7f4747fff640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 0x7f47501989d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f473800c890 tx=0x7f473800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.929+0000 7f474dffb640 1 -- 172.21.15.122:0/2459679849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738004220 con 0x7f4750101420 2024-09-12T02:08:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.929+0000 7f474dffb640 1 -- 172.21.15.122:0/2459679849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47380043c0 con 0x7f4750101420 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.929+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475019d730 con 0x7f4750101420 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.929+0000 7f474dffb640 1 -- 172.21.15.122:0/2459679849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380049e0 con 0x7f4750101420 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.929+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47501a5010 con 0x7f4750101420 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.931+0000 7f474dffb640 1 -- 172.21.15.122:0/2459679849 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f47380079b0 con 0x7f4750101420 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.931+0000 7f475864b740 1 Processor -- start 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.931+0000 7f475864b740 1 -- start start 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475864b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475864b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475864b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929ed9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475864b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d929e2d620 con 0x55d929ec8840 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475864b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d929dd4e50 con 0x55d929ed7650 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475864b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d929e295e0 con 0x55d929ec7f20 2024-09-12T02:08:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474dffb640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f473c0a2c00 0x7f473c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42220/0 (socket says 172.21.15.122:42220) 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475480f640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f473c0a2c00 0x7f473c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 -- 172.21.15.122:0/749452316 learned_addr learned my addr 172.21.15.122:0/749452316 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474dffb640 1 -- 172.21.15.122:0/2459679849 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f47380cf110 con 0x7f4750101420 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475480f640 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f4747fff640 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929ed9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 -- 172.21.15.122:0/749452316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 msgr2=0x55d929ec8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 -- 172.21.15.122:0/749452316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 msgr2=0x55d929ed9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929ed9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f474ffff640 1 -- 172.21.15.122:0/749452316 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d929dc5d40 con 0x55d929ec8840 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.932+0000 7f475480f640 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475480f640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f473c0a2c00 0x7f473c0a50c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4748009fd0 tx=0x7f4748009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f474ffff640 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed7110 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4740005ab0 tx=0x7f4740031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371d8a4d8c505ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f47467fc640 1 -- 172.21.15.122:0/749452316 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474003f070 con 0x55d929ec8840 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f47467fc640 1 -- 172.21.15.122:0/749452316 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740031a20 con 0x55d929ec8840 2024-09-12T02:08:47.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f47467fc640 1 -- 172.21.15.122:0/749452316 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740035070 con 0x55d929ec8840 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 -- 172.21.15.122:0/749452316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 msgr2=0x55d929ed7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed7110 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4740005ab0 tx=0x7f4740031630 comp rx=0 tx=0).stop 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 -- 172.21.15.122:0/749452316 shutdown_connections 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929ed9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed7110 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 --2- 172.21.15.122:0/749452316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.933+0000 7f475864b740 1 -- 172.21.15.122:0/749452316 >> 172.21.15.122:0/749452316 conn(0x55d929ec45f0 msgr2=0x55d929ec49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 -- 172.21.15.122:0/749452316 shutdown_connections 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 -- 172.21.15.122:0/749452316 wait complete. 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 -- start start 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 Processor -- start 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec7f20 0x55d929ed4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec8840 0x55d929ed4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929ed4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d929eda1c0 con 0x55d929ed7650 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d929db5f10 con 0x55d929ec7f20 2024-09-12T02:08:47.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d929eda340 con 0x55d929ec8840 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475864b740 10 client.0 _ll_register_callbacks cb 0x55d929dd9db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475480f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929ed4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475480f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929ed4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42236/0 (socket says 172.21.15.122:42236) 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.934+0000 7f475480f640 1 -- 172.21.15.122:0/4081391369 learned_addr learned my addr 172.21.15.122:0/4081391369 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f4747fff640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec8840 0x55d929ed4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f474ffff640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec7f20 0x55d929ed4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475480f640 1 -- 172.21.15.122:0/4081391369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec8840 msgr2=0x55d929ed4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475480f640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec8840 0x55d929ed4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475480f640 1 -- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec7f20 msgr2=0x55d929ed4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475480f640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec7f20 0x55d929ed4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475480f640 1 -- 172.21.15.122:0/4081391369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d929b3c580 con 0x55d929ed7650 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475480f640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929ed4ff0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4748007090 tx=0x7f4748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754c0736cfacfb76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f472f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f472f7fe640 1 -- 172.21.15.122:0/4081391369 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748029060 con 0x55d929ed7650 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f472f7fe640 1 -- 172.21.15.122:0/4081391369 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4748019920 con 0x55d929ed7650 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4740031a20 con 0x55d929ed7650 2024-09-12T02:08:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f472f7fe640 1 -- 172.21.15.122:0/4081391369 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4748013040 con 0x55d929ed7650 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475864b740 10 client.18516 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.935+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d929c06960 con 0x55d929ed7650 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f472f7fe640 1 -- 172.21.15.122:0/4081391369 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740031a20 con 0x55d929ed7650 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f472f7fe640 1 -- 172.21.15.122:0/4081391369 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f474802e070 con 0x55d929ed7650 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f475864b740 10 client.18516 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f475864b740 10 client.18516 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:08:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f475864b740 20 client.18516 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f475864b740 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929ee7230 0x55d929ee9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f475864b740 4 client.18516 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d929e3ab90 con 0x55d929ee7230 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f474ffff640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929ee7230 0x55d929ee9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f472f7fe640 10 client.18516 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.936+0000 7f474ffff640 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929ee7230 0x55d929ee9680 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c3502b867bc813 server_cookie=c2a55df5e2e5219c in_seq=0 out_seq=0 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f472f7fe640 1 -- 172.21.15.122:0/4081391369 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d929e3ab90 con 0x55d929ee7230 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f472f7fe640 10 client.18516 handle_command_reply: tid=0 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 client.18516 shutdown 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 msgr2=0x55d929ed4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929ed4ff0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4748007090 tx=0x7f4748004a40 comp rx=0 tx=0).stop 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 shutdown_connections 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929ee7230 0x55d929ee9680 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929ed4ff0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec8840 0x55d929ed4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 --2- 172.21.15.122:0/4081391369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec7f20 0x55d929ed4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f472f7fe640 0 client.18516 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 >> 172.21.15.122:0/4081391369 conn(0x55d929ec45f0 msgr2=0x55d929eca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 shutdown_connections 2024-09-12T02:08:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.937+0000 7f475864b740 1 -- 172.21.15.122:0/4081391369 wait complete. 2024-09-12T02:08:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.938+0000 7f475864b740 20 client.18516 trim_cache size 0 max 16384 2024-09-12T02:08:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 Processor -- start 2024-09-12T02:08:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 -- start start 2024-09-12T02:08:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929eeef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47501adcf0 con 0x55d929ec8840 2024-09-12T02:08:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47501ab5c0 con 0x55d929ed7650 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f475864b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47501a93f0 con 0x55d929ec7f20 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f4747fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f4747fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42240/0 (socket says 172.21.15.122:42240) 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.954+0000 7f4747fff640 1 -- 172.21.15.122:0/2861372012 learned_addr learned my addr 172.21.15.122:0/2861372012 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f474ffff640 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4747fff640 1 -- 172.21.15.122:0/2861372012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 msgr2=0x55d929ec3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f475480f640 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929eeef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4747fff640 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4747fff640 1 -- 172.21.15.122:0/2861372012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 msgr2=0x55d929eeef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4747fff640 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929eeef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4747fff640 1 -- 172.21.15.122:0/2861372012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d929ec2bb0 con 0x55d929ec8840 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f475480f640 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929eeef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4747fff640 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed4c40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4738005450 tx=0x7f4738086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97aaa54c2a134f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4745ffb640 1 -- 172.21.15.122:0/2861372012 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473801d040 con 0x55d929ec8840 2024-09-12T02:08:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4745ffb640 1 -- 172.21.15.122:0/2861372012 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473809c070 con 0x55d929ec8840 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.955+0000 7f4745ffb640 1 -- 172.21.15.122:0/2861372012 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380a34d0 con 0x55d929ec8840 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 -- 172.21.15.122:0/2861372012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 msgr2=0x55d929ed4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed4c40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4738005450 tx=0x7f4738086040 comp rx=0 tx=0).stop 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 -- 172.21.15.122:0/2861372012 shutdown_connections 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ed7650 0x55d929eeef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ec8840 0x55d929ed4c40 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 --2- 172.21.15.122:0/2861372012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929ec3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 -- 172.21.15.122:0/2861372012 >> 172.21.15.122:0/2861372012 conn(0x55d929ec45f0 msgr2=0x55d929ed4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 -- 172.21.15.122:0/2861372012 shutdown_connections 2024-09-12T02:08:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 -- 172.21.15.122:0/2861372012 wait complete. 2024-09-12T02:08:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 -- start start 2024-09-12T02:08:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.956+0000 7f475864b740 1 Processor -- start 2024-09-12T02:08:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929edfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec8840 0x55d929edc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929edc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47501a9a80 con 0x55d929ed7650 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47501aaef0 con 0x55d929ec8840 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47501aa150 con 0x55d929ec7f20 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 10 client.0 _ll_register_callbacks cb 0x7f4748014d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929edc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929edc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42244/0 (socket says 172.21.15.122:42244) 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 -- 172.21.15.122:0/1054537437 learned_addr learned my addr 172.21.15.122:0/1054537437 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:47.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f474ffff640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929edfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f4747fff640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec8840 0x55d929edc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 -- 172.21.15.122:0/1054537437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 msgr2=0x55d929edfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929edfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 -- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec8840 msgr2=0x55d929edc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec8840 0x55d929edc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 -- 172.21.15.122:0/1054537437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d929edcf40 con 0x55d929ed7650 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f4747fff640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec8840 0x55d929edc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475480f640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929edc740 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4748009c40 tx=0x7f4748019f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7094858a8b463542 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f472f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f472f7fe640 1 -- 172.21.15.122:0/1054537437 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748034020 con 0x55d929ed7650 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f472f7fe640 1 -- 172.21.15.122:0/1054537437 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f474802f5f0 con 0x55d929ed7650 2024-09-12T02:08:47.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.957+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473809c070 con 0x55d929ed7650 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f472f7fe640 1 -- 172.21.15.122:0/1054537437 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f474800a6b0 con 0x55d929ed7650 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 10 client.18522 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d929f52fc0 con 0x55d929ed7650 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f472f7fe640 1 -- 172.21.15.122:0/1054537437 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473809c070 con 0x55d929ed7650 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f472f7fe640 1 -- 172.21.15.122:0/1054537437 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f474802e070 con 0x55d929ed7650 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 10 client.18522 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 10 client.18522 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 20 client.18522 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929f53710 0x55d929f55b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 4 client.18522 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dea7d3f4", "release": true, "await": true}] 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dea7d3f4", "release": true, "await": true}) -- 0x55d929e350a0 con 0x55d929f53710 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.958+0000 7f474ffff640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929f53710 0x55d929f55b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:47.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.959+0000 7f474ffff640 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929f53710 0x55d929f55b30 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14148384cd3b69b2 server_cookie=a75cd17ce009bac in_seq=0 out_seq=0 2024-09-12T02:08:47.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:47.959+0000 7f472f7fe640 10 client.18522 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.074+0000 7f472f7fe640 1 -- 172.21.15.122:0/1054537437 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d929e350a0 con 0x55d929f53710 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f472f7fe640 10 client.18522 handle_command_reply: tid=0 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 client.18522 shutdown 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 msgr2=0x55d929edc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929edc740 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4748009c40 tx=0x7f4748019f30 comp rx=0 tx=0).stop 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 shutdown_connections 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d929f53710 0x55d929f55b30 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d929ed7650 0x55d929edc740 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d929ec8840 0x55d929edc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 --2- 172.21.15.122:0/1054537437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d929ec7f20 0x55d929edfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 >> 172.21.15.122:0/1054537437 conn(0x55d929ec45f0 msgr2=0x55d929ed4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f472f7fe640 0 client.18522 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 shutdown_connections 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 1 -- 172.21.15.122:0/1054537437 wait complete. 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.075+0000 7f475864b740 20 client.18522 trim_cache size 0 max 16384 2024-09-12T02:08:48.077 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 80, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "dea7d3f4": { 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 80, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:08:48.078 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:48.079 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:08:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f473c0a2c00 msgr2=0x7f473c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f473c0a2c00 0x7f473c0a50c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4748009fd0 tx=0x7f4748009540 comp rx=0 tx=0).stop 2024-09-12T02:08:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 msgr2=0x7f47501989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 0x7f47501989d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f473800c890 tx=0x7f473800cd50 comp rx=0 tx=0).stop 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 shutdown_connections 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f473c0a2c00 0x7f473c0a50c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750106a80 0x7f475019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4750101420 0x7f47501989d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 --2- 172.21.15.122:0/2459679849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750100a80 0x7f4750198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 >> 172.21.15.122:0/2459679849 conn(0x7f47500fc7a0 msgr2=0x7f475006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 shutdown_connections 2024-09-12T02:08:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:48.076+0000 7f4756272640 1 -- 172.21.15.122:0/2459679849 wait complete. 2024-09-12T02:08:48.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196211 19.56 MB/sec execute 395 sec latency 29785.304 ms 2024-09-12T02:08:48.941 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:48 smithi122 ceph-mon[30925]: pgmap v1010: 97 pgs: 97 active+clean; 512 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.7 MiB/s wr, 181 op/s 2024-09-12T02:08:48.971 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dea7d3f4', total seconds elapsed: 33.8 2024-09-12T02:08:48.971 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T02:08:48.971 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:08:49.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196519 19.56 MB/sec execute 396 sec latency 29792.566 ms 2024-09-12T02:08:49.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:48 smithi183 ceph-mon[35127]: pgmap v1010: 97 pgs: 97 active+clean; 512 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.7 MiB/s wr, 181 op/s 2024-09-12T02:08:49.248 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:49.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:48 smithi151 ceph-mon[35229]: pgmap v1010: 97 pgs: 97 active+clean; 512 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.7 MiB/s wr, 181 op/s 2024-09-12T02:08:49.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.776+0000 7fab6e7fc640 1 -- 172.21.15.122:0/15932905 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680319b0 con 0x7fab70106050 2024-09-12T02:08:49.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 -- 172.21.15.122:0/15932905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70106050 msgr2=0x7fab7010dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:49.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 --2- 172.21.15.122:0/15932905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70106050 0x7fab7010dbd0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fab68009900 tx=0x7fab6802a410 comp rx=0 tx=0).stop 2024-09-12T02:08:49.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 -- 172.21.15.122:0/15932905 shutdown_connections 2024-09-12T02:08:49.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 --2- 172.21.15.122:0/15932905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70106050 0x7fab7010dbd0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 --2- 172.21.15.122:0/15932905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab70101df0 0x7fab70105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 --2- 172.21.15.122:0/15932905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab70101440 0x7fab70101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 -- 172.21.15.122:0/15932905 >> 172.21.15.122:0/15932905 conn(0x7fab70077220 msgr2=0x7fab700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 -- 172.21.15.122:0/15932905 shutdown_connections 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.777+0000 7fab75e1c640 1 -- 172.21.15.122:0/15932905 wait complete. 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.778+0000 7fab75e1c640 1 Processor -- start 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.778+0000 7fab75e1c640 1 -- start start 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab75e1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab70101440 0x7fab70198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab75e1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 0x7fab70198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab75e1c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab70106050 0x7fab7019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab75e1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab70112220 con 0x7fab70101df0 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab75e1c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab701120a0 con 0x7fab70101440 2024-09-12T02:08:49.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab75e1c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab701123a0 con 0x7fab70106050 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 0x7fab70198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 0x7fab70198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42254/0 (socket says 172.21.15.122:42254) 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 -- 172.21.15.122:0/2398662689 learned_addr learned my addr 172.21.15.122:0/2398662689 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6ffff640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab70106050 0x7fab7019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6f7fe640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab70101440 0x7fab70198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 -- 172.21.15.122:0/2398662689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab70106050 msgr2=0x7fab7019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab70106050 0x7fab7019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 -- 172.21.15.122:0/2398662689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab70101440 msgr2=0x7fab70198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab70101440 0x7fab70198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.779+0000 7fab6effd640 1 -- 172.21.15.122:0/2398662689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7019d620 con 0x7fab70101df0 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.780+0000 7fab6effd640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 0x7fab70198bb0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fab6000e9f0 tx=0x7fab6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:49.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.780+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6000cd80 con 0x7fab70101df0 2024-09-12T02:08:49.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.780+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab60010910 con 0x7fab70101df0 2024-09-12T02:08:49.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.780+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7019d910 con 0x7fab70101df0 2024-09-12T02:08:49.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.781+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60018990 con 0x7fab70101df0 2024-09-12T02:08:49.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.781+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab701a51f0 con 0x7fab70101df0 2024-09-12T02:08:49.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.782+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab70102490 con 0x7fab70101df0 2024-09-12T02:08:49.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.783+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fab60018b30 con 0x7fab70101df0 2024-09-12T02:08:49.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.784+0000 7fab6cff9640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab580a2c00 0x7fab580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:49.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.784+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fab600cf310 con 0x7fab70101df0 2024-09-12T02:08:49.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.784+0000 7fab6f7fe640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab580a2c00 0x7fab580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:49.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.785+0000 7fab6f7fe640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab580a2c00 0x7fab580a50c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fab6400a400 tx=0x7fab6400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:49.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:49.790+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab6009bc20 con 0x7fab70101df0 2024-09-12T02:08:50.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.046+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab7006a570 con 0x7fab70101df0 2024-09-12T02:08:50.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.046+0000 7fab6cff9640 1 -- 172.21.15.122:0/2398662689 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7fab60086090 con 0x7fab70101df0 2024-09-12T02:08:50.047 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:08:50.047 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:08:50.047 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:08:50.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab580a2c00 msgr2=0x7fab580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:50.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab580a2c00 0x7fab580a50c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fab6400a400 tx=0x7fab6400d040 comp rx=0 tx=0).stop 2024-09-12T02:08:50.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 msgr2=0x7fab70198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:50.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 0x7fab70198bb0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fab6000e9f0 tx=0x7fab6000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 shutdown_connections 2024-09-12T02:08:50.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fab580a2c00 0x7fab580a50c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab70106050 0x7fab7019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab70101df0 0x7fab70198bb0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 --2- 172.21.15.122:0/2398662689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab70101440 0x7fab70198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:50.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 >> 172.21.15.122:0/2398662689 conn(0x7fab70077220 msgr2=0x7fab70104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:50.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 shutdown_connections 2024-09-12T02:08:50.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:50.049+0000 7fab75e1c640 1 -- 172.21.15.122:0/2398662689 wait complete. 2024-09-12T02:08:50.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196995 19.58 MB/sec execute 397 sec latency 108.065 ms 2024-09-12T02:08:50.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:50 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2398662689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:50.368 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:50 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2398662689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:50.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:50 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2398662689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:08:50.705 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T02:08:50.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T02:08:50.976 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:08:51.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197468 19.60 MB/sec execute 398 sec latency 89.729 ms 2024-09-12T02:08:51.347 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:51 smithi122 ceph-mon[30925]: pgmap v1011: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 5.2 MiB/s wr, 198 op/s 2024-09-12T02:08:51.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:51 smithi151 ceph-mon[35229]: pgmap v1011: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 5.2 MiB/s wr, 198 op/s 2024-09-12T02:08:51.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:51 smithi183 ceph-mon[35127]: pgmap v1011: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 5.2 MiB/s wr, 198 op/s 2024-09-12T02:08:51.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 -- 172.21.15.122:0/3564437496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5301036e0 msgr2=0x7fa530107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 --2- 172.21.15.122:0/3564437496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5301036e0 0x7fa530107b90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa518009930 tx=0x7fa51802a420 comp rx=0 tx=0).stop 2024-09-12T02:08:51.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 -- 172.21.15.122:0/3564437496 shutdown_connections 2024-09-12T02:08:51.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 --2- 172.21.15.122:0/3564437496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5301082a0 0x7fa53010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 --2- 172.21.15.122:0/3564437496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5301036e0 0x7fa530107b90 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 --2- 172.21.15.122:0/3564437496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa530102d30 0x7fa530103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 -- 172.21.15.122:0/3564437496 >> 172.21.15.122:0/3564437496 conn(0x7fa5300fc7d0 msgr2=0x7fa5300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 -- 172.21.15.122:0/3564437496 shutdown_connections 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.863+0000 7fa5354aa640 1 -- 172.21.15.122:0/3564437496 wait complete. 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 Processor -- start 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 -- start start 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 0x7fa530100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5301036e0 0x7fa530100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5301082a0 0x7fa530101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301120b0 con 0x7fa530102d30 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa530111f30 con 0x7fa5301082a0 2024-09-12T02:08:51.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa5354aa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa530112230 con 0x7fa5301036e0 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa52effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 0x7fa530100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa52effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 0x7fa530100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40726/0 (socket says 172.21.15.122:40726) 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa52effd640 1 -- 172.21.15.122:0/1211814033 learned_addr learned my addr 172.21.15.122:0/1211814033 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa527fff640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5301036e0 0x7fa530100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.864+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5301082a0 0x7fa530101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52effd640 1 -- 172.21.15.122:0/1211814033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5301036e0 msgr2=0x7fa530100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52effd640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5301036e0 0x7fa530100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52effd640 1 -- 172.21.15.122:0/1211814033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5301082a0 msgr2=0x7fa530101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52effd640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5301082a0 0x7fa530101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52effd640 1 -- 172.21.15.122:0/1211814033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301a6a70 con 0x7fa530102d30 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52effd640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 0x7fa530100990 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa52000e9f0 tx=0x7fa52000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52cff9640 1 -- 172.21.15.122:0/1211814033 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52000cd80 con 0x7fa530102d30 2024-09-12T02:08:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52cff9640 1 -- 172.21.15.122:0/1211814033 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa520005500 con 0x7fa530102d30 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301a6d60 con 0x7fa530102d30 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa52cff9640 1 -- 172.21.15.122:0/1211814033 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520010690 con 0x7fa530102d30 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.865+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a71c0 con 0x7fa530102d30 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.868+0000 7fa52cff9640 1 -- 172.21.15.122:0/1211814033 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa5200046c0 con 0x7fa530102d30 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.868+0000 7fa537883740 1 Processor -- start 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.868+0000 7fa537883740 1 -- start start 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.868+0000 7fa52cff9640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.868+0000 7fa537883740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a825b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.868+0000 7fa537883740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a830400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa537883740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a832d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa537883740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55664a78b2a0 con 0x55664a826080 2024-09-12T02:08:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa537883740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55664a732b10 con 0x55664a830940 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa537883740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55664a787260 con 0x55664a825760 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa527fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a830400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52cff9640 1 -- 172.21.15.122:0/1211814033 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa5200cf3c0 con 0x7fa530102d30 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa527fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a830400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40734/0 (socket says 172.21.15.122:40734) 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a825b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa527fff640 1 -- 172.21.15.122:0/1096536496 learned_addr learned my addr 172.21.15.122:0/1096536496 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a832d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 -- 172.21.15.122:0/1096536496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 msgr2=0x55664a832d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a832d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 -- 172.21.15.122:0/1096536496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 msgr2=0x55664a830400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a830400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 -- 172.21.15.122:0/1096536496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664a4fcb90 con 0x55664a825760 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa527fff640 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a830400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52effd640 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a825b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c6a0 tx=0x7fa52009b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18730481ec8e2e4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.869+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa528009fd0 tx=0x7fa528009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.870+0000 7fa5257fa640 1 -- 172.21.15.122:0/1096536496 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200b5000 con 0x55664a825760 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.870+0000 7fa5257fa640 1 -- 172.21.15.122:0/1096536496 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52009c320 con 0x55664a825760 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.870+0000 7fa5257fa640 1 -- 172.21.15.122:0/1096536496 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200b4e20 con 0x55664a825760 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.870+0000 7fa537883740 1 -- 172.21.15.122:0/1096536496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 msgr2=0x55664a825b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.870+0000 7fa537883740 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a825b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c6a0 tx=0x7fa52009b760 comp rx=0 tx=0).stop 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- 172.21.15.122:0/1096536496 shutdown_connections 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a832d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a830400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 --2- 172.21.15.122:0/1096536496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a825b40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- 172.21.15.122:0/1096536496 >> 172.21.15.122:0/1096536496 conn(0x55664a821cf0 msgr2=0x55664a8220d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- 172.21.15.122:0/1096536496 shutdown_connections 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- 172.21.15.122:0/1096536496 wait complete. 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- start start 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 Processor -- start 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a82d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a82dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a82e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55664a8334b0 con 0x55664a826080 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55664a7136e0 con 0x55664a830940 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55664a833630 con 0x55664a825760 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.871+0000 7fa537883740 10 client.0 _ll_register_callbacks cb 0x55664a737a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:51.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a82dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a82dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40744/0 (socket says 172.21.15.122:40744) 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 -- 172.21.15.122:0/1295120424 learned_addr learned my addr 172.21.15.122:0/1295120424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a82e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa52effd640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a82d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 -- 172.21.15.122:0/1295120424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 msgr2=0x55664a82d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a82d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 -- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 msgr2=0x55664a82e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a82e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 -- 172.21.15.122:0/1295120424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664a83c9a0 con 0x55664a826080 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa52effd640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a82d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa527fff640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a82dd60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa518009230 tx=0x7fa518041e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed5e096fbc3f7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa5167fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:51.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa5167fc640 1 -- 172.21.15.122:0/1295120424 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518035040 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa5167fc640 1 -- 172.21.15.122:0/1295120424 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa518004320 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.872+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52009c320 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa5167fc640 1 -- 172.21.15.122:0/1295120424 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa518005020 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 10 client.18540 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55664a83f380 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa5167fc640 1 -- 172.21.15.122:0/1295120424 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52009c320 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa5167fc640 1 -- 172.21.15.122:0/1295120424 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa5180035f0 con 0x55664a826080 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 10 client.18540 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 10 client.18540 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 20 client.18540 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a840c60 0x55664a843120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 4 client.18540 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:51.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55664a7989d0 con 0x55664a840c60 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.873+0000 7fa52effd640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a840c60 0x55664a843120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa52effd640 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a840c60 0x55664a843120 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c49597375fab706 server_cookie=fd1e561bb0d90dde in_seq=0 out_seq=0 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa5167fc640 10 client.18540 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa5167fc640 1 -- 172.21.15.122:0/1295120424 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55664a7989d0 con 0x55664a840c60 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa5167fc640 10 client.18540 handle_command_reply: tid=0 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 client.18540 shutdown 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 msgr2=0x55664a82dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a82dd60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa518009230 tx=0x7fa518041e60 comp rx=0 tx=0).stop 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 shutdown_connections 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a840c60 0x55664a843120 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a82e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a82dd60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 --2- 172.21.15.122:0/1295120424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a825760 0x55664a82d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 >> 172.21.15.122:0/1295120424 conn(0x55664a821cf0 msgr2=0x55664a827a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa5167fc640 0 client.18540 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.874+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 shutdown_connections 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.875+0000 7fa537883740 1 -- 172.21.15.122:0/1295120424 wait complete. 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.875+0000 7fa537883740 20 client.18540 trim_cache size 0 max 16384 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.892+0000 7fa537883740 1 Processor -- start 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 -- start start 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 0x55664a82aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a826080 0x55664a82b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a8487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301adbc0 con 0x55664a825760 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301ab490 con 0x55664a830940 2024-09-12T02:08:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa537883740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301a92c0 con 0x55664a826080 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 0x55664a82aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 0x55664a82aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40748/0 (socket says 172.21.15.122:40748) 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 -- 172.21.15.122:0/3702811817 learned_addr learned my addr 172.21.15.122:0/3702811817 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa527fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a826080 0x55664a82b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a8487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 -- 172.21.15.122:0/3702811817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a826080 msgr2=0x55664a82b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a826080 0x55664a82b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 -- 172.21.15.122:0/3702811817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 msgr2=0x55664a8487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a8487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52effd640 1 -- 172.21.15.122:0/3702811817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664a82c810 con 0x55664a825760 2024-09-12T02:08:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa527fff640 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a826080 0x55664a82b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.893+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a8487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.894+0000 7fa52effd640 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 0x55664a82aae0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa52000e9c0 tx=0x7fa520004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360f821f101234b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.894+0000 7fa524ff9640 1 -- 172.21.15.122:0/3702811817 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52009c320 con 0x55664a825760 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.894+0000 7fa524ff9640 1 -- 172.21.15.122:0/3702811817 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52009ffd0 con 0x55664a825760 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.894+0000 7fa524ff9640 1 -- 172.21.15.122:0/3702811817 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520098290 con 0x55664a825760 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.894+0000 7fa537883740 1 -- 172.21.15.122:0/3702811817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 msgr2=0x55664a82aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.894+0000 7fa537883740 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 0x55664a82aae0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa52000e9c0 tx=0x7fa520004a40 comp rx=0 tx=0).stop 2024-09-12T02:08:51.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 -- 172.21.15.122:0/3702811817 shutdown_connections 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a830940 0x55664a8487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a826080 0x55664a82b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 --2- 172.21.15.122:0/3702811817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a825760 0x55664a82aae0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 -- 172.21.15.122:0/3702811817 >> 172.21.15.122:0/3702811817 conn(0x55664a821cf0 msgr2=0x55664a82f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 -- 172.21.15.122:0/3702811817 shutdown_connections 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 -- 172.21.15.122:0/3702811817 wait complete. 2024-09-12T02:08:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 -- start start 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 Processor -- start 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.895+0000 7fa537883740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 0x55664a846f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa537883740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a8474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa537883740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a830940 0x55664a8a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa537883740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301a9950 con 0x55664a826080 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa537883740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301aadc0 con 0x55664a825760 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa537883740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301aa020 con 0x55664a830940 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa537883740 10 client.0 _ll_register_callbacks cb 0x7fa518041b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 0x55664a846f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 0x55664a846f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:51602/0 (socket says 172.21.15.122:51602) 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 -- 172.21.15.122:0/4028569569 learned_addr learned my addr 172.21.15.122:0/4028569569 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a830940 0x55664a8a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa527fff640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a8474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 -- 172.21.15.122:0/4028569569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a830940 msgr2=0x55664a8a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a830940 0x55664a8a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 -- 172.21.15.122:0/4028569569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 msgr2=0x55664a8474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a8474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 -- 172.21.15.122:0/4028569569 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664a825eb0 con 0x55664a825760 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52f7fe640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a830940 0x55664a8a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa52effd640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 0x55664a846f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa520098080 tx=0x7fa520099f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a49632366f291d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.896+0000 7fa5167fc640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa5167fc640 1 -- 172.21.15.122:0/4028569569 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52009c320 con 0x55664a825760 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa5167fc640 1 -- 172.21.15.122:0/4028569569 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa520098290 con 0x55664a825760 2024-09-12T02:08:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52009ffd0 con 0x55664a825760 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa5167fc640 1 -- 172.21.15.122:0/4028569569 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa5200a4400 con 0x55664a825760 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 10 client.27509 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55664a8ac0d0 con 0x55664a825760 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa5167fc640 1 -- 172.21.15.122:0/4028569569 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52009ffd0 con 0x55664a825760 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa5167fc640 1 -- 172.21.15.122:0/4028569569 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fa52009b070 con 0x55664a825760 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 10 client.27509 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 10 client.27509 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 20 client.27509 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a8ac350 0x55664a8ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 4 client.27509 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T02:08:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.897+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55664a792d20 con 0x55664a8ac350 2024-09-12T02:08:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.898+0000 7fa527fff640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a8ac350 0x55664a8ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.898+0000 7fa5167fc640 10 client.27509 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:51.898+0000 7fa527fff640 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a8ac350 0x55664a8ae740 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e5525d75736995 server_cookie=91c9fc28806dfe4b in_seq=0 out_seq=0 2024-09-12T02:08:52.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197804 19.58 MB/sec execute 399 sec latency 173.464 ms 2024-09-12T02:08:52.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.885+0000 7fa5167fc640 1 -- 172.21.15.122:0/4028569569 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55664a792d20 con 0x55664a8ac350 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.885+0000 7fa5167fc640 10 client.27509 handle_command_reply: tid=0 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.885+0000 7fa537883740 1 client.27509 shutdown 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 msgr2=0x55664a846f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 0x55664a846f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa520098080 tx=0x7fa520099f20 comp rx=0 tx=0).stop 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 shutdown_connections 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55664a8ac350 0x55664a8ae740 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55664a830940 0x55664a8a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa5167fc640 0 client.27509 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55664a826080 0x55664a8474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 --2- 172.21.15.122:0/4028569569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55664a825760 0x55664a846f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 >> 172.21.15.122:0/4028569569 conn(0x55664a821cf0 msgr2=0x55664a820650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 shutdown_connections 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 1 -- 172.21.15.122:0/4028569569 wait complete. 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.886+0000 7fa537883740 20 client.27509 trim_cache size 0 max 16384 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 82, 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout: "70bccc12": { 2024-09-12T02:08:52.889 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 82, 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:08:52.890 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:08:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa51c0a2c00 msgr2=0x7fa51c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa528009fd0 tx=0x7fa528009da0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 msgr2=0x7fa530100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 0x7fa530100990 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa52000e9f0 tx=0x7fa52000edf0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 shutdown_connections 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa51c0a2c00 0x7fa51c0a50c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5301082a0 0x7fa530101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5301036e0 0x7fa530100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 --2- 172.21.15.122:0/1211814033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa530102d30 0x7fa530100990 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 >> 172.21.15.122:0/1211814033 conn(0x7fa5300fc7d0 msgr2=0x7fa530106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 shutdown_connections 2024-09-12T02:08:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:08:52.887+0000 7fa5354aa640 1 -- 172.21.15.122:0/1211814033 wait complete. 2024-09-12T02:08:53.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.54 MB/sec execute 400 sec latency 1114.268 ms 2024-09-12T02:08:53.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:52 smithi183 ceph-mon[35127]: pgmap v1012: 97 pgs: 97 active+clean; 511 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 218 op/s 2024-09-12T02:08:53.222 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:52 smithi122 ceph-mon[30925]: pgmap v1012: 97 pgs: 97 active+clean; 511 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 218 op/s 2024-09-12T02:08:53.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:52 smithi151 ceph-mon[35229]: pgmap v1012: 97 pgs: 97 active+clean; 511 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 218 op/s 2024-09-12T02:08:53.585 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '70bccc12', quiesce took 4.6 seconds. Will release after: 25.4 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 -- 172.21.15.122:0/3757109978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8048101c30 msgr2=0x7f80481020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 --2- 172.21.15.122:0/3757109978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8048101c30 0x7f80481020b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8038009900 tx=0x7f803802a410 comp rx=0 tx=0).stop 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 -- 172.21.15.122:0/3757109978 shutdown_connections 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 --2- 172.21.15.122:0/3757109978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80481025f0 0x7f8048108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 --2- 172.21.15.122:0/3757109978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8048101c30 0x7f80481020b0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 --2- 172.21.15.122:0/3757109978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048100a40 0x7f8048100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 -- 172.21.15.122:0/3757109978 >> 172.21.15.122:0/3757109978 conn(0x7f80480fc1e0 msgr2=0x7f80480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 -- 172.21.15.122:0/3757109978 shutdown_connections 2024-09-12T02:08:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.938+0000 7f804f474640 1 -- 172.21.15.122:0/3757109978 wait complete. 2024-09-12T02:08:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 Processor -- start 2024-09-12T02:08:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 -- start start 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8048100a40 0x7f804806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048101c30 0x7f804819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 0x7f804819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f804810c6c0 con 0x7f80481025f0 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f804810c540 con 0x7f8048100a40 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804f474640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f804810c840 con 0x7f8048101c30 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804da11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 0x7f804819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804da11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 0x7f804819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40758/0 (socket says 172.21.15.122:40758) 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804da11640 1 -- 172.21.15.122:0/1723243855 learned_addr learned my addr 172.21.15.122:0/1723243855 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804d210640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8048100a40 0x7f804806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.939+0000 7f804ca0f640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048101c30 0x7f804819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804da11640 1 -- 172.21.15.122:0/1723243855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048101c30 msgr2=0x7f804819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804da11640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048101c30 0x7f804819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804da11640 1 -- 172.21.15.122:0/1723243855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8048100a40 msgr2=0x7f804806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804da11640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8048100a40 0x7f804806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804da11640 1 -- 172.21.15.122:0/1723243855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819fa60 con 0x7f80481025f0 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804da11640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 0x7f804819f150 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f804000eb10 tx=0x7f804000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f803e7fc640 1 -- 172.21.15.122:0/1723243855 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040004b20 con 0x7f80481025f0 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f803e7fc640 1 -- 172.21.15.122:0/1723243855 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8040004cc0 con 0x7f80481025f0 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f803e7fc640 1 -- 172.21.15.122:0/1723243855 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040010760 con 0x7f80481025f0 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819fcf0 con 0x7f80481025f0 2024-09-12T02:08:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.940+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481a01c0 con 0x7f80481025f0 2024-09-12T02:08:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.942+0000 7f803e7fc640 1 -- 172.21.15.122:0/1723243855 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8040005020 con 0x7f80481025f0 2024-09-12T02:08:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 Processor -- start 2024-09-12T02:08:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 -- start start 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 0x55a628d4e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d50c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a628ca82f0 con 0x55a628d496f0 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a628c50270 con 0x55a628d48db0 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f8052215740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a628ca42b0 con 0x55a628d4e780 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 0x55a628d4e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f803e7fc640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80340a2cb0 0x7f80340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 0x55a628d4e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40768/0 (socket says 172.21.15.122:40768) 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 -- 172.21.15.122:0/516097066 learned_addr learned my addr 172.21.15.122:0/516097066 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804ca0f640 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804ca0f640 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d50c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804da11640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80340a2cb0 0x7f80340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 -- 172.21.15.122:0/516097066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 msgr2=0x55a628d50c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d50c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 -- 172.21.15.122:0/516097066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 msgr2=0x55a628d491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f804d210640 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f804d210640 1 -- 172.21.15.122:0/516097066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a628c4f340 con 0x55a628d496f0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.943+0000 7f803e7fc640 1 -- 172.21.15.122:0/1723243855 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f80400cf570 con 0x7f80481025f0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f804ca0f640 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d50c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f804d210640 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 0x55a628d4e1b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f804400bcb0 tx=0x7f80440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a03f24aed11f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f804da11640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80340a2cb0 0x7f80340a5170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f804009bd60 tx=0x7f804006bc80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f80337fe640 1 -- 172.21.15.122:0/516097066 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044005020 con 0x55a628d496f0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f80337fe640 1 -- 172.21.15.122:0/516097066 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80440047a0 con 0x55a628d496f0 2024-09-12T02:08:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.944+0000 7f80337fe640 1 -- 172.21.15.122:0/516097066 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044011780 con 0x55a628d496f0 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- 172.21.15.122:0/516097066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 msgr2=0x55a628d4e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 0x55a628d4e1b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f804400bcb0 tx=0x7f80440107a0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- 172.21.15.122:0/516097066 shutdown_connections 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d50c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d496f0 0x55a628d4e1b0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- 172.21.15.122:0/516097066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- 172.21.15.122:0/516097066 >> 172.21.15.122:0/516097066 conn(0x55a628d40010 msgr2=0x55a628d40410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- 172.21.15.122:0/516097066 shutdown_connections 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- 172.21.15.122:0/516097066 wait complete. 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- start start 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 Processor -- start 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d52fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 0x55a628d5a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a628d51750 con 0x55a628d4e780 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a628c31290 con 0x55a628d48db0 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.945+0000 7f8052215740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a628d518d0 con 0x55a628d496f0 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8052215740 10 client.0 _ll_register_callbacks cb 0x55a628c62140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 0x55a628d5a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 0x55a628d5a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40776/0 (socket says 172.21.15.122:40776) 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 -- 172.21.15.122:0/130598612 learned_addr learned my addr 172.21.15.122:0/130598612 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d52fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804d210640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 -- 172.21.15.122:0/130598612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 msgr2=0x55a628d534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 -- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 msgr2=0x55a628d52fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d52fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 -- 172.21.15.122:0/130598612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6289bb210 con 0x55a628d4e780 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f804ca0f640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 0x55a628d5a520 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f80380096c0 tx=0x7f803802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c757eec39ebcfd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8030ff9640 1 -- 172.21.15.122:0/130598612 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038035070 con 0x55a628d4e780 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8030ff9640 1 -- 172.21.15.122:0/130598612 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8038004220 con 0x55a628d4e780 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80440047a0 con 0x55a628d4e780 2024-09-12T02:08:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8030ff9640 1 -- 172.21.15.122:0/130598612 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8038051050 con 0x55a628d4e780 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8052215740 10 client.18558 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a628a87620 con 0x55a628d4e780 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.946+0000 7f8030ff9640 1 -- 172.21.15.122:0/130598612 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80440047a0 con 0x55a628d4e780 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8030ff9640 1 -- 172.21.15.122:0/130598612 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f8038004780 con 0x55a628d4e780 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8052215740 10 client.18558 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8052215740 10 client.18558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8052215740 20 client.18558 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8052215740 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628d5ebf0 0x55a628d61080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8052215740 4 client.18558 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a628cb5a20 con 0x55a628d5ebf0 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f804d210640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628d5ebf0 0x55a628d61080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f8030ff9640 10 client.18558 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.947+0000 7f804d210640 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628d5ebf0 0x55a628d61080 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4007d4bd04217de9 server_cookie=bb41020d64da4dab in_seq=0 out_seq=0 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.948+0000 7f8030ff9640 1 -- 172.21.15.122:0/130598612 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a628cb5a20 con 0x55a628d5ebf0 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.948+0000 7f8030ff9640 10 client.18558 handle_command_reply: tid=0 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.948+0000 7f8052215740 1 client.18558 shutdown 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.948+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 msgr2=0x55a628d5a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.948+0000 7f8052215740 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 0x55a628d5a520 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f80380096c0 tx=0x7f803802a410 comp rx=0 tx=0).stop 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 shutdown_connections 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628d5ebf0 0x55a628d61080 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d4e780 0x55a628d5a520 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8030ff9640 0 client.18558 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 --2- 172.21.15.122:0/130598612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d48db0 0x55a628d52fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 >> 172.21.15.122:0/130598612 conn(0x55a628d40010 msgr2=0x55a628d41820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 shutdown_connections 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 1 -- 172.21.15.122:0/130598612 wait complete. 2024-09-12T02:08:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.949+0000 7f8052215740 20 client.18558 trim_cache size 0 max 16384 2024-09-12T02:08:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 Processor -- start 2024-09-12T02:08:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 -- start start 2024-09-12T02:08:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d40ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d5a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d4e780 0x55a628d41400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80481a95a0 con 0x55a628d48db0 2024-09-12T02:08:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f80481a6e70 con 0x55a628d4e780 2024-09-12T02:08:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8052215740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80481a4ca0 con 0x55a628d496f0 2024-09-12T02:08:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d40ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d40ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40792/0 (socket says 172.21.15.122:40792) 2024-09-12T02:08:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 -- 172.21.15.122:0/1165823378 learned_addr learned my addr 172.21.15.122:0/1165823378 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804d210640 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d5a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 -- 172.21.15.122:0/1165823378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 msgr2=0x55a628d5a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d5a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 -- 172.21.15.122:0/1165823378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d4e780 msgr2=0x55a628d41400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d4e780 0x55a628d41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 -- 172.21.15.122:0/1165823378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a628d49bd0 con 0x55a628d48db0 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f804ca0f640 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d40ec0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8038009850 tx=0x7f8038031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144157754aa4bee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.963+0000 7f8032ffd640 1 -- 172.21.15.122:0/1165823378 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803805a070 con 0x55a628d48db0 2024-09-12T02:08:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8032ffd640 1 -- 172.21.15.122:0/1165823378 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038004980 con 0x55a628d48db0 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8032ffd640 1 -- 172.21.15.122:0/1165823378 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803804e070 con 0x55a628d48db0 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 -- 172.21.15.122:0/1165823378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 msgr2=0x55a628d40ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d40ec0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8038009850 tx=0x7f8038031ad0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 -- 172.21.15.122:0/1165823378 shutdown_connections 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d4e780 0x55a628d41400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d496f0 0x55a628d5a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 --2- 172.21.15.122:0/1165823378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d40ec0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 -- 172.21.15.122:0/1165823378 >> 172.21.15.122:0/1165823378 conn(0x55a628d40010 msgr2=0x55a628d52fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 -- 172.21.15.122:0/1165823378 shutdown_connections 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 -- 172.21.15.122:0/1165823378 wait complete. 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 -- start start 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.964+0000 7f8052215740 1 Processor -- start 2024-09-12T02:08:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d496f0 0x55a628d57f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d58e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80481a5330 con 0x55a628d48db0 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f80481a67a0 con 0x55a628d496f0 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80481a5a00 con 0x55a628d4e780 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 10 client.0 _ll_register_callbacks cb 0x7f8038009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40802/0 (socket says 172.21.15.122:40802) 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 -- 172.21.15.122:0/171606989 learned_addr learned my addr 172.21.15.122:0/171606989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804d210640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d496f0 0x55a628d57f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d58e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 -- 172.21.15.122:0/171606989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 msgr2=0x55a628d58e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d58e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 -- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d496f0 msgr2=0x55a628d57f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d496f0 0x55a628d57f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 -- 172.21.15.122:0/171606989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a628d59470 con 0x55a628d48db0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804d210640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d496f0 0x55a628d57f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f804ca0f640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d579d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f803802a450 tx=0x7f803803e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ba30fb5ce59dab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8030ff9640 1 -- 172.21.15.122:0/171606989 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803804e070 con 0x55a628d48db0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8030ff9640 1 -- 172.21.15.122:0/171606989 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8038004250 con 0x55a628d48db0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.965+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8038004980 con 0x55a628d48db0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8030ff9640 1 -- 172.21.15.122:0/171606989 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f803805a040 con 0x55a628d48db0 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 10 client.18564 fetch_fsmap learned FSMap version 45 2024-09-12T02:08:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a628d5a780 con 0x55a628d48db0 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8030ff9640 1 -- 172.21.15.122:0/171606989 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038004980 con 0x55a628d48db0 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8030ff9640 1 -- 172.21.15.122:0/171606989 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f8038003710 con 0x55a628d48db0 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 10 client.18564 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 10 client.18564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 20 client.18564 populate_metadata read hostname 'smithi122' 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628dc9fd0 0x55a628dcc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 4 client.18564 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a628cafd70 con 0x55a628dc9fd0 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.966+0000 7f804d210640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628dc9fd0 0x55a628dcc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.967+0000 7f804d210640 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628dc9fd0 0x55a628dcc3f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae7e6b1aa15c4556 server_cookie=ff7e78299210125a in_seq=0 out_seq=0 2024-09-12T02:08:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:53.967+0000 7f8030ff9640 10 client.18564 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8030ff9640 1 -- 172.21.15.122:0/171606989 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a628cafd70 con 0x55a628dc9fd0 2024-09-12T02:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8030ff9640 10 client.18564 handle_command_reply: tid=0 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 client.18564 shutdown 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 msgr2=0x55a628d579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d579d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f803802a450 tx=0x7f803803e920 comp rx=0 tx=0).stop 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 shutdown_connections 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55a628dc9fd0 0x55a628dcc3f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a628d4e780 0x55a628d58e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a628d496f0 0x55a628d57f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 --2- 172.21.15.122:0/171606989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a628d48db0 0x55a628d579d0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8030ff9640 0 client.18564 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 >> 172.21.15.122:0/171606989 conn(0x55a628d40010 msgr2=0x55a628d3e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 shutdown_connections 2024-09-12T02:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 1 -- 172.21.15.122:0/171606989 wait complete. 2024-09-12T02:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f8052215740 20 client.18564 trim_cache size 0 max 16384 2024-09-12T02:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80340a2cb0 msgr2=0x7f80340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80340a2cb0 0x7f80340a5170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f804009bd60 tx=0x7f804006bc80 comp rx=0 tx=0).stop 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 msgr2=0x7f804819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 0x7f804819f150 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f804000eb10 tx=0x7f804000c2d0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 shutdown_connections 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f80340a2cb0 0x7f80340a5170 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80481025f0 0x7f804819f150 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048101c30 0x7f804819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 --2- 172.21.15.122:0/1723243855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8048100a40 0x7f804806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.070+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 >> 172.21.15.122:0/1723243855 conn(0x7f80480fc1e0 msgr2=0x7f8048104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.071+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 shutdown_connections 2024-09-12T02:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:08:54.071+0000 7f804f474640 1 -- 172.21.15.122:0/1723243855 wait complete. 2024-09-12T02:08:54.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.49 MB/sec execute 401 sec latency 2114.311 ms 2024-09-12T02:08:55.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.44 MB/sec execute 402 sec latency 3114.414 ms 2024-09-12T02:08:55.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:54 smithi151 ceph-mon[35229]: pgmap v1013: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 340 op/s 2024-09-12T02:08:55.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:54 smithi183 ceph-mon[35127]: pgmap v1013: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 340 op/s 2024-09-12T02:08:55.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:54 smithi122 ceph-mon[30925]: pgmap v1013: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 340 op/s 2024-09-12T02:08:56.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.39 MB/sec execute 403 sec latency 4114.515 ms 2024-09-12T02:08:57.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.34 MB/sec execute 404 sec latency 5114.603 ms 2024-09-12T02:08:57.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:56 smithi183 ceph-mon[35127]: pgmap v1014: 97 pgs: 97 active+clean; 509 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 301 op/s 2024-09-12T02:08:57.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:56 smithi122 ceph-mon[30925]: pgmap v1014: 97 pgs: 97 active+clean; 509 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 301 op/s 2024-09-12T02:08:57.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:56 smithi151 ceph-mon[35229]: pgmap v1014: 97 pgs: 97 active+clean; 509 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 301 op/s 2024-09-12T02:08:58.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.30 MB/sec execute 405 sec latency 6114.712 ms 2024-09-12T02:08:59.081 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.25 MB/sec execute 406 sec latency 7114.821 ms 2024-09-12T02:08:59.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:08:58 smithi183 ceph-mon[35127]: pgmap v1015: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 292 op/s 2024-09-12T02:08:59.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:08:58 smithi122 ceph-mon[30925]: pgmap v1015: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 292 op/s 2024-09-12T02:08:59.366 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:08:58 smithi151 ceph-mon[35229]: pgmap v1015: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 292 op/s 2024-09-12T02:09:00.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.20 MB/sec execute 407 sec latency 8114.910 ms 2024-09-12T02:09:01.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.15 MB/sec execute 408 sec latency 9115.017 ms 2024-09-12T02:09:01.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:00 smithi183 ceph-mon[35127]: pgmap v1016: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-09-12T02:09:01.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:00 smithi122 ceph-mon[30925]: pgmap v1016: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-09-12T02:09:01.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:00 smithi151 ceph-mon[35229]: pgmap v1016: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-09-12T02:09:02.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.11 MB/sec execute 409 sec latency 10115.134 ms 2024-09-12T02:09:03.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.06 MB/sec execute 410 sec latency 11115.203 ms 2024-09-12T02:09:03.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:02 smithi183 ceph-mon[35127]: pgmap v1017: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-09-12T02:09:03.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:02 smithi122 ceph-mon[30925]: pgmap v1017: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-09-12T02:09:03.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:02 smithi151 ceph-mon[35229]: pgmap v1017: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-09-12T02:09:04.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 19.01 MB/sec execute 411 sec latency 12115.236 ms 2024-09-12T02:09:05.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.97 MB/sec execute 412 sec latency 13115.361 ms 2024-09-12T02:09:05.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:04 smithi183 ceph-mon[35127]: pgmap v1018: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-12T02:09:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:04 smithi122 ceph-mon[30925]: pgmap v1018: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-12T02:09:05.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:04 smithi151 ceph-mon[35229]: pgmap v1018: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-12T02:09:06.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.92 MB/sec execute 413 sec latency 14115.443 ms 2024-09-12T02:09:07.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.88 MB/sec execute 414 sec latency 15115.491 ms 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 -- 172.21.15.122:0/437277420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 msgr2=0x7f71141020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 --2- 172.21.15.122:0/437277420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f71141020d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009900 tx=0x7f70fc02a410 comp rx=0 tx=0).stop 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 -- 172.21.15.122:0/437277420 shutdown_connections 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 --2- 172.21.15.122:0/437277420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7114102610 0x7f7114108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 --2- 172.21.15.122:0/437277420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f71141020d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 --2- 172.21.15.122:0/437277420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7114100a60 0x7f7114100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 -- 172.21.15.122:0/437277420 >> 172.21.15.122:0/437277420 conn(0x7f71140fc1e0 msgr2=0x7f71140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 -- 172.21.15.122:0/437277420 shutdown_connections 2024-09-12T02:09:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.167+0000 7f7118c06640 1 -- 172.21.15.122:0/437277420 wait complete. 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 Processor -- start 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 -- start start 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7114100a60 0x7f7114197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f7114198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7114102610 0x7f711419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d830 con 0x7f7114101c50 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d6b0 con 0x7f7114102610 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7118c06640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f711410d9b0 con 0x7f7114100a60 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f7114198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f7114198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57396/0 (socket says 172.21.15.122:57396) 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7111d9b640 1 -- 172.21.15.122:0/583384658 learned_addr learned my addr 172.21.15.122:0/583384658 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f7112d9d640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7114102610 0x7f711419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.168+0000 7f711259c640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7114100a60 0x7f7114197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7111d9b640 1 -- 172.21.15.122:0/583384658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7114100a60 msgr2=0x7f7114197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7111d9b640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7114100a60 0x7f7114197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7111d9b640 1 -- 172.21.15.122:0/583384658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7114102610 msgr2=0x7f711419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7111d9b640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7114102610 0x7f711419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7111d9b640 1 -- 172.21.15.122:0/583384658 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711419fa60 con 0x7f7114101c50 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7111d9b640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f7114198380 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009230 tx=0x7f70fc031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f710b7fe640 1 -- 172.21.15.122:0/583384658 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc03f030 con 0x7f7114101c50 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f710b7fe640 1 -- 172.21.15.122:0/583384658 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc003ad0 con 0x7f7114101c50 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711419fcf0 con 0x7f7114101c50 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f710b7fe640 1 -- 172.21.15.122:0/583384658 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc035070 con 0x7f7114101c50 2024-09-12T02:09:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.169+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141a01d0 con 0x7f7114101c50 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.171+0000 7f710b7fe640 1 -- 172.21.15.122:0/583384658 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f70fc042030 con 0x7f7114101c50 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 Processor -- start 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 -- start start 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3946cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec39471da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec39474830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec393d02f0 con 0x55ec39472370 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec39378270 con 0x55ec3946d2e0 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711ba82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec393cc2b0 con 0x55ec3946c9a0 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec39474830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f710b7fe640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71000a2cb0 0x7f71000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec39474830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57410/0 (socket says 172.21.15.122:57410) 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f7111d9b640 1 -- 172.21.15.122:0/1579254437 learned_addr learned my addr 172.21.15.122:0/1579254437 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f711259c640 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec39471da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f7112d9d640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71000a2cb0 0x7f71000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f710b7fe640 1 -- 172.21.15.122:0/583384658 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f70fc0ef4b0 con 0x7f7114101c50 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.172+0000 7f7112d9d640 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3946cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f711259c640 1 -- 172.21.15.122:0/1579254437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 msgr2=0x55ec3946cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f711259c640 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3946cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f711259c640 1 -- 172.21.15.122:0/1579254437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 msgr2=0x55ec39474830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f711259c640 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec39474830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f711259c640 1 -- 172.21.15.122:0/1579254437 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec39377340 con 0x55ec3946d2e0 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f7111d9b640 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec39474830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f7112d9d640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71000a2cb0 0x7f71000a5170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f710c00a780 tx=0x7f710c005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f711259c640 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec39471da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7104009bd0 tx=0x7f7104011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5aa451b24a39fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f7108ff9640 1 -- 172.21.15.122:0/1579254437 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104004230 con 0x55ec3946d2e0 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f7108ff9640 1 -- 172.21.15.122:0/1579254437 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7104009e40 con 0x55ec3946d2e0 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.173+0000 7f7108ff9640 1 -- 172.21.15.122:0/1579254437 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040197e0 con 0x55ec3946d2e0 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 -- 172.21.15.122:0/1579254437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 msgr2=0x55ec39471da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec39471da0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7104009bd0 tx=0x7f7104011810 comp rx=0 tx=0).stop 2024-09-12T02:09:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 -- 172.21.15.122:0/1579254437 shutdown_connections 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec39474830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec39471da0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 --2- 172.21.15.122:0/1579254437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3946cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 -- 172.21.15.122:0/1579254437 >> 172.21.15.122:0/1579254437 conn(0x55ec39468010 msgr2=0x55ec39468410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 -- 172.21.15.122:0/1579254437 shutdown_connections 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 -- 172.21.15.122:0/1579254437 wait complete. 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 -- start start 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.174+0000 7f711ba82740 1 Processor -- start 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec39476bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 0x55ec39477110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec3947e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec39475340 con 0x55ec3946d2e0 2024-09-12T02:09:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec39359290 con 0x55ec39472370 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec394754c0 con 0x55ec3946c9a0 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 10 client.0 _ll_register_callbacks cb 0x55ec3938a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 0x55ec39477110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 0x55ec39477110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57422/0 (socket says 172.21.15.122:57422) 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 -- 172.21.15.122:0/2450853475 learned_addr learned my addr 172.21.15.122:0/2450853475 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711259c640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec39476bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7112d9d640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec3947e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 -- 172.21.15.122:0/2450853475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 msgr2=0x55ec39476bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec39476bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 -- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 msgr2=0x55ec3947e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec3947e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 -- 172.21.15.122:0/2450853475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec390e3210 con 0x55ec3946d2e0 2024-09-12T02:09:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7112d9d640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec3947e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f7111d9b640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 0x55ec39477110 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0314e0 tx=0x7f70fc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9559f5a636aa1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f70f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f70f9ffb640 1 -- 172.21.15.122:0/2450853475 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc049040 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f70f9ffb640 1 -- 172.21.15.122:0/2450853475 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f70fc0b8430 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.175+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ec393dda20 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f70f9ffb640 1 -- 172.21.15.122:0/2450853475 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f70fc0c00c0 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 10 client.18576 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ec391af620 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f70f9ffb640 1 -- 172.21.15.122:0/2450853475 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ec393dda20 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f70f9ffb640 1 -- 172.21.15.122:0/2450853475 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f70fc0ef0d0 con 0x55ec3946d2e0 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 10 client.18576 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 10 client.18576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 20 client.18576 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec39482720 0x55ec39484b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 4 client.18576 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec393d7d70 con 0x55ec39482720 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.176+0000 7f711259c640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec39482720 0x55ec39484b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.177+0000 7f711259c640 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec39482720 0x55ec39484b30 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a904750223442ad server_cookie=d58e92cdb05bd5a in_seq=0 out_seq=0 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.177+0000 7f70f9ffb640 10 client.18576 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.177+0000 7f70f9ffb640 1 -- 172.21.15.122:0/2450853475 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec393d7d70 con 0x55ec39482720 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.177+0000 7f70f9ffb640 10 client.18576 handle_command_reply: tid=0 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.177+0000 7f711ba82740 1 client.18576 shutdown 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 msgr2=0x55ec39477110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 0x55ec39477110 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0314e0 tx=0x7f70fc009650 comp rx=0 tx=0).stop 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 shutdown_connections 2024-09-12T02:09:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec39482720 0x55ec39484b30 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec3947e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946d2e0 0x55ec39477110 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 --2- 172.21.15.122:0/2450853475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec39476bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f70f9ffb640 0 client.18576 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 >> 172.21.15.122:0/2450853475 conn(0x55ec39468010 msgr2=0x55ec39469820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 shutdown_connections 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 1 -- 172.21.15.122:0/2450853475 wait complete. 2024-09-12T02:09:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.178+0000 7f711ba82740 20 client.18576 trim_cache size 0 max 16384 2024-09-12T02:09:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 Processor -- start 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 -- start start 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 0x55ec39468ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946d2e0 0x55ec3947e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec39469400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71141ab8a0 con 0x55ec3946c9a0 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71141a9170 con 0x55ec39472370 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711ba82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71141a6fa0 con 0x55ec3946d2e0 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 0x55ec39468ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 0x55ec39468ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57430/0 (socket says 172.21.15.122:57430) 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 -- 172.21.15.122:0/732194338 learned_addr learned my addr 172.21.15.122:0/732194338 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f7112d9d640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec39469400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 -- 172.21.15.122:0/732194338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946d2e0 msgr2=0x55ec3947e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f7111d9b640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946d2e0 0x55ec3947e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946d2e0 0x55ec3947e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 -- 172.21.15.122:0/732194338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 msgr2=0x55ec39469400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec39469400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f711259c640 1 -- 172.21.15.122:0/732194338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec39468ca0 con 0x55ec3946c9a0 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f7112d9d640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec39469400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.192+0000 7f7111d9b640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946d2e0 0x55ec3947e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711259c640 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 0x55ec39468ec0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7104004240 tx=0x7f7104011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b83df478f6b131 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f70fbfff640 1 -- 172.21.15.122:0/732194338 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104030020 con 0x55ec3946c9a0 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f70fbfff640 1 -- 172.21.15.122:0/732194338 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71040035f0 con 0x55ec3946c9a0 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f70fbfff640 1 -- 172.21.15.122:0/732194338 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710402c520 con 0x55ec3946c9a0 2024-09-12T02:09:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 -- 172.21.15.122:0/732194338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 msgr2=0x55ec39468ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 0x55ec39468ec0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7104004240 tx=0x7f7104011810 comp rx=0 tx=0).stop 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 -- 172.21.15.122:0/732194338 shutdown_connections 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec39472370 0x55ec39469400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946d2e0 0x55ec3947e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 --2- 172.21.15.122:0/732194338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec3946c9a0 0x55ec39468ec0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 -- 172.21.15.122:0/732194338 >> 172.21.15.122:0/732194338 conn(0x55ec39468010 msgr2=0x55ec39476bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 -- 172.21.15.122:0/732194338 shutdown_connections 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.193+0000 7f711ba82740 1 -- 172.21.15.122:0/732194338 wait complete. 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 -- start start 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 Processor -- start 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3947b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec3947be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec3947ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71141a7630 con 0x55ec39472370 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71141a8aa0 con 0x55ec3946d2e0 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71141a7d00 con 0x55ec3946c9a0 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711ba82740 10 client.0 _ll_register_callbacks cb 0x7f70fc0ecef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec3947ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec3947ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57446/0 (socket says 172.21.15.122:57446) 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 -- 172.21.15.122:0/349306303 learned_addr learned my addr 172.21.15.122:0/349306303 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f711259c640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3947b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7111d9b640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec3947be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 -- 172.21.15.122:0/349306303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 msgr2=0x55ec3947b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3947b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 -- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 msgr2=0x55ec3947be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec3947be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 -- 172.21.15.122:0/349306303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3947d480 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7111d9b640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec3947be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f7112d9d640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec3947ce70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f710c014c80 tx=0x7f710c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbecad37af112c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.194+0000 7f70f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f70f9ffb640 1 -- 172.21.15.122:0/349306303 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c026070 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f70f9ffb640 1 -- 172.21.15.122:0/349306303 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f710c02b660 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71040035f0 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f70f9ffb640 1 -- 172.21.15.122:0/349306303 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f710c02b860 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 10 client.18582 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ec394ed540 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f70f9ffb640 1 -- 172.21.15.122:0/349306303 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71040035f0 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f70f9ffb640 1 -- 172.21.15.122:0/349306303 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f710c0224a0 con 0x55ec39472370 2024-09-12T02:09:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 10 client.18582 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 20 client.18582 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec394ede40 0x55ec394f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 4 client.18582 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec39354900 con 0x55ec394ede40 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.195+0000 7f711259c640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec394ede40 0x55ec394f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.196+0000 7f70f9ffb640 10 client.18582 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.196+0000 7f711259c640 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec394ede40 0x55ec394f0260 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aeed72941252e3f server_cookie=54a7daa45d68be7c in_seq=0 out_seq=0 2024-09-12T02:09:07.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:06 smithi183 ceph-mon[35127]: pgmap v1019: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 67 KiB/s wr, 38 op/s 2024-09-12T02:09:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f70f9ffb640 1 -- 172.21.15.122:0/349306303 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec39354900 con 0x55ec394ede40 2024-09-12T02:09:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f70f9ffb640 10 client.18582 handle_command_reply: tid=0 2024-09-12T02:09:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 client.18582 shutdown 2024-09-12T02:09:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 msgr2=0x55ec3947ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec3947ce70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f710c014c80 tx=0x7f710c004a40 comp rx=0 tx=0).stop 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 shutdown_connections 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ec394ede40 0x55ec394f0260 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec39472370 0x55ec3947ce70 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f70f9ffb640 0 client.18582 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec3946d2e0 0x55ec3947be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 --2- 172.21.15.122:0/349306303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec3946c9a0 0x55ec3947b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 >> 172.21.15.122:0/349306303 conn(0x55ec39468010 msgr2=0x55ec394669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 shutdown_connections 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.222+0000 7f711ba82740 1 -- 172.21.15.122:0/349306303 wait complete. 2024-09-12T02:09:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f711ba82740 20 client.18582 trim_cache size 0 max 16384 2024-09-12T02:09:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:09:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:09:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:09:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71000a2cb0 msgr2=0x7f71000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71000a2cb0 0x7f71000a5170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f710c00a780 tx=0x7f710c005f60 comp rx=0 tx=0).stop 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 msgr2=0x7f7114198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f7114198380 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009230 tx=0x7f70fc031b60 comp rx=0 tx=0).stop 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 shutdown_connections 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f71000a2cb0 0x7f71000a5170 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7114102610 0x7f711419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7114101c50 0x7f7114198380 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 --2- 172.21.15.122:0/583384658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7114100a60 0x7f7114197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 >> 172.21.15.122:0/583384658 conn(0x7f71140fc1e0 msgr2=0x7f7114104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 shutdown_connections 2024-09-12T02:09:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:07.223+0000 7f7118c06640 1 -- 172.21.15.122:0/583384658 wait complete. 2024-09-12T02:09:07.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:06 smithi122 ceph-mon[30925]: pgmap v1019: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 67 KiB/s wr, 38 op/s 2024-09-12T02:09:07.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:06 smithi151 ceph-mon[35229]: pgmap v1019: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 67 KiB/s wr, 38 op/s 2024-09-12T02:09:08.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.83 MB/sec execute 415 sec latency 16115.611 ms 2024-09-12T02:09:09.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.79 MB/sec execute 416 sec latency 17115.686 ms 2024-09-12T02:09:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:08 smithi183 ceph-mon[35127]: pgmap v1020: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-12T02:09:09.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:08 smithi122 ceph-mon[30925]: pgmap v1020: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-12T02:09:09.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:08 smithi151 ceph-mon[35229]: pgmap v1020: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-12T02:09:10.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.74 MB/sec execute 417 sec latency 18115.731 ms 2024-09-12T02:09:10.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:09 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:10.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:09 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:10.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:09 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:11.082 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.70 MB/sec execute 418 sec latency 19115.803 ms 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:11 smithi183 ceph-mon[35127]: pgmap v1021: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-09-12T02:09:11.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:11.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:11.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:11 smithi122 ceph-mon[30925]: pgmap v1021: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-09-12T02:09:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:09:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:11 smithi151 ceph-mon[35229]: pgmap v1021: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 9.8 KiB/s wr, 15 op/s 2024-09-12T02:09:12.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.65 MB/sec execute 419 sec latency 20115.902 ms 2024-09-12T02:09:13.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.61 MB/sec execute 420 sec latency 21116.023 ms 2024-09-12T02:09:13.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:12 smithi183 ceph-mon[35127]: pgmap v1022: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-12T02:09:13.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:12 smithi122 ceph-mon[30925]: pgmap v1022: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-12T02:09:13.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:12 smithi151 ceph-mon[35229]: pgmap v1022: 97 pgs: 97 active+clean; 508 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-12T02:09:14.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.56 MB/sec execute 421 sec latency 22116.095 ms 2024-09-12T02:09:15.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.52 MB/sec execute 422 sec latency 23116.180 ms 2024-09-12T02:09:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:14 smithi183 ceph-mon[35127]: pgmap v1023: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 262 KiB/s wr, 16 op/s 2024-09-12T02:09:15.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:14 smithi122 ceph-mon[30925]: pgmap v1023: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 262 KiB/s wr, 16 op/s 2024-09-12T02:09:15.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:14 smithi151 ceph-mon[35229]: pgmap v1023: 97 pgs: 97 active+clean; 510 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 262 KiB/s wr, 16 op/s 2024-09-12T02:09:16.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.47 MB/sec execute 423 sec latency 24116.310 ms 2024-09-12T02:09:17.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.43 MB/sec execute 424 sec latency 25116.393 ms 2024-09-12T02:09:17.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:16 smithi183 ceph-mon[35127]: pgmap v1024: 97 pgs: 97 active+clean; 513 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 824 KiB/s wr, 25 op/s 2024-09-12T02:09:17.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:16 smithi122 ceph-mon[30925]: pgmap v1024: 97 pgs: 97 active+clean; 513 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 824 KiB/s wr, 25 op/s 2024-09-12T02:09:17.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:16 smithi151 ceph-mon[35229]: pgmap v1024: 97 pgs: 97 active+clean; 513 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 824 KiB/s wr, 25 op/s 2024-09-12T02:09:18.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.39 MB/sec execute 425 sec latency 26116.485 ms 2024-09-12T02:09:18.986 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:09:19.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.34 MB/sec execute 426 sec latency 27116.558 ms 2024-09-12T02:09:19.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:18 smithi183 ceph-mon[35127]: pgmap v1025: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 46 op/s 2024-09-12T02:09:19.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:18 smithi122 ceph-mon[30925]: pgmap v1025: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 46 op/s 2024-09-12T02:09:19.242 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:19.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:18 smithi151 ceph-mon[35229]: pgmap v1025: 97 pgs: 97 active+clean; 515 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 46 op/s 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 -- 172.21.15.122:0/4021414019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102570 msgr2=0x7f6b781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/4021414019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102570 0x7f6b781029b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009900 tx=0x7f6b6002a480 comp rx=0 tx=0).stop 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 -- 172.21.15.122:0/4021414019 shutdown_connections 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/4021414019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b78102ef0 0x7f6b7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/4021414019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102570 0x7f6b781029b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/4021414019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b78100e20 0x7f6b78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 -- 172.21.15.122:0/4021414019 >> 172.21.15.122:0/4021414019 conn(0x7f6b780fc7d0 msgr2=0x7f6b780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 -- 172.21.15.122:0/4021414019 shutdown_connections 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.729+0000 7f6b7cedb640 1 -- 172.21.15.122:0/4021414019 wait complete. 2024-09-12T02:09:19.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 Processor -- start 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 -- start start 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b78100e20 0x7f6b780fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b78102570 0x7f6b780fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 0x7f6b780fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78110f80 con 0x7f6b78102ef0 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78110e00 con 0x7f6b78100e20 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7cedb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78111100 con 0x7f6b78102570 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 0x7f6b780fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 0x7f6b780fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54506/0 (socket says 172.21.15.122:54506) 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 -- 172.21.15.122:0/1038731804 learned_addr learned my addr 172.21.15.122:0/1038731804 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:19.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b6ffff640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b78102570 0x7f6b780fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b7659c640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b78100e20 0x7f6b780fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 -- 172.21.15.122:0/1038731804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b78102570 msgr2=0x7f6b780fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b78102570 0x7f6b780fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 -- 172.21.15.122:0/1038731804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b78100e20 msgr2=0x7f6b780fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b78100e20 0x7f6b780fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.730+0000 7f6b76d9d640 1 -- 172.21.15.122:0/1038731804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b781a68b0 con 0x7f6b78102ef0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.731+0000 7f6b76d9d640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 0x7f6b780fd700 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000cce0 tx=0x7f6b7000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.731+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000f450 con 0x7f6b78102ef0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.731+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7000fad0 con 0x7f6b78102ef0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.731+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70018c10 con 0x7f6b78102ef0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.731+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b781a6ba0 con 0x7f6b78102ef0 2024-09-12T02:09:19.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.731+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b781a7060 con 0x7f6b78102ef0 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.732+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b78101200 con 0x7f6b78102ef0 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.733+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6b7000f5f0 con 0x7f6b78102ef0 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.734+0000 7f6b6f7fe640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6b640a2b80 0x7f6b640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.734+0000 7f6b7659c640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6b640a2b80 0x7f6b640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.734+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6b700cf700 con 0x7f6b78102ef0 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.734+0000 7f6b7659c640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6b640a2b80 0x7f6b640a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005e10 tx=0x7f6b68005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:19.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.737+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b70014030 con 0x7f6b78102ef0 2024-09-12T02:09:19.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.951+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b780fe1b0 con 0x7f6b78102ef0 2024-09-12T02:09:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.952+0000 7f6b6f7fe640 1 -- 172.21.15.122:0/1038731804 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7f6b780fe1b0 con 0x7f6b78102ef0 2024-09-12T02:09:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:09:19.953 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:09:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:09:19.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6b640a2b80 msgr2=0x7f6b640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:19.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6b640a2b80 0x7f6b640a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005e10 tx=0x7f6b68005da0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 msgr2=0x7f6b780fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:19.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 0x7f6b780fd700 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000cce0 tx=0x7f6b7000a620 comp rx=0 tx=0).stop 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 shutdown_connections 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f6b640a2b80 0x7f6b640a5040 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b78102ef0 0x7f6b780fd700 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b78102570 0x7f6b780fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 --2- 172.21.15.122:0/1038731804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b78100e20 0x7f6b780fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 >> 172.21.15.122:0/1038731804 conn(0x7f6b780fc7d0 msgr2=0x7f6b7806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 shutdown_connections 2024-09-12T02:09:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:19.956+0000 7f6b7cedb640 1 -- 172.21.15.122:0/1038731804 wait complete. 2024-09-12T02:09:20.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.30 MB/sec execute 427 sec latency 28116.590 ms 2024-09-12T02:09:20.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1038731804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 -- 172.21.15.122:0/1393308379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c104fb0 msgr2=0x7f541c1073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 --2- 172.21.15.122:0/1393308379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c104fb0 0x7f541c1073a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f5408009900 tx=0x7f540802a410 comp rx=0 tx=0).stop 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 -- 172.21.15.122:0/1393308379 shutdown_connections 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 --2- 172.21.15.122:0/1393308379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c1078e0 0x7f541c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 --2- 172.21.15.122:0/1393308379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c104fb0 0x7f541c1073a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 --2- 172.21.15.122:0/1393308379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c102680 0x7f541c104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 -- 172.21.15.122:0/1393308379 >> 172.21.15.122:0/1393308379 conn(0x7f541c0fc1e0 msgr2=0x7f541c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 -- 172.21.15.122:0/1393308379 shutdown_connections 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.312+0000 7f5422bd8640 1 -- 172.21.15.122:0/1393308379 wait complete. 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 Processor -- start 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 -- start start 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 0x7f541c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c104fb0 0x7f541c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1078e0 0x7f541c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f541c10d7c0 con 0x7f541c102680 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f541c10d640 con 0x7f541c104fb0 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5422bd8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f541c10d940 con 0x7f541c1078e0 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 0x7f541c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 0x7f541c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57872/0 (socket says 172.21.15.122:57872) 2024-09-12T02:09:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 -- 172.21.15.122:0/734648179 learned_addr learned my addr 172.21.15.122:0/734648179 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f541bfff640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c104fb0 0x7f541c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5421175640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1078e0 0x7f541c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 -- 172.21.15.122:0/734648179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1078e0 msgr2=0x7f541c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1078e0 0x7f541c19f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 -- 172.21.15.122:0/734648179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c104fb0 msgr2=0x7f541c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.313+0000 7f5420974640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c104fb0 0x7f541c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5420974640 1 -- 172.21.15.122:0/734648179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c19fa70 con 0x7f541c102680 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5420974640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 0x7f541c197e50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f541000c970 tx=0x7f541000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5419ffb640 1 -- 172.21.15.122:0/734648179 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410007bb0 con 0x7f541c102680 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5419ffb640 1 -- 172.21.15.122:0/734648179 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5410007d50 con 0x7f541c102680 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5419ffb640 1 -- 172.21.15.122:0/734648179 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541000f450 con 0x7f541c102680 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c19fd60 con 0x7f541c102680 2024-09-12T02:09:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.314+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1a01d0 con 0x7f541c102680 2024-09-12T02:09:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.316+0000 7f5419ffb640 1 -- 172.21.15.122:0/734648179 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5410005020 con 0x7f541c102680 2024-09-12T02:09:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.317+0000 7f5419ffb640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f540c0a2c30 0x7f540c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.317+0000 7f5419ffb640 1 -- 172.21.15.122:0/734648179 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f54100cef10 con 0x7f541c102680 2024-09-12T02:09:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.317+0000 7f541bfff640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f540c0a2c30 0x7f540c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.317+0000 7f5425a4c740 1 Processor -- start 2024-09-12T02:09:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.317+0000 7f5425a4c740 1 -- start start 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20d9a0 0x55ca9f20dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20e2e0 0x55ca9f212da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 0x55ca9f215830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5425a4c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f1712f0 con 0x55ca9f213370 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5425a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f119270 con 0x55ca9f20d9a0 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5425a4c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f16d2b0 con 0x55ca9f20e2e0 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 0x55ca9f215830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f541bfff640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f540c0a2c30 0x7f540c0a50f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5408006010 tx=0x7f5408000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 0x55ca9f215830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57876/0 (socket says 172.21.15.122:57876) 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 -- 172.21.15.122:0/782992168 learned_addr learned my addr 172.21.15.122:0/782992168 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5420974640 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20d9a0 0x55ca9f20dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f541bfff640 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20e2e0 0x55ca9f212da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 -- 172.21.15.122:0/782992168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20e2e0 msgr2=0x55ca9f212da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20e2e0 0x55ca9f212da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 -- 172.21.15.122:0/782992168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20d9a0 msgr2=0x55ca9f20dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20d9a0 0x55ca9f20dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 -- 172.21.15.122:0/782992168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9f118340 con 0x55ca9f213370 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5421175640 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 0x55ca9f215830 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5414008d00 tx=0x7f541400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7875d1851c73c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5406ffd640 1 -- 172.21.15.122:0/782992168 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414018680 con 0x55ca9f213370 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5406ffd640 1 -- 172.21.15.122:0/782992168 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541400f900 con 0x55ca9f213370 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.318+0000 7f5406ffd640 1 -- 172.21.15.122:0/782992168 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541400ea10 con 0x55ca9f213370 2024-09-12T02:09:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 -- 172.21.15.122:0/782992168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 msgr2=0x55ca9f215830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 0x55ca9f215830 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5414008d00 tx=0x7f541400d710 comp rx=0 tx=0).stop 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 -- 172.21.15.122:0/782992168 shutdown_connections 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f213370 0x55ca9f215830 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20e2e0 0x55ca9f212da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 --2- 172.21.15.122:0/782992168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20d9a0 0x55ca9f20dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.319+0000 7f5425a4c740 1 -- 172.21.15.122:0/782992168 >> 172.21.15.122:0/782992168 conn(0x55ca9f209010 msgr2=0x55ca9f209410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 -- 172.21.15.122:0/782992168 shutdown_connections 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 -- 172.21.15.122:0/782992168 wait complete. 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 -- start start 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 Processor -- start 2024-09-12T02:09:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f217bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f218100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f21f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f216340 con 0x55ca9f20e2e0 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f0fa290 con 0x55ca9f213370 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.320+0000 7f5425a4c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca9f2164c0 con 0x55ca9f20d9a0 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f5425a4c740 10 client.0 _ll_register_callbacks cb 0x55ca9f12b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f218100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f218100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57880/0 (socket says 172.21.15.122:57880) 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 -- 172.21.15.122:0/238395641 learned_addr learned my addr 172.21.15.122:0/238395641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f5421175640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f21f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f5420974640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f217bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 -- 172.21.15.122:0/238395641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 msgr2=0x55ca9f217bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f217bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 -- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 msgr2=0x55ca9f21f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f21f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 -- 172.21.15.122:0/238395641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9ee84210 con 0x55ca9f20e2e0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f541bfff640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f218100 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f54080096c0 tx=0x7f5408009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49960ddb0eb6735 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f53fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f53fbfff640 1 -- 172.21.15.122:0/238395641 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408035050 con 0x55ca9f20e2e0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f53fbfff640 1 -- 172.21.15.122:0/238395641 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5408003c00 con 0x55ca9f20e2e0 2024-09-12T02:09:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541400f900 con 0x55ca9f20e2e0 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f53fbfff640 1 -- 172.21.15.122:0/238395641 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5408003e00 con 0x55ca9f20e2e0 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f5425a4c740 10 client.18600 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.321+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ca9ef50620 con 0x55ca9f20e2e0 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f53fbfff640 1 -- 172.21.15.122:0/238395641 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541400f900 con 0x55ca9f20e2e0 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f53fbfff640 1 -- 172.21.15.122:0/238395641 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f540803b690 con 0x55ca9f20e2e0 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f5425a4c740 10 client.18600 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f5425a4c740 10 client.18600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f5425a4c740 20 client.18600 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f5425a4c740 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f223830 0x55ca9f225c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f5425a4c740 4 client.18600 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.322+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca9f17ea20 con 0x55ca9f223830 2024-09-12T02:09:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.323+0000 7f5420974640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f223830 0x55ca9f225c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.323+0000 7f5420974640 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f223830 0x55ca9f225c20 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9496a46a09edf5e server_cookie=ad9fe7e183e42af8 in_seq=0 out_seq=0 2024-09-12T02:09:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.323+0000 7f53fbfff640 10 client.18600 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f53fbfff640 1 -- 172.21.15.122:0/238395641 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca9f17ea20 con 0x55ca9f223830 2024-09-12T02:09:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f53fbfff640 10 client.18600 handle_command_reply: tid=0 2024-09-12T02:09:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 client.18600 shutdown 2024-09-12T02:09:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 msgr2=0x55ca9f218100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f218100 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f54080096c0 tx=0x7f5408009260 comp rx=0 tx=0).stop 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 shutdown_connections 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f223830 0x55ca9f225c20 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f21f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f218100 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f53fbfff640 0 client.18600 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 --2- 172.21.15.122:0/238395641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f217bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 >> 172.21.15.122:0/238395641 conn(0x55ca9f209010 msgr2=0x55ca9f20a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 shutdown_connections 2024-09-12T02:09:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 1 -- 172.21.15.122:0/238395641 wait complete. 2024-09-12T02:09:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.324+0000 7f5425a4c740 20 client.18600 trim_cache size 0 max 16384 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 Processor -- start 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 -- start start 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f209ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f20a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ab9c0 con 0x55ca9f20e2e0 2024-09-12T02:09:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1a9290 con 0x55ca9f213370 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.338+0000 7f5425a4c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1a70c0 con 0x55ca9f20d9a0 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57896/0 (socket says 172.21.15.122:57896) 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 -- 172.21.15.122:0/750910313 learned_addr learned my addr 172.21.15.122:0/750910313 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f5420974640 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f209ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f5421175640 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f20a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 -- 172.21.15.122:0/750910313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 msgr2=0x55ca9f209ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f209ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 -- 172.21.15.122:0/750910313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 msgr2=0x55ca9f20a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f20a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 -- 172.21.15.122:0/750910313 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9f2127a0 con 0x55ca9f20e2e0 2024-09-12T02:09:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f541bfff640 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21f100 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5408031610 tx=0x7f5408031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5139132faf523424 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f54067fc640 1 -- 172.21.15.122:0/750910313 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408035070 con 0x55ca9f20e2e0 2024-09-12T02:09:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f54067fc640 1 -- 172.21.15.122:0/750910313 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5408060070 con 0x55ca9f20e2e0 2024-09-12T02:09:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.339+0000 7f54067fc640 1 -- 172.21.15.122:0/750910313 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408049aa0 con 0x55ca9f20e2e0 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 -- 172.21.15.122:0/750910313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 msgr2=0x55ca9f21f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21f100 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5408031610 tx=0x7f5408031b60 comp rx=0 tx=0).stop 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 -- 172.21.15.122:0/750910313 shutdown_connections 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f213370 0x55ca9f20a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21f100 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 --2- 172.21.15.122:0/750910313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f20d9a0 0x55ca9f209ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 -- 172.21.15.122:0/750910313 >> 172.21.15.122:0/750910313 conn(0x55ca9f209010 msgr2=0x55ca9f20f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 -- 172.21.15.122:0/750910313 shutdown_connections 2024-09-12T02:09:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 -- 172.21.15.122:0/750910313 wait complete. 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 -- start start 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.340+0000 7f5425a4c740 1 Processor -- start 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 0x55ca9f21cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f213370 0x55ca9f21e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1a7750 con 0x55ca9f20d9a0 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1a8bc0 con 0x55ca9f20e2e0 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1a7e20 con 0x55ca9f213370 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5425a4c740 10 client.0 _ll_register_callbacks cb 0x7f540803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 0x55ca9f21cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 0x55ca9f21cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57912/0 (socket says 172.21.15.122:57912) 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 -- 172.21.15.122:0/2009578323 learned_addr learned my addr 172.21.15.122:0/2009578323 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f541bfff640 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 -- 172.21.15.122:0/2009578323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f213370 msgr2=0x55ca9f21e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f213370 0x55ca9f21e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 -- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20e2e0 msgr2=0x55ca9f21d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 -- 172.21.15.122:0/2009578323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9f21e830 con 0x55ca9f20d9a0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f5420974640 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 0x55ca9f21cd40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f54100cc9d0 tx=0x7f54100ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaf4e57f5c67b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f53fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.341+0000 7f53fbfff640 1 -- 172.21.15.122:0/2009578323 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541001d040 con 0x55ca9f20d9a0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f53fbfff640 1 -- 172.21.15.122:0/2009578323 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5410099a30 con 0x55ca9f20d9a0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5408060070 con 0x55ca9f20d9a0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f53fbfff640 1 -- 172.21.15.122:0/2009578323 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f54100ccae0 con 0x55ca9f20d9a0 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 10 client.18606 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ca9f28ea60 con 0x55ca9f20d9a0 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f53fbfff640 1 -- 172.21.15.122:0/2009578323 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5408060070 con 0x55ca9f20d9a0 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f53fbfff640 1 -- 172.21.15.122:0/2009578323 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f541009ab80 con 0x55ca9f20d9a0 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 10 client.18606 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 20 client.18606 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f28f410 0x55ca9f291850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 4 client.18606 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca9f2130c0 con 0x55ca9f28f410 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.342+0000 7f541bfff640 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f28f410 0x55ca9f291850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.343+0000 7f541bfff640 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f28f410 0x55ca9f291850 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7808629f7ce5e3f9 server_cookie=ba36a107c8347083 in_seq=0 out_seq=0 2024-09-12T02:09:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.343+0000 7f53fbfff640 10 client.18606 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:20.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1038731804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.393+0000 7f53fbfff640 1 -- 172.21.15.122:0/2009578323 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca9f2130c0 con 0x55ca9f28f410 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.393+0000 7f53fbfff640 10 client.18606 handle_command_reply: tid=0 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.393+0000 7f5425a4c740 1 client.18606 shutdown 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.393+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 msgr2=0x55ca9f21cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 0x55ca9f21cd40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f54100cc9d0 tx=0x7f54100ce040 comp rx=0 tx=0).stop 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 shutdown_connections 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca9f28f410 0x55ca9f291850 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca9f213370 0x55ca9f21e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca9f20e2e0 0x55ca9f21d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 --2- 172.21.15.122:0/2009578323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca9f20d9a0 0x55ca9f21cd40 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f53fbfff640 0 client.18606 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 >> 172.21.15.122:0/2009578323 conn(0x55ca9f209010 msgr2=0x55ca9f2079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 shutdown_connections 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 1 -- 172.21.15.122:0/2009578323 wait complete. 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5425a4c740 20 client.18606 trim_cache size 0 max 16384 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:09:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:09:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f540c0a2c30 msgr2=0x7f540c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f540c0a2c30 0x7f540c0a50f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5408006010 tx=0x7f5408000bc0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 msgr2=0x7f541c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 0x7f541c197e50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f541000c970 tx=0x7f541000ce30 comp rx=0 tx=0).stop 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 shutdown_connections 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f540c0a2c30 0x7f540c0a50f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1078e0 0x7f541c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c104fb0 0x7f541c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 --2- 172.21.15.122:0/734648179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f541c102680 0x7f541c197e50 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.394+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 >> 172.21.15.122:0/734648179 conn(0x7f541c0fc1e0 msgr2=0x7f541c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.395+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 shutdown_connections 2024-09-12T02:09:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:20.395+0000 7f5422bd8640 1 -- 172.21.15.122:0/734648179 wait complete. 2024-09-12T02:09:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1038731804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:09:20.599 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 70bccc12 --release --await' 2024-09-12T02:09:20.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 70bccc12 --release --await 2024-09-12T02:09:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:21.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197827 18.26 MB/sec execute 428 sec latency 29116.676 ms 2024-09-12T02:09:21.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:21 smithi122 ceph-mon[30925]: pgmap v1026: 97 pgs: 97 active+clean; 516 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 48 op/s 2024-09-12T02:09:21.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- 172.21.15.122:0/410074266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc105f40 msgr2=0x7f9bcc1063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/410074266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc105f40 0x7f9bcc1063d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009ca0 tx=0x7f9bc402a940 comp rx=0 tx=0).stop 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- 172.21.15.122:0/410074266 shutdown_connections 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/410074266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc105f40 0x7f9bcc1063d0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/410074266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bcc101450 0x7f9bcc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/410074266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bcc100b30 0x7f9bcc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- 172.21.15.122:0/410074266 >> 172.21.15.122:0/410074266 conn(0x7f9bcc0fc7a0 msgr2=0x7f9bcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- 172.21.15.122:0/410074266 shutdown_connections 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- 172.21.15.122:0/410074266 wait complete. 2024-09-12T02:09:21.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 Processor -- start 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- start start 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 0x7f9bcc1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bcc101450 0x7f9bcc1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bcc105f40 0x7f9bcc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc118560 con 0x7f9bcc100b30 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1183e0 con 0x7f9bcc101450 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.348+0000 7f9bd3c24640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1186e0 con 0x7f9bcc105f40 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 0x7f9bcc1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 0x7f9bcc1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57942/0 (socket says 172.21.15.122:57942) 2024-09-12T02:09:21.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 -- 172.21.15.122:0/2901658066 learned_addr learned my addr 172.21.15.122:0/2901658066 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bcc101450 0x7f9bcc1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bcc105f40 0x7f9bcc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 -- 172.21.15.122:0/2901658066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bcc105f40 msgr2=0x7f9bcc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bcc105f40 0x7f9bcc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 -- 172.21.15.122:0/2901658066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bcc101450 msgr2=0x7f9bcc1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bcc101450 0x7f9bcc1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 -- 172.21.15.122:0/2901658066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc19efe0 con 0x7f9bcc100b30 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd19c0640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 0x7f9bcc1a46c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800e9f0 tx=0x7f9bc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bc2ffd640 1 -- 172.21.15.122:0/2901658066 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800cd80 con 0x7f9bcc100b30 2024-09-12T02:09:21.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bc2ffd640 1 -- 172.21.15.122:0/2901658066 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc8010910 con 0x7f9bcc100b30 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bc2ffd640 1 -- 172.21.15.122:0/2901658066 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8018990 con 0x7f9bcc100b30 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bcc19f2d0 con 0x7f9bcc100b30 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.349+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bcc1ab520 con 0x7f9bcc100b30 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.352+0000 7f9bc2ffd640 1 -- 172.21.15.122:0/2901658066 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9bc8010ac0 con 0x7f9bcc100b30 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.352+0000 7f9bd5ffd740 1 Processor -- start 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.352+0000 7f9bd5ffd740 1 -- start start 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe67440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67980 0x55623fe71d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55623fdcc620 con 0x55623fe67060 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55623fd73e50 con 0x55623fe72240 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55623fdc85e0 con 0x55623fe67980 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe67440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe67440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57946/0 (socket says 172.21.15.122:57946) 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 -- 172.21.15.122:0/3677528511 learned_addr learned my addr 172.21.15.122:0/3677528511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd19c0640 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67980 0x55623fe71d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 -- 172.21.15.122:0/3677528511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67980 msgr2=0x55623fe71d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67980 0x55623fe71d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 -- 172.21.15.122:0/3677528511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 msgr2=0x55623fe74630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe74630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 -- 172.21.15.122:0/3677528511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623fd64d40 con 0x55623fe67060 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bc2ffd640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bb80a2c00 0x7f9bb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd19c0640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bb80a2c00 0x7f9bb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe67440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4005f70 tx=0x7f9bc40093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62ea21826707eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bc2ffd640 1 -- 172.21.15.122:0/2901658066 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9bc80cf360 con 0x7f9bcc100b30 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bb7fff640 1 -- 172.21.15.122:0/3677528511 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4034040 con 0x55623fe67060 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bb7fff640 1 -- 172.21.15.122:0/3677528511 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc402f590 con 0x55623fe67060 2024-09-12T02:09:21.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.353+0000 7f9bb7fff640 1 -- 172.21.15.122:0/3677528511 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40308a0 con 0x55623fe67060 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/3677528511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 msgr2=0x55623fe67440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd19c0640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bb80a2c00 0x7f9bb80a50c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9bc801e000 tx=0x7f9bc80d2de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe67440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4005f70 tx=0x7f9bc40093f0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/3677528511 shutdown_connections 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe74630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67980 0x55623fe71d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/3677528511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe67440 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/3677528511 >> 172.21.15.122:0/3677528511 conn(0x55623fe635f0 msgr2=0x55623fe639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/3677528511 shutdown_connections 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.354+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/3677528511 wait complete. 2024-09-12T02:09:21.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 -- start start 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 Processor -- start 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe6f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67980 0x55623fe6f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe6fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55623fe74e10 con 0x55623fe67060 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55623fd54f10 con 0x55623fe67980 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55623fe74f90 con 0x55623fe72240 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd5ffd740 10 client.0 _ll_register_callbacks cb 0x55623fd78db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd21c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe6f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd21c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe6f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57960/0 (socket says 172.21.15.122:57960) 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd21c1640 1 -- 172.21.15.122:0/2461743279 learned_addr learned my addr 172.21.15.122:0/2461743279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe6fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.355+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67980 0x55623fe6f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd21c1640 1 -- 172.21.15.122:0/2461743279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 msgr2=0x55623fe6fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe6fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd21c1640 1 -- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67980 msgr2=0x55623fe6f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67980 0x55623fe6f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd21c1640 1 -- 172.21.15.122:0/2461743279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623fadb580 con 0x55623fe67060 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67980 0x55623fe6f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe6f180 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4003e30 tx=0x7f9bc4003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8e29063418adf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:21.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bb57fa640 1 -- 172.21.15.122:0/2461743279 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4034040 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bb57fa640 1 -- 172.21.15.122:0/2461743279 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9bc4030a80 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc402f590 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bb57fa640 1 -- 172.21.15.122:0/2461743279 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9bc402f740 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 10 client.18618 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55623fba5960 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bb57fa640 1 -- 172.21.15.122:0/2461743279 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc402f590 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bb57fa640 1 -- 172.21.15.122:0/2461743279 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9bc404f020 con 0x55623fe67060 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 10 client.18618 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:21.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 10 client.18618 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 20 client.18618 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623fe81c70 0x55623fe84060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 4 client.18618 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.356+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55623fdd9b90 con 0x55623fe81c70 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.357+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623fe81c70 0x55623fe84060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.357+0000 7f9bb57fa640 10 client.18618 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.357+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623fe81c70 0x55623fe84060 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3765a34108da04 server_cookie=f8cac17daba339c2 in_seq=0 out_seq=0 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bb57fa640 1 -- 172.21.15.122:0/2461743279 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55623fdd9b90 con 0x55623fe81c70 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bb57fa640 10 client.18618 handle_command_reply: tid=0 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 client.18618 shutdown 2024-09-12T02:09:21.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 msgr2=0x55623fe6f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe6f180 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4003e30 tx=0x7f9bc4003d30 comp rx=0 tx=0).stop 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 shutdown_connections 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623fe81c70 0x55623fe84060 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe6fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67980 0x55623fe6f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/2461743279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67060 0x55623fe6f180 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bb57fa640 0 client.18618 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 >> 172.21.15.122:0/2461743279 conn(0x55623fe635f0 msgr2=0x55623fe692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 shutdown_connections 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/2461743279 wait complete. 2024-09-12T02:09:21.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.358+0000 7f9bd5ffd740 20 client.18618 trim_cache size 0 max 16384 2024-09-12T02:09:21.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:21 smithi151 ceph-mon[35229]: pgmap v1026: 97 pgs: 97 active+clean; 516 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 48 op/s 2024-09-12T02:09:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 Processor -- start 2024-09-12T02:09:21.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 -- start start 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67060 0x55623fe62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe6c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1b6560 con 0x55623fe67980 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1b3e30 con 0x55623fe72240 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1b1c60 con 0x55623fe67060 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd11bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd11bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57974/0 (socket says 172.21.15.122:57974) 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd11bf640 1 -- 172.21.15.122:0/4153661495 learned_addr learned my addr 172.21.15.122:0/4153661495 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.375+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe6c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67060 0x55623fe62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd11bf640 1 -- 172.21.15.122:0/4153661495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67060 msgr2=0x55623fe62f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67060 0x55623fe62f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd11bf640 1 -- 172.21.15.122:0/4153661495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 msgr2=0x55623fe6c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe6c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd11bf640 1 -- 172.21.15.122:0/4153661495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623fe74e90 con 0x55623fe67980 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe6c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe710f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc19fb90 tx=0x7f9bbc00b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fb4065a294e503 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bb77fe640 1 -- 172.21.15.122:0/4153661495 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc01e070 con 0x55623fe67980 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bb77fe640 1 -- 172.21.15.122:0/4153661495 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bbc004c60 con 0x55623fe67980 2024-09-12T02:09:21.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.376+0000 7f9bb77fe640 1 -- 172.21.15.122:0/4153661495 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011440 con 0x55623fe67980 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/4153661495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 msgr2=0x55623fe710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe710f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc19fb90 tx=0x7f9bbc00b360 comp rx=0 tx=0).stop 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/4153661495 shutdown_connections 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe72240 0x55623fe6c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe710f0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/4153661495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe67060 0x55623fe62f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/4153661495 >> 172.21.15.122:0/4153661495 conn(0x55623fe635f0 msgr2=0x55623fe709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:21.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/4153661495 shutdown_connections 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/4153661495 wait complete. 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 -- start start 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.377+0000 7f9bd5ffd740 1 Processor -- start 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67060 0x55623fe876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe85b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1b22f0 con 0x55623fe67980 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1b3760 con 0x55623fe67060 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1b29c0 con 0x55623fe72240 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd5ffd740 10 client.0 _ll_register_callbacks cb 0x7f9bc402fd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe85b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe85b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57976/0 (socket says 172.21.15.122:57976) 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 -- 172.21.15.122:0/670390587 learned_addr learned my addr 172.21.15.122:0/670390587 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:21.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67060 0x55623fe876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 -- 172.21.15.122:0/670390587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 msgr2=0x55623fe86080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe86080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 -- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67060 msgr2=0x55623fe876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67060 0x55623fe876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 -- 172.21.15.122:0/670390587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623fe86880 con 0x55623fe67980 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67060 0x55623fe876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bd11bf640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe85b40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc005d00 tx=0x7f9bbc00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b886f288653a22a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.378+0000 7f9bb57fa640 1 -- 172.21.15.122:0/670390587 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc030060 con 0x55623fe67980 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bb57fa640 1 -- 172.21.15.122:0/670390587 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9bbc0055f0 con 0x55623fe67980 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bbc004c60 con 0x55623fe67980 2024-09-12T02:09:21.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bb57fa640 1 -- 172.21.15.122:0/670390587 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9bbc01e040 con 0x55623fe67980 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 10 client.18624 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55623feed520 con 0x55623fe67980 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bb57fa640 1 -- 172.21.15.122:0/670390587 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bbc004c60 con 0x55623fe67980 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bb57fa640 1 -- 172.21.15.122:0/670390587 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9bbc0117f0 con 0x55623fe67980 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 10 client.18624 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 10 client.18624 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 20 client.18624 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623feedbe0 0x55623fef0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 4 client.18624 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "70bccc12", "release": true, "await": true}] 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "70bccc12", "release": true, "await": true}) -- 0x55623fdd40a0 con 0x55623feedbe0 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.379+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623feedbe0 0x55623fef0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.380+0000 7f9bb57fa640 10 client.18624 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:21.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.380+0000 7f9bd21c1640 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623feedbe0 0x55623fef0000 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541e9b357a469ad9 server_cookie=3edff570c7568975 in_seq=0 out_seq=0 2024-09-12T02:09:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.441+0000 7f9bb57fa640 1 -- 172.21.15.122:0/670390587 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55623fdd40a0 con 0x55623feedbe0 2024-09-12T02:09:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bb57fa640 10 client.18624 handle_command_reply: tid=0 2024-09-12T02:09:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 client.18624 shutdown 2024-09-12T02:09:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 msgr2=0x55623fe85b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe85b40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc005d00 tx=0x7f9bbc00bd10 comp rx=0 tx=0).stop 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 shutdown_connections 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55623feedbe0 0x55623fef0000 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55623fe72240 0x55623fe86080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55623fe67980 0x55623fe85b40 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bb57fa640 0 client.18624 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 --2- 172.21.15.122:0/670390587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55623fe67060 0x55623fe876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 >> 172.21.15.122:0/670390587 conn(0x55623fe635f0 msgr2=0x55623fe6f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 shutdown_connections 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 1 -- 172.21.15.122:0/670390587 wait complete. 2024-09-12T02:09:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.442+0000 7f9bd5ffd740 20 client.18624 trim_cache size 0 max 16384 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 84, 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "70bccc12": { 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 84, 2024-09-12T02:09:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 150.0, 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:09:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:09:21.447 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bb80a2c00 msgr2=0x7f9bb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bb80a2c00 0x7f9bb80a50c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9bc801e000 tx=0x7f9bc80d2de0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 msgr2=0x7f9bcc1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 0x7f9bcc1a46c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800e9f0 tx=0x7f9bc800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 shutdown_connections 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f9bb80a2c00 0x7f9bb80a50c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bcc105f40 0x7f9bcc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bcc101450 0x7f9bcc1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 --2- 172.21.15.122:0/2901658066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bcc100b30 0x7f9bcc1a46c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 >> 172.21.15.122:0/2901658066 conn(0x7f9bcc0fc7a0 msgr2=0x7f9bcc10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 shutdown_connections 2024-09-12T02:09:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:09:21.443+0000 7f9bd3c24640 1 -- 172.21.15.122:0/2901658066 wait complete. 2024-09-12T02:09:21.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:21 smithi183 ceph-mon[35127]: pgmap v1026: 97 pgs: 97 active+clean; 516 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.8 MiB/s wr, 48 op/s 2024-09-12T02:09:22.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 198153 18.25 MB/sec execute 429 sec latency 29483.204 ms 2024-09-12T02:09:22.157 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '70bccc12', total seconds elapsed: 33.2 2024-09-12T02:09:23.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 198610 18.27 MB/sec execute 430 sec latency 100.751 ms 2024-09-12T02:09:23.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:22 smithi183 ceph-mon[35127]: pgmap v1027: 97 pgs: 97 active+clean; 517 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-09-12T02:09:23.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:22 smithi122 ceph-mon[30925]: pgmap v1027: 97 pgs: 97 active+clean; 517 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-09-12T02:09:23.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:22 smithi151 ceph-mon[35229]: pgmap v1027: 97 pgs: 97 active+clean; 517 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.0 MiB/s wr, 71 op/s 2024-09-12T02:09:24.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 199036 18.29 MB/sec execute 431 sec latency 117.413 ms 2024-09-12T02:09:25.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 199506 18.30 MB/sec execute 432 sec latency 100.327 ms 2024-09-12T02:09:25.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:24 smithi151 ceph-mon[35229]: pgmap v1028: 97 pgs: 97 active+clean; 512 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-12T02:09:25.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:24 smithi183 ceph-mon[35127]: pgmap v1028: 97 pgs: 97 active+clean; 512 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-12T02:09:25.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:24 smithi122 ceph-mon[30925]: pgmap v1028: 97 pgs: 97 active+clean; 512 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 106 op/s 2024-09-12T02:09:26.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 200108 18.33 MB/sec execute 433 sec latency 121.989 ms 2024-09-12T02:09:27.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 200732 18.38 MB/sec execute 434 sec latency 62.767 ms 2024-09-12T02:09:27.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:26 smithi183 ceph-mon[35127]: pgmap v1029: 97 pgs: 97 active+clean; 508 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 118 op/s 2024-09-12T02:09:27.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:26 smithi122 ceph-mon[30925]: pgmap v1029: 97 pgs: 97 active+clean; 508 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 118 op/s 2024-09-12T02:09:27.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:26 smithi151 ceph-mon[35229]: pgmap v1029: 97 pgs: 97 active+clean; 508 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 118 op/s 2024-09-12T02:09:28.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 201264 18.41 MB/sec execute 435 sec latency 57.241 ms 2024-09-12T02:09:29.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 201787 18.41 MB/sec execute 436 sec latency 70.531 ms 2024-09-12T02:09:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:28 smithi183 ceph-mon[35127]: pgmap v1030: 97 pgs: 97 active+clean; 513 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-09-12T02:09:29.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:28 smithi122 ceph-mon[30925]: pgmap v1030: 97 pgs: 97 active+clean; 513 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-09-12T02:09:29.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:28 smithi151 ceph-mon[35229]: pgmap v1030: 97 pgs: 97 active+clean; 513 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-09-12T02:09:30.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 202284 18.43 MB/sec execute 437 sec latency 61.545 ms 2024-09-12T02:09:31.083 INFO:tasks.workunit.client.0.smithi122.stdout: 10 202912 18.47 MB/sec execute 438 sec latency 74.231 ms 2024-09-12T02:09:31.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:30 smithi183 ceph-mon[35127]: pgmap v1031: 97 pgs: 97 active+clean; 521 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-09-12T02:09:31.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:30 smithi122 ceph-mon[30925]: pgmap v1031: 97 pgs: 97 active+clean; 521 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-09-12T02:09:31.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:30 smithi151 ceph-mon[35229]: pgmap v1031: 97 pgs: 97 active+clean; 521 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-09-12T02:09:32.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 203689 18.53 MB/sec execute 439 sec latency 68.087 ms 2024-09-12T02:09:33.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 204254 18.55 MB/sec execute 440 sec latency 60.394 ms 2024-09-12T02:09:33.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:32 smithi183 ceph-mon[35127]: pgmap v1032: 97 pgs: 97 active+clean; 519 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-09-12T02:09:33.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:32 smithi122 ceph-mon[30925]: pgmap v1032: 97 pgs: 97 active+clean; 519 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-09-12T02:09:33.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:32 smithi151 ceph-mon[35229]: pgmap v1032: 97 pgs: 97 active+clean; 519 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-09-12T02:09:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 -- 172.21.15.122:0/4212178505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 msgr2=0x7f73ac109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/4212178505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac109f90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009ca0 tx=0x7f73a402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 -- 172.21.15.122:0/4212178505 shutdown_connections 2024-09-12T02:09:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/4212178505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac109f90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/4212178505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac102d40 0x7f73ac1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/4212178505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac101b50 0x7f73ac101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 -- 172.21.15.122:0/4212178505 >> 172.21.15.122:0/4212178505 conn(0x7f73ac0fd2b0 msgr2=0x7f73ac0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 -- 172.21.15.122:0/4212178505 shutdown_connections 2024-09-12T02:09:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.492+0000 7f73b0b8d640 1 -- 172.21.15.122:0/4212178505 wait complete. 2024-09-12T02:09:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.493+0000 7f73b0b8d640 1 Processor -- start 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.493+0000 7f73b0b8d640 1 -- start start 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73b0b8d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac101b50 0x7f73ac197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73b0b8d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac102d40 0x7f73ac198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73b0b8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73b0b8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac10d7b0 con 0x7f73ac103700 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73b0b8d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac10d630 con 0x7f73ac102d40 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73b0b8d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac10d930 con 0x7f73ac101b50 2024-09-12T02:09:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36736/0 (socket says 172.21.15.122:36736) 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 -- 172.21.15.122:0/2842910207 learned_addr learned my addr 172.21.15.122:0/2842910207 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aa59c640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac101b50 0x7f73ac197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac102d40 0x7f73ac198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 -- 172.21.15.122:0/2842910207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac101b50 msgr2=0x7f73ac197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac101b50 0x7f73ac197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 -- 172.21.15.122:0/2842910207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac102d40 msgr2=0x7f73ac198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac102d40 0x7f73ac198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aad9d640 1 -- 172.21.15.122:0/2842910207 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac19f950 con 0x7f73ac103700 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73aa59c640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac101b50 0x7f73ac197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.494+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac102d40 0x7f73ac198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.495+0000 7f73aad9d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac19f2b0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f73a4031ae0 tx=0x7f73a4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.495+0000 7f739b7fe640 1 -- 172.21.15.122:0/2842910207 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003c30 con 0x7f73ac103700 2024-09-12T02:09:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.495+0000 7f739b7fe640 1 -- 172.21.15.122:0/2842910207 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4003dd0 con 0x7f73ac103700 2024-09-12T02:09:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.495+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac074420 con 0x7f73ac103700 2024-09-12T02:09:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.495+0000 7f739b7fe640 1 -- 172.21.15.122:0/2842910207 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a402fcb0 con 0x7f73ac103700 2024-09-12T02:09:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.495+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac074970 con 0x7f73ac103700 2024-09-12T02:09:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.498+0000 7f73b3a04740 1 Processor -- start 2024-09-12T02:09:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f739b7fe640 1 -- 172.21.15.122:0/2842910207 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f73a4004050 con 0x7f73ac103700 2024-09-12T02:09:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 -- start start 2024-09-12T02:09:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27fa2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fa7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27faa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ae27f062f0 con 0x55ae27fa32e0 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ae27eae270 con 0x55ae27fa29a0 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73b3a04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ae27f022b0 con 0x55ae27fa8370 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27fa2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27fa2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:55242/0 (socket says 172.21.15.122:55242) 2024-09-12T02:09:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.499+0000 7f73a9d9b640 1 -- 172.21.15.122:0/64163684 learned_addr learned my addr 172.21.15.122:0/64163684 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27faa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73aa59c640 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fa7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 -- 172.21.15.122:0/64163684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 msgr2=0x55ae27faa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27faa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 -- 172.21.15.122:0/64163684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 msgr2=0x55ae27fa7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fa7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 -- 172.21.15.122:0/64163684 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae27ead340 con 0x55ae27fa29a0 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f739b7fe640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73940a2c30 0x7f73940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27fa2da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73a00058c0 tx=0x7f73a000bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bb7b0eae2e0c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f739b7fe640 1 -- 172.21.15.122:0/2842910207 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f73a40f0320 con 0x7f73ac103700 2024-09-12T02:09:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f73aa59c640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73940a2c30 0x7f73940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f7398ff9640 1 -- 172.21.15.122:0/64163684 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0012cd0 con 0x55ae27fa29a0 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f7398ff9640 1 -- 172.21.15.122:0/64163684 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a0005500 con 0x55ae27fa29a0 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.500+0000 7f7398ff9640 1 -- 172.21.15.122:0/64163684 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004d00 con 0x55ae27fa29a0 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.501+0000 7f73b3a04740 1 -- 172.21.15.122:0/64163684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 msgr2=0x55ae27fa2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.501+0000 7f73aa59c640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73940a2c30 0x7f73940a50f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55ae27fa3d10 tx=0x7f739c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.501+0000 7f73b3a04740 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27fa2da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73a00058c0 tx=0x7f73a000bbd0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 -- 172.21.15.122:0/64163684 shutdown_connections 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27faa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fa7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 --2- 172.21.15.122:0/64163684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27fa2da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 -- 172.21.15.122:0/64163684 >> 172.21.15.122:0/64163684 conn(0x55ae27f9e010 msgr2=0x55ae27f9e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 -- 172.21.15.122:0/64163684 shutdown_connections 2024-09-12T02:09:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 -- 172.21.15.122:0/64163684 wait complete. 2024-09-12T02:09:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 -- start start 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.502+0000 7f73b3a04740 1 Processor -- start 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa29a0 0x55ae27fb54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 0x55ae27fb59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ae27fab340 con 0x55ae27fa29a0 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ae27e8f290 con 0x55ae27fa8370 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ae27fab4c0 con 0x55ae27fa32e0 2024-09-12T02:09:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73b3a04740 10 client.0 _ll_register_callbacks cb 0x55ae27ec0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 0x55ae27fb59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73aad9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 0x55ae27fb59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33564/0 (socket says 172.21.15.122:33564) 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 -- 172.21.15.122:0/446947359 learned_addr learned my addr 172.21.15.122:0/446947359 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73aa59c640 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa29a0 0x55ae27fb54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 -- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 msgr2=0x55ae27fbc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 -- 172.21.15.122:0/446947359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa29a0 msgr2=0x55ae27fb54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa29a0 0x55ae27fb54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 -- 172.21.15.122:0/446947359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae27c19210 con 0x55ae27fa32e0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 0x55ae27fb59e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005250 tx=0x7f73a0005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f58bb05175b58e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.503+0000 7f7391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:09:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f7391ffb640 1 -- 172.21.15.122:0/446947359 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0010bd0 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f7391ffb640 1 -- 172.21.15.122:0/446947359 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f73a0004020 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a0005500 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f7391ffb640 1 -- 172.21.15.122:0/446947359 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f73a0004260 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 10 client.27565 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ae27ce5620 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f7391ffb640 1 -- 172.21.15.122:0/446947359 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a0005500 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f7391ffb640 1 -- 172.21.15.122:0/446947359 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f73a0013ce0 con 0x55ae27fa32e0 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 10 client.27565 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 10 client.27565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 20 client.27565 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae27fc0e40 0x55ae27fc3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 4 client.27565 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae27f13a20 con 0x55ae27fc0e40 2024-09-12T02:09:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.504+0000 7f73aa59c640 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae27fc0e40 0x55ae27fc3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.505+0000 7f7391ffb640 10 client.27565 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.505+0000 7f73aa59c640 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae27fc0e40 0x55ae27fc3250 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7c7e984ec8dc5 server_cookie=28bfd6cef260b1ef in_seq=0 out_seq=0 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.505+0000 7f7391ffb640 1 -- 172.21.15.122:0/446947359 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae27f13a20 con 0x55ae27fc0e40 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.505+0000 7f7391ffb640 10 client.27565 handle_command_reply: tid=0 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 client.27565 shutdown 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 msgr2=0x55ae27fb59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 0x55ae27fb59e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005250 tx=0x7f73a0005cc0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 shutdown_connections 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae27fc0e40 0x55ae27fc3250 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f7391ffb640 0 client.27565 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa32e0 0x55ae27fb59e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 --2- 172.21.15.122:0/446947359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa29a0 0x55ae27fb54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 >> 172.21.15.122:0/446947359 conn(0x55ae27f9e010 msgr2=0x55ae27f9f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 shutdown_connections 2024-09-12T02:09:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 1 -- 172.21.15.122:0/446947359 wait complete. 2024-09-12T02:09:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.506+0000 7f73b3a04740 20 client.27565 trim_cache size 0 max 16384 2024-09-12T02:09:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 Processor -- start 2024-09-12T02:09:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 -- start start 2024-09-12T02:09:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27f9d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fbc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27fc7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1a9570 con 0x55ae27fa32e0 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1a6e40 con 0x55ae27fa29a0 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73b3a04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1a4c70 con 0x55ae27fa8370 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fbc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fbc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36764/0 (socket says 172.21.15.122:36764) 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73aad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27fc7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73a9d9b640 1 -- 172.21.15.122:0/1130632110 learned_addr learned my addr 172.21.15.122:0/1130632110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.524+0000 7f73aa59c640 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27f9d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73aad9d640 1 -- 172.21.15.122:0/1130632110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 msgr2=0x55ae27f9d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73aad9d640 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27f9d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73aad9d640 1 -- 172.21.15.122:0/1130632110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 msgr2=0x55ae27fbc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73aad9d640 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fbc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73aad9d640 1 -- 172.21.15.122:0/1130632110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae27f992d0 con 0x55ae27fa8370 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fbc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f73aad9d640 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27fc7b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73a40bc9a0 tx=0x7f73a403d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ed5b936e7c5bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f7393fff640 1 -- 172.21.15.122:0/1130632110 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40da070 con 0x55ae27fa8370 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f7393fff640 1 -- 172.21.15.122:0/1130632110 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4036040 con 0x55ae27fa8370 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.525+0000 7f7393fff640 1 -- 172.21.15.122:0/1130632110 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40b9280 con 0x55ae27fa8370 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 -- 172.21.15.122:0/1130632110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 msgr2=0x55ae27fc7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27fc7b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73a40bc9a0 tx=0x7f73a403d040 comp rx=0 tx=0).stop 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 -- 172.21.15.122:0/1130632110 shutdown_connections 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa8370 0x55ae27fc7b10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fbc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 --2- 172.21.15.122:0/1130632110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa29a0 0x55ae27f9d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 -- 172.21.15.122:0/1130632110 >> 172.21.15.122:0/1130632110 conn(0x55ae27f9e010 msgr2=0x55ae27f9c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 -- 172.21.15.122:0/1130632110 shutdown_connections 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 -- 172.21.15.122:0/1130632110 wait complete. 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 -- start start 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.526+0000 7f73b3a04740 1 Processor -- start 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa29a0 0x55ae27fb9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1a5300 con 0x55ae27fa32e0 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1a6770 con 0x55ae27fa8370 2024-09-12T02:09:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1a59d0 con 0x55ae27fa29a0 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73aa59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa29a0 0x55ae27fb9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73aa59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa29a0 0x55ae27fb9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:33592/0 (socket says 172.21.15.122:33592) 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73aa59c640 1 -- 172.21.15.122:0/2683043759 learned_addr learned my addr 172.21.15.122:0/2683043759 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73aad9d640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73b3a04740 10 client.0 _ll_register_callbacks cb 0x7f739c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 -- 172.21.15.122:0/2683043759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa29a0 msgr2=0x55ae27fb9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa29a0 0x55ae27fb9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 -- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 msgr2=0x55ae27fbb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 -- 172.21.15.122:0/2683043759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae27fbba50 con 0x55ae27fa32e0 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73aad9d640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f73a9d9b640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fba430 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f73a00058c0 tx=0x7f73a0012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789420072736dc99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.527+0000 7f7391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f7391ffb640 1 -- 172.21.15.122:0/2683043759 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a003b060 con 0x55ae27fa32e0 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f7391ffb640 1 -- 172.21.15.122:0/2683043759 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f73a0004660 con 0x55ae27fa32e0 2024-09-12T02:09:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a4036040 con 0x55ae27fa32e0 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f7391ffb640 1 -- 172.21.15.122:0/2683043759 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f73a0004070 con 0x55ae27fa32e0 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 10 client.18642 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ae2802baf0 con 0x55ae27fa32e0 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f7391ffb640 1 -- 172.21.15.122:0/2683043759 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4036040 con 0x55ae27fa32e0 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f7391ffb640 1 -- 172.21.15.122:0/2683043759 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f73a0040070 con 0x55ae27fa32e0 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 10 client.18642 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 10 client.18642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 20 client.18642 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae2802c290 0x55ae2802e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 4 client.18642 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae27f0dd70 con 0x55ae2802c290 2024-09-12T02:09:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.528+0000 7f73aa59c640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae2802c290 0x55ae2802e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.529+0000 7f7391ffb640 10 client.18642 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.529+0000 7f73aa59c640 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae2802c290 0x55ae2802e6b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0db75876cc05bb server_cookie=dc6b30b110ea5a65 in_seq=0 out_seq=0 2024-09-12T02:09:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f7391ffb640 1 -- 172.21.15.122:0/2683043759 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae27f0dd70 con 0x55ae2802c290 2024-09-12T02:09:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f7391ffb640 10 client.18642 handle_command_reply: tid=0 2024-09-12T02:09:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 client.18642 shutdown 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 msgr2=0x55ae27fba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fba430 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f73a00058c0 tx=0x7f73a0012b80 comp rx=0 tx=0).stop 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 shutdown_connections 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ae2802c290 0x55ae2802e6b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae27fa8370 0x55ae27fbb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae27fa32e0 0x55ae27fba430 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 --2- 172.21.15.122:0/2683043759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae27fa29a0 0x55ae27fb9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 >> 172.21.15.122:0/2683043759 conn(0x55ae27f9e010 msgr2=0x55ae27f9f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.760+0000 7f7391ffb640 0 client.18642 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 shutdown_connections 2024-09-12T02:09:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b3a04740 1 -- 172.21.15.122:0/2683043759 wait complete. 2024-09-12T02:09:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b3a04740 20 client.18642 trim_cache size 0 max 16384 2024-09-12T02:09:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:09:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:09:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:09:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:09:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73940a2c30 msgr2=0x7f73940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73940a2c30 0x7f73940a50f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55ae27fa3d10 tx=0x7f739c005f90 comp rx=0 tx=0).stop 2024-09-12T02:09:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 msgr2=0x7f73ac19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac19f2b0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f73a4031ae0 tx=0x7f73a4031ee0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 shutdown_connections 2024-09-12T02:09:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f73940a2c30 0x7f73940a50f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac103700 0x7f73ac19f2b0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ac102d40 0x7f73ac198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 --2- 172.21.15.122:0/2842910207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ac101b50 0x7f73ac197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 >> 172.21.15.122:0/2842910207 conn(0x7f73ac0fd2b0 msgr2=0x7f73ac106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 shutdown_connections 2024-09-12T02:09:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:33.761+0000 7f73b0b8d640 1 -- 172.21.15.122:0/2842910207 wait complete. 2024-09-12T02:09:34.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 204812 18.59 MB/sec execute 441 sec latency 205.692 ms 2024-09-12T02:09:35.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 205351 18.62 MB/sec execute 442 sec latency 92.129 ms 2024-09-12T02:09:35.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:34 smithi183 ceph-mon[35127]: pgmap v1033: 97 pgs: 97 active+clean; 506 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 445 op/s 2024-09-12T02:09:35.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:34 smithi122 ceph-mon[30925]: pgmap v1033: 97 pgs: 97 active+clean; 506 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 445 op/s 2024-09-12T02:09:35.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:34 smithi151 ceph-mon[35229]: pgmap v1033: 97 pgs: 97 active+clean; 506 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.1 MiB/s wr, 445 op/s 2024-09-12T02:09:36.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 205752 18.61 MB/sec execute 443 sec latency 92.748 ms 2024-09-12T02:09:37.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 206325 18.62 MB/sec execute 444 sec latency 56.189 ms 2024-09-12T02:09:37.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:36 smithi183 ceph-mon[35127]: pgmap v1034: 97 pgs: 97 active+clean; 509 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 459 op/s 2024-09-12T02:09:37.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:36 smithi122 ceph-mon[30925]: pgmap v1034: 97 pgs: 97 active+clean; 509 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 459 op/s 2024-09-12T02:09:37.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:36 smithi151 ceph-mon[35229]: pgmap v1034: 97 pgs: 97 active+clean; 509 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.1 MiB/s wr, 459 op/s 2024-09-12T02:09:38.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 206870 18.65 MB/sec execute 445 sec latency 55.272 ms 2024-09-12T02:09:39.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 207590 18.69 MB/sec execute 446 sec latency 79.900 ms 2024-09-12T02:09:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:38 smithi183 ceph-mon[35127]: pgmap v1035: 97 pgs: 97 active+clean; 507 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.3 MiB/s wr, 498 op/s 2024-09-12T02:09:39.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:38 smithi122 ceph-mon[30925]: pgmap v1035: 97 pgs: 97 active+clean; 507 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.3 MiB/s wr, 498 op/s 2024-09-12T02:09:39.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:38 smithi151 ceph-mon[35229]: pgmap v1035: 97 pgs: 97 active+clean; 507 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.3 MiB/s wr, 498 op/s 2024-09-12T02:09:40.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 208283 18.75 MB/sec execute 447 sec latency 54.276 ms 2024-09-12T02:09:41.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 208774 18.77 MB/sec execute 448 sec latency 190.522 ms 2024-09-12T02:09:41.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:40 smithi183 ceph-mon[35127]: pgmap v1036: 97 pgs: 97 active+clean; 518 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-09-12T02:09:41.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:40 smithi122 ceph-mon[30925]: pgmap v1036: 97 pgs: 97 active+clean; 518 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-09-12T02:09:41.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:40 smithi151 ceph-mon[35229]: pgmap v1036: 97 pgs: 97 active+clean; 518 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-09-12T02:09:42.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 209369 18.81 MB/sec execute 449 sec latency 55.182 ms 2024-09-12T02:09:43.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 209777 18.81 MB/sec execute 450 sec latency 64.919 ms 2024-09-12T02:09:43.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:42 smithi183 ceph-mon[35127]: pgmap v1037: 97 pgs: 97 active+clean; 521 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.0 MiB/s wr, 428 op/s 2024-09-12T02:09:43.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:42 smithi122 ceph-mon[30925]: pgmap v1037: 97 pgs: 97 active+clean; 521 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.0 MiB/s wr, 428 op/s 2024-09-12T02:09:43.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:42 smithi151 ceph-mon[35229]: pgmap v1037: 97 pgs: 97 active+clean; 521 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.0 MiB/s wr, 428 op/s 2024-09-12T02:09:44.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 210153 18.81 MB/sec execute 451 sec latency 122.527 ms 2024-09-12T02:09:45.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 210739 18.82 MB/sec execute 452 sec latency 82.679 ms 2024-09-12T02:09:46.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 211400 18.86 MB/sec execute 453 sec latency 101.038 ms 2024-09-12T02:09:46.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:45 smithi183 ceph-mon[35127]: pgmap v1038: 97 pgs: 97 active+clean; 527 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 476 op/s 2024-09-12T02:09:46.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:45 smithi122 ceph-mon[30925]: pgmap v1038: 97 pgs: 97 active+clean; 527 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 476 op/s 2024-09-12T02:09:46.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:45 smithi151 ceph-mon[35229]: pgmap v1038: 97 pgs: 97 active+clean; 527 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 476 op/s 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 -- 172.21.15.122:0/48053469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c101b60 msgr2=0x7f132c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 --2- 172.21.15.122:0/48053469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c101b60 0x7f132c101f60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f131c009900 tx=0x7f131c02a410 comp rx=0 tx=0).stop 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 -- 172.21.15.122:0/48053469 shutdown_connections 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 --2- 172.21.15.122:0/48053469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 0x7f132c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 --2- 172.21.15.122:0/48053469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c102d50 0x7f132c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 --2- 172.21.15.122:0/48053469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c101b60 0x7f132c101f60 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 -- 172.21.15.122:0/48053469 >> 172.21.15.122:0/48053469 conn(0x7f132c0fd2e0 msgr2=0x7f132c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 -- 172.21.15.122:0/48053469 shutdown_connections 2024-09-12T02:09:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 -- 172.21.15.122:0/48053469 wait complete. 2024-09-12T02:09:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.878+0000 7f13321a6640 1 Processor -- start 2024-09-12T02:09:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 -- start start 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c101b60 0x7f132c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 0x7f132c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 0x7f132c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f132c10d830 con 0x7f132c102d50 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f132c10d6b0 con 0x7f132c103710 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f13321a6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f132c10d9b0 con 0x7f132c101b60 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f132affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 0x7f132c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f132affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 0x7f132c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38034/0 (socket says 172.21.15.122:38034) 2024-09-12T02:09:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f132affd640 1 -- 172.21.15.122:0/318487790 learned_addr learned my addr 172.21.15.122:0/318487790 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f132b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c101b60 0x7f132c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.879+0000 7f132bfff640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 0x7f132c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132b7fe640 1 -- 172.21.15.122:0/318487790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 msgr2=0x7f132c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132b7fe640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 0x7f132c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132b7fe640 1 -- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 msgr2=0x7f132c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132b7fe640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 0x7f132c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132b7fe640 1 -- 172.21.15.122:0/318487790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c19fa10 con 0x7f132c101b60 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132bfff640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 0x7f132c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132affd640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 0x7f132c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f132b7fe640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c101b60 0x7f132c197df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f131c009230 tx=0x7f131c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f1328ff9640 1 -- 172.21.15.122:0/318487790 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c03f030 con 0x7f132c101b60 2024-09-12T02:09:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c19fca0 con 0x7f132c101b60 2024-09-12T02:09:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f1328ff9640 1 -- 172.21.15.122:0/318487790 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c003ce0 con 0x7f132c101b60 2024-09-12T02:09:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f1328ff9640 1 -- 172.21.15.122:0/318487790 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c035070 con 0x7f132c101b60 2024-09-12T02:09:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.880+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1a01b0 con 0x7f132c101b60 2024-09-12T02:09:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f1328ff9640 1 -- 172.21.15.122:0/318487790 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f131c042030 con 0x7f132c101b60 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 Processor -- start 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 -- start start 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b91da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b96da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66b99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d66af52f0 con 0x561d66b922e0 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561d66a9d270 con 0x561d66b97370 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f133501e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561d66af12b0 con 0x561d66b919a0 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f132bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b91da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f132b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b96da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f132bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b91da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41134/0 (socket says 172.21.15.122:41134) 2024-09-12T02:09:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f132bfff640 1 -- 172.21.15.122:0/4157991983 learned_addr learned my addr 172.21.15.122:0/4157991983 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f132affd640 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66b99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.883+0000 7f1328ff9640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13140a2c30 0x7f13140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 -- 172.21.15.122:0/4157991983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 msgr2=0x561d66b99830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66b99830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 -- 172.21.15.122:0/4157991983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 msgr2=0x561d66b96da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f1328ff9640 1 -- 172.21.15.122:0/318487790 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f131c0ef4a0 con 0x7f132c101b60 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b96da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 -- 172.21.15.122:0/4157991983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d66a9c340 con 0x561d66b919a0 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132affd640 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66b99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132b7fe640 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13140a2c30 0x7f13140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b91da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1324011940 tx=0x7f1324011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99faa11fc2eaaba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f1311ffb640 1 -- 172.21.15.122:0/4157991983 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324017070 con 0x561d66b919a0 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f132bfff640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13140a2c30 0x7f13140a50f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1324023000 tx=0x7f1324024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f1311ffb640 1 -- 172.21.15.122:0/4157991983 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1324011250 con 0x561d66b919a0 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.884+0000 7f1311ffb640 1 -- 172.21.15.122:0/4157991983 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324010c00 con 0x561d66b919a0 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 -- 172.21.15.122:0/4157991983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 msgr2=0x561d66b91da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b91da0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1324011940 tx=0x7f1324011e00 comp rx=0 tx=0).stop 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 -- 172.21.15.122:0/4157991983 shutdown_connections 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66b99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 --2- 172.21.15.122:0/4157991983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b91da0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 -- 172.21.15.122:0/4157991983 >> 172.21.15.122:0/4157991983 conn(0x561d66b8d010 msgr2=0x561d66b8d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 -- 172.21.15.122:0/4157991983 shutdown_connections 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.885+0000 7f133501e740 1 -- 172.21.15.122:0/4157991983 wait complete. 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 -- start start 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 Processor -- start 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b9bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b9c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66ba31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d66b9a340 con 0x561d66b922e0 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561d66a7e290 con 0x561d66b97370 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561d66b9a4c0 con 0x561d66b919a0 2024-09-12T02:09:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f133501e740 10 client.0 _ll_register_callbacks cb 0x561d66aaf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b9c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b9bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b9bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:41140/0 (socket says 172.21.15.122:41140) 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b9c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38054/0 (socket says 172.21.15.122:38054) 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132b7fe640 1 -- 172.21.15.122:0/1952910821 learned_addr learned my addr 172.21.15.122:0/1952910821 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132bfff640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66ba31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.886+0000 7f132b7fe640 1 -- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 msgr2=0x561d66ba31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f132b7fe640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66ba31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f132b7fe640 1 -- 172.21.15.122:0/1952910821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 msgr2=0x561d66b9c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f132b7fe640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b9c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f132b7fe640 1 -- 172.21.15.122:0/1952910821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d66808210 con 0x561d66b919a0 2024-09-12T02:09:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f132affd640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b9c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f132b7fe640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b9bc20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f131c000c00 tx=0x7f131c032f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab925aeb7d144e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f130affd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f130affd640 1 -- 172.21.15.122:0/1952910821 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c049040 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f130affd640 1 -- 172.21.15.122:0/1952910821 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f131c0b83e0 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1324011250 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f130affd640 1 -- 172.21.15.122:0/1952910821 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f131c0bb8b0 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 10 client.27583 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561d668d4620 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f130affd640 1 -- 172.21.15.122:0/1952910821 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1324011250 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f130affd640 1 -- 172.21.15.122:0/1952910821 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f131c0ef110 con 0x561d66b919a0 2024-09-12T02:09:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 10 client.27583 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 10 client.27583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 20 client.27583 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66ba78d0 0x561d66ba9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 4 client.27583 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.887+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d66b02a20 con 0x561d66ba78d0 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.888+0000 7f132affd640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66ba78d0 0x561d66ba9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.888+0000 7f132affd640 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66ba78d0 0x561d66ba9ce0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff6cc1f87ced779 server_cookie=4991821ab9a648d in_seq=0 out_seq=0 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.888+0000 7f130affd640 10 client.27583 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f130affd640 1 -- 172.21.15.122:0/1952910821 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d66b02a20 con 0x561d66ba78d0 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f130affd640 10 client.27583 handle_command_reply: tid=0 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 client.27583 shutdown 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 msgr2=0x561d66b9bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b9bc20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f131c000c00 tx=0x7f131c032f90 comp rx=0 tx=0).stop 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 shutdown_connections 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66ba78d0 0x561d66ba9ce0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b97370 0x561d66ba31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66b9c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f130affd640 0 client.27583 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 --2- 172.21.15.122:0/1952910821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b919a0 0x561d66b9bc20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 >> 172.21.15.122:0/1952910821 conn(0x561d66b8d010 msgr2=0x561d66b8e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 shutdown_connections 2024-09-12T02:09:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 1 -- 172.21.15.122:0/1952910821 wait complete. 2024-09-12T02:09:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.889+0000 7f133501e740 20 client.27583 trim_cache size 0 max 16384 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 Processor -- start 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 -- start start 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66b9c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66ba3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b97370 0x561d66b8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f132c1a9600 con 0x561d66b922e0 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f132c1a6ed0 con 0x561d66b919a0 2024-09-12T02:09:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f133501e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f132c1a4d00 con 0x561d66b97370 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66ba3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b97370 0x561d66b8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66ba3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38056/0 (socket says 172.21.15.122:38056) 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132affd640 1 -- 172.21.15.122:0/1831923588 learned_addr learned my addr 172.21.15.122:0/1831923588 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132b7fe640 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66b9c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132bfff640 1 -- 172.21.15.122:0/1831923588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 msgr2=0x561d66b9c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132bfff640 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66b9c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132bfff640 1 -- 172.21.15.122:0/1831923588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 msgr2=0x561d66ba3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132bfff640 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66ba3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132bfff640 1 -- 172.21.15.122:0/1831923588 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d66baa310 con 0x561d66b97370 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.905+0000 7f132affd640 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66ba3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.906+0000 7f132bfff640 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b97370 0x561d66b8c9b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1324009560 tx=0x7f1324005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21bf7466b9e900c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.906+0000 7f13117fa640 1 -- 172.21.15.122:0/1831923588 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324017070 con 0x561d66b97370 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.906+0000 7f13117fa640 1 -- 172.21.15.122:0/1831923588 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13240104a0 con 0x561d66b97370 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.906+0000 7f13117fa640 1 -- 172.21.15.122:0/1831923588 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324010eb0 con 0x561d66b97370 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- 172.21.15.122:0/1831923588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b97370 msgr2=0x561d66b8c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b97370 0x561d66b8c9b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1324009560 tx=0x7f1324005fd0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- 172.21.15.122:0/1831923588 shutdown_connections 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b97370 0x561d66b8c9b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b922e0 0x561d66ba3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- 172.21.15.122:0/1831923588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66b9c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- 172.21.15.122:0/1831923588 >> 172.21.15.122:0/1831923588 conn(0x561d66b8d010 msgr2=0x561d66b8dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- 172.21.15.122:0/1831923588 shutdown_connections 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- 172.21.15.122:0/1831923588 wait complete. 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- start start 2024-09-12T02:09:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 Processor -- start 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66ba0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b922e0 0x561d66ba1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 0x561d66ba23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f132c1a5390 con 0x561d66b97370 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f132c1a6800 con 0x561d66b919a0 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.907+0000 7f133501e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f132c1a5a60 con 0x561d66b922e0 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f133501e740 10 client.0 _ll_register_callbacks cb 0x7f131c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 0x561d66ba23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66ba0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66ba0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:35638/0 (socket says 172.21.15.122:35638) 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132b7fe640 1 -- 172.21.15.122:0/3609072485 learned_addr learned my addr 172.21.15.122:0/3609072485 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 0x561d66ba23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38058/0 (socket says 172.21.15.122:38058) 2024-09-12T02:09:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132affd640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b922e0 0x561d66ba1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 -- 172.21.15.122:0/3609072485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b922e0 msgr2=0x561d66ba1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b922e0 0x561d66ba1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 -- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 msgr2=0x561d66ba0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66ba0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 -- 172.21.15.122:0/3609072485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d66ba29b0 con 0x561d66b97370 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132affd640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b922e0 0x561d66ba1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f132bfff640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 0x561d66ba23a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1324013010 tx=0x7f13240046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88873102a557ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f130affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f130affd640 1 -- 172.21.15.122:0/3609072485 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132403a060 con 0x561d66b97370 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f130affd640 1 -- 172.21.15.122:0/3609072485 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1324005b70 con 0x561d66b97370 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13240104a0 con 0x561d66b97370 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f130affd640 1 -- 172.21.15.122:0/3609072485 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1324017030 con 0x561d66b97370 2024-09-12T02:09:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f133501e740 10 client.18660 fetch_fsmap learned FSMap version 45 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.908+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561d66c12b70 con 0x561d66b97370 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f130affd640 1 -- 172.21.15.122:0/3609072485 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13240104a0 con 0x561d66b97370 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f130affd640 1 -- 172.21.15.122:0/3609072485 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f132403f070 con 0x561d66b97370 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f133501e740 10 client.18660 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f133501e740 10 client.18660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f133501e740 20 client.18660 populate_metadata read hostname 'smithi122' 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f133501e740 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66c13350 0x561d66c15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f133501e740 4 client.18660 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d66afcd70 con 0x561d66c13350 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f132b7fe640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66c13350 0x561d66c15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f130affd640 10 client.18660 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:46.909+0000 7f132b7fe640 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66c13350 0x561d66c15770 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74073f213f682372 server_cookie=45c3b7ff51bbfade in_seq=0 out_seq=0 2024-09-12T02:09:47.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 211891 18.88 MB/sec execute 454 sec latency 125.523 ms 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f130affd640 1 -- 172.21.15.122:0/3609072485 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d66afcd70 con 0x561d66c13350 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f130affd640 10 client.18660 handle_command_reply: tid=0 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 client.18660 shutdown 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 msgr2=0x561d66ba23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 0x561d66ba23a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1324013010 tx=0x7f13240046d0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 shutdown_connections 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x561d66c13350 0x561d66c15770 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d66b97370 0x561d66ba23a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d66b922e0 0x561d66ba1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f130affd640 0 client.18660 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 --2- 172.21.15.122:0/3609072485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d66b919a0 0x561d66ba0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 >> 172.21.15.122:0/3609072485 conn(0x561d66b8d010 msgr2=0x561d66b8e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 shutdown_connections 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 1 -- 172.21.15.122:0/3609072485 wait complete. 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.234+0000 7f133501e740 20 client.18660 trim_cache size 0 max 16384 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:09:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:09:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13140a2c30 msgr2=0x7f13140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13140a2c30 0x7f13140a50f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1324023000 tx=0x7f1324024000 comp rx=0 tx=0).stop 2024-09-12T02:09:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c101b60 msgr2=0x7f132c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c101b60 0x7f132c197df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f131c009230 tx=0x7f131c032ce0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 shutdown_connections 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f13140a2c30 0x7f13140a50f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f132c103710 0x7f132c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f132c102d50 0x7f132c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 --2- 172.21.15.122:0/318487790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c101b60 0x7f132c197df0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 >> 172.21.15.122:0/318487790 conn(0x7f132c0fd2e0 msgr2=0x7f132c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 shutdown_connections 2024-09-12T02:09:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:09:47.235+0000 7f13321a6640 1 -- 172.21.15.122:0/318487790 wait complete. 2024-09-12T02:09:48.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 212432 18.92 MB/sec execute 455 sec latency 189.285 ms 2024-09-12T02:09:48.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:47 smithi183 ceph-mon[35127]: pgmap v1039: 97 pgs: 97 active+clean; 534 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.3 MiB/s wr, 448 op/s 2024-09-12T02:09:48.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:47 smithi122 ceph-mon[30925]: pgmap v1039: 97 pgs: 97 active+clean; 534 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.3 MiB/s wr, 448 op/s 2024-09-12T02:09:48.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:47 smithi151 ceph-mon[35229]: pgmap v1039: 97 pgs: 97 active+clean; 534 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.3 MiB/s wr, 448 op/s 2024-09-12T02:09:49.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 212995 18.95 MB/sec execute 456 sec latency 95.381 ms 2024-09-12T02:09:50.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 213389 18.96 MB/sec execute 457 sec latency 112.045 ms 2024-09-12T02:09:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:49 smithi183 ceph-mon[35127]: pgmap v1040: 97 pgs: 97 active+clean; 536 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2024-09-12T02:09:50.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:49 smithi122 ceph-mon[30925]: pgmap v1040: 97 pgs: 97 active+clean; 536 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2024-09-12T02:09:50.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:49 smithi151 ceph-mon[35229]: pgmap v1040: 97 pgs: 97 active+clean; 536 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2024-09-12T02:09:51.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 213758 18.95 MB/sec execute 458 sec latency 110.562 ms 2024-09-12T02:09:52.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 214258 18.96 MB/sec execute 459 sec latency 98.077 ms 2024-09-12T02:09:52.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:51 smithi183 ceph-mon[35127]: pgmap v1041: 97 pgs: 97 active+clean; 534 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 442 op/s 2024-09-12T02:09:52.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:51 smithi122 ceph-mon[30925]: pgmap v1041: 97 pgs: 97 active+clean; 534 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 442 op/s 2024-09-12T02:09:52.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:51 smithi151 ceph-mon[35229]: pgmap v1041: 97 pgs: 97 active+clean; 534 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 442 op/s 2024-09-12T02:09:53.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 214665 18.96 MB/sec execute 460 sec latency 99.675 ms 2024-09-12T02:09:54.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 215343 18.98 MB/sec execute 461 sec latency 74.439 ms 2024-09-12T02:09:54.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:53 smithi183 ceph-mon[35127]: pgmap v1042: 97 pgs: 97 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.9 MiB/s wr, 425 op/s 2024-09-12T02:09:54.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:53 smithi122 ceph-mon[30925]: pgmap v1042: 97 pgs: 97 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.9 MiB/s wr, 425 op/s 2024-09-12T02:09:54.365 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:53 smithi151 ceph-mon[35229]: pgmap v1042: 97 pgs: 97 active+clean; 541 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.9 MiB/s wr, 425 op/s 2024-09-12T02:09:55.084 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216008 19.04 MB/sec execute 462 sec latency 60.283 ms 2024-09-12T02:09:56.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216612 19.08 MB/sec execute 463 sec latency 94.208 ms 2024-09-12T02:09:56.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:56 smithi151 ceph-mon[35229]: pgmap v1043: 97 pgs: 97 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 485 op/s 2024-09-12T02:09:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:56 smithi183 ceph-mon[35127]: pgmap v1043: 97 pgs: 97 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 485 op/s 2024-09-12T02:09:56.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:56 smithi122 ceph-mon[30925]: pgmap v1043: 97 pgs: 97 active+clean; 543 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 485 op/s 2024-09-12T02:09:57.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 217218 19.12 MB/sec execute 464 sec latency 65.004 ms 2024-09-12T02:09:58.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 217482 19.11 MB/sec execute 465 sec latency 324.350 ms 2024-09-12T02:09:58.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:09:58 smithi151 ceph-mon[35229]: pgmap v1044: 97 pgs: 97 active+clean; 547 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 454 op/s 2024-09-12T02:09:58.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:09:58 smithi183 ceph-mon[35127]: pgmap v1044: 97 pgs: 97 active+clean; 547 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 454 op/s 2024-09-12T02:09:58.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:09:58 smithi122 ceph-mon[30925]: pgmap v1044: 97 pgs: 97 active+clean; 547 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 454 op/s 2024-09-12T02:09:59.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 217876 19.11 MB/sec execute 466 sec latency 137.699 ms 2024-09-12T02:10:00.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 218444 19.11 MB/sec execute 467 sec latency 55.386 ms 2024-09-12T02:10:00.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T02:09:59.999+0000 7f42afbd7640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 -- 172.21.15.122:0/1586821999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 msgr2=0x7f45541005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 --2- 172.21.15.122:0/1586821999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f45541005d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4544009860 tx=0x7f454402a340 comp rx=0 tx=0).stop 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 -- 172.21.15.122:0/1586821999 shutdown_connections 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 --2- 172.21.15.122:0/1586821999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4554101d80 0x7f4554108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 --2- 172.21.15.122:0/1586821999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 0x7f4554101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 --2- 172.21.15.122:0/1586821999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f45541005d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 -- 172.21.15.122:0/1586821999 >> 172.21.15.122:0/1586821999 conn(0x7f45540fb930 msgr2=0x7f45540fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 -- 172.21.15.122:0/1586821999 shutdown_connections 2024-09-12T02:10:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.332+0000 7f4559a82640 1 -- 172.21.15.122:0/1586821999 wait complete. 2024-09-12T02:10:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 Processor -- start 2024-09-12T02:10:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 -- start start 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f455419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 0x7f455419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4554101d80 0x7f45541a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45541099e0 con 0x7f4554101d80 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4554109860 con 0x7f45541001d0 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4559a82640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4554109b60 con 0x7f45541013c0 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f45537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f455419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4552ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 0x7f455419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f45537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f455419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:54278/0 (socket says 172.21.15.122:54278) 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f45537fe640 1 -- 172.21.15.122:0/3168574598 learned_addr learned my addr 172.21.15.122:0/3168574598 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.333+0000 7f4553fff640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4554101d80 0x7f45541a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f45537fe640 1 -- 172.21.15.122:0/3168574598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 msgr2=0x7f455419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f45537fe640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 0x7f455419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f45537fe640 1 -- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4554101d80 msgr2=0x7f45541a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f45537fe640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4554101d80 0x7f45541a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f45537fe640 1 -- 172.21.15.122:0/3168574598 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45541a1ca0 con 0x7f45541001d0 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f4552ffd640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 0x7f455419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f45537fe640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f455419a080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4544035040 tx=0x7f4544033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f4550ff9640 1 -- 172.21.15.122:0/3168574598 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544037040 con 0x7f45541001d0 2024-09-12T02:10:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f4550ff9640 1 -- 172.21.15.122:0/3168574598 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544030850 con 0x7f45541001d0 2024-09-12T02:10:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45541a1f30 con 0x7f45541001d0 2024-09-12T02:10:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f4550ff9640 1 -- 172.21.15.122:0/3168574598 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544004920 con 0x7f45541001d0 2024-09-12T02:10:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.334+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a2340 con 0x7f45541001d0 2024-09-12T02:10:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.337+0000 7f4550ff9640 1 -- 172.21.15.122:0/3168574598 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4544004050 con 0x7f45541001d0 2024-09-12T02:10:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.337+0000 7f455c81f740 1 Processor -- start 2024-09-12T02:10:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.337+0000 7f455c81f740 1 -- start start 2024-09-12T02:10:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f455c81f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e330ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f455c81f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330b2e0 0x55f0e330fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f455c81f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e3310370 0x55f0e3312830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f455c81f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0e326e2f0 con 0x55f0e330b2e0 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f455c81f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f0e3216270 con 0x55f0e330a9a0 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f455c81f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f0e326a2b0 con 0x55f0e3310370 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f45537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330b2e0 0x55f0e330fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4553fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e330ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4553fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e330ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:54294/0 (socket says 172.21.15.122:54294) 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4553fff640 1 -- 172.21.15.122:0/207877101 learned_addr learned my addr 172.21.15.122:0/207877101 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e3310370 0x55f0e3312830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 -- 172.21.15.122:0/207877101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 msgr2=0x55f0e330ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e330ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 -- 172.21.15.122:0/207877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330b2e0 msgr2=0x55f0e330fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330b2e0 0x55f0e330fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 -- 172.21.15.122:0/207877101 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e3215340 con 0x55f0e3310370 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4550ff9640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f453c0a2c30 0x7f453c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4553fff640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f453c0a2c30 0x7f453c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4550ff9640 1 -- 172.21.15.122:0/3168574598 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f45440eeea0 con 0x7f45541001d0 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.338+0000 7f4552ffd640 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e3310370 0x55f0e3312830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4548009fa0 tx=0x7f45480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce9cef0607c3883 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f4539ffb640 1 -- 172.21.15.122:0/207877101 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548004310 con 0x55f0e3310370 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f4539ffb640 1 -- 172.21.15.122:0/207877101 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45480044b0 con 0x55f0e3310370 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f4539ffb640 1 -- 172.21.15.122:0/207877101 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480035f0 con 0x55f0e3310370 2024-09-12T02:10:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 -- 172.21.15.122:0/207877101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e3310370 msgr2=0x55f0e3312830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f4553fff640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f453c0a2c30 0x7f453c0a50f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f454c005c40 tx=0x7f454c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e3310370 0x55f0e3312830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4548009fa0 tx=0x7f45480118d0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 -- 172.21.15.122:0/207877101 shutdown_connections 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e3310370 0x55f0e3312830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330b2e0 0x55f0e330fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 --2- 172.21.15.122:0/207877101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e330ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 -- 172.21.15.122:0/207877101 >> 172.21.15.122:0/207877101 conn(0x55f0e3306010 msgr2=0x55f0e3306410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 -- 172.21.15.122:0/207877101 shutdown_connections 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.339+0000 7f455c81f740 1 -- 172.21.15.122:0/207877101 wait complete. 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 -- start start 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 Processor -- start 2024-09-12T02:10:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e3314b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e33150d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0e3313340 con 0x55f0e3310370 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f0e31f7290 con 0x55f0e330a9a0 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f0e33134c0 con 0x55f0e330b2e0 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.340+0000 7f455c81f740 10 client.0 _ll_register_callbacks cb 0x55f0e3228140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f45537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e3314b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f45537fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e3314b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:54298/0 (socket says 172.21.15.122:54298) 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43910/0 (socket says 172.21.15.122:43910) 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f45537fe640 1 -- 172.21.15.122:0/2887168351 learned_addr learned my addr 172.21.15.122:0/2887168351 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4552ffd640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e33150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 -- 172.21.15.122:0/2887168351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 msgr2=0x55f0e33150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e33150d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 -- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 msgr2=0x55f0e3314b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e3314b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 -- 172.21.15.122:0/2887168351 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e2f81210 con 0x55f0e3310370 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f45537fe640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e3314b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4553fff640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331c110 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f454c00eee0 tx=0x7f454c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820540f445c3f371 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4532ffd640 1 -- 172.21.15.122:0/2887168351 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c020040 con 0x55f0e3310370 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f4532ffd640 1 -- 172.21.15.122:0/2887168351 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f454c00dbe0 con 0x55f0e3310370 2024-09-12T02:10:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.341+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45480044b0 con 0x55f0e3310370 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f4532ffd640 1 -- 172.21.15.122:0/2887168351 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f454c004810 con 0x55f0e3310370 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 10 client.18672 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f0e304d620 con 0x55f0e3310370 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f4532ffd640 1 -- 172.21.15.122:0/2887168351 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45480044b0 con 0x55f0e3310370 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f4532ffd640 1 -- 172.21.15.122:0/2887168351 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f454c02d020 con 0x55f0e3310370 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 10 client.18672 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 20 client.18672 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3322cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 4 client.18672 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e327ba20 con 0x55f0e33208d0 2024-09-12T02:10:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.342+0000 7f45537fe640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3322cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f45537fe640 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3322cc0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730cd26d51649a69 server_cookie=fff58ad8a30b066a in_seq=0 out_seq=0 2024-09-12T02:10:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f4532ffd640 10 client.18672 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f4532ffd640 1 -- 172.21.15.122:0/2887168351 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0e327ba20 con 0x55f0e33208d0 2024-09-12T02:10:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f4532ffd640 10 client.18672 handle_command_reply: tid=0 2024-09-12T02:10:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 client.18672 shutdown 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 msgr2=0x55f0e331c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331c110 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f454c00eee0 tx=0x7f454c004ab0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 shutdown_connections 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3322cc0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f4532ffd640 0 client.18672 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331c110 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e33150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 --2- 172.21.15.122:0/2887168351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330a9a0 0x55f0e3314b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.343+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 >> 172.21.15.122:0/2887168351 conn(0x55f0e3306010 msgr2=0x55f0e3307820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.344+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 shutdown_connections 2024-09-12T02:10:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.344+0000 7f455c81f740 1 -- 172.21.15.122:0/2887168351 wait complete. 2024-09-12T02:10:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.344+0000 7f455c81f740 20 client.18672 trim_cache size 0 max 16384 2024-09-12T02:10:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 Processor -- start 2024-09-12T02:10:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 -- start start 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 0x55f0e3312870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e331c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e3310370 0x55f0e3306ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45541adac0 con 0x55f0e330a9a0 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ab390 con 0x55f0e3310370 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f455c81f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a91c0 con 0x55f0e330b2e0 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f45537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 0x55f0e3312870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f45537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 0x55f0e3312870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43926/0 (socket says 172.21.15.122:43926) 2024-09-12T02:10:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f45537fe640 1 -- 172.21.15.122:0/516649444 learned_addr learned my addr 172.21.15.122:0/516649444 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f4553fff640 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e3310370 0x55f0e3306ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.359+0000 7f4552ffd640 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e331c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45537fe640 1 -- 172.21.15.122:0/516649444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 msgr2=0x55f0e331c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45537fe640 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e331c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45537fe640 1 -- 172.21.15.122:0/516649444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e3310370 msgr2=0x55f0e3306ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45537fe640 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e3310370 0x55f0e3306ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45537fe640 1 -- 172.21.15.122:0/516649444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e3323030 con 0x55f0e330a9a0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45537fe640 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 0x55f0e3312870 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4544009830 tx=0x7f45440a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9508bec38e27936d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45397fa640 1 -- 172.21.15.122:0/516649444 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440ece60 con 0x55f0e330a9a0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45397fa640 1 -- 172.21.15.122:0/516649444 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544030a00 con 0x55f0e330a9a0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.360+0000 7f45397fa640 1 -- 172.21.15.122:0/516649444 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440c4370 con 0x55f0e330a9a0 2024-09-12T02:10:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 -- 172.21.15.122:0/516649444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 msgr2=0x55f0e3312870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 0x55f0e3312870 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4544009830 tx=0x7f45440a6040 comp rx=0 tx=0).stop 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 -- 172.21.15.122:0/516649444 shutdown_connections 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e3310370 0x55f0e3306ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330b2e0 0x55f0e331c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 --2- 172.21.15.122:0/516649444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e330a9a0 0x55f0e3312870 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 -- 172.21.15.122:0/516649444 >> 172.21.15.122:0/516649444 conn(0x55f0e3306010 msgr2=0x55f0e3314b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 -- 172.21.15.122:0/516649444 shutdown_connections 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 -- 172.21.15.122:0/516649444 wait complete. 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 -- start start 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.361+0000 7f455c81f740 1 Processor -- start 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 0x55f0e3319310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 0x55f0e3319850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a9850 con 0x55f0e3310370 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45541aacc0 con 0x55f0e330b2e0 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a9f20 con 0x55f0e330a9a0 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f455c81f740 10 client.0 _ll_register_callbacks cb 0x7f454c01d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f45537fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 0x55f0e3319310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 0x55f0e3319850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 0x55f0e3319850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:54308/0 (socket says 172.21.15.122:54308) 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43936/0 (socket says 172.21.15.122:43936) 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f45537fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 0x55f0e3319310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:38932/0 (socket says 172.21.15.122:38932) 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 -- 172.21.15.122:0/3854855291 learned_addr learned my addr 172.21.15.122:0/3854855291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 -- 172.21.15.122:0/3854855291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 msgr2=0x55f0e3319310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 0x55f0e3319310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 -- 172.21.15.122:0/3854855291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 msgr2=0x55f0e331a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4552ffd640 1 -- 172.21.15.122:0/3854855291 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e331add0 con 0x55f0e330b2e0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f4553fff640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.362+0000 7f45537fe640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 0x55f0e3319310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f4552ffd640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 0x55f0e3319850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4548009710 tx=0x7f4548009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408bc1f8aca98423 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f4532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f4532ffd640 1 -- 172.21.15.122:0/3854855291 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454801d040 con 0x55f0e330b2e0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f4532ffd640 1 -- 172.21.15.122:0/3854855291 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4548005510 con 0x55f0e330b2e0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4544030a00 con 0x55f0e330b2e0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f4532ffd640 1 -- 172.21.15.122:0/3854855291 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f45480035f0 con 0x55f0e330b2e0 2024-09-12T02:10:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f455c81f740 10 client.27614 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f0e331c0e0 con 0x55f0e330b2e0 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.363+0000 7f4532ffd640 1 -- 172.21.15.122:0/3854855291 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544030a00 con 0x55f0e330b2e0 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f4532ffd640 1 -- 172.21.15.122:0/3854855291 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f454800f9c0 con 0x55f0e330b2e0 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f455c81f740 10 client.27614 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f455c81f740 10 client.27614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f455c81f740 20 client.27614 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f455c81f740 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3396060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f455c81f740 4 client.27614 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0e3275d70 con 0x55f0e33208d0 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f45537fe640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3396060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f4532ffd640 10 client.27614 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.364+0000 7f45537fe640 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3396060 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9666528dd8ddae server_cookie=77e9ef34df090791 in_seq=0 out_seq=0 2024-09-12T02:10:00.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:00 smithi151 ceph-mon[35229]: pgmap v1045: 97 pgs: 97 active+clean; 556 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-09-12T02:10:00.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:00 smithi151 ceph-mon[35229]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:10:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:00 smithi183 ceph-mon[35127]: pgmap v1045: 97 pgs: 97 active+clean; 556 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-09-12T02:10:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:00 smithi183 ceph-mon[35127]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:10:00.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:00 smithi122 ceph-mon[30925]: pgmap v1045: 97 pgs: 97 active+clean; 556 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-09-12T02:10:00.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:00 smithi122 ceph-mon[30925]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:10:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f4532ffd640 1 -- 172.21.15.122:0/3854855291 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0e3275d70 con 0x55f0e33208d0 2024-09-12T02:10:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f4532ffd640 10 client.27614 handle_command_reply: tid=0 2024-09-12T02:10:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 client.27614 shutdown 2024-09-12T02:10:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 msgr2=0x55f0e3319850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 0x55f0e3319850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4548009710 tx=0x7f4548009c20 comp rx=0 tx=0).stop 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 shutdown_connections 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f0e33208d0 0x55f0e3396060 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0e3310370 0x55f0e331a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f4532ffd640 0 client.27614 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f0e330b2e0 0x55f0e3319850 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 --2- 172.21.15.122:0/3854855291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0e330a9a0 0x55f0e3319310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 >> 172.21.15.122:0/3854855291 conn(0x55f0e3306010 msgr2=0x55f0e33049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.754+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 shutdown_connections 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f455c81f740 1 -- 172.21.15.122:0/3854855291 wait complete. 2024-09-12T02:10:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f455c81f740 20 client.27614 trim_cache size 0 max 16384 2024-09-12T02:10:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:10:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:10:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:10:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f453c0a2c30 msgr2=0x7f453c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f453c0a2c30 0x7f453c0a50f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f454c005c40 tx=0x7f454c010040 comp rx=0 tx=0).stop 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 msgr2=0x7f455419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f455419a080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4544035040 tx=0x7f4544033d90 comp rx=0 tx=0).stop 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 shutdown_connections 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f453c0a2c30 0x7f453c0a50f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4554101d80 0x7f45541a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45541013c0 0x7f455419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 --2- 172.21.15.122:0/3168574598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45541001d0 0x7f455419a080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 >> 172.21.15.122:0/3168574598 conn(0x7f45540fb930 msgr2=0x7f45541025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 shutdown_connections 2024-09-12T02:10:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:00.755+0000 7f4559a82640 1 -- 172.21.15.122:0/3168574598 wait complete. 2024-09-12T02:10:01.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 218868 19.11 MB/sec execute 468 sec latency 179.209 ms 2024-09-12T02:10:02.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 219626 19.15 MB/sec execute 469 sec latency 64.939 ms 2024-09-12T02:10:02.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:02 smithi151 ceph-mon[35229]: pgmap v1046: 97 pgs: 97 active+clean; 548 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.5 MiB/s wr, 480 op/s 2024-09-12T02:10:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:02 smithi183 ceph-mon[35127]: pgmap v1046: 97 pgs: 97 active+clean; 548 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.5 MiB/s wr, 480 op/s 2024-09-12T02:10:02.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:02 smithi122 ceph-mon[30925]: pgmap v1046: 97 pgs: 97 active+clean; 548 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.5 MiB/s wr, 480 op/s 2024-09-12T02:10:03.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 220214 19.19 MB/sec execute 470 sec latency 275.413 ms 2024-09-12T02:10:04.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 220763 19.23 MB/sec execute 471 sec latency 64.041 ms 2024-09-12T02:10:04.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:04 smithi151 ceph-mon[35229]: pgmap v1047: 97 pgs: 97 active+clean; 532 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-09-12T02:10:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:04 smithi183 ceph-mon[35127]: pgmap v1047: 97 pgs: 97 active+clean; 532 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-09-12T02:10:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:04 smithi122 ceph-mon[30925]: pgmap v1047: 97 pgs: 97 active+clean; 532 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-09-12T02:10:05.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 221320 19.25 MB/sec execute 472 sec latency 122.418 ms 2024-09-12T02:10:06.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 221744 19.27 MB/sec execute 473 sec latency 142.737 ms 2024-09-12T02:10:06.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:06 smithi151 ceph-mon[35229]: pgmap v1048: 97 pgs: 97 active+clean; 538 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 585 op/s 2024-09-12T02:10:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:06 smithi183 ceph-mon[35127]: pgmap v1048: 97 pgs: 97 active+clean; 538 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 585 op/s 2024-09-12T02:10:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:06 smithi122 ceph-mon[30925]: pgmap v1048: 97 pgs: 97 active+clean; 538 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 585 op/s 2024-09-12T02:10:07.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 222203 19.27 MB/sec execute 474 sec latency 86.014 ms 2024-09-12T02:10:08.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 222625 19.28 MB/sec execute 475 sec latency 239.560 ms 2024-09-12T02:10:08.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:08 smithi151 ceph-mon[35229]: pgmap v1049: 97 pgs: 97 active+clean; 537 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 531 op/s 2024-09-12T02:10:08.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:08 smithi183 ceph-mon[35127]: pgmap v1049: 97 pgs: 97 active+clean; 537 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 531 op/s 2024-09-12T02:10:08.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:08 smithi122 ceph-mon[30925]: pgmap v1049: 97 pgs: 97 active+clean; 537 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.3 MiB/s wr, 531 op/s 2024-09-12T02:10:09.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223200 19.29 MB/sec execute 476 sec latency 92.055 ms 2024-09-12T02:10:10.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223963 19.32 MB/sec execute 477 sec latency 63.858 ms 2024-09-12T02:10:10.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:10 smithi151 ceph-mon[35229]: pgmap v1050: 97 pgs: 97 active+clean; 541 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.1 MiB/s wr, 564 op/s 2024-09-12T02:10:10.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:10 smithi183 ceph-mon[35127]: pgmap v1050: 97 pgs: 97 active+clean; 541 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.1 MiB/s wr, 564 op/s 2024-09-12T02:10:10.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:10 smithi122 ceph-mon[30925]: pgmap v1050: 97 pgs: 97 active+clean; 541 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.1 MiB/s wr, 564 op/s 2024-09-12T02:10:11.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 224625 19.37 MB/sec execute 478 sec latency 73.022 ms 2024-09-12T02:10:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:11.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:11.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:10:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:10:11.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:11.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:11.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:11.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:10:12.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 225358 19.43 MB/sec execute 479 sec latency 71.038 ms 2024-09-12T02:10:12.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:12 smithi151 ceph-mon[35229]: pgmap v1051: 97 pgs: 97 active+clean; 547 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 628 op/s 2024-09-12T02:10:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:12 smithi183 ceph-mon[35127]: pgmap v1051: 97 pgs: 97 active+clean; 547 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 628 op/s 2024-09-12T02:10:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:12 smithi122 ceph-mon[30925]: pgmap v1051: 97 pgs: 97 active+clean; 547 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 628 op/s 2024-09-12T02:10:13.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 225729 19.43 MB/sec execute 480 sec latency 238.125 ms 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 -- 172.21.15.122:0/3758958998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 msgr2=0x7f17901020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 --2- 172.21.15.122:0/3758958998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 0x7f17901020b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402a410 comp rx=0 tx=0).stop 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 -- 172.21.15.122:0/3758958998 shutdown_connections 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 --2- 172.21.15.122:0/3758958998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17901025f0 0x7f1790108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 --2- 172.21.15.122:0/3758958998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 0x7f17901020b0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 --2- 172.21.15.122:0/3758958998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1790100a40 0x7f1790100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 -- 172.21.15.122:0/3758958998 >> 172.21.15.122:0/3758958998 conn(0x7f17900fc1e0 msgr2=0x7f17900fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 -- 172.21.15.122:0/3758958998 shutdown_connections 2024-09-12T02:10:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.875+0000 7f1794d11640 1 -- 172.21.15.122:0/3758958998 wait complete. 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 Processor -- start 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 -- start start 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1790100a40 0x7f1790199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 0x7f179019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 0x7f17901a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f179010b600 con 0x7f1790101c30 2024-09-12T02:10:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f179010b480 con 0x7f1790100a40 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f1794d11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f179010b780 con 0x7f17901025f0 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 0x7f17901a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f178dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 0x7f179019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f178e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1790100a40 0x7f1790199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 0x7f17901a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51528/0 (socket says 172.21.15.122:51528) 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f178ed9d640 1 -- 172.21.15.122:0/1673508178 learned_addr learned my addr 172.21.15.122:0/1673508178 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.876+0000 7f178e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1790100a40 0x7f1790199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40300/0 (socket says 172.21.15.122:40300) 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f178ed9d640 1 -- 172.21.15.122:0/1673508178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1790100a40 msgr2=0x7f1790199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1790100a40 0x7f1790199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f178ed9d640 1 -- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 msgr2=0x7f179019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 0x7f179019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f178ed9d640 1 -- 172.21.15.122:0/1673508178 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a1bd0 con 0x7f17901025f0 2024-09-12T02:10:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 0x7f17901a1560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f178800eb10 tx=0x7f178800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f177f7fe640 1 -- 172.21.15.122:0/1673508178 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788004b20 con 0x7f17901025f0 2024-09-12T02:10:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179006bc70 con 0x7f17901025f0 2024-09-12T02:10:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.877+0000 7f177f7fe640 1 -- 172.21.15.122:0/1673508178 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788004cc0 con 0x7f17901025f0 2024-09-12T02:10:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.878+0000 7f177f7fe640 1 -- 172.21.15.122:0/1673508178 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788010700 con 0x7f17901025f0 2024-09-12T02:10:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.878+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179006c140 con 0x7f17901025f0 2024-09-12T02:10:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.880+0000 7f177f7fe640 1 -- 172.21.15.122:0/1673508178 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1788005000 con 0x7f17901025f0 2024-09-12T02:10:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.880+0000 7f177f7fe640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17780a2c30 0x7f17780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178e59c640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17780a2c30 0x7f17780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 Processor -- start 2024-09-12T02:10:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f177f7fe640 1 -- 172.21.15.122:0/1673508178 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f17880cf590 con 0x7f17901025f0 2024-09-12T02:10:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 -- start start 2024-09-12T02:10:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f313cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3141da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c5f30a02f0 con 0x55c5f3142370 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5f3048270 con 0x55c5f313d2e0 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f1797aa6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c5f309c2b0 con 0x55c5f313c9a0 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178dd9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58534/0 (socket says 172.21.15.122:58534) 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178ed9d640 1 -- 172.21.15.122:0/1245469031 learned_addr learned my addr 172.21.15.122:0/1245469031 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178e59c640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17780a2c30 0x7f17780a50f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1790101a90 tx=0x7f1780005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.881+0000 7f178e59c640 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f313cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178ed9d640 1 -- 172.21.15.122:0/1245469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 msgr2=0x55c5f313cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f313cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178ed9d640 1 -- 172.21.15.122:0/1245469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 msgr2=0x55c5f3141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178ed9d640 1 -- 172.21.15.122:0/1245469031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f3047340 con 0x55c5f3142370 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178e59c640 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f313cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178dd9b640 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3144830 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f178809bd80 tx=0x7f17880981c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4095992b38c914c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f177cff9640 1 -- 172.21.15.122:0/1245469031 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178809bdf0 con 0x55c5f3142370 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f177cff9640 1 -- 172.21.15.122:0/1245469031 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17880a01a0 con 0x55c5f3142370 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f177cff9640 1 -- 172.21.15.122:0/1245469031 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880052f0 con 0x55c5f3142370 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.882+0000 7f1797aa6740 1 -- 172.21.15.122:0/1245469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 msgr2=0x55c5f3144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3144830 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f178809bd80 tx=0x7f17880981c0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- 172.21.15.122:0/1245469031 shutdown_connections 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3144830 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1245469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f313cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- 172.21.15.122:0/1245469031 >> 172.21.15.122:0/1245469031 conn(0x55c5f3138010 msgr2=0x55c5f3138410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- 172.21.15.122:0/1245469031 shutdown_connections 2024-09-12T02:10:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- 172.21.15.122:0/1245469031 wait complete. 2024-09-12T02:10:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- start start 2024-09-12T02:10:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 Processor -- start 2024-09-12T02:10:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f3146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3147130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c5f3145340 con 0x55c5f3142370 2024-09-12T02:10:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5f3029290 con 0x55c5f313d2e0 2024-09-12T02:10:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c5f31454c0 con 0x55c5f313c9a0 2024-09-12T02:10:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f1797aa6740 10 client.0 _ll_register_callbacks cb 0x55c5f305a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58544/0 (socket says 172.21.15.122:58544) 2024-09-12T02:10:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f178ed9d640 1 -- 172.21.15.122:0/1925514600 learned_addr learned my addr 172.21.15.122:0/1925514600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f178dd9b640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.883+0000 7f178e59c640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f3146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f178ed9d640 1 -- 172.21.15.122:0/1925514600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 msgr2=0x55c5f3146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f3146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f178ed9d640 1 -- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 msgr2=0x55c5f3147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f178ed9d640 1 -- 172.21.15.122:0/1925514600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f2db3210 con 0x55c5f3142370 2024-09-12T02:10:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314e170 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f17880bc060 tx=0x7f178809b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d109faf70852574f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1775ffb640 1 -- 172.21.15.122:0/1925514600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880cf2f0 con 0x55c5f3142370 2024-09-12T02:10:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1775ffb640 1 -- 172.21.15.122:0/1925514600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f17880052f0 con 0x55c5f3142370 2024-09-12T02:10:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17880a01a0 con 0x55c5f3142370 2024-09-12T02:10:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1775ffb640 1 -- 172.21.15.122:0/1925514600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f178809bdf0 con 0x55c5f3142370 2024-09-12T02:10:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1797aa6740 10 client.18690 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c5f2e7f620 con 0x55c5f3142370 2024-09-12T02:10:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1775ffb640 1 -- 172.21.15.122:0/1925514600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17880a01a0 con 0x55c5f3142370 2024-09-12T02:10:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1775ffb640 1 -- 172.21.15.122:0/1925514600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f178809add0 con 0x55c5f3142370 2024-09-12T02:10:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1797aa6740 10 client.18690 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.884+0000 7f1797aa6740 10 client.18690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f1797aa6740 20 client.18690 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f3154860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f1797aa6740 4 client.18690 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5f30ada20 con 0x55c5f3152470 2024-09-12T02:10:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f178e59c640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f3154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f1775ffb640 10 client.18690 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.885+0000 7f178e59c640 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f3154860 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43263779f770c2be server_cookie=bd4d15d1aa759412 in_seq=0 out_seq=0 2024-09-12T02:10:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.886+0000 7f1775ffb640 1 -- 172.21.15.122:0/1925514600 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5f30ada20 con 0x55c5f3152470 2024-09-12T02:10:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.886+0000 7f1775ffb640 10 client.18690 handle_command_reply: tid=0 2024-09-12T02:10:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.886+0000 7f1797aa6740 1 client.18690 shutdown 2024-09-12T02:10:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 msgr2=0x55c5f314e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314e170 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f17880bc060 tx=0x7f178809b850 comp rx=0 tx=0).stop 2024-09-12T02:10:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 shutdown_connections 2024-09-12T02:10:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f3154860 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1775ffb640 0 client.18690 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314e170 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f3147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1925514600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f3146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 >> 172.21.15.122:0/1925514600 conn(0x55c5f3138010 msgr2=0x55c5f3139820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 shutdown_connections 2024-09-12T02:10:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 1 -- 172.21.15.122:0/1925514600 wait complete. 2024-09-12T02:10:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.887+0000 7f1797aa6740 20 client.18690 trim_cache size 0 max 16384 2024-09-12T02:10:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 Processor -- start 2024-09-12T02:10:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 -- start start 2024-09-12T02:10:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 0x55c5f31383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313d2e0 0x55c5f314e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3138930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a5180 con 0x55c5f3142370 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a2a50 con 0x55c5f313c9a0 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f1797aa6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1790072f00 con 0x55c5f313d2e0 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f178e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 0x55c5f31383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f178e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 0x55c5f31383f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40330/0 (socket says 172.21.15.122:40330) 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58552/0 (socket says 172.21.15.122:58552) 2024-09-12T02:10:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f178e59c640 1 -- 172.21.15.122:0/3081646061 learned_addr learned my addr 172.21.15.122:0/3081646061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.903+0000 7f178dd9b640 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313d2e0 0x55c5f314e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178ed9d640 1 -- 172.21.15.122:0/3081646061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313d2e0 msgr2=0x55c5f314e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178ed9d640 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313d2e0 0x55c5f314e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178ed9d640 1 -- 172.21.15.122:0/3081646061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 msgr2=0x55c5f31383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178ed9d640 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 0x55c5f31383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178ed9d640 1 -- 172.21.15.122:0/3081646061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f3137e30 con 0x55c5f3142370 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178e59c640 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 0x55c5f31383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f178ed9d640 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3138930 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f178809bd80 tx=0x7f178809a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0ea2d58f8f1217 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f1777fff640 1 -- 172.21.15.122:0/3081646061 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178809aeb0 con 0x55c5f3142370 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f1777fff640 1 -- 172.21.15.122:0/3081646061 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17880a01a0 con 0x55c5f3142370 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.904+0000 7f1777fff640 1 -- 172.21.15.122:0/3081646061 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880052f0 con 0x55c5f3142370 2024-09-12T02:10:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 -- 172.21.15.122:0/3081646061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 msgr2=0x55c5f3138930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3138930 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f178809bd80 tx=0x7f178809a300 comp rx=0 tx=0).stop 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 -- 172.21.15.122:0/3081646061 shutdown_connections 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f3138930 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313d2e0 0x55c5f314e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 --2- 172.21.15.122:0/3081646061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313c9a0 0x55c5f31383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 -- 172.21.15.122:0/3081646061 >> 172.21.15.122:0/3081646061 conn(0x55c5f3138010 msgr2=0x55c5f3146080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 -- 172.21.15.122:0/3081646061 shutdown_connections 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 -- 172.21.15.122:0/3081646061 wait complete. 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.905+0000 7f1797aa6740 1 -- start start 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 Processor -- start 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f314b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f314b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1790073590 con 0x55c5f3142370 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17901a2380 con 0x55c5f313d2e0 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1790073c60 con 0x55c5f313c9a0 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f178e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f314b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f178e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f314b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51576/0 (socket says 172.21.15.122:51576) 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f178e59c640 1 -- 172.21.15.122:0/1172835429 learned_addr learned my addr 172.21.15.122:0/1172835429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f178dd9b640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f314b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f178ed9d640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.906+0000 7f1797aa6740 10 client.0 _ll_register_callbacks cb 0x7f1780005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f178e59c640 1 -- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 msgr2=0x55c5f314b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f178e59c640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f314b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f178e59c640 1 -- 172.21.15.122:0/1172835429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 msgr2=0x55c5f314c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f178e59c640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f178e59c640 1 -- 172.21.15.122:0/1172835429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f314cf20 con 0x55c5f313c9a0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f178e59c640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f314b430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f178001eaa0 tx=0x7f1780004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7818a0d951a3df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1775ffb640 1 -- 172.21.15.122:0/1172835429 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780038060 con 0x55c5f313c9a0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17880a01a0 con 0x55c5f313c9a0 2024-09-12T02:10:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1775ffb640 1 -- 172.21.15.122:0/1172835429 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1780009040 con 0x55c5f313c9a0 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1775ffb640 1 -- 172.21.15.122:0/1172835429 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f178002a570 con 0x55c5f313c9a0 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1797aa6740 10 client.27625 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.907+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c5f314e2c0 con 0x55c5f313c9a0 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1775ffb640 1 -- 172.21.15.122:0/1172835429 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17880a01a0 con 0x55c5f313c9a0 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1775ffb640 1 -- 172.21.15.122:0/1172835429 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f178001cc00 con 0x55c5f313c9a0 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1797aa6740 10 client.27625 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1797aa6740 10 client.27625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1797aa6740 20 client.27625 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f31bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1797aa6740 4 client.27625 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5f31420f0 con 0x55c5f3152470 2024-09-12T02:10:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f178dd9b640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f31bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f178dd9b640 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f31bf5f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a2953b5b5d541e server_cookie=ac06be982d7d9d95 in_seq=0 out_seq=0 2024-09-12T02:10:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:13.908+0000 7f1775ffb640 10 client.27625 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:14.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 226164 19.44 MB/sec execute 481 sec latency 113.561 ms 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.195+0000 7f1775ffb640 1 -- 172.21.15.122:0/1172835429 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5f31420f0 con 0x55c5f3152470 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.195+0000 7f1775ffb640 10 client.27625 handle_command_reply: tid=0 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.195+0000 7f1797aa6740 1 client.27625 shutdown 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.195+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 msgr2=0x55c5f314b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.195+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f314b430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f178001eaa0 tx=0x7f1780004730 comp rx=0 tx=0).stop 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.195+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 shutdown_connections 2024-09-12T02:10:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55c5f3152470 0x55c5f31bf5f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1775ffb640 0 client.27625 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5f3142370 0x55c5f314c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5f313d2e0 0x55c5f314b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 --2- 172.21.15.122:0/1172835429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5f313c9a0 0x55c5f314b430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 >> 172.21.15.122:0/1172835429 conn(0x55c5f3138010 msgr2=0x55c5f3145860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 shutdown_connections 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 1 -- 172.21.15.122:0/1172835429 wait complete. 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.196+0000 7f1797aa6740 20 client.27625 trim_cache size 0 max 16384 2024-09-12T02:10:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:10:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:10:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:10:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17780a2c30 msgr2=0x7f17780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17780a2c30 0x7f17780a50f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1790101a90 tx=0x7f1780005f90 comp rx=0 tx=0).stop 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 msgr2=0x7f17901a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 0x7f17901a1560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f178800eb10 tx=0x7f178800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 shutdown_connections 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f17780a2c30 0x7f17780a50f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17901025f0 0x7f17901a1560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1790101c30 0x7f179019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 --2- 172.21.15.122:0/1673508178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1790100a40 0x7f1790199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.197+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 >> 172.21.15.122:0/1673508178 conn(0x7f17900fc1e0 msgr2=0x7f1790104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.198+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 shutdown_connections 2024-09-12T02:10:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:14.198+0000 7f1794d11640 1 -- 172.21.15.122:0/1673508178 wait complete. 2024-09-12T02:10:14.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:14 smithi151 ceph-mon[35229]: pgmap v1052: 97 pgs: 97 active+clean; 548 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.2 MiB/s wr, 549 op/s 2024-09-12T02:10:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:14 smithi183 ceph-mon[35127]: pgmap v1052: 97 pgs: 97 active+clean; 548 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.2 MiB/s wr, 549 op/s 2024-09-12T02:10:14.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:14 smithi122 ceph-mon[30925]: pgmap v1052: 97 pgs: 97 active+clean; 548 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.2 MiB/s wr, 549 op/s 2024-09-12T02:10:15.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 226581 19.44 MB/sec execute 482 sec latency 188.944 ms 2024-09-12T02:10:16.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 227025 19.45 MB/sec execute 483 sec latency 163.485 ms 2024-09-12T02:10:16.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:16 smithi151 ceph-mon[35229]: pgmap v1053: 97 pgs: 97 active+clean; 557 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 679 op/s 2024-09-12T02:10:16.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:16 smithi183 ceph-mon[35127]: pgmap v1053: 97 pgs: 97 active+clean; 557 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 679 op/s 2024-09-12T02:10:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:16 smithi122 ceph-mon[30925]: pgmap v1053: 97 pgs: 97 active+clean; 557 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 679 op/s 2024-09-12T02:10:17.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 227721 19.46 MB/sec execute 484 sec latency 62.620 ms 2024-09-12T02:10:18.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 228269 19.48 MB/sec execute 485 sec latency 145.393 ms 2024-09-12T02:10:18.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:18 smithi151 ceph-mon[35229]: pgmap v1054: 97 pgs: 97 active+clean; 557 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 529 op/s 2024-09-12T02:10:18.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:18 smithi183 ceph-mon[35127]: pgmap v1054: 97 pgs: 97 active+clean; 557 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 529 op/s 2024-09-12T02:10:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:18 smithi122 ceph-mon[30925]: pgmap v1054: 97 pgs: 97 active+clean; 557 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 529 op/s 2024-09-12T02:10:19.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 228877 19.52 MB/sec execute 486 sec latency 238.298 ms 2024-09-12T02:10:20.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 229502 19.56 MB/sec execute 487 sec latency 77.386 ms 2024-09-12T02:10:20.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:20 smithi151 ceph-mon[35229]: pgmap v1055: 97 pgs: 97 active+clean; 552 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 610 op/s 2024-09-12T02:10:20.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:20 smithi183 ceph-mon[35127]: pgmap v1055: 97 pgs: 97 active+clean; 552 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 610 op/s 2024-09-12T02:10:20.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:20 smithi122 ceph-mon[30925]: pgmap v1055: 97 pgs: 97 active+clean; 552 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 610 op/s 2024-09-12T02:10:21.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 229952 19.58 MB/sec execute 488 sec latency 166.793 ms 2024-09-12T02:10:22.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 230342 19.59 MB/sec execute 489 sec latency 104.350 ms 2024-09-12T02:10:22.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:22 smithi151 ceph-mon[35229]: pgmap v1056: 97 pgs: 97 active+clean; 547 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 666 op/s 2024-09-12T02:10:22.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:22 smithi183 ceph-mon[35127]: pgmap v1056: 97 pgs: 97 active+clean; 547 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 666 op/s 2024-09-12T02:10:22.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:22 smithi122 ceph-mon[30925]: pgmap v1056: 97 pgs: 97 active+clean; 547 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 666 op/s 2024-09-12T02:10:23.085 INFO:tasks.workunit.client.0.smithi122.stdout: 10 230703 19.57 MB/sec execute 490 sec latency 121.789 ms 2024-09-12T02:10:24.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231002 19.57 MB/sec execute 491 sec latency 266.072 ms 2024-09-12T02:10:24.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:24 smithi151 ceph-mon[35229]: pgmap v1057: 97 pgs: 97 active+clean; 534 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 596 op/s 2024-09-12T02:10:24.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:24 smithi183 ceph-mon[35127]: pgmap v1057: 97 pgs: 97 active+clean; 534 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 596 op/s 2024-09-12T02:10:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:24 smithi122 ceph-mon[30925]: pgmap v1057: 97 pgs: 97 active+clean; 534 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 596 op/s 2024-09-12T02:10:25.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231419 19.57 MB/sec execute 492 sec latency 131.245 ms 2024-09-12T02:10:26.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231916 19.56 MB/sec execute 493 sec latency 76.478 ms 2024-09-12T02:10:26.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:26 smithi151 ceph-mon[35229]: pgmap v1058: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 678 op/s 2024-09-12T02:10:26.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:26 smithi183 ceph-mon[35127]: pgmap v1058: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 678 op/s 2024-09-12T02:10:26.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:26 smithi122 ceph-mon[30925]: pgmap v1058: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.1 MiB/s wr, 678 op/s 2024-09-12T02:10:27.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 232672 19.62 MB/sec execute 494 sec latency 49.769 ms 2024-09-12T02:10:27.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 -- 172.21.15.122:0/1673153436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c101c30 msgr2=0x7fca5c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.294 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 --2- 172.21.15.122:0/1673153436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c101c30 0x7fca5c1020b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fca48009900 tx=0x7fca4802a410 comp rx=0 tx=0).stop 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 -- 172.21.15.122:0/1673153436 shutdown_connections 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 --2- 172.21.15.122:0/1673153436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca5c1025f0 0x7fca5c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 --2- 172.21.15.122:0/1673153436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c101c30 0x7fca5c1020b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 --2- 172.21.15.122:0/1673153436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca5c100a40 0x7fca5c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 -- 172.21.15.122:0/1673153436 >> 172.21.15.122:0/1673153436 conn(0x7fca5c0fc1e0 msgr2=0x7fca5c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 -- 172.21.15.122:0/1673153436 shutdown_connections 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.292+0000 7fca63023640 1 -- 172.21.15.122:0/1673153436 wait complete. 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 Processor -- start 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 -- start start 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 0x7fca5c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca5c101c30 0x7fca5c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca5c1025f0 0x7fca5c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10c6c0 con 0x7fca5c100a40 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10c540 con 0x7fca5c1025f0 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca63023640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10c840 con 0x7fca5c101c30 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 0x7fca5c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 0x7fca5c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34394/0 (socket says 172.21.15.122:34394) 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 -- 172.21.15.122:0/1200227610 learned_addr learned my addr 172.21.15.122:0/1200227610 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca5bfff640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca5c101c30 0x7fca5c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca615c0640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca5c1025f0 0x7fca5c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 -- 172.21.15.122:0/1200227610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca5c101c30 msgr2=0x7fca5c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca5c101c30 0x7fca5c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 -- 172.21.15.122:0/1200227610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca5c1025f0 msgr2=0x7fca5c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca5c1025f0 0x7fca5c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 -- 172.21.15.122:0/1200227610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c19fa90 con 0x7fca5c100a40 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca60dbf640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 0x7fca5c06a410 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9c0 tx=0x7fca5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.293+0000 7fca59ffb640 1 -- 172.21.15.122:0/1200227610 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cd80 con 0x7fca5c100a40 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.294+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c19fd20 con 0x7fca5c100a40 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.294+0000 7fca59ffb640 1 -- 172.21.15.122:0/1200227610 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca500035f0 con 0x7fca5c100a40 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.294+0000 7fca59ffb640 1 -- 172.21.15.122:0/1200227610 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50010690 con 0x7fca5c100a40 2024-09-12T02:10:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.294+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c1a0190 con 0x7fca5c100a40 2024-09-12T02:10:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.296+0000 7fca65dbc740 1 Processor -- start 2024-09-12T02:10:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 -- start start 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca59ffb640 1 -- 172.21.15.122:0/1200227610 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fca500037a0 con 0x7fca5c100a40 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f2a84472f0 con 0x55f2a84e42e0 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f2a83ef270 con 0x55f2a84e9370 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca65dbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2a84432b0 con 0x55f2a84e39a0 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca60dbf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca60dbf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84e8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34396/0 (socket says 172.21.15.122:34396) 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca60dbf640 1 -- 172.21.15.122:0/1360487300 learned_addr learned my addr 172.21.15.122:0/1360487300 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca615c0640 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca5bfff640 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca615c0640 1 -- 172.21.15.122:0/1360487300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 msgr2=0x55f2a84eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca615c0640 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca615c0640 1 -- 172.21.15.122:0/1360487300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 msgr2=0x55f2a84e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca615c0640 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84e8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca615c0640 1 -- 172.21.15.122:0/1360487300 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a83ee340 con 0x55f2a84e39a0 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.297+0000 7fca60dbf640 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca615c0640 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca54009ec0 tx=0x7fca5400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6b15dad9261f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca59ffb640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca4c0a2c30 0x7fca4c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca46ffd640 1 -- 172.21.15.122:0/1360487300 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54015070 con 0x55f2a84e39a0 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca60dbf640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca4c0a2c30 0x7fca4c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca46ffd640 1 -- 172.21.15.122:0/1360487300 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca54004d20 con 0x55f2a84e39a0 2024-09-12T02:10:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca46ffd640 1 -- 172.21.15.122:0/1360487300 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca540117c0 con 0x55f2a84e39a0 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.298+0000 7fca59ffb640 1 -- 172.21.15.122:0/1200227610 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fca500cf440 con 0x7fca5c100a40 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 -- 172.21.15.122:0/1360487300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 msgr2=0x55f2a84e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca54009ec0 tx=0x7fca5400f730 comp rx=0 tx=0).stop 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca60dbf640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca4c0a2c30 0x7fca4c0a50f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fca500ccf40 tx=0x7fca500d2ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 -- 172.21.15.122:0/1360487300 shutdown_connections 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1360487300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3da0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 -- 172.21.15.122:0/1360487300 >> 172.21.15.122:0/1360487300 conn(0x55f2a84df010 msgr2=0x55f2a84df410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 -- 172.21.15.122:0/1360487300 shutdown_connections 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 -- 172.21.15.122:0/1360487300 wait complete. 2024-09-12T02:10:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 -- start start 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.299+0000 7fca65dbc740 1 Processor -- start 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84edba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e42e0 0x55f2a84ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 0x55f2a84f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f2a84ec340 con 0x55f2a84e9370 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f2a83d0290 con 0x55f2a84e42e0 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2a84ec4c0 con 0x55f2a84e39a0 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 10 client.0 _ll_register_callbacks cb 0x55f2a8401140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 0x55f2a84f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca5bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e42e0 0x55f2a84ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca5bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e42e0 0x55f2a84ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:55280/0 (socket says 172.21.15.122:55280) 2024-09-12T02:10:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 0x55f2a84f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34410/0 (socket says 172.21.15.122:34410) 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca5bfff640 1 -- 172.21.15.122:0/172392082 learned_addr learned my addr 172.21.15.122:0/172392082 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 -- 172.21.15.122:0/172392082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 msgr2=0x55f2a84edba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84edba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 -- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e42e0 msgr2=0x55f2a84ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e42e0 0x55f2a84ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 -- 172.21.15.122:0/172392082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a815a210 con 0x55f2a84e9370 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca615c0640 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 0x55f2a84f5120 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fca54009c80 tx=0x7fca54000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272949a4f29f642a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca3bfff640 1 -- 172.21.15.122:0/172392082 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54015030 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca3bfff640 1 -- 172.21.15.122:0/172392082 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fca540053d0 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.300+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca54004d20 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca3bfff640 1 -- 172.21.15.122:0/172392082 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fca54022040 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 10 client.18708 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f2a8226620 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca3bfff640 1 -- 172.21.15.122:0/172392082 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca54004d20 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca3bfff640 1 -- 172.21.15.122:0/172392082 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fca540044f0 con 0x55f2a84e9370 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 10 client.18708 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 10 client.18708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 20 client.18708 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a84fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 4 client.18708 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2a8454a20 con 0x55f2a84f9600 2024-09-12T02:10:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.301+0000 7fca5bfff640 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a84fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.302+0000 7fca3bfff640 10 client.18708 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.302+0000 7fca5bfff640 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a84fb9f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d2d9488224a46e server_cookie=dd484b1c7558bf35 in_seq=0 out_seq=0 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca3bfff640 1 -- 172.21.15.122:0/172392082 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2a8454a20 con 0x55f2a84f9600 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca3bfff640 10 client.18708 handle_command_reply: tid=0 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 client.18708 shutdown 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 msgr2=0x55f2a84f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 0x55f2a84f5120 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fca54009c80 tx=0x7fca54000f40 comp rx=0 tx=0).stop 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 shutdown_connections 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a84fb9f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e9370 0x55f2a84f5120 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca3bfff640 0 client.18708 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e42e0 0x55f2a84ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 --2- 172.21.15.122:0/172392082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 >> 172.21.15.122:0/172392082 conn(0x55f2a84df010 msgr2=0x55f2a84e0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 shutdown_connections 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 1 -- 172.21.15.122:0/172392082 wait complete. 2024-09-12T02:10:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.303+0000 7fca65dbc740 20 client.18708 trim_cache size 0 max 16384 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.317+0000 7fca65dbc740 1 Processor -- start 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 -- start start 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1ab990 con 0x55f2a84e42e0 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1a9260 con 0x55f2a84e9370 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca65dbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1a7090 con 0x55f2a84e39a0 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34422/0 (socket says 172.21.15.122:34422) 2024-09-12T02:10:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 -- 172.21.15.122:0/2474485170 learned_addr learned my addr 172.21.15.122:0/2474485170 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca615c0640 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca615c0640 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 -- 172.21.15.122:0/2474485170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 msgr2=0x55f2a84e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 -- 172.21.15.122:0/2474485170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 msgr2=0x55f2a84e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 -- 172.21.15.122:0/2474485170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a84ee280 con 0x55f2a84e42e0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca5bfff640 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f50b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fca480314e0 tx=0x7fca48005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1639cad97f8247d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca467fc640 1 -- 172.21.15.122:0/2474485170 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48035040 con 0x55f2a84e42e0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.318+0000 7fca467fc640 1 -- 172.21.15.122:0/2474485170 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca48005040 con 0x55f2a84e42e0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca467fc640 1 -- 172.21.15.122:0/2474485170 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480049e0 con 0x55f2a84e42e0 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 -- 172.21.15.122:0/2474485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 msgr2=0x55f2a84f50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f50b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fca480314e0 tx=0x7fca48005fd0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 -- 172.21.15.122:0/2474485170 shutdown_connections 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e9370 0x55f2a84e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f50b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 --2- 172.21.15.122:0/2474485170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e39a0 0x55f2a84e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 -- 172.21.15.122:0/2474485170 >> 172.21.15.122:0/2474485170 conn(0x55f2a84df010 msgr2=0x55f2a84de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 -- 172.21.15.122:0/2474485170 shutdown_connections 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.319+0000 7fca65dbc740 1 -- 172.21.15.122:0/2474485170 wait complete. 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 -- start start 2024-09-12T02:10:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 Processor -- start 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e39a0 0x55f2a84f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e9370 0x55f2a8562270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1a7720 con 0x55f2a84e42e0 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1a8b90 con 0x55f2a84e39a0 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1a7df0 con 0x55f2a84e9370 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca65dbc740 10 client.0 _ll_register_callbacks cb 0x7fca500d2d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34430/0 (socket says 172.21.15.122:34430) 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 -- 172.21.15.122:0/1057555885 learned_addr learned my addr 172.21.15.122:0/1057555885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca615c0640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e9370 0x55f2a8562270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca615c0640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e39a0 0x55f2a84f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 -- 172.21.15.122:0/1057555885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e9370 msgr2=0x55f2a8562270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e9370 0x55f2a8562270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 -- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e39a0 msgr2=0x55f2a84f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e39a0 0x55f2a84f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.320+0000 7fca5bfff640 1 -- 172.21.15.122:0/1057555885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a85627b0 con 0x55f2a84e42e0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca5bfff640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f25d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fca4802a930 tx=0x7fca480051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fba0f29419573e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca3bfff640 1 -- 172.21.15.122:0/1057555885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48054070 con 0x55f2a84e42e0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca3bfff640 1 -- 172.21.15.122:0/1057555885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fca48004780 con 0x55f2a84e42e0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca48005040 con 0x55f2a84e42e0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca3bfff640 1 -- 172.21.15.122:0/1057555885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fca48060050 con 0x55f2a84e42e0 2024-09-12T02:10:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 10 client.18714 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f2a8563a60 con 0x55f2a84e42e0 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca3bfff640 1 -- 172.21.15.122:0/1057555885 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca48005040 con 0x55f2a84e42e0 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca3bfff640 1 -- 172.21.15.122:0/1057555885 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fca48046b60 con 0x55f2a84e42e0 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 10 client.18714 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 10 client.18714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 20 client.18714 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a8566430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 4 client.18714 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2a844ed70 con 0x55f2a84f9600 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.321+0000 7fca615c0640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a8566430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.322+0000 7fca615c0640 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a8566430 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab7a884f45698c1 server_cookie=9b8f457f47ff510 in_seq=0 out_seq=0 2024-09-12T02:10:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.322+0000 7fca3bfff640 10 client.18714 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca3bfff640 1 -- 172.21.15.122:0/1057555885 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2a844ed70 con 0x55f2a84f9600 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca3bfff640 10 client.18714 handle_command_reply: tid=0 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 client.18714 shutdown 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 msgr2=0x55f2a84f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f25d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fca4802a930 tx=0x7fca480051f0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 shutdown_connections 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f2a84f9600 0x55f2a8566430 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2a84e9370 0x55f2a8562270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2a84e42e0 0x55f2a84f25d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca3bfff640 0 client.18714 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 --2- 172.21.15.122:0/1057555885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2a84e39a0 0x55f2a84f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 >> 172.21.15.122:0/1057555885 conn(0x55f2a84df010 msgr2=0x55f2a84de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 shutdown_connections 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 1 -- 172.21.15.122:0/1057555885 wait complete. 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.596+0000 7fca65dbc740 20 client.18714 trim_cache size 0 max 16384 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:10:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:10:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca4c0a2c30 msgr2=0x7fca4c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca4c0a2c30 0x7fca4c0a50f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fca500ccf40 tx=0x7fca500d2ec0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 msgr2=0x7fca5c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 0x7fca5c06a410 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9c0 tx=0x7fca5000ee80 comp rx=0 tx=0).stop 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 shutdown_connections 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fca4c0a2c30 0x7fca4c0a50f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca5c1025f0 0x7fca5c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca5c101c30 0x7fca5c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 --2- 172.21.15.122:0/1200227610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca5c100a40 0x7fca5c06a410 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 >> 172.21.15.122:0/1200227610 conn(0x7fca5c0fc1e0 msgr2=0x7fca5c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 shutdown_connections 2024-09-12T02:10:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:27.597+0000 7fca63023640 1 -- 172.21.15.122:0/1200227610 wait complete. 2024-09-12T02:10:28.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 233258 19.65 MB/sec execute 495 sec latency 200.604 ms 2024-09-12T02:10:28.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:28 smithi151 ceph-mon[35229]: pgmap v1059: 97 pgs: 97 active+clean; 539 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 535 op/s 2024-09-12T02:10:28.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:28 smithi183 ceph-mon[35127]: pgmap v1059: 97 pgs: 97 active+clean; 539 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 535 op/s 2024-09-12T02:10:28.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:28 smithi122 ceph-mon[30925]: pgmap v1059: 97 pgs: 97 active+clean; 539 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.4 MiB/s wr, 535 op/s 2024-09-12T02:10:29.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 233547 19.64 MB/sec execute 496 sec latency 222.560 ms 2024-09-12T02:10:30.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 234092 19.67 MB/sec execute 497 sec latency 76.839 ms 2024-09-12T02:10:30.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:30 smithi151 ceph-mon[35229]: pgmap v1060: 97 pgs: 97 active+clean; 539 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-09-12T02:10:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:30 smithi183 ceph-mon[35127]: pgmap v1060: 97 pgs: 97 active+clean; 539 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-09-12T02:10:30.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:30 smithi122 ceph-mon[30925]: pgmap v1060: 97 pgs: 97 active+clean; 539 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-09-12T02:10:31.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 234549 19.69 MB/sec execute 498 sec latency 71.304 ms 2024-09-12T02:10:32.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 235002 19.68 MB/sec execute 499 sec latency 74.042 ms 2024-09-12T02:10:32.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:32 smithi151 ceph-mon[35229]: pgmap v1061: 97 pgs: 97 active+clean; 543 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 551 op/s 2024-09-12T02:10:32.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:32 smithi183 ceph-mon[35127]: pgmap v1061: 97 pgs: 97 active+clean; 543 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 551 op/s 2024-09-12T02:10:32.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:32 smithi122 ceph-mon[30925]: pgmap v1061: 97 pgs: 97 active+clean; 543 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 551 op/s 2024-09-12T02:10:33.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 235682 19.71 MB/sec execute 500 sec latency 71.664 ms 2024-09-12T02:10:34.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 236014 19.71 MB/sec execute 501 sec latency 290.004 ms 2024-09-12T02:10:34.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:34 smithi151 ceph-mon[35229]: pgmap v1062: 97 pgs: 97 active+clean; 550 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 541 op/s 2024-09-12T02:10:34.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:34 smithi183 ceph-mon[35127]: pgmap v1062: 97 pgs: 97 active+clean; 550 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 541 op/s 2024-09-12T02:10:34.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:34 smithi122 ceph-mon[30925]: pgmap v1062: 97 pgs: 97 active+clean; 550 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 541 op/s 2024-09-12T02:10:35.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 236610 19.73 MB/sec execute 502 sec latency 136.371 ms 2024-09-12T02:10:36.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 237147 19.75 MB/sec execute 503 sec latency 354.554 ms 2024-09-12T02:10:36.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:36 smithi151 ceph-mon[35229]: pgmap v1063: 97 pgs: 97 active+clean; 552 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-09-12T02:10:36.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:36 smithi183 ceph-mon[35127]: pgmap v1063: 97 pgs: 97 active+clean; 552 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-09-12T02:10:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:36 smithi122 ceph-mon[30925]: pgmap v1063: 97 pgs: 97 active+clean; 552 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-09-12T02:10:37.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 237743 19.78 MB/sec execute 504 sec latency 69.205 ms 2024-09-12T02:10:38.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 238396 19.82 MB/sec execute 505 sec latency 55.437 ms 2024-09-12T02:10:38.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:38 smithi151 ceph-mon[35229]: pgmap v1064: 97 pgs: 97 active+clean; 552 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.6 MiB/s wr, 536 op/s 2024-09-12T02:10:38.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:38 smithi183 ceph-mon[35127]: pgmap v1064: 97 pgs: 97 active+clean; 552 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.6 MiB/s wr, 536 op/s 2024-09-12T02:10:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:38 smithi122 ceph-mon[30925]: pgmap v1064: 97 pgs: 97 active+clean; 552 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.6 MiB/s wr, 536 op/s 2024-09-12T02:10:39.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 238753 19.82 MB/sec execute 506 sec latency 231.900 ms 2024-09-12T02:10:40.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 239176 19.82 MB/sec execute 507 sec latency 165.279 ms 2024-09-12T02:10:40.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:40 smithi151 ceph-mon[35229]: pgmap v1065: 97 pgs: 97 active+clean; 555 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.2 MiB/s wr, 565 op/s 2024-09-12T02:10:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 -- 172.21.15.122:0/1048047723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728101970 msgr2=0x7f7728101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 --2- 172.21.15.122:0/1048047723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728101970 0x7f7728101d70 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f771c009900 tx=0x7f771c02a410 comp rx=0 tx=0).stop 2024-09-12T02:10:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 -- 172.21.15.122:0/1048047723 shutdown_connections 2024-09-12T02:10:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 --2- 172.21.15.122:0/1048047723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7728103520 0x7f7728109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 --2- 172.21.15.122:0/1048047723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7728102b60 0x7f7728102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 --2- 172.21.15.122:0/1048047723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728101970 0x7f7728101d70 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 -- 172.21.15.122:0/1048047723 >> 172.21.15.122:0/1048047723 conn(0x7f77280fd0d0 msgr2=0x7f77280ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 -- 172.21.15.122:0/1048047723 shutdown_connections 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 -- 172.21.15.122:0/1048047723 wait complete. 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 Processor -- start 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.696+0000 7f772df9d640 1 -- start start 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7728101970 0x7f7728197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 0x7f77281981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7728103520 0x7f772819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f772810d5f0 con 0x7f7728102b60 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f772810d470 con 0x7f7728101970 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f772810d770 con 0x7f7728103520 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 0x7f77281981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772cf9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7728101970 0x7f7728197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 0x7f77281981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34618/0 (socket says 172.21.15.122:34618) 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 -- 172.21.15.122:0/4084460847 learned_addr learned my addr 172.21.15.122:0/4084460847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772d79c640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7728103520 0x7f772819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 -- 172.21.15.122:0/4084460847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7728103520 msgr2=0x7f772819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7728103520 0x7f772819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 -- 172.21.15.122:0/4084460847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7728101970 msgr2=0x7f7728197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7728101970 0x7f7728197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 -- 172.21.15.122:0/4084460847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772819f8c0 con 0x7f7728102b60 2024-09-12T02:10:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7727fff640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 0x7f77281981c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f771400d950 tx=0x7f771400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7725ffb640 1 -- 172.21.15.122:0/4084460847 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714017030 con 0x7f7728102b60 2024-09-12T02:10:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7725ffb640 1 -- 172.21.15.122:0/4084460847 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7714004020 con 0x7f7728102b60 2024-09-12T02:10:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772819fbb0 con 0x7f7728102b60 2024-09-12T02:10:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.697+0000 7f7725ffb640 1 -- 172.21.15.122:0/4084460847 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77140047a0 con 0x7f7728102b60 2024-09-12T02:10:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.698+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77281a00d0 con 0x7f7728102b60 2024-09-12T02:10:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:40 smithi183 ceph-mon[35127]: pgmap v1065: 97 pgs: 97 active+clean; 555 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.2 MiB/s wr, 565 op/s 2024-09-12T02:10:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f7725ffb640 1 -- 172.21.15.122:0/4084460847 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f77140041d0 con 0x7f7728102b60 2024-09-12T02:10:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 Processor -- start 2024-09-12T02:10:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 -- start start 2024-09-12T02:10:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa24f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 0x5641aa254380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa254950 0x5641aa256e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5641aa1ae2f0 con 0x5641aa24ef80 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5641aa156270 con 0x5641aa24f8c0 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.700+0000 7f773206e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641aa1aa2b0 con 0x5641aa254950 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772d79c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa24f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 0x5641aa254380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f7725ffb640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f77180a2b80 0x7f77180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f7727fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa254950 0x5641aa256e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772d79c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa24f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34634/0 (socket says 172.21.15.122:34634) 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772d79c640 1 -- 172.21.15.122:0/502755621 learned_addr learned my addr 172.21.15.122:0/502755621 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 0x5641aa254380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:47270/0 (socket says 172.21.15.122:47270) 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772d79c640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f77180a2b80 0x7f77180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 -- 172.21.15.122:0/502755621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa254950 msgr2=0x5641aa256e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa254950 0x5641aa256e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 -- 172.21.15.122:0/502755621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 msgr2=0x5641aa24f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa24f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 -- 172.21.15.122:0/502755621 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641aa155340 con 0x5641aa24f8c0 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f7725ffb640 1 -- 172.21.15.122:0/4084460847 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f77140cea00 con 0x7f7728102b60 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772cf9b640 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 0x5641aa254380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f771c009230 tx=0x7f771c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4d046bcb873e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f772d79c640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f77180a2b80 0x7f77180a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7720005f40 tx=0x7f7720004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f7712ffd640 1 -- 172.21.15.122:0/502755621 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c040070 con 0x5641aa24f8c0 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f7712ffd640 1 -- 172.21.15.122:0/502755621 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c0318b0 con 0x5641aa24f8c0 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.701+0000 7f7712ffd640 1 -- 172.21.15.122:0/502755621 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c035040 con 0x5641aa24f8c0 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 -- 172.21.15.122:0/502755621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 msgr2=0x5641aa254380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 0x5641aa254380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f771c009230 tx=0x7f771c005c00 comp rx=0 tx=0).stop 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 -- 172.21.15.122:0/502755621 shutdown_connections 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa254950 0x5641aa256e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24f8c0 0x5641aa254380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 --2- 172.21.15.122:0/502755621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa24f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 -- 172.21.15.122:0/502755621 >> 172.21.15.122:0/502755621 conn(0x5641aa2461e0 msgr2=0x5641aa2465e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 -- 172.21.15.122:0/502755621 shutdown_connections 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.702+0000 7f773206e740 1 -- 172.21.15.122:0/502755621 wait complete. 2024-09-12T02:10:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 -- start start 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 Processor -- start 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa2591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa2596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa260720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5641aa2578c0 con 0x5641aa24ef80 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5641aa137290 con 0x5641aa254950 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641aa257a40 con 0x5641aa24f8c0 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f773206e740 10 client.0 _ll_register_callbacks cb 0x5641aa168140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f772cf9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa2591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f7727fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa2596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f772cf9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa2591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34636/0 (socket says 172.21.15.122:34636) 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.703+0000 7f772cf9b640 1 -- 172.21.15.122:0/505654432 learned_addr learned my addr 172.21.15.122:0/505654432 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772d79c640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa260720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772cf9b640 1 -- 172.21.15.122:0/505654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 msgr2=0x5641aa2596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772cf9b640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa2596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772cf9b640 1 -- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 msgr2=0x5641aa260720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772cf9b640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa260720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772cf9b640 1 -- 172.21.15.122:0/505654432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a9ec1210 con 0x5641aa24ef80 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f7727fff640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa2596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f772cf9b640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa2591a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f771c009900 tx=0x7f771c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d881d18003be7d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f7707fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f7707fff640 1 -- 172.21.15.122:0/505654432 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c04b020 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f7707fff640 1 -- 172.21.15.122:0/505654432 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f771c004250 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c0318b0 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f7707fff640 1 -- 172.21.15.122:0/505654432 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f771c035070 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f773206e740 10 client.18726 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.704+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5641a9f8d620 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f7707fff640 1 -- 172.21.15.122:0/505654432 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c0318b0 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f7707fff640 1 -- 172.21.15.122:0/505654432 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f771c0048f0 con 0x5641aa24ef80 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f773206e740 10 client.18726 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f773206e740 10 client.18726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f773206e740 20 client.18726 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f773206e740 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa264a20 0x5641aa266e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f773206e740 4 client.18726 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641aa1bba20 con 0x5641aa264a20 2024-09-12T02:10:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f7727fff640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa264a20 0x5641aa266e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f7707fff640 10 client.18726 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.705+0000 7f7727fff640 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa264a20 0x5641aa266e10 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802de5fc89f6ff83 server_cookie=a48ebfd01d6d981c in_seq=0 out_seq=0 2024-09-12T02:10:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f7707fff640 1 -- 172.21.15.122:0/505654432 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641aa1bba20 con 0x5641aa264a20 2024-09-12T02:10:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f7707fff640 10 client.18726 handle_command_reply: tid=0 2024-09-12T02:10:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f773206e740 1 client.18726 shutdown 2024-09-12T02:10:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 msgr2=0x5641aa2591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f773206e740 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa2591a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f771c009900 tx=0x7f771c005740 comp rx=0 tx=0).stop 2024-09-12T02:10:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 shutdown_connections 2024-09-12T02:10:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f773206e740 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa264a20 0x5641aa266e10 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.707+0000 7f773206e740 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa260720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f773206e740 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa2596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f7707fff640 0 client.18726 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f773206e740 1 --2- 172.21.15.122:0/505654432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24ef80 0x5641aa2591a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 >> 172.21.15.122:0/505654432 conn(0x5641aa2461e0 msgr2=0x5641aa2479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 shutdown_connections 2024-09-12T02:10:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f773206e740 1 -- 172.21.15.122:0/505654432 wait complete. 2024-09-12T02:10:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.708+0000 7f773206e740 20 client.18726 trim_cache size 0 max 16384 2024-09-12T02:10:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.723+0000 7f773206e740 1 Processor -- start 2024-09-12T02:10:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.723+0000 7f773206e740 1 -- start start 2024-09-12T02:10:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f773206e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24ef80 0x5641aa259770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f773206e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 0x5641aa2606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f773206e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa2451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f773206e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77281a94d0 con 0x5641aa24f8c0 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f773206e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77281a6da0 con 0x5641aa254950 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f773206e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77281a4bd0 con 0x5641aa24ef80 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 0x5641aa2606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 0x5641aa2606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34642/0 (socket says 172.21.15.122:34642) 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 -- 172.21.15.122:0/1024509878 learned_addr learned my addr 172.21.15.122:0/1024509878 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f772cf9b640 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24ef80 0x5641aa259770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f772d79c640 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa2451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 -- 172.21.15.122:0/1024509878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24ef80 msgr2=0x5641aa259770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24ef80 0x5641aa259770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 -- 172.21.15.122:0/1024509878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 msgr2=0x5641aa2451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa2451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 -- 172.21.15.122:0/1024509878 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641aa267180 con 0x5641aa24f8c0 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f772cf9b640 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24ef80 0x5641aa259770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f7727fff640 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 0x5641aa2606e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f771409af60 tx=0x7f77140cc3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c89785d9893116 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.724+0000 7f77127fc640 1 -- 172.21.15.122:0/1024509878 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771409abc0 con 0x5641aa24f8c0 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f77127fc640 1 -- 172.21.15.122:0/1024509878 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771409b960 con 0x5641aa24f8c0 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f77127fc640 1 -- 172.21.15.122:0/1024509878 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77140978d0 con 0x5641aa24f8c0 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 -- 172.21.15.122:0/1024509878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 msgr2=0x5641aa2606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 0x5641aa2606e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f771409af60 tx=0x7f77140cc3d0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 -- 172.21.15.122:0/1024509878 shutdown_connections 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa254950 0x5641aa2451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa24f8c0 0x5641aa2606e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 --2- 172.21.15.122:0/1024509878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24ef80 0x5641aa259770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 -- 172.21.15.122:0/1024509878 >> 172.21.15.122:0/1024509878 conn(0x5641aa2461e0 msgr2=0x5641aa244a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 -- 172.21.15.122:0/1024509878 shutdown_connections 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.725+0000 7f773206e740 1 -- 172.21.15.122:0/1024509878 wait complete. 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 -- start start 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 Processor -- start 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24ef80 0x5641aa25dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa25e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 0x5641aa25f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77281a5260 con 0x5641aa254950 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77281a66d0 con 0x5641aa24ef80 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77281a5930 con 0x5641aa24f8c0 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f773206e740 10 client.0 _ll_register_callbacks cb 0x7f7720009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f772d79c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 0x5641aa25f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f772d79c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 0x5641aa25f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34650/0 (socket says 172.21.15.122:34650) 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f772cf9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24ef80 0x5641aa25dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f772d79c640 1 -- 172.21.15.122:0/3735982326 learned_addr learned my addr 172.21.15.122:0/3735982326 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.726+0000 7f7727fff640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa25e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f772d79c640 1 -- 172.21.15.122:0/3735982326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 msgr2=0x5641aa25e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f772d79c640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa25e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f772d79c640 1 -- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24ef80 msgr2=0x5641aa25dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f772d79c640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24ef80 0x5641aa25dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f772d79c640 1 -- 172.21.15.122:0/3735982326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641aa25f7a0 con 0x5641aa254950 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7727fff640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa25e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f772d79c640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 0x5641aa25f190 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f7720014390 tx=0x7f7720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3088a9fef67f55cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7707fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7707fff640 1 -- 172.21.15.122:0/3735982326 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772001a040 con 0x5641aa254950 2024-09-12T02:10:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7707fff640 1 -- 172.21.15.122:0/3735982326 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7720024be0 con 0x5641aa254950 2024-09-12T02:10:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771409b960 con 0x5641aa254950 2024-09-12T02:10:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7707fff640 1 -- 172.21.15.122:0/3735982326 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7720024de0 con 0x5641aa254950 2024-09-12T02:10:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 10 client.18732 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5641aa2cf6f0 con 0x5641aa254950 2024-09-12T02:10:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7707fff640 1 -- 172.21.15.122:0/3735982326 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771409b960 con 0x5641aa254950 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f7707fff640 1 -- 172.21.15.122:0/3735982326 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f7720020040 con 0x5641aa254950 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 10 client.18732 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 10 client.18732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 20 client.18732 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.727+0000 7f773206e740 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa2d0070 0x5641aa2d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.728+0000 7f773206e740 4 client.18732 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.728+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641aa1b5d70 con 0x5641aa2d0070 2024-09-12T02:10:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.728+0000 7f772cf9b640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa2d0070 0x5641aa2d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.728+0000 7f7707fff640 10 client.18732 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.728+0000 7f772cf9b640 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa2d0070 0x5641aa2d2490 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c2cff772bd2eeb server_cookie=c2c63cdabe74bef0 in_seq=0 out_seq=0 2024-09-12T02:10:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:40 smithi122 ceph-mon[30925]: pgmap v1065: 97 pgs: 97 active+clean; 555 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.2 MiB/s wr, 565 op/s 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f7707fff640 1 -- 172.21.15.122:0/3735982326 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641aa1b5d70 con 0x5641aa2d0070 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f7707fff640 10 client.18732 handle_command_reply: tid=0 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 client.18732 shutdown 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 msgr2=0x5641aa25f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 0x5641aa25f190 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f7720014390 tx=0x7f7720004a40 comp rx=0 tx=0).stop 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 shutdown_connections 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5641aa2d0070 0x5641aa2d2490 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5641aa254950 0x5641aa25f190 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641aa24f8c0 0x5641aa25e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 --2- 172.21.15.122:0/3735982326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5641aa24ef80 0x5641aa25dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 >> 172.21.15.122:0/3735982326 conn(0x5641aa2461e0 msgr2=0x5641aa2452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f7707fff640 0 client.18732 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 shutdown_connections 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 1 -- 172.21.15.122:0/3735982326 wait complete. 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.985+0000 7f773206e740 20 client.18732 trim_cache size 0 max 16384 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:10:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:10:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f77180a2b80 msgr2=0x7f77180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f77180a2b80 0x7f77180a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7720005f40 tx=0x7f7720004ab0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 msgr2=0x7f77281981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 0x7f77281981c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f771400d950 tx=0x7f771400de10 comp rx=0 tx=0).stop 2024-09-12T02:10:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 shutdown_connections 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f77180a2b80 0x7f77180a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7728103520 0x7f772819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7728102b60 0x7f77281981c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 --2- 172.21.15.122:0/4084460847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7728101970 0x7f7728197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 >> 172.21.15.122:0/4084460847 conn(0x7f77280fd0d0 msgr2=0x7f7728105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 shutdown_connections 2024-09-12T02:10:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:40.986+0000 7f772df9d640 1 -- 172.21.15.122:0/4084460847 wait complete. 2024-09-12T02:10:41.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 239689 19.84 MB/sec execute 508 sec latency 157.365 ms 2024-09-12T02:10:42.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 240228 19.84 MB/sec execute 509 sec latency 57.182 ms 2024-09-12T02:10:42.615 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:42 smithi151 ceph-mon[35229]: pgmap v1066: 97 pgs: 97 active+clean; 544 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.9 MiB/s wr, 624 op/s 2024-09-12T02:10:42.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:42 smithi183 ceph-mon[35127]: pgmap v1066: 97 pgs: 97 active+clean; 544 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.9 MiB/s wr, 624 op/s 2024-09-12T02:10:42.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:42 smithi122 ceph-mon[30925]: pgmap v1066: 97 pgs: 97 active+clean; 544 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.9 MiB/s wr, 624 op/s 2024-09-12T02:10:43.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 240885 19.87 MB/sec execute 510 sec latency 51.492 ms 2024-09-12T02:10:44.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 241351 19.88 MB/sec execute 511 sec latency 350.828 ms 2024-09-12T02:10:44.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:44 smithi183 ceph-mon[35127]: pgmap v1067: 97 pgs: 97 active+clean; 532 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-09-12T02:10:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:44 smithi122 ceph-mon[30925]: pgmap v1067: 97 pgs: 97 active+clean; 532 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-09-12T02:10:44.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:44 smithi151 ceph-mon[35229]: pgmap v1067: 97 pgs: 97 active+clean; 532 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-09-12T02:10:45.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 241858 19.90 MB/sec execute 512 sec latency 79.530 ms 2024-09-12T02:10:46.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 242621 19.95 MB/sec execute 513 sec latency 55.450 ms 2024-09-12T02:10:46.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:46 smithi183 ceph-mon[35127]: pgmap v1068: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.4 MiB/s wr, 642 op/s 2024-09-12T02:10:46.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:46 smithi122 ceph-mon[30925]: pgmap v1068: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.4 MiB/s wr, 642 op/s 2024-09-12T02:10:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:46 smithi151 ceph-mon[35229]: pgmap v1068: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.4 MiB/s wr, 642 op/s 2024-09-12T02:10:47.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243068 19.96 MB/sec execute 514 sec latency 62.715 ms 2024-09-12T02:10:48.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243643 19.98 MB/sec execute 515 sec latency 177.994 ms 2024-09-12T02:10:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:48 smithi183 ceph-mon[35127]: pgmap v1069: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 568 op/s 2024-09-12T02:10:48.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:48 smithi122 ceph-mon[30925]: pgmap v1069: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 568 op/s 2024-09-12T02:10:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:48 smithi151 ceph-mon[35229]: pgmap v1069: 97 pgs: 97 active+clean; 537 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 568 op/s 2024-09-12T02:10:49.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243972 19.98 MB/sec execute 516 sec latency 297.152 ms 2024-09-12T02:10:50.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 244508 19.99 MB/sec execute 517 sec latency 81.839 ms 2024-09-12T02:10:50.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:50 smithi183 ceph-mon[35127]: pgmap v1070: 97 pgs: 97 active+clean; 538 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-09-12T02:10:50.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:50 smithi122 ceph-mon[30925]: pgmap v1070: 97 pgs: 97 active+clean; 538 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-09-12T02:10:50.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:50 smithi151 ceph-mon[35229]: pgmap v1070: 97 pgs: 97 active+clean; 538 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-09-12T02:10:51.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 245153 20.02 MB/sec execute 518 sec latency 96.690 ms 2024-09-12T02:10:52.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 245769 20.03 MB/sec execute 519 sec latency 83.157 ms 2024-09-12T02:10:52.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:52 smithi183 ceph-mon[35127]: pgmap v1071: 97 pgs: 97 active+clean; 543 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 655 op/s 2024-09-12T02:10:52.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:52 smithi122 ceph-mon[30925]: pgmap v1071: 97 pgs: 97 active+clean; 543 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 655 op/s 2024-09-12T02:10:52.870 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:52 smithi151 ceph-mon[35229]: pgmap v1071: 97 pgs: 97 active+clean; 543 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 655 op/s 2024-09-12T02:10:53.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 246431 20.08 MB/sec execute 520 sec latency 57.106 ms 2024-09-12T02:10:54.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 246864 20.08 MB/sec execute 521 sec latency 283.683 ms 2024-09-12T02:10:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 -- 172.21.15.122:0/1092225239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 msgr2=0x7f49a81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 --2- 172.21.15.122:0/1092225239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a81020b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4998009900 tx=0x7f499802a410 comp rx=0 tx=0).stop 2024-09-12T02:10:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 -- 172.21.15.122:0/1092225239 shutdown_connections 2024-09-12T02:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 --2- 172.21.15.122:0/1092225239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49a81025f0 0x7f49a8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 --2- 172.21.15.122:0/1092225239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a81020b0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 --2- 172.21.15.122:0/1092225239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8100a40 0x7f49a8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 -- 172.21.15.122:0/1092225239 >> 172.21.15.122:0/1092225239 conn(0x7f49a80fc1e0 msgr2=0x7f49a80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 -- 172.21.15.122:0/1092225239 shutdown_connections 2024-09-12T02:10:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.095+0000 7f49af4f1640 1 -- 172.21.15.122:0/1092225239 wait complete. 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 Processor -- start 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 -- start start 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8100a40 0x7f49a806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49a81025f0 0x7f49a819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810c6a0 con 0x7f49a8101c30 2024-09-12T02:10:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810c520 con 0x7f49a81025f0 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49af4f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810c820 con 0x7f49a8100a40 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55682/0 (socket says 172.21.15.122:55682) 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49aca8c640 1 -- 172.21.15.122:0/2390002261 learned_addr learned my addr 172.21.15.122:0/2390002261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49ad28d640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8100a40 0x7f49a806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49ada8e640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49a81025f0 0x7f49a819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.096+0000 7f49aca8c640 1 -- 172.21.15.122:0/2390002261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8100a40 msgr2=0x7f49a806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49aca8c640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8100a40 0x7f49a806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49aca8c640 1 -- 172.21.15.122:0/2390002261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49a81025f0 msgr2=0x7f49a819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49aca8c640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49a81025f0 0x7f49a819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49aca8c640 1 -- 172.21.15.122:0/2390002261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a819fa40 con 0x7f49a8101c30 2024-09-12T02:10:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49aca8c640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a819cb10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4998009230 tx=0x7f4998004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f499e7fc640 1 -- 172.21.15.122:0/2390002261 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499803f030 con 0x7f49a8101c30 2024-09-12T02:10:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a819fc70 con 0x7f49a8101c30 2024-09-12T02:10:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f499e7fc640 1 -- 172.21.15.122:0/2390002261 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4998003ce0 con 0x7f49a8101c30 2024-09-12T02:10:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f499e7fc640 1 -- 172.21.15.122:0/2390002261 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998035070 con 0x7f49a8101c30 2024-09-12T02:10:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.097+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a01b0 con 0x7f49a8101c30 2024-09-12T02:10:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.099+0000 7f499e7fc640 1 -- 172.21.15.122:0/2390002261 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4998042030 con 0x7f49a8101c30 2024-09-12T02:10:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 Processor -- start 2024-09-12T02:10:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 -- start start 2024-09-12T02:10:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 0x55d2cff54da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d2cfeb82f0 con 0x55d2cff549a0 2024-09-12T02:10:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d2cfe60270 con 0x55d2cff552e0 2024-09-12T02:10:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49b2291740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2cfeb42b0 con 0x55d2cff5a370 2024-09-12T02:10:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49ada8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 0x55d2cff54da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49ada8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 0x55d2cff54da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55688/0 (socket says 172.21.15.122:55688) 2024-09-12T02:10:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49ada8e640 1 -- 172.21.15.122:0/1757532530 learned_addr learned my addr 172.21.15.122:0/1757532530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f49ad28d640 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.100+0000 7f499e7fc640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49940a2cb0 0x7f49940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 -- 172.21.15.122:0/1757532530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 msgr2=0x55d2cff5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 -- 172.21.15.122:0/1757532530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 msgr2=0x55d2cff59da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff59da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 -- 172.21.15.122:0/1757532530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cfe5f340 con 0x55d2cff549a0 2024-09-12T02:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f499e7fc640 1 -- 172.21.15.122:0/2390002261 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49980ef900 con 0x7f49a8101c30 2024-09-12T02:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49940a2cb0 0x7f49940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49ada8e640 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 0x55d2cff54da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f49a0010510 tx=0x7f49a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd86b91939ac5d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49937fe640 1 -- 172.21.15.122:0/1757532530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0023070 con 0x55d2cff549a0 2024-09-12T02:10:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49937fe640 1 -- 172.21.15.122:0/1757532530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a00134c0 con 0x55d2cff549a0 2024-09-12T02:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.101+0000 7f49937fe640 1 -- 172.21.15.122:0/1757532530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a001d820 con 0x55d2cff549a0 2024-09-12T02:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 -- 172.21.15.122:0/1757532530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 msgr2=0x55d2cff54da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49ada8e640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49940a2cb0 0x7f49940a5170 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f49a0020000 tx=0x7f49a001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 0x55d2cff54da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f49a0010510 tx=0x7f49a0009f90 comp rx=0 tx=0).stop 2024-09-12T02:10:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 -- 172.21.15.122:0/1757532530 shutdown_connections 2024-09-12T02:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff59da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 --2- 172.21.15.122:0/1757532530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff549a0 0x55d2cff54da0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 -- 172.21.15.122:0/1757532530 >> 172.21.15.122:0/1757532530 conn(0x55d2cff50010 msgr2=0x55d2cff50410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 -- 172.21.15.122:0/1757532530 shutdown_connections 2024-09-12T02:10:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 -- 172.21.15.122:0/1757532530 wait complete. 2024-09-12T02:10:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.102+0000 7f49b2291740 1 -- start start 2024-09-12T02:10:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 Processor -- start 2024-09-12T02:10:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff5ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff5f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d2cff5d340 con 0x55d2cff552e0 2024-09-12T02:10:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d2cfe41290 con 0x55d2cff549a0 2024-09-12T02:10:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2cff5d4c0 con 0x55d2cff5a370 2024-09-12T02:10:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49b2291740 10 client.0 _ll_register_callbacks cb 0x55d2cfe72140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff5f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff5f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55704/0 (socket says 172.21.15.122:55704) 2024-09-12T02:10:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 -- 172.21.15.122:0/568701066 learned_addr learned my addr 172.21.15.122:0/568701066 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49ada8e640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49ad28d640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff5ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 -- 172.21.15.122:0/568701066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 msgr2=0x55d2cff661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 -- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 msgr2=0x55d2cff5ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff5ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49aca8c640 1 -- 172.21.15.122:0/568701066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cfbcb210 con 0x55d2cff552e0 2024-09-12T02:10:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.103+0000 7f49ad28d640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff5ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49aca8c640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff5f160 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f49980314e0 tx=0x7f4998004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f24987e7aa3387 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f4990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f4990ff9640 1 -- 172.21.15.122:0/568701066 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998049040 con 0x55d2cff552e0 2024-09-12T02:10:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f4990ff9640 1 -- 172.21.15.122:0/568701066 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49980b8880 con 0x55d2cff552e0 2024-09-12T02:10:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a00134c0 con 0x55d2cff552e0 2024-09-12T02:10:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f4990ff9640 1 -- 172.21.15.122:0/568701066 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f49980bc210 con 0x55d2cff552e0 2024-09-12T02:10:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 10 client.18744 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d2cfc97620 con 0x55d2cff552e0 2024-09-12T02:10:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f4990ff9640 1 -- 172.21.15.122:0/568701066 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a00134c0 con 0x55d2cff552e0 2024-09-12T02:10:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f4990ff9640 1 -- 172.21.15.122:0/568701066 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f49980c0150 con 0x55d2cff552e0 2024-09-12T02:10:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 10 client.18744 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 10 client.18744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 20 client.18744 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cff6a8b0 0x55d2cff6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 4 client.18744 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:10:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2cfec5a20 con 0x55d2cff6a8b0 2024-09-12T02:10:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.104+0000 7f49ad28d640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cff6a8b0 0x55d2cff6cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.105+0000 7f49ad28d640 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cff6a8b0 0x55d2cff6cd00 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0bdeb4fa652658d server_cookie=abb25765e44c46a in_seq=0 out_seq=0 2024-09-12T02:10:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.105+0000 7f4990ff9640 10 client.18744 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f4990ff9640 1 -- 172.21.15.122:0/568701066 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2cfec5a20 con 0x55d2cff6a8b0 2024-09-12T02:10:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f4990ff9640 10 client.18744 handle_command_reply: tid=0 2024-09-12T02:10:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 client.18744 shutdown 2024-09-12T02:10:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 msgr2=0x55d2cff5f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff5f160 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f49980314e0 tx=0x7f4998004610 comp rx=0 tx=0).stop 2024-09-12T02:10:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 shutdown_connections 2024-09-12T02:10:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cff6a8b0 0x55d2cff6cd00 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff5f160 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f4990ff9640 0 client.18744 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 --2- 172.21.15.122:0/568701066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff5ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 >> 172.21.15.122:0/568701066 conn(0x55d2cff50010 msgr2=0x55d2cff51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 shutdown_connections 2024-09-12T02:10:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 1 -- 172.21.15.122:0/568701066 wait complete. 2024-09-12T02:10:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.107+0000 7f49b2291740 20 client.18744 trim_cache size 0 max 16384 2024-09-12T02:10:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.121+0000 7f49b2291740 1 Processor -- start 2024-09-12T02:10:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 -- start start 2024-09-12T02:10:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff549a0 0x55d2cff503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 0x55d2cff4f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a9640 con 0x55d2cff5a370 2024-09-12T02:10:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a6f10 con 0x55d2cff552e0 2024-09-12T02:10:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49b2291740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a4d40 con 0x55d2cff549a0 2024-09-12T02:10:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 0x55d2cff4f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 0x55d2cff4f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55714/0 (socket says 172.21.15.122:55714) 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ad28d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff549a0 0x55d2cff503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 -- 172.21.15.122:0/922477912 learned_addr learned my addr 172.21.15.122:0/922477912 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49aca8c640 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 -- 172.21.15.122:0/922477912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff549a0 msgr2=0x55d2cff503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff549a0 0x55d2cff503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 -- 172.21.15.122:0/922477912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 msgr2=0x55d2cff66160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff66160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 -- 172.21.15.122:0/922477912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cff6d380 con 0x55d2cff5a370 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f49ada8e640 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 0x55d2cff4f0b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f49a0010510 tx=0x7f49a0026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8968418491ac164a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f4992ffd640 1 -- 172.21.15.122:0/922477912 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0034070 con 0x55d2cff5a370 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f4992ffd640 1 -- 172.21.15.122:0/922477912 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a0013d40 con 0x55d2cff5a370 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.122+0000 7f4992ffd640 1 -- 172.21.15.122:0/922477912 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0023070 con 0x55d2cff5a370 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 -- 172.21.15.122:0/922477912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 msgr2=0x55d2cff4f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 0x55d2cff4f0b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f49a0010510 tx=0x7f49a0026000 comp rx=0 tx=0).stop 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 -- 172.21.15.122:0/922477912 shutdown_connections 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff5a370 0x55d2cff4f0b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff552e0 0x55d2cff66160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 --2- 172.21.15.122:0/922477912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff549a0 0x55d2cff503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 -- 172.21.15.122:0/922477912 >> 172.21.15.122:0/922477912 conn(0x55d2cff50010 msgr2=0x55d2cff4e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 -- 172.21.15.122:0/922477912 shutdown_connections 2024-09-12T02:10:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 -- 172.21.15.122:0/922477912 wait complete. 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 -- start start 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.123+0000 7f49b2291740 1 Processor -- start 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff64520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff65530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a53d0 con 0x55d2cff552e0 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a6840 con 0x55d2cff549a0 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a5aa0 con 0x55d2cff5a370 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 10 client.0 _ll_register_callbacks cb 0x7f49980bc0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff64520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff64520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55722/0 (socket says 172.21.15.122:55722) 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 -- 172.21.15.122:0/3394058555 learned_addr learned my addr 172.21.15.122:0/3394058555 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:10:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49ada8e640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff65530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49ad28d640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 -- 172.21.15.122:0/3394058555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 msgr2=0x55d2cff65530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff65530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 -- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 msgr2=0x55d2cff63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 -- 172.21.15.122:0/3394058555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cff65b40 con 0x55d2cff552e0 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49ad28d640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49aca8c640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff64520 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f49980096c0 tx=0x7f499802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5939b0a350121be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f4990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f4990ff9640 1 -- 172.21.15.122:0/3394058555 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998049070 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f4990ff9640 1 -- 172.21.15.122:0/3394058555 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49980c4b40 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.124+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a0013d40 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f4990ff9640 1 -- 172.21.15.122:0/3394058555 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f49980dc0b0 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 10 client.18750 fetch_fsmap learned FSMap version 45 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d2cffd5c90 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f4990ff9640 1 -- 172.21.15.122:0/3394058555 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a0013d40 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f4990ff9640 1 -- 172.21.15.122:0/3394058555 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4998031ba0 con 0x55d2cff552e0 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 10 client.18750 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:10:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 10 client.18750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 20 client.18750 populate_metadata read hostname 'smithi122' 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cffd6280 0x55d2cffd86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 4 client.18750 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2cfebfd70 con 0x55d2cffd6280 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.125+0000 7f49ad28d640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cffd6280 0x55d2cffd86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.126+0000 7f4990ff9640 10 client.18750 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.126+0000 7f49ad28d640 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cffd6280 0x55d2cffd86a0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ec1b553c401530 server_cookie=5fb7804e91b883c in_seq=0 out_seq=0 2024-09-12T02:10:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f4990ff9640 1 -- 172.21.15.122:0/3394058555 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2cfebfd70 con 0x55d2cffd6280 2024-09-12T02:10:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f4990ff9640 10 client.18750 handle_command_reply: tid=0 2024-09-12T02:10:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 client.18750 shutdown 2024-09-12T02:10:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 msgr2=0x55d2cff64520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff64520 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f49980096c0 tx=0x7f499802a410 comp rx=0 tx=0).stop 2024-09-12T02:10:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 shutdown_connections 2024-09-12T02:10:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55d2cffd6280 0x55d2cffd86a0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2cff5a370 0x55d2cff65530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f4990ff9640 0 client.18750 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:10:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2cff552e0 0x55d2cff64520 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 --2- 172.21.15.122:0/3394058555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2cff549a0 0x55d2cff63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 >> 172.21.15.122:0/3394058555 conn(0x55d2cff50010 msgr2=0x55d2cff4f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 shutdown_connections 2024-09-12T02:10:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.586+0000 7f49b2291740 1 -- 172.21.15.122:0/3394058555 wait complete. 2024-09-12T02:10:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49b2291740 20 client.18750 trim_cache size 0 max 16384 2024-09-12T02:10:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:10:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:10:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:10:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:10:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49940a2cb0 msgr2=0x7f49940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49940a2cb0 0x7f49940a5170 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f49a0020000 tx=0x7f49a001b000 comp rx=0 tx=0).stop 2024-09-12T02:10:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 msgr2=0x7f49a819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a819cb10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4998009230 tx=0x7f4998004360 comp rx=0 tx=0).stop 2024-09-12T02:10:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 shutdown_connections 2024-09-12T02:10:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f49940a2cb0 0x7f49940a5170 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49a81025f0 0x7f49a819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101c30 0x7f49a819cb10 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 --2- 172.21.15.122:0/2390002261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8100a40 0x7f49a806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 >> 172.21.15.122:0/2390002261 conn(0x7f49a80fc1e0 msgr2=0x7f49a8104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 shutdown_connections 2024-09-12T02:10:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:10:54.587+0000 7f49af4f1640 1 -- 172.21.15.122:0/2390002261 wait complete. 2024-09-12T02:10:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:54 smithi183 ceph-mon[35127]: pgmap v1072: 97 pgs: 97 active+clean; 543 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 577 op/s 2024-09-12T02:10:54.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:54 smithi122 ceph-mon[30925]: pgmap v1072: 97 pgs: 97 active+clean; 543 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 577 op/s 2024-09-12T02:10:54.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:54 smithi151 ceph-mon[35229]: pgmap v1072: 97 pgs: 97 active+clean; 543 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 577 op/s 2024-09-12T02:10:55.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 247154 20.07 MB/sec execute 522 sec latency 296.180 ms 2024-09-12T02:10:56.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 247677 20.09 MB/sec execute 523 sec latency 159.381 ms 2024-09-12T02:10:56.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:56 smithi151 ceph-mon[35229]: pgmap v1073: 97 pgs: 97 active+clean; 547 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-09-12T02:10:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:56 smithi183 ceph-mon[35127]: pgmap v1073: 97 pgs: 97 active+clean; 547 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-09-12T02:10:56.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:56 smithi122 ceph-mon[30925]: pgmap v1073: 97 pgs: 97 active+clean; 547 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-09-12T02:10:57.086 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248140 20.10 MB/sec execute 524 sec latency 108.459 ms 2024-09-12T02:10:58.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248757 20.12 MB/sec execute 525 sec latency 53.750 ms 2024-09-12T02:10:58.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:10:58 smithi183 ceph-mon[35127]: pgmap v1074: 97 pgs: 97 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-09-12T02:10:58.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:10:58 smithi122 ceph-mon[30925]: pgmap v1074: 97 pgs: 97 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-09-12T02:10:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:10:58 smithi151 ceph-mon[35229]: pgmap v1074: 97 pgs: 97 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-09-12T02:10:59.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 249238 20.13 MB/sec execute 526 sec latency 194.976 ms 2024-09-12T02:11:00.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 249655 20.14 MB/sec execute 527 sec latency 332.504 ms 2024-09-12T02:11:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:00 smithi183 ceph-mon[35127]: pgmap v1075: 97 pgs: 97 active+clean; 552 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 558 op/s 2024-09-12T02:11:00.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:00 smithi122 ceph-mon[30925]: pgmap v1075: 97 pgs: 97 active+clean; 552 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 558 op/s 2024-09-12T02:11:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:00 smithi151 ceph-mon[35229]: pgmap v1075: 97 pgs: 97 active+clean; 552 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 558 op/s 2024-09-12T02:11:01.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 250289 20.15 MB/sec execute 528 sec latency 87.199 ms 2024-09-12T02:11:02.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 250822 20.18 MB/sec execute 529 sec latency 149.338 ms 2024-09-12T02:11:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:02 smithi183 ceph-mon[35127]: pgmap v1076: 97 pgs: 97 active+clean; 553 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 620 op/s 2024-09-12T02:11:02.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:02 smithi122 ceph-mon[30925]: pgmap v1076: 97 pgs: 97 active+clean; 553 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 620 op/s 2024-09-12T02:11:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:02 smithi151 ceph-mon[35229]: pgmap v1076: 97 pgs: 97 active+clean; 553 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 620 op/s 2024-09-12T02:11:03.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 251491 20.20 MB/sec execute 530 sec latency 215.079 ms 2024-09-12T02:11:04.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 252034 20.23 MB/sec execute 531 sec latency 78.109 ms 2024-09-12T02:11:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:04 smithi183 ceph-mon[35127]: pgmap v1077: 97 pgs: 97 active+clean; 545 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-09-12T02:11:04.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:04 smithi122 ceph-mon[30925]: pgmap v1077: 97 pgs: 97 active+clean; 545 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-09-12T02:11:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:04 smithi151 ceph-mon[35229]: pgmap v1077: 97 pgs: 97 active+clean; 545 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-09-12T02:11:05.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 252469 20.23 MB/sec execute 532 sec latency 312.234 ms 2024-09-12T02:11:06.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 253047 20.26 MB/sec execute 533 sec latency 73.771 ms 2024-09-12T02:11:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:06 smithi183 ceph-mon[35127]: pgmap v1078: 97 pgs: 97 active+clean; 537 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.8 MiB/s wr, 651 op/s 2024-09-12T02:11:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:06 smithi122 ceph-mon[30925]: pgmap v1078: 97 pgs: 97 active+clean; 537 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.8 MiB/s wr, 651 op/s 2024-09-12T02:11:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:06 smithi151 ceph-mon[35229]: pgmap v1078: 97 pgs: 97 active+clean; 537 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.8 MiB/s wr, 651 op/s 2024-09-12T02:11:07.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 253445 20.25 MB/sec execute 534 sec latency 92.230 ms 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 -- 172.21.15.122:0/2497164111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 msgr2=0x7fdce0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 --2- 172.21.15.122:0/2497164111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce0108e80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ca0 tx=0x7fdcd802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 -- 172.21.15.122:0/2497164111 shutdown_connections 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 --2- 172.21.15.122:0/2497164111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce0108e80 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 --2- 172.21.15.122:0/2497164111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0101c30 0x7fdce01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 --2- 172.21.15.122:0/2497164111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdce0100a40 0x7fdce0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 -- 172.21.15.122:0/2497164111 >> 172.21.15.122:0/2497164111 conn(0x7fdce00fc1e0 msgr2=0x7fdce00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 -- 172.21.15.122:0/2497164111 shutdown_connections 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 -- 172.21.15.122:0/2497164111 wait complete. 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.682+0000 7fdce60f1640 1 Processor -- start 2024-09-12T02:11:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 -- start start 2024-09-12T02:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 0x7fdce006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdce0101c30 0x7fdce019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdce010c6f0 con 0x7fdce01025f0 2024-09-12T02:11:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdce010c570 con 0x7fdce0101c30 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdce60f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdce010c870 con 0x7fdce0100a40 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 0x7fdce006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 0x7fdce006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57290/0 (socket says 172.21.15.122:57290) 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/113633746 learned_addr learned my addr 172.21.15.122:0/113633746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdce0101c30 0x7fdce019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/113633746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdce0101c30 msgr2=0x7fdce019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdce0101c30 0x7fdce019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 msgr2=0x7fdce019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/113633746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce019f9d0 con 0x7fdce0100a40 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.683+0000 7fdcdffff640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.684+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 0x7fdce006a410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e9c0 tx=0x7fdcd000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.684+0000 7fdcdcff9640 1 -- 172.21.15.122:0/113633746 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000cde0 con 0x7fdce0100a40 2024-09-12T02:11:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.684+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce019fc60 con 0x7fdce0100a40 2024-09-12T02:11:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.684+0000 7fdcdcff9640 1 -- 172.21.15.122:0/113633746 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd0010910 con 0x7fdce0100a40 2024-09-12T02:11:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.684+0000 7fdcdcff9640 1 -- 172.21.15.122:0/113633746 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0018a10 con 0x7fdce0100a40 2024-09-12T02:11:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.684+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01a01b0 con 0x7fdce0100a40 2024-09-12T02:11:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdcff9640 1 -- 172.21.15.122:0/113633746 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdcd001f020 con 0x7fdce0100a40 2024-09-12T02:11:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 Processor -- start 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 -- start start 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 0x56463dbbcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbc4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56463db202f0 con 0x56463dbbc9a0 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56463dac8270 con 0x56463dbc2370 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56463db1c2b0 con 0x56463dbbd2e0 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 0x56463dbbcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 0x56463dbbcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38694/0 (socket says 172.21.15.122:38694) 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 -- 172.21.15.122:0/388513727 learned_addr learned my addr 172.21.15.122:0/388513727 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbc4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdcff9640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2c30 0x7fdcc80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 -- 172.21.15.122:0/388513727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 msgr2=0x56463dbc1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 -- 172.21.15.122:0/388513727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 msgr2=0x56463dbc4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbc4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.687+0000 7fdcdffff640 1 -- 172.21.15.122:0/388513727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463dac7340 con 0x56463dbbc9a0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.688+0000 7fdcdcff9640 1 -- 172.21.15.122:0/113633746 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdcd00cf290 con 0x7fdce0100a40 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.688+0000 7fdcdffff640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2c30 0x7fdcc80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.688+0000 7fdcdffff640 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 0x56463dbbcda0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ca0 tx=0x7fdcd80098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf33374d1a21e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.688+0000 7fdcc5ffb640 1 -- 172.21.15.122:0/388513727 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8043040 con 0x56463dbbc9a0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.688+0000 7fdcc5ffb640 1 -- 172.21.15.122:0/388513727 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd8036040 con 0x56463dbbc9a0 2024-09-12T02:11:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.688+0000 7fdcc5ffb640 1 -- 172.21.15.122:0/388513727 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd802fb70 con 0x56463dbbc9a0 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 -- 172.21.15.122:0/388513727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 msgr2=0x56463dbbcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdcdffff640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2c30 0x7fdcc80a50f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdcd804e000 tx=0x7fdcd804f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 0x56463dbbcda0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009ca0 tx=0x7fdcd80098d0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 -- 172.21.15.122:0/388513727 shutdown_connections 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbc4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/388513727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbc9a0 0x56463dbbcda0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 -- 172.21.15.122:0/388513727 >> 172.21.15.122:0/388513727 conn(0x56463dbb8010 msgr2=0x56463dbb8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 -- 172.21.15.122:0/388513727 shutdown_connections 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 -- 172.21.15.122:0/388513727 wait complete. 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 -- start start 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.689+0000 7fdce8e8e740 1 Processor -- start 2024-09-12T02:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbbc9a0 0x56463dbc6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 0x56463dbce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56463dbc5340 con 0x56463dbc2370 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56463daa9290 con 0x56463dbbc9a0 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56463dbc54c0 con 0x56463dbbd2e0 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 10 client.0 _ll_register_callbacks cb 0x56463dada140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 0x56463dbce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbbc9a0 0x56463dbc6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 0x56463dbce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38710/0 (socket says 172.21.15.122:38710) 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 -- 172.21.15.122:0/1828501423 learned_addr learned my addr 172.21.15.122:0/1828501423 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 -- 172.21.15.122:0/1828501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 msgr2=0x56463dbc7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 -- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbbc9a0 msgr2=0x56463dbc6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbbc9a0 0x56463dbc6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 -- 172.21.15.122:0/1828501423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463d833210 con 0x56463dbc2370 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbbc9a0 0x56463dbc6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcdffff640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 0x56463dbce150 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005e30 tx=0x7fdcd80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c982bc4c290492 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1828501423 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8043040 con 0x56463dbc2370 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1828501423 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdcd8004040 con 0x56463dbc2370 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.690+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd8036040 con 0x56463dbc2370 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1828501423 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdcd8004240 con 0x56463dbc2370 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 10 client.18762 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56463d8ff620 con 0x56463dbc2370 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1828501423 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd8036040 con 0x56463dbc2370 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1828501423 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdcd8030d30 con 0x56463dbc2370 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 10 client.18762 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 10 client.18762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 20 client.18762 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dbd2450 0x56463dbd4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 4 client.18762 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56463db2da20 con 0x56463dbd2450 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.691+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dbd2450 0x56463dbd4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dbd2450 0x56463dbd4840 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa34ddece0dc404b server_cookie=27401e4628073dcb in_seq=0 out_seq=0 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdcbeffd640 10 client.18762 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1828501423 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56463db2da20 con 0x56463dbd2450 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdcbeffd640 10 client.18762 handle_command_reply: tid=0 2024-09-12T02:11:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdce8e8e740 1 client.18762 shutdown 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 msgr2=0x56463dbce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.692+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 0x56463dbce150 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005e30 tx=0x7fdcd80095c0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 shutdown_connections 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dbd2450 0x56463dbd4840 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbc2370 0x56463dbce150 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbd2e0 0x56463dbc7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1828501423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbbc9a0 0x56463dbc6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 >> 172.21.15.122:0/1828501423 conn(0x56463dbb8010 msgr2=0x56463dbb9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdcbeffd640 0 client.18762 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 shutdown_connections 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1828501423 wait complete. 2024-09-12T02:11:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.693+0000 7fdce8e8e740 20 client.18762 trim_cache size 0 max 16384 2024-09-12T02:11:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 Processor -- start 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 -- start start 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbb70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbd90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01ab810 con 0x56463dbbd2e0 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01a90e0 con 0x56463dbc2370 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.708+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01a6f10 con 0x56463dbbc9a0 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbb70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbb70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:57324/0 (socket says 172.21.15.122:57324) 2024-09-12T02:11:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/2275353075 learned_addr learned my addr 172.21.15.122:0/2275353075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdffff640 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbd90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/2275353075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 msgr2=0x56463dbd90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbd90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/2275353075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 msgr2=0x56463dbce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 -- 172.21.15.122:0/2275353075 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463dbb7e30 con 0x56463dbbc9a0 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbb70b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0098480 tx=0x7fdcd00d16e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec5dd23455bca0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcc57fa640 1 -- 172.21.15.122:0/2275353075 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd009c1b0 con 0x56463dbbc9a0 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcc57fa640 1 -- 172.21.15.122:0/2275353075 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd009bab0 con 0x56463dbbc9a0 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.709+0000 7fdcc57fa640 1 -- 172.21.15.122:0/2275353075 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0018bb0 con 0x56463dbbc9a0 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 -- 172.21.15.122:0/2275353075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 msgr2=0x56463dbb70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbb70b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0098480 tx=0x7fdcd00d16e0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 -- 172.21.15.122:0/2275353075 shutdown_connections 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbd90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/2275353075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbb70b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 -- 172.21.15.122:0/2275353075 >> 172.21.15.122:0/2275353075 conn(0x56463dbb8010 msgr2=0x56463dbb69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 -- 172.21.15.122:0/2275353075 shutdown_connections 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 -- 172.21.15.122:0/2275353075 wait complete. 2024-09-12T02:11:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.710+0000 7fdce8e8e740 1 -- start start 2024-09-12T02:11:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 Processor -- start 2024-09-12T02:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbcb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbcbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbcce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01a75a0 con 0x56463dbbd2e0 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01a8a10 con 0x56463dbc2370 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdce01a7c70 con 0x56463dbbc9a0 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdce8e8e740 10 client.0 _ll_register_callbacks cb 0x7fdcd400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbcbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbcbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38726/0 (socket says 172.21.15.122:38726) 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 -- 172.21.15.122:0/1737749502 learned_addr learned my addr 172.21.15.122:0/1737749502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbcb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdffff640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbcce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 -- 172.21.15.122:0/1737749502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 msgr2=0x56463dbcb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbcb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 -- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 msgr2=0x56463dbcce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbcce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 -- 172.21.15.122:0/1737749502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463dbcd430 con 0x56463dbbd2e0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbcb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcdeffd640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbcbe10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005d00 tx=0x7fdcd4012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddae28b636f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.711+0000 7fdcbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1737749502 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4021030 con 0x56463dbbd2e0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1737749502 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdcd4010bd0 con 0x56463dbbd2e0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd009bab0 con 0x56463dbbd2e0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1737749502 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdcd4010dd0 con 0x56463dbbd2e0 2024-09-12T02:11:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 10 client.18768 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56463dc3d270 con 0x56463dbbd2e0 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1737749502 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd009bab0 con 0x56463dbbd2e0 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1737749502 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdcd4026450 con 0x56463dbbd2e0 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 10 client.18768 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 10 client.18768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 20 client.18768 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dc3dcd0 0x56463dc400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 4 client.18768 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56463db27d70 con 0x56463dc3dcd0 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.712+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dc3dcd0 0x56463dc400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.713+0000 7fdcdf7fe640 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dc3dcd0 0x56463dc400f0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffea318cd12bbc3 server_cookie=b46ff48972bc8d in_seq=0 out_seq=0 2024-09-12T02:11:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.713+0000 7fdcbeffd640 10 client.18768 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdcbeffd640 1 -- 172.21.15.122:0/1737749502 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56463db27d70 con 0x56463dc3dcd0 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdcbeffd640 10 client.18768 handle_command_reply: tid=0 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 client.18768 shutdown 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 msgr2=0x56463dbcbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbcbe10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005d00 tx=0x7fdcd4012bd0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 shutdown_connections 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x56463dc3dcd0 0x56463dc400f0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463dbc2370 0x56463dbcce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56463dbbd2e0 0x56463dbcbe10 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdcbeffd640 0 client.18768 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 --2- 172.21.15.122:0/1737749502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56463dbbc9a0 0x56463dbcb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 >> 172.21.15.122:0/1737749502 conn(0x56463dbb8010 msgr2=0x56463dbb71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 shutdown_connections 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 1 -- 172.21.15.122:0/1737749502 wait complete. 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.963+0000 7fdce8e8e740 20 client.18768 trim_cache size 0 max 16384 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:11:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:11:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2c30 msgr2=0x7fdcc80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2c30 0x7fdcc80a50f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdcd804e000 tx=0x7fdcd804f000 comp rx=0 tx=0).stop 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 msgr2=0x7fdce006a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 0x7fdce006a410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e9c0 tx=0x7fdcd000ee80 comp rx=0 tx=0).stop 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 shutdown_connections 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdcc80a2c30 0x7fdcc80a50f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdce01025f0 0x7fdce019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdce0101c30 0x7fdce019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 --2- 172.21.15.122:0/113633746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0100a40 0x7fdce006a410 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 >> 172.21.15.122:0/113633746 conn(0x7fdce00fc1e0 msgr2=0x7fdce0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 shutdown_connections 2024-09-12T02:11:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:07.964+0000 7fdce60f1640 1 -- 172.21.15.122:0/113633746 wait complete. 2024-09-12T02:11:08.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 253929 20.26 MB/sec execute 535 sec latency 207.492 ms 2024-09-12T02:11:08.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:08 smithi183 ceph-mon[35127]: pgmap v1079: 97 pgs: 97 active+clean; 537 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.2 MiB/s wr, 521 op/s 2024-09-12T02:11:08.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:08 smithi122 ceph-mon[30925]: pgmap v1079: 97 pgs: 97 active+clean; 537 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.2 MiB/s wr, 521 op/s 2024-09-12T02:11:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:08 smithi151 ceph-mon[35229]: pgmap v1079: 97 pgs: 97 active+clean; 537 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.2 MiB/s wr, 521 op/s 2024-09-12T02:11:09.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 254503 20.29 MB/sec execute 536 sec latency 56.367 ms 2024-09-12T02:11:10.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 254919 20.28 MB/sec execute 537 sec latency 330.863 ms 2024-09-12T02:11:10.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:10 smithi183 ceph-mon[35127]: pgmap v1080: 97 pgs: 97 active+clean; 542 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.6 MiB/s wr, 530 op/s 2024-09-12T02:11:10.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:10 smithi122 ceph-mon[30925]: pgmap v1080: 97 pgs: 97 active+clean; 542 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.6 MiB/s wr, 530 op/s 2024-09-12T02:11:10.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:10 smithi151 ceph-mon[35229]: pgmap v1080: 97 pgs: 97 active+clean; 542 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.6 MiB/s wr, 530 op/s 2024-09-12T02:11:11.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 255434 20.30 MB/sec execute 538 sec latency 78.615 ms 2024-09-12T02:11:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:11 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:11.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:11 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:11.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:11 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:12.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 256076 20.32 MB/sec execute 539 sec latency 80.971 ms 2024-09-12T02:11:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:12 smithi183 ceph-mon[35127]: pgmap v1081: 97 pgs: 97 active+clean; 538 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2024-09-12T02:11:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:11:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:12 smithi122 ceph-mon[30925]: pgmap v1081: 97 pgs: 97 active+clean; 538 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2024-09-12T02:11:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:12.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:11:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:12 smithi151 ceph-mon[35229]: pgmap v1081: 97 pgs: 97 active+clean; 538 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2024-09-12T02:11:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:11:13.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 256693 20.36 MB/sec execute 540 sec latency 120.233 ms 2024-09-12T02:11:13.586 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.7 2024-09-12T02:11:13.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:14.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 257310 20.39 MB/sec execute 541 sec latency 69.682 ms 2024-09-12T02:11:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:14 smithi183 ceph-mon[35127]: pgmap v1082: 97 pgs: 97 active+clean; 541 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 592 op/s 2024-09-12T02:11:14.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:14 smithi122 ceph-mon[30925]: pgmap v1082: 97 pgs: 97 active+clean; 541 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 592 op/s 2024-09-12T02:11:14.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:14 smithi151 ceph-mon[35229]: pgmap v1082: 97 pgs: 97 active+clean; 541 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 592 op/s 2024-09-12T02:11:15.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 257679 20.38 MB/sec execute 542 sec latency 312.549 ms 2024-09-12T02:11:16.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 258057 20.37 MB/sec execute 543 sec latency 159.442 ms 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 -- 172.21.15.122:0/2621822597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954077040 msgr2=0x7ff954112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 --2- 172.21.15.122:0/2621822597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954077040 0x7ff954112330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff94c01acb0 tx=0x7ff94c03b8e0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 -- 172.21.15.122:0/2621822597 shutdown_connections 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 --2- 172.21.15.122:0/2621822597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff954112870 0x7ff954114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 --2- 172.21.15.122:0/2621822597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954077040 0x7ff954112330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 --2- 172.21.15.122:0/2621822597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff954076720 0x7ff954076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.227+0000 7ff95359e640 1 -- 172.21.15.122:0/2621822597 >> 172.21.15.122:0/2621822597 conn(0x7ff9540ff910 msgr2=0x7ff954101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.228+0000 7ff95359e640 1 -- 172.21.15.122:0/2621822597 shutdown_connections 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.228+0000 7ff95359e640 1 -- 172.21.15.122:0/2621822597 wait complete. 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.228+0000 7ff95359e640 1 Processor -- start 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 -- start start 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 0x7ff954111fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff954077040 0x7ff95410e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff954112870 0x7ff95410e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541064c0 con 0x7ff954112870 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff954106340 con 0x7ff954077040 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95359e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff954106640 con 0x7ff954076720 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 0x7ff954111fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 0x7ff954111fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:38680/0 (socket says 172.21.15.122:38680) 2024-09-12T02:11:16.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 -- 172.21.15.122:0/2475325119 learned_addr learned my addr 172.21.15.122:0/2475325119 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff952d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff954112870 0x7ff95410e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff951d9b640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff954077040 0x7ff95410e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 -- 172.21.15.122:0/2475325119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff954077040 msgr2=0x7ff95410e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff954077040 0x7ff95410e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 -- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff954112870 msgr2=0x7ff95410e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff954112870 0x7ff95410e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.229+0000 7ff95259c640 1 -- 172.21.15.122:0/2475325119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95410ed20 con 0x7ff954076720 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.230+0000 7ff95259c640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 0x7ff954111fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff94400e9f0 tx=0x7ff94400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.230+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94400cd80 con 0x7ff954076720 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.230+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff944010910 con 0x7ff954076720 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.230+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944018980 con 0x7ff954076720 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.231+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9541a27a0 con 0x7ff954076720 2024-09-12T02:11:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.231+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a2ba0 con 0x7ff954076720 2024-09-12T02:11:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.234+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff944010ac0 con 0x7ff954076720 2024-09-12T02:11:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.234+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff954077e60 con 0x7ff954076720 2024-09-12T02:11:16.154 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:11:16.154 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.235+0000 7ff94b7fe640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9400a2c80 0x7ff9400a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.236+0000 7ff951d9b640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9400a2c80 0x7ff9400a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.236+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff9440cf360 con 0x7ff954076720 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.236+0000 7ff951d9b640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9400a2c80 0x7ff9400a5140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff95410f820 tx=0x7ff94c042490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.241+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff94409c280 con 0x7ff954076720 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.515+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9541083f0 con 0x7ff954076720 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.516+0000 7ff94b7fe640 1 -- 172.21.15.122:0/2475325119 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7ff94409bb80 con 0x7ff954076720 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:11:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.519+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9400a2c80 msgr2=0x7ff9400a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.519+0000 7ff95359e640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9400a2c80 0x7ff9400a5140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff95410f820 tx=0x7ff94c042490 comp rx=0 tx=0).stop 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.519+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 msgr2=0x7ff954111fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.519+0000 7ff95359e640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 0x7ff954111fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff94400e9f0 tx=0x7ff94400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.520+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 shutdown_connections 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.520+0000 7ff95359e640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7ff9400a2c80 0x7ff9400a5140 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.520+0000 7ff95359e640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff954112870 0x7ff95410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.520+0000 7ff95359e640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff954077040 0x7ff95410e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.520+0000 7ff95359e640 1 --2- 172.21.15.122:0/2475325119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff954076720 0x7ff954111fc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.520+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 >> 172.21.15.122:0/2475325119 conn(0x7ff9540ff910 msgr2=0x7ff954101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.522+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 shutdown_connections 2024-09-12T02:11:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:15.523+0000 7ff95359e640 1 -- 172.21.15.122:0/2475325119 wait complete. 2024-09-12T02:11:16.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:16 smithi183 ceph-mon[35127]: pgmap v1083: 97 pgs: 97 active+clean; 545 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 669 op/s 2024-09-12T02:11:16.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:16 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2475325119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:16 smithi122 ceph-mon[30925]: pgmap v1083: 97 pgs: 97 active+clean; 545 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 669 op/s 2024-09-12T02:11:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:16 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2475325119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:16 smithi151 ceph-mon[35229]: pgmap v1083: 97 pgs: 97 active+clean; 545 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 669 op/s 2024-09-12T02:11:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:16 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2475325119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:17.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 258616 20.40 MB/sec execute 544 sec latency 115.548 ms 2024-09-12T02:11:17.355 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 153.7 --await-for 45.0' 2024-09-12T02:11:17.355 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 153.7 --await-for 45.0 2024-09-12T02:11:17.689 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:18.087 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259224 20.41 MB/sec execute 545 sec latency 93.411 ms 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 -- 172.21.15.122:0/765489648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658106a70 msgr2=0x7fd658106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/765489648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658106a70 0x7fd658106e50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd648009930 tx=0x7fd64802a420 comp rx=0 tx=0).stop 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 -- 172.21.15.122:0/765489648 shutdown_connections 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/765489648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd65810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/765489648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658100a70 0x7fd658100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/765489648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658106a70 0x7fd658106e50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 -- 172.21.15.122:0/765489648 >> 172.21.15.122:0/765489648 conn(0x7fd6580fc7d0 msgr2=0x7fd6580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 -- 172.21.15.122:0/765489648 shutdown_connections 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.204+0000 7fd65d2ea640 1 -- 172.21.15.122:0/765489648 wait complete. 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 Processor -- start 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 -- start start 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658100a70 0x7fd658197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd658197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658106a70 0x7fd65819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd658110e70 con 0x7fd658100a70 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd658110cf0 con 0x7fd658106a70 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd65d2ea640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd658110ff0 con 0x7fd658101410 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd64e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd658197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd64e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd658197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:38710/0 (socket says 172.21.15.122:38710) 2024-09-12T02:11:18.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd64e7fc640 1 -- 172.21.15.122:0/721502893 learned_addr learned my addr 172.21.15.122:0/721502893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd656ffd640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658100a70 0x7fd658197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.205+0000 7fd6577fe640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658106a70 0x7fd65819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd64e7fc640 1 -- 172.21.15.122:0/721502893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658106a70 msgr2=0x7fd65819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658106a70 0x7fd65819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd64e7fc640 1 -- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658100a70 msgr2=0x7fd658197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658100a70 0x7fd658197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd64e7fc640 1 -- 172.21.15.122:0/721502893 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65819c2e0 con 0x7fd658101410 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd656ffd640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658100a70 0x7fd658197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd658197870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd64000eb00 tx=0x7fd640009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd654ff9640 1 -- 172.21.15.122:0/721502893 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64001c070 con 0x7fd658101410 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd654ff9640 1 -- 172.21.15.122:0/721502893 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64000bc30 con 0x7fd658101410 2024-09-12T02:11:18.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd654ff9640 1 -- 172.21.15.122:0/721502893 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400166a0 con 0x7fd658101410 2024-09-12T02:11:18.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65819c5d0 con 0x7fd658101410 2024-09-12T02:11:18.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.206+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65806a790 con 0x7fd658101410 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.209+0000 7fd654ff9640 1 -- 172.21.15.122:0/721502893 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd64001f070 con 0x7fd658101410 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 Processor -- start 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 -- start start 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317756760 0x55f317756b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd654ff9640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd6440a2c00 0x7fd6440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 0x55f317761400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f317763d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3176bc2a0 con 0x55f317757080 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f317663b10 con 0x55f317756760 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f3176b8260 con 0x55f317761940 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd64e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 0x55f317761400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd6577fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317756760 0x55f317756b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd64e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 0x55f317761400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40020/0 (socket says 172.21.15.122:40020) 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.210+0000 7fd64e7fc640 1 -- 172.21.15.122:0/160673562 learned_addr learned my addr 172.21.15.122:0/160673562 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd656ffd640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd6440a2c00 0x7fd6440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd6577fe640 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f317763d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd654ff9640 1 -- 172.21.15.122:0/721502893 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd6400d0630 con 0x7fd658101410 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64e7fc640 1 -- 172.21.15.122:0/160673562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 msgr2=0x55f317763d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f317763d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64e7fc640 1 -- 172.21.15.122:0/160673562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317756760 msgr2=0x55f317756b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317756760 0x55f317756b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64e7fc640 1 -- 172.21.15.122:0/160673562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f31742db90 con 0x55f317757080 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 0x55f317761400 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd6400979a0 tx=0x7fd640017d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49e37910c2df8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64d7fa640 1 -- 172.21.15.122:0/160673562 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640026040 con 0x55f317757080 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64d7fa640 1 -- 172.21.15.122:0/160673562 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64000bde0 con 0x55f317757080 2024-09-12T02:11:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.211+0000 7fd64d7fa640 1 -- 172.21.15.122:0/160673562 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64009d3a0 con 0x55f317757080 2024-09-12T02:11:18.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.212+0000 7fd65f6c3740 1 -- 172.21.15.122:0/160673562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 msgr2=0x55f317761400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:18.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.212+0000 7fd656ffd640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd6440a2c00 0x7fd6440a50c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd648009900 tx=0x7fd6480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.212+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 0x55f317761400 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd6400979a0 tx=0x7fd640017d60 comp rx=0 tx=0).stop 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- 172.21.15.122:0/160673562 shutdown_connections 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f317763d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317757080 0x55f317761400 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/160673562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317756760 0x55f317756b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- 172.21.15.122:0/160673562 >> 172.21.15.122:0/160673562 conn(0x55f317752cf0 msgr2=0x55f3177530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- 172.21.15.122:0/160673562 shutdown_connections 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- 172.21.15.122:0/160673562 wait complete. 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- start start 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 Processor -- start 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f31775ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f31775f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3177644b0 con 0x55f317756760 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3176446e0 con 0x55f317761940 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.213+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f317764630 con 0x55f317757080 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f31775ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40034/0 (socket says 172.21.15.122:40034) 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd656ffd640 1 -- 172.21.15.122:0/3876652606 learned_addr learned my addr 172.21.15.122:0/3876652606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd65f6c3740 10 client.0 _ll_register_callbacks cb 0x55f317668a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd6577fe640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f31775f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 -- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 msgr2=0x55f31775f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f31775f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 -- 172.21.15.122:0/3876652606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 msgr2=0x55f31775e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 -- 172.21.15.122:0/3876652606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f31776da20 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd6577fe640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f31775f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f31775ee20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd6400979a0 tx=0x7fd640098af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1479bb59360f4b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd63e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd63e7fc640 1 -- 172.21.15.122:0/3876652606 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400a1200 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd63e7fc640 1 -- 172.21.15.122:0/3876652606 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd640099570 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.214+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd64000bde0 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.215+0000 7fd63e7fc640 1 -- 172.21.15.122:0/3876652606 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd6400a5ac0 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.215+0000 7fd65f6c3740 10 client.27715 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.215+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f3177644b0 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.215+0000 7fd63e7fc640 1 -- 172.21.15.122:0/3876652606 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64000bde0 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd63e7fc640 1 -- 172.21.15.122:0/3876652606 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fd64009dc60 con 0x55f317757080 2024-09-12T02:11:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd65f6c3740 10 client.27715 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd65f6c3740 10 client.27715 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd65f6c3740 20 client.27715 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f317771530 0x55f317773920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd65f6c3740 4 client.27715 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3176c99d0 con 0x55f317771530 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.216+0000 7fd656ffd640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f317771530 0x55f317773920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.217+0000 7fd63e7fc640 10 client.27715 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.217+0000 7fd656ffd640 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f317771530 0x55f317773920 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ded61155f4dda21 server_cookie=fcd6d989076fd216 in_seq=0 out_seq=0 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.217+0000 7fd63e7fc640 1 -- 172.21.15.122:0/3876652606 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3176c99d0 con 0x55f317771530 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.217+0000 7fd63e7fc640 10 client.27715 handle_command_reply: tid=0 2024-09-12T02:11:18.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.217+0000 7fd65f6c3740 1 client.27715 shutdown 2024-09-12T02:11:18.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 msgr2=0x55f31775ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:18.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f31775ee20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd6400979a0 tx=0x7fd640098af0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 shutdown_connections 2024-09-12T02:11:18.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f317771530 0x55f317773920 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f31775f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd63e7fc640 0 client.27715 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f31775ee20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/3876652606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 >> 172.21.15.122:0/3876652606 conn(0x55f317752cf0 msgr2=0x55f317751e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 shutdown_connections 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 1 -- 172.21.15.122:0/3876652606 wait complete. 2024-09-12T02:11:18.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.218+0000 7fd65f6c3740 20 client.27715 trim_cache size 0 max 16384 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.240+0000 7fd65f6c3740 1 Processor -- start 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.240+0000 7fd65f6c3740 1 -- start start 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317757080 0x55f31776ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f31776d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581add40 con 0x55f317756760 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581ab610 con 0x55f317757080 2024-09-12T02:11:18.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581a9440 con 0x55f317761940 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40038/0 (socket says 172.21.15.122:40038) 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 -- 172.21.15.122:0/2802809861 learned_addr learned my addr 172.21.15.122:0/2802809861 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317757080 0x55f31776ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd6577fe640 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f31776d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 -- 172.21.15.122:0/2802809861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 msgr2=0x55f31776d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f31776d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 -- 172.21.15.122:0/2802809861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317757080 msgr2=0x55f31776ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317757080 0x55f31776ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.241+0000 7fd656ffd640 1 -- 172.21.15.122:0/2802809861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f31774e260 con 0x55f317756760 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.242+0000 7fd656ffd640 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775f1b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd648009230 tx=0x7fd648041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8844f656adffcfd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:18.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.242+0000 7fd64cff9640 1 -- 172.21.15.122:0/2802809861 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648035040 con 0x55f317756760 2024-09-12T02:11:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.242+0000 7fd64cff9640 1 -- 172.21.15.122:0/2802809861 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd648004770 con 0x55f317756760 2024-09-12T02:11:18.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.242+0000 7fd64cff9640 1 -- 172.21.15.122:0/2802809861 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648003850 con 0x55f317756760 2024-09-12T02:11:18.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.242+0000 7fd65f6c3740 1 -- 172.21.15.122:0/2802809861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 msgr2=0x55f31775f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.242+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775f1b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd648009230 tx=0x7fd648041000 comp rx=0 tx=0).stop 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- 172.21.15.122:0/2802809861 shutdown_connections 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317761940 0x55f31776d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317757080 0x55f31776ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/2802809861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f31775f1b0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- 172.21.15.122:0/2802809861 >> 172.21.15.122:0/2802809861 conn(0x55f317752cf0 msgr2=0x55f31776c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- 172.21.15.122:0/2802809861 shutdown_connections 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- 172.21.15.122:0/2802809861 wait complete. 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- start start 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 Processor -- start 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f317765e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f317766360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f3177d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581a9ad0 con 0x55f317756760 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581aaf40 con 0x55f317761940 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581aa1a0 con 0x55f317757080 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd65f6c3740 10 client.0 _ll_register_callbacks cb 0x7fd64009ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.243+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f317765e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd6577fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f3177d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f317765e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40042/0 (socket says 172.21.15.122:40042) 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 -- 172.21.15.122:0/4095649369 learned_addr learned my addr 172.21.15.122:0/4095649369 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f317766360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 -- 172.21.15.122:0/4095649369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 msgr2=0x55f317766360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f317766360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 -- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 msgr2=0x55f3177d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f3177d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 -- 172.21.15.122:0/4095649369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3177d5450 con 0x55f317756760 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd6577fe640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f3177d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f317766360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd656ffd640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f317765e20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd648005cd0 tx=0x7fd6480316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b7c56ab93eb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd63e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd63e7fc640 1 -- 172.21.15.122:0/4095649369 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648035040 con 0x55f317756760 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd63e7fc640 1 -- 172.21.15.122:0/4095649369 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd648005000 con 0x55f317756760 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd648004770 con 0x55f317756760 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd63e7fc640 1 -- 172.21.15.122:0/4095649369 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd648005200 con 0x55f317756760 2024-09-12T02:11:18.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd65f6c3740 10 client.18792 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.244+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f3177dc7b0 con 0x55f317756760 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd63e7fc640 1 -- 172.21.15.122:0/4095649369 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd648004770 con 0x55f317756760 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd63e7fc640 1 -- 172.21.15.122:0/4095649369 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fd6480523a0 con 0x55f317756760 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd65f6c3740 10 client.18792 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd65f6c3740 10 client.18792 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd65f6c3740 20 client.18792 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3177dcf30 0x55f3177df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd65f6c3740 4 client.18792 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.7, "await_for": 45.0}] 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.7, "await_for": 45.0}) -- 0x55f3176c3d20 con 0x55f3177dcf30 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3177dcf30 0x55f3177df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd64e7fc640 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3177dcf30 0x55f3177df350 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66747bc539034748 server_cookie=a0339b8a37854568 in_seq=0 out_seq=0 2024-09-12T02:11:18.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:18.245+0000 7fd63e7fc640 10 client.18792 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:18.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:18 smithi183 ceph-mon[35127]: pgmap v1084: 97 pgs: 97 active+clean; 546 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 549 op/s 2024-09-12T02:11:18.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:18 smithi122 ceph-mon[30925]: pgmap v1084: 97 pgs: 97 active+clean; 546 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 549 op/s 2024-09-12T02:11:18.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:18 smithi151 ceph-mon[35229]: pgmap v1084: 97 pgs: 97 active+clean; 546 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 549 op/s 2024-09-12T02:11:19.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd63e7fc640 1 -- 172.21.15.122:0/4095649369 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f3176c3d20 con 0x55f3177dcf30 2024-09-12T02:11:19.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd63e7fc640 10 client.18792 handle_command_reply: tid=0 2024-09-12T02:11:19.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 client.18792 shutdown 2024-09-12T02:11:19.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 msgr2=0x55f317765e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:19.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f317765e20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd648005cd0 tx=0x7fd6480316e0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 shutdown_connections 2024-09-12T02:11:19.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55f3177dcf30 0x55f3177df350 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f317761940 0x55f3177d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f317757080 0x55f317766360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 --2- 172.21.15.122:0/4095649369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f317756760 0x55f317765e20 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 >> 172.21.15.122:0/4095649369 conn(0x55f317752cf0 msgr2=0x55f31776cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd63e7fc640 0 client.18792 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 shutdown_connections 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 1 -- 172.21.15.122:0/4095649369 wait complete. 2024-09-12T02:11:19.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.050+0000 7fd65f6c3740 20 client.18792 trim_cache size 0 max 16384 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 86, 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "c507febe": { 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 86, 2024-09-12T02:11:19.053 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:11:19.054 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:11:19.054 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:11:19.054 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:11:19.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:11:19.054 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:11:19.055 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 153.7, 2024-09-12T02:11:19.055 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:11:19.055 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:11:19.055 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:11:19.055 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:11:19.056 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "QUIESCED", 2024-09-12T02:11:19.056 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:11:19.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:19.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:19.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:19.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:19.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:19.057 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:11:19.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd6440a2c00 msgr2=0x7fd6440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd6440a2c00 0x7fd6440a50c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd648009900 tx=0x7fd6480058d0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 msgr2=0x7fd658197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd658197870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd64000eb00 tx=0x7fd640009f20 comp rx=0 tx=0).stop 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 shutdown_connections 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd6440a2c00 0x7fd6440a50c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658106a70 0x7fd65819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd658101410 0x7fd658197870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 --2- 172.21.15.122:0/721502893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd658100a70 0x7fd658197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 >> 172.21.15.122:0/721502893 conn(0x7fd6580fc7d0 msgr2=0x7fd6580fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 shutdown_connections 2024-09-12T02:11:19.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:19.051+0000 7fd65d2ea640 1 -- 172.21.15.122:0/721502893 wait complete. 2024-09-12T02:11:19.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.38 MB/sec execute 546 sec latency 831.109 ms 2024-09-12T02:11:19.722 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c507febe', quiesce took 6.1 seconds. Will release after: 27.6 2024-09-12T02:11:20.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.34 MB/sec execute 547 sec latency 1831.191 ms 2024-09-12T02:11:20.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:20 smithi183 ceph-mon[35127]: pgmap v1085: 97 pgs: 97 active+clean; 551 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2024-09-12T02:11:20.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:20 smithi122 ceph-mon[30925]: pgmap v1085: 97 pgs: 97 active+clean; 551 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2024-09-12T02:11:20.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:20 smithi151 ceph-mon[35229]: pgmap v1085: 97 pgs: 97 active+clean; 551 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2024-09-12T02:11:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 -- 172.21.15.122:0/101673926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 msgr2=0x7fdc84100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 --2- 172.21.15.122:0/101673926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84100e60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009900 tx=0x7fdc7402a410 comp rx=0 tx=0).stop 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 -- 172.21.15.122:0/101673926 shutdown_connections 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 --2- 172.21.15.122:0/101673926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84102610 0x7fdc84108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 --2- 172.21.15.122:0/101673926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84101c50 0x7fdc841020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 --2- 172.21.15.122:0/101673926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84100e60 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 -- 172.21.15.122:0/101673926 >> 172.21.15.122:0/101673926 conn(0x7fdc840fc1e0 msgr2=0x7fdc840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 -- 172.21.15.122:0/101673926 shutdown_connections 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.050+0000 7fdc8a598640 1 -- 172.21.15.122:0/101673926 wait complete. 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 Processor -- start 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 -- start start 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84101c50 0x7fdc84197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84102610 0x7fdc8419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8410c740 con 0x7fdc84100a60 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8410c5c0 con 0x7fdc84101c50 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc8a598640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8410c8c0 con 0x7fdc84102610 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50440/0 (socket says 172.21.15.122:50440) 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc83fff640 1 -- 172.21.15.122:0/1795376551 learned_addr learned my addr 172.21.15.122:0/1795376551 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.051+0000 7fdc88b35640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84102610 0x7fdc8419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc837fe640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84101c50 0x7fdc84197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc83fff640 1 -- 172.21.15.122:0/1795376551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84102610 msgr2=0x7fdc8419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc83fff640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84102610 0x7fdc8419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc83fff640 1 -- 172.21.15.122:0/1795376551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84101c50 msgr2=0x7fdc84197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc83fff640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84101c50 0x7fdc84197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc83fff640 1 -- 172.21.15.122:0/1795376551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8419e930 con 0x7fdc84100a60 2024-09-12T02:11:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc837fe640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84101c50 0x7fdc84197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc88b35640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84102610 0x7fdc8419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc83fff640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84196d20 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdc74031610 tx=0x7fdc740043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc817fa640 1 -- 172.21.15.122:0/1795376551 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7403f030 con 0x7fdc84100a60 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8419ebc0 con 0x7fdc84100a60 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc817fa640 1 -- 172.21.15.122:0/1795376551 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc74003ce0 con 0x7fdc84100a60 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc817fa640 1 -- 172.21.15.122:0/1795376551 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74035070 con 0x7fdc84100a60 2024-09-12T02:11:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.052+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8419f100 con 0x7fdc84100a60 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.054+0000 7fdc817fa640 1 -- 172.21.15.122:0/1795376551 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdc74042030 con 0x7fdc84100a60 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 Processor -- start 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 -- start start 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc5afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 0x55561cc5da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55561cbb72f0 con 0x55561cc5b580 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55561cb5f270 con 0x55561cc564f0 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc8d412740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55561cbb32b0 con 0x55561cc55bb0 2024-09-12T02:11:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc817fa640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc6c0a2c30 0x7fdc6c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 0x55561cc5da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 0x55561cc5da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50454/0 (socket says 172.21.15.122:50454) 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 -- 172.21.15.122:0/2033548172 learned_addr learned my addr 172.21.15.122:0/2033548172 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc88b35640 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc83fff640 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc5afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc88b35640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc6c0a2c30 0x7fdc6c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc817fa640 1 -- 172.21.15.122:0/1795376551 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdc740ef830 con 0x7fdc84100a60 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 -- 172.21.15.122:0/2033548172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 msgr2=0x55561cc55fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc55fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 -- 172.21.15.122:0/2033548172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 msgr2=0x55561cc5afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc5afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 -- 172.21.15.122:0/2033548172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561cb5e340 con 0x55561cc5b580 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc88b35640 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.055+0000 7fdc837fe640 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 0x55561cc5da40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800aa10 tx=0x7fdc7800aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9520e382eff03161 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc6a7fc640 1 -- 172.21.15.122:0/2033548172 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78015070 con 0x55561cc5b580 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc88b35640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc6c0a2c30 0x7fdc6c0a50f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c007fd0 tx=0x7fdc7c0072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc6a7fc640 1 -- 172.21.15.122:0/2033548172 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7800cc60 con 0x55561cc5b580 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc6a7fc640 1 -- 172.21.15.122:0/2033548172 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78018760 con 0x55561cc5b580 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 -- 172.21.15.122:0/2033548172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 msgr2=0x55561cc5da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 0x55561cc5da40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800aa10 tx=0x7fdc7800aed0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 -- 172.21.15.122:0/2033548172 shutdown_connections 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc5b580 0x55561cc5da40 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc5afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 --2- 172.21.15.122:0/2033548172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 -- 172.21.15.122:0/2033548172 >> 172.21.15.122:0/2033548172 conn(0x55561cc4f010 msgr2=0x55561cc4f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 -- 172.21.15.122:0/2033548172 shutdown_connections 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.056+0000 7fdc8d412740 1 -- 172.21.15.122:0/2033548172 wait complete. 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 -- start start 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 Processor -- start 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561cc5fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc5b580 0x55561cc67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55561cc5e550 con 0x55561cc55bb0 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55561cb40290 con 0x55561cc564f0 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55561cc5e6d0 con 0x55561cc5b580 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc8d412740 10 client.0 _ll_register_callbacks cb 0x55561cb71140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561cc5fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561cc5fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50464/0 (socket says 172.21.15.122:50464) 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc83fff640 1 -- 172.21.15.122:0/267804818 learned_addr learned my addr 172.21.15.122:0/267804818 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc88b35640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc5b580 0x55561cc67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.057+0000 7fdc837fe640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc83fff640 1 -- 172.21.15.122:0/267804818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc5b580 msgr2=0x55561cc67320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc83fff640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc5b580 0x55561cc67320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc83fff640 1 -- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 msgr2=0x55561cc602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc83fff640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc83fff640 1 -- 172.21.15.122:0/267804818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561c8ca210 con 0x55561cc55bb0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc83fff640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561cc5fda0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdc740314e0 tx=0x7fdc74047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9368ee3b5236af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc637fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc637fe640 1 -- 172.21.15.122:0/267804818 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74049070 con 0x55561cc55bb0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc637fe640 1 -- 172.21.15.122:0/267804818 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdc740c00b0 con 0x55561cc55bb0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7800cc60 con 0x55561cc55bb0 2024-09-12T02:11:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc637fe640 1 -- 172.21.15.122:0/267804818 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc740ba1d0 con 0x55561cc55bb0 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 10 client.18804 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55561c996620 con 0x55561cc55bb0 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc637fe640 1 -- 172.21.15.122:0/267804818 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7800cc60 con 0x55561cc55bb0 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc637fe640 1 -- 172.21.15.122:0/267804818 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdc740b8670 con 0x55561cc55bb0 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 10 client.18804 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 10 client.18804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 20 client.18804 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561cc6b9b0 0x55561cc6de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 4 client.18804 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.058+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55561cbc4a20 con 0x55561cc6b9b0 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.059+0000 7fdc837fe640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561cc6b9b0 0x55561cc6de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.059+0000 7fdc637fe640 10 client.18804 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.059+0000 7fdc837fe640 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561cc6b9b0 0x55561cc6de20 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793640e1cce5f78 server_cookie=91ef24298ea411bb in_seq=0 out_seq=0 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc637fe640 1 -- 172.21.15.122:0/267804818 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55561cbc4a20 con 0x55561cc6b9b0 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc637fe640 10 client.18804 handle_command_reply: tid=0 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 client.18804 shutdown 2024-09-12T02:11:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 msgr2=0x55561cc5fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561cc5fda0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdc740314e0 tx=0x7fdc74047000 comp rx=0 tx=0).stop 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 shutdown_connections 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561cc6b9b0 0x55561cc6de20 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc5b580 0x55561cc67320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc637fe640 0 client.18804 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc564f0 0x55561cc602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 --2- 172.21.15.122:0/267804818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561cc5fda0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 >> 172.21.15.122:0/267804818 conn(0x55561cc4f010 msgr2=0x55561cc50820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 shutdown_connections 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 1 -- 172.21.15.122:0/267804818 wait complete. 2024-09-12T02:11:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.060+0000 7fdc8d412740 20 client.18804 trim_cache size 0 max 16384 2024-09-12T02:11:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 Processor -- start 2024-09-12T02:11:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 -- start start 2024-09-12T02:11:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc4fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 0x55561cc50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 0x55561cc50940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841a9730 con 0x55561cc564f0 2024-09-12T02:11:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841a7000 con 0x55561cc5b580 2024-09-12T02:11:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc8d412740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841a4e30 con 0x55561cc55bb0 2024-09-12T02:11:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 0x55561cc50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 0x55561cc50400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50468/0 (socket says 172.21.15.122:50468) 2024-09-12T02:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 -- 172.21.15.122:0/74446631 learned_addr learned my addr 172.21.15.122:0/74446631 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc83fff640 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc4fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 -- 172.21.15.122:0/74446631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 msgr2=0x55561cc4fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc4fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 -- 172.21.15.122:0/74446631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 msgr2=0x55561cc50940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 0x55561cc50940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc837fe640 1 -- 172.21.15.122:0/74446631 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561cc5f8e0 con 0x55561cc564f0 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.074+0000 7fdc83fff640 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc4fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc837fe640 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 0x55561cc50400 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800c520 tx=0x7fdc78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dede5e723a10bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc69ffb640 1 -- 172.21.15.122:0/74446631 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78015040 con 0x55561cc564f0 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc69ffb640 1 -- 172.21.15.122:0/74446631 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78006be0 con 0x55561cc564f0 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc69ffb640 1 -- 172.21.15.122:0/74446631 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801a7d0 con 0x55561cc564f0 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 -- 172.21.15.122:0/74446631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 msgr2=0x55561cc50400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 0x55561cc50400 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800c520 tx=0x7fdc78008040 comp rx=0 tx=0).stop 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 -- 172.21.15.122:0/74446631 shutdown_connections 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 0x55561cc50940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc564f0 0x55561cc50400 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 --2- 172.21.15.122:0/74446631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc55bb0 0x55561cc4fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 -- 172.21.15.122:0/74446631 >> 172.21.15.122:0/74446631 conn(0x55561cc4f010 msgr2=0x55561cc57690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 -- 172.21.15.122:0/74446631 shutdown_connections 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.075+0000 7fdc8d412740 1 -- 172.21.15.122:0/74446631 wait complete. 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 -- start start 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 Processor -- start 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561ccd26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc564f0 0x55561ccd4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 0x55561ccd7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841a54c0 con 0x55561cc55bb0 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841a6930 con 0x55561cc5b580 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841a5b90 con 0x55561cc564f0 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc8d412740 10 client.0 _ll_register_callbacks cb 0x7fdc74005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561ccd26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561ccd26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50470/0 (socket says 172.21.15.122:50470) 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 -- 172.21.15.122:0/1771809610 learned_addr learned my addr 172.21.15.122:0/1771809610 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc837fe640 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc564f0 0x55561ccd4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 -- 172.21.15.122:0/1771809610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc564f0 msgr2=0x55561ccd4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc564f0 0x55561ccd4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 -- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 msgr2=0x55561ccd7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 0x55561ccd7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.076+0000 7fdc83fff640 1 -- 172.21.15.122:0/1771809610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561ccd7930 con 0x55561cc55bb0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc83fff640 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561ccd26b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdc740c4d00 tx=0x7fdc7402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c071296f88ed5e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc637fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc637fe640 1 -- 172.21.15.122:0/1771809610 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74049040 con 0x55561cc55bb0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc637fe640 1 -- 172.21.15.122:0/1771809610 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdc74031dd0 con 0x55561cc55bb0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc78006be0 con 0x55561cc55bb0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc637fe640 1 -- 172.21.15.122:0/1771809610 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc740b82a0 con 0x55561cc55bb0 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 10 client.18810 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55561ccd8c90 con 0x55561cc55bb0 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc637fe640 1 -- 172.21.15.122:0/1771809610 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78006be0 con 0x55561cc55bb0 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc637fe640 1 -- 172.21.15.122:0/1771809610 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fdc740bc2f0 con 0x55561cc55bb0 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 10 client.18810 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 10 client.18810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 20 client.18810 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561ccd94d0 0x55561ccdb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 4 client.18810 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55561cbbed70 con 0x55561ccd94d0 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.077+0000 7fdc837fe640 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561ccd94d0 0x55561ccdb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.078+0000 7fdc637fe640 10 client.18810 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.078+0000 7fdc837fe640 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561ccd94d0 0x55561ccdb8f0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b717c8c818a12acf server_cookie=ca73aa37b58cfab2 in_seq=0 out_seq=0 2024-09-12T02:11:21.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.30 MB/sec execute 548 sec latency 2831.236 ms 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc637fe640 1 -- 172.21.15.122:0/1771809610 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55561cbbed70 con 0x55561ccd94d0 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc637fe640 10 client.18810 handle_command_reply: tid=0 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 client.18810 shutdown 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 msgr2=0x55561ccd26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561ccd26b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdc740c4d00 tx=0x7fdc7402a410 comp rx=0 tx=0).stop 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 shutdown_connections 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55561ccd94d0 0x55561ccdb8f0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55561cc5b580 0x55561ccd7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55561cc564f0 0x55561ccd4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc637fe640 0 client.18810 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 --2- 172.21.15.122:0/1771809610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55561cc55bb0 0x55561ccd26b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 >> 172.21.15.122:0/1771809610 conn(0x55561cc4f010 msgr2=0x55561cc4d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 shutdown_connections 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 1 -- 172.21.15.122:0/1771809610 wait complete. 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.255+0000 7fdc8d412740 20 client.18810 trim_cache size 0 max 16384 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:11:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:11:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:11:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc6c0a2c30 msgr2=0x7fdc6c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc6c0a2c30 0x7fdc6c0a50f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c007fd0 tx=0x7fdc7c0072a0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 msgr2=0x7fdc84196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84196d20 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdc74031610 tx=0x7fdc740043b0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 shutdown_connections 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fdc6c0a2c30 0x7fdc6c0a50f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc84102610 0x7fdc8419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc84101c50 0x7fdc84197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 --2- 172.21.15.122:0/1795376551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc84100a60 0x7fdc84196d20 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 >> 172.21.15.122:0/1795376551 conn(0x7fdc840fc1e0 msgr2=0x7fdc84104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 shutdown_connections 2024-09-12T02:11:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:21.256+0000 7fdc8a598640 1 -- 172.21.15.122:0/1795376551 wait complete. 2024-09-12T02:11:22.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.27 MB/sec execute 549 sec latency 3831.277 ms 2024-09-12T02:11:22.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:22 smithi183 ceph-mon[35127]: pgmap v1086: 97 pgs: 97 active+clean; 558 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.4 MiB/s wr, 659 op/s 2024-09-12T02:11:22.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:22 smithi122 ceph-mon[30925]: pgmap v1086: 97 pgs: 97 active+clean; 558 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.4 MiB/s wr, 659 op/s 2024-09-12T02:11:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:22 smithi151 ceph-mon[35229]: pgmap v1086: 97 pgs: 97 active+clean; 558 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.4 MiB/s wr, 659 op/s 2024-09-12T02:11:23.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.23 MB/sec execute 550 sec latency 4831.394 ms 2024-09-12T02:11:24.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.19 MB/sec execute 551 sec latency 5831.513 ms 2024-09-12T02:11:24.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:24 smithi183 ceph-mon[35127]: pgmap v1087: 97 pgs: 97 active+clean; 557 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.3 MiB/s wr, 542 op/s 2024-09-12T02:11:24.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:24 smithi122 ceph-mon[30925]: pgmap v1087: 97 pgs: 97 active+clean; 557 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.3 MiB/s wr, 542 op/s 2024-09-12T02:11:24.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:24 smithi151 ceph-mon[35229]: pgmap v1087: 97 pgs: 97 active+clean; 557 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.3 MiB/s wr, 542 op/s 2024-09-12T02:11:25.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.16 MB/sec execute 552 sec latency 6831.612 ms 2024-09-12T02:11:26.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.12 MB/sec execute 553 sec latency 7831.659 ms 2024-09-12T02:11:26.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:26 smithi183 ceph-mon[35127]: pgmap v1088: 97 pgs: 97 active+clean; 549 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 521 op/s 2024-09-12T02:11:26.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:26 smithi122 ceph-mon[30925]: pgmap v1088: 97 pgs: 97 active+clean; 549 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 521 op/s 2024-09-12T02:11:26.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:26 smithi151 ceph-mon[35229]: pgmap v1088: 97 pgs: 97 active+clean; 549 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.1 MiB/s wr, 521 op/s 2024-09-12T02:11:27.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.08 MB/sec execute 554 sec latency 8831.747 ms 2024-09-12T02:11:28.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.05 MB/sec execute 555 sec latency 9831.834 ms 2024-09-12T02:11:28.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:28 smithi183 ceph-mon[35127]: pgmap v1089: 97 pgs: 97 active+clean; 552 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 422 op/s 2024-09-12T02:11:28.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:28 smithi122 ceph-mon[30925]: pgmap v1089: 97 pgs: 97 active+clean; 552 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 422 op/s 2024-09-12T02:11:28.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:28 smithi151 ceph-mon[35229]: pgmap v1089: 97 pgs: 97 active+clean; 552 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.9 MiB/s wr, 422 op/s 2024-09-12T02:11:29.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 20.01 MB/sec execute 556 sec latency 10831.884 ms 2024-09-12T02:11:30.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.98 MB/sec execute 557 sec latency 11831.947 ms 2024-09-12T02:11:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:30 smithi183 ceph-mon[35127]: pgmap v1090: 97 pgs: 97 active+clean; 550 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.8 MiB/s wr, 417 op/s 2024-09-12T02:11:30.740 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:30 smithi122 ceph-mon[30925]: pgmap v1090: 97 pgs: 97 active+clean; 550 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.8 MiB/s wr, 417 op/s 2024-09-12T02:11:30.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:30 smithi151 ceph-mon[35229]: pgmap v1090: 97 pgs: 97 active+clean; 550 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.8 MiB/s wr, 417 op/s 2024-09-12T02:11:31.088 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.94 MB/sec execute 558 sec latency 12832.005 ms 2024-09-12T02:11:32.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.90 MB/sec execute 559 sec latency 13832.113 ms 2024-09-12T02:11:32.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:32 smithi183 ceph-mon[35127]: pgmap v1091: 97 pgs: 97 active+clean; 546 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-09-12T02:11:32.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:32 smithi122 ceph-mon[30925]: pgmap v1091: 97 pgs: 97 active+clean; 546 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-09-12T02:11:32.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:32 smithi151 ceph-mon[35229]: pgmap v1091: 97 pgs: 97 active+clean; 546 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-09-12T02:11:33.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.87 MB/sec execute 560 sec latency 14832.188 ms 2024-09-12T02:11:34.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.83 MB/sec execute 561 sec latency 15832.292 ms 2024-09-12T02:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.352+0000 7f14b69fd640 1 -- 172.21.15.122:0/3704059208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0103710 msgr2=0x7f14b0109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.352+0000 7f14b69fd640 1 --2- 172.21.15.122:0/3704059208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0103710 0x7f14b0109fa0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009ca0 tx=0x7f14a802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 -- 172.21.15.122:0/3704059208 shutdown_connections 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 --2- 172.21.15.122:0/3704059208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0103710 0x7f14b0109fa0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 --2- 172.21.15.122:0/3704059208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b0102d50 0x7f14b01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 --2- 172.21.15.122:0/3704059208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0101b60 0x7f14b0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 -- 172.21.15.122:0/3704059208 >> 172.21.15.122:0/3704059208 conn(0x7f14b00fd2e0 msgr2=0x7f14b00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 -- 172.21.15.122:0/3704059208 shutdown_connections 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 -- 172.21.15.122:0/3704059208 wait complete. 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 Processor -- start 2024-09-12T02:11:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.353+0000 7f14b69fd640 1 -- start start 2024-09-12T02:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 0x7f14b0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b0102d50 0x7f14b0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0103710 0x7f14b019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010d7c0 con 0x7f14b0101b60 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010d640 con 0x7f14b0103710 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010d940 con 0x7f14b0102d50 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 0x7f14b0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 0x7f14b0197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49130/0 (socket says 172.21.15.122:49130) 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 -- 172.21.15.122:0/2722790894 learned_addr learned my addr 172.21.15.122:0/2722790894 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14af7fe640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b0102d50 0x7f14b0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0103710 0x7f14b019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 -- 172.21.15.122:0/2722790894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b0102d50 msgr2=0x7f14b0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b0102d50 0x7f14b0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 -- 172.21.15.122:0/2722790894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0103710 msgr2=0x7f14b019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0103710 0x7f14b019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 -- 172.21.15.122:0/2722790894 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b019f960 con 0x7f14b0101b60 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0103710 0x7f14b019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14affff640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 0x7f14b0197d50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9c0 tx=0x7f14a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14ad7fa640 1 -- 172.21.15.122:0/2722790894 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a000cd80 con 0x7f14b0101b60 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b019fc50 con 0x7f14b0101b60 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14ad7fa640 1 -- 172.21.15.122:0/2722790894 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a00035f0 con 0x7f14b0101b60 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.354+0000 7f14ad7fa640 1 -- 172.21.15.122:0/2722790894 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0010690 con 0x7f14b0101b60 2024-09-12T02:11:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.355+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b01a01a0 con 0x7f14b0101b60 2024-09-12T02:11:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14ad7fa640 1 -- 172.21.15.122:0/2722790894 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f14a00037a0 con 0x7f14b0101b60 2024-09-12T02:11:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 Processor -- start 2024-09-12T02:11:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 -- start start 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca55637da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca5563cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca5563f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca5559b2f0 con 0x55ca556382e0 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ca55543270 con 0x55ca5563d370 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.357+0000 7f14b987f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca555972b0 con 0x55ca556379a0 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca5563cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca5563cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49144/0 (socket says 172.21.15.122:49144) 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14affff640 1 -- 172.21.15.122:0/3172864529 learned_addr learned my addr 172.21.15.122:0/3172864529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14b4f9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca55637da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca5563f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14ad7fa640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f14980a2c30 0x7f14980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f14980a2c30 0x7f14980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 -- 172.21.15.122:0/3172864529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 msgr2=0x55ca55637da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca55637da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 -- 172.21.15.122:0/3172864529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 msgr2=0x55ca5563cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca5563cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 -- 172.21.15.122:0/3172864529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca55542340 con 0x55ca5563d370 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14affff640 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca5563cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14ad7fa640 1 -- 172.21.15.122:0/2722790894 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f14a00cf590 con 0x7f14b0101b60 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14af7fe640 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca5563f830 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009c20 tx=0x7f14a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791eac0e34eaec2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f14980a2c30 0x7f14980a50f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f14a802ade0 tx=0x7f14a802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.358+0000 7f14967fc640 1 -- 172.21.15.122:0/3172864529 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004050 con 0x55ca5563d370 2024-09-12T02:11:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14967fc640 1 -- 172.21.15.122:0/3172864529 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4005500 con 0x55ca5563d370 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14967fc640 1 -- 172.21.15.122:0/3172864529 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004dd0 con 0x55ca5563d370 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 -- 172.21.15.122:0/3172864529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 msgr2=0x55ca5563f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca5563f830 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009c20 tx=0x7f14a4011810 comp rx=0 tx=0).stop 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 -- 172.21.15.122:0/3172864529 shutdown_connections 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca5563f830 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca5563cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 --2- 172.21.15.122:0/3172864529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca55637da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 -- 172.21.15.122:0/3172864529 >> 172.21.15.122:0/3172864529 conn(0x55ca55633010 msgr2=0x55ca55633410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 -- 172.21.15.122:0/3172864529 shutdown_connections 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.359+0000 7f14b987f740 1 -- 172.21.15.122:0/3172864529 wait complete. 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 -- start start 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 Processor -- start 2024-09-12T02:11:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca55641b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55642040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55649080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca55640340 con 0x55ca5563d370 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ca55524290 con 0x55ca556379a0 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca556404c0 con 0x55ca556382e0 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b987f740 10 client.0 _ll_register_callbacks cb 0x55ca55555140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b4f9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55649080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14affff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca55641b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14af7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55642040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b4f9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55649080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49158/0 (socket says 172.21.15.122:49158) 2024-09-12T02:11:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.360+0000 7f14b4f9a640 1 -- 172.21.15.122:0/820677026 learned_addr learned my addr 172.21.15.122:0/820677026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14affff640 1 -- 172.21.15.122:0/820677026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 msgr2=0x55ca55642040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14affff640 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55642040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14affff640 1 -- 172.21.15.122:0/820677026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 msgr2=0x55ca55649080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14affff640 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55649080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14affff640 1 -- 172.21.15.122:0/820677026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca552ae210 con 0x55ca556379a0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14affff640 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca55641b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14a00986f0 tx=0x7f14a009bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb49e2739244d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f148f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f148f7fe640 1 -- 172.21.15.122:0/820677026 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a00cf2c0 con 0x55ca556379a0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f148f7fe640 1 -- 172.21.15.122:0/820677026 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f14a009b200 con 0x55ca556379a0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a4005500 con 0x55ca556379a0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f148f7fe640 1 -- 172.21.15.122:0/820677026 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f14a009c4b0 con 0x55ca556379a0 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14b987f740 10 client.27743 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.361+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ca5537a620 con 0x55ca556379a0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f148f7fe640 1 -- 172.21.15.122:0/820677026 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4005500 con 0x55ca556379a0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f148f7fe640 1 -- 172.21.15.122:0/820677026 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f14a009ac70 con 0x55ca556379a0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14b987f740 10 client.27743 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14b987f740 10 client.27743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14b987f740 20 client.27743 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14b987f740 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca5564d7e0 0x55ca5564fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14b987f740 4 client.27743 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca555a8a20 con 0x55ca5564d7e0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14af7fe640 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca5564d7e0 0x55ca5564fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f14af7fe640 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca5564d7e0 0x55ca5564fbd0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd68195c05143811 server_cookie=50de472d9eae3533 in_seq=0 out_seq=0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.362+0000 7f148f7fe640 10 client.27743 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f148f7fe640 1 -- 172.21.15.122:0/820677026 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca555a8a20 con 0x55ca5564d7e0 2024-09-12T02:11:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f148f7fe640 10 client.27743 handle_command_reply: tid=0 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 client.27743 shutdown 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 msgr2=0x55ca55641b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca55641b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14a00986f0 tx=0x7f14a009bc40 comp rx=0 tx=0).stop 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 shutdown_connections 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca5564d7e0 0x55ca5564fbd0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55649080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55642040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 --2- 172.21.15.122:0/820677026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca55641b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f148f7fe640 0 client.27743 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.363+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 >> 172.21.15.122:0/820677026 conn(0x55ca55633010 msgr2=0x55ca55634820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.364+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 shutdown_connections 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.364+0000 7f14b987f740 1 -- 172.21.15.122:0/820677026 wait complete. 2024-09-12T02:11:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.364+0000 7f14b987f740 20 client.27743 trim_cache size 0 max 16384 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 Processor -- start 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 -- start start 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca556418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55641df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55633ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01ab8a0 con 0x55ca5563d370 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a9170 con 0x55ca556379a0 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b987f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a6fa0 con 0x55ca556382e0 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55633ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.383 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55633ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49168/0 (socket says 172.21.15.122:49168) 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 -- 172.21.15.122:0/4133978683 learned_addr learned my addr 172.21.15.122:0/4133978683 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14af7fe640 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55641df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14affff640 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca556418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 -- 172.21.15.122:0/4133978683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 msgr2=0x55ca55641df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55641df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 -- 172.21.15.122:0/4133978683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 msgr2=0x55ca556418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca556418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.378+0000 7f14b4f9a640 1 -- 172.21.15.122:0/4133978683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca556387c0 con 0x55ca5563d370 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55633ec0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f14a8031690 tx=0x7f14a8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a10459a5c879fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f1495ffb640 1 -- 172.21.15.122:0/4133978683 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8044070 con 0x55ca5563d370 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f1495ffb640 1 -- 172.21.15.122:0/4133978683 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a8036040 con 0x55ca5563d370 2024-09-12T02:11:34.384 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f1495ffb640 1 -- 172.21.15.122:0/4133978683 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a802fa30 con 0x55ca5563d370 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 -- 172.21.15.122:0/4133978683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 msgr2=0x55ca55633ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55633ec0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f14a8031690 tx=0x7f14a8005d80 comp rx=0 tx=0).stop 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 -- 172.21.15.122:0/4133978683 shutdown_connections 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca5563d370 0x55ca55633ec0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556382e0 0x55ca55641df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 --2- 172.21.15.122:0/4133978683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca556379a0 0x55ca556418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.379+0000 7f14b987f740 1 -- 172.21.15.122:0/4133978683 >> 172.21.15.122:0/4133978683 conn(0x55ca55633010 msgr2=0x55ca556393c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 -- 172.21.15.122:0/4133978683 shutdown_connections 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 -- 172.21.15.122:0/4133978683 wait complete. 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 -- start start 2024-09-12T02:11:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 Processor -- start 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca556b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca556b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca556ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a7630 con 0x55ca556382e0 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a8aa0 con 0x55ca5563d370 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a7d00 con 0x55ca556379a0 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14b987f740 10 client.0 _ll_register_callbacks cb 0x7f14a00d2cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca556b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.380+0000 7f14af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca556b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49172/0 (socket says 172.21.15.122:49172) 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 -- 172.21.15.122:0/2990206358 learned_addr learned my addr 172.21.15.122:0/2990206358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14affff640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca556b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14b4f9a640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca556ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 -- 172.21.15.122:0/2990206358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 msgr2=0x55ca556b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca556b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 -- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 msgr2=0x55ca556ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca556ba8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 -- 172.21.15.122:0/2990206358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca556baec0 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14af7fe640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca556b3870 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009f10 tx=0x7f14a4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d911742cb12989 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f148f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f148f7fe640 1 -- 172.21.15.122:0/2990206358 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004240 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f148f7fe640 1 -- 172.21.15.122:0/2990206358 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f14a40043e0 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a8036040 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f148f7fe640 1 -- 172.21.15.122:0/2990206358 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f14a40035f0 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14b987f740 10 client.18828 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ca556bc240 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.381+0000 7f148f7fe640 1 -- 172.21.15.122:0/2990206358 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a8036040 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f148f7fe640 1 -- 172.21.15.122:0/2990206358 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f14a4019880 con 0x55ca556382e0 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14b987f740 10 client.18828 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:34.387 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14b987f740 10 client.18828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14b987f740 20 client.18828 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14b987f740 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca556bca80 0x55ca556beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14b987f740 4 client.18828 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca555a2d70 con 0x55ca556bca80 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14affff640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca556bca80 0x55ca556beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f148f7fe640 10 client.18828 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:34.388 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.382+0000 7f14affff640 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca556bca80 0x55ca556beea0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9702ab5c04780309 server_cookie=144d81a487befd87 in_seq=0 out_seq=0 2024-09-12T02:11:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f148f7fe640 1 -- 172.21.15.122:0/2990206358 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca555a2d70 con 0x55ca556bca80 2024-09-12T02:11:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f148f7fe640 10 client.18828 handle_command_reply: tid=0 2024-09-12T02:11:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 client.18828 shutdown 2024-09-12T02:11:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 msgr2=0x55ca556b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca556b3870 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009f10 tx=0x7f14a4009810 comp rx=0 tx=0).stop 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 shutdown_connections 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55ca556bca80 0x55ca556beea0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca5563d370 0x55ca556ba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca556382e0 0x55ca556b3870 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 --2- 172.21.15.122:0/2990206358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca556379a0 0x55ca556b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f148f7fe640 0 client.18828 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 >> 172.21.15.122:0/2990206358 conn(0x55ca55633010 msgr2=0x55ca556418b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 shutdown_connections 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.419+0000 7f14b987f740 1 -- 172.21.15.122:0/2990206358 wait complete. 2024-09-12T02:11:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b987f740 20 client.18828 trim_cache size 0 max 16384 2024-09-12T02:11:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:11:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:11:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:11:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f14980a2c30 msgr2=0x7f14980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f14980a2c30 0x7f14980a50f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f14a802ade0 tx=0x7f14a802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 msgr2=0x7f14b0197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 0x7f14b0197d50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9c0 tx=0x7f14a000ee80 comp rx=0 tx=0).stop 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 shutdown_connections 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f14980a2c30 0x7f14980a50f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14b0103710 0x7f14b019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b0102d50 0x7f14b0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 --2- 172.21.15.122:0/2722790894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b0101b60 0x7f14b0197d50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 >> 172.21.15.122:0/2722790894 conn(0x7f14b00fd2e0 msgr2=0x7f14b0106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 shutdown_connections 2024-09-12T02:11:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:34.420+0000 7f14b69fd640 1 -- 172.21.15.122:0/2722790894 wait complete. 2024-09-12T02:11:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:34 smithi183 ceph-mon[35127]: pgmap v1092: 97 pgs: 97 active+clean; 538 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 4.4 MiB/s wr, 239 op/s 2024-09-12T02:11:34.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:34 smithi122 ceph-mon[30925]: pgmap v1092: 97 pgs: 97 active+clean; 538 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 4.4 MiB/s wr, 239 op/s 2024-09-12T02:11:34.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:34 smithi151 ceph-mon[35229]: pgmap v1092: 97 pgs: 97 active+clean; 538 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 4.4 MiB/s wr, 239 op/s 2024-09-12T02:11:35.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.80 MB/sec execute 562 sec latency 16832.389 ms 2024-09-12T02:11:36.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.76 MB/sec execute 563 sec latency 17832.499 ms 2024-09-12T02:11:36.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:36 smithi122 ceph-mon[30925]: pgmap v1093: 97 pgs: 97 active+clean; 515 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 4.3 MiB/s wr, 216 op/s 2024-09-12T02:11:36.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:36 smithi151 ceph-mon[35229]: pgmap v1093: 97 pgs: 97 active+clean; 515 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 4.3 MiB/s wr, 216 op/s 2024-09-12T02:11:36.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:36 smithi183 ceph-mon[35127]: pgmap v1093: 97 pgs: 97 active+clean; 515 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 4.3 MiB/s wr, 216 op/s 2024-09-12T02:11:37.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.73 MB/sec execute 564 sec latency 18832.574 ms 2024-09-12T02:11:38.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.69 MB/sec execute 565 sec latency 19832.652 ms 2024-09-12T02:11:38.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:38 smithi122 ceph-mon[30925]: pgmap v1094: 97 pgs: 97 active+clean; 515 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 3.7 MiB/s wr, 169 op/s 2024-09-12T02:11:38.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:38 smithi151 ceph-mon[35229]: pgmap v1094: 97 pgs: 97 active+clean; 515 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 3.7 MiB/s wr, 169 op/s 2024-09-12T02:11:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:38 smithi183 ceph-mon[35127]: pgmap v1094: 97 pgs: 97 active+clean; 515 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 3.7 MiB/s wr, 169 op/s 2024-09-12T02:11:39.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.66 MB/sec execute 566 sec latency 20832.739 ms 2024-09-12T02:11:40.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.62 MB/sec execute 567 sec latency 21832.793 ms 2024-09-12T02:11:40.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:40 smithi122 ceph-mon[30925]: pgmap v1095: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 4.1 MiB/s wr, 179 op/s 2024-09-12T02:11:40.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:40 smithi151 ceph-mon[35229]: pgmap v1095: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 4.1 MiB/s wr, 179 op/s 2024-09-12T02:11:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:40 smithi183 ceph-mon[35127]: pgmap v1095: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 4.1 MiB/s wr, 179 op/s 2024-09-12T02:11:41.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.59 MB/sec execute 568 sec latency 22832.856 ms 2024-09-12T02:11:42.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.55 MB/sec execute 569 sec latency 23832.965 ms 2024-09-12T02:11:42.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:42 smithi122 ceph-mon[30925]: pgmap v1096: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-09-12T02:11:42.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:42 smithi151 ceph-mon[35229]: pgmap v1096: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-09-12T02:11:42.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:42 smithi183 ceph-mon[35127]: pgmap v1096: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-09-12T02:11:43.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.52 MB/sec execute 570 sec latency 24833.066 ms 2024-09-12T02:11:44.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.49 MB/sec execute 571 sec latency 25833.165 ms 2024-09-12T02:11:44.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:44 smithi122 ceph-mon[30925]: pgmap v1097: 97 pgs: 97 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.2 MiB/s wr, 159 op/s 2024-09-12T02:11:44.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:44 smithi151 ceph-mon[35229]: pgmap v1097: 97 pgs: 97 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.2 MiB/s wr, 159 op/s 2024-09-12T02:11:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:44 smithi183 ceph-mon[35127]: pgmap v1097: 97 pgs: 97 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.2 MiB/s wr, 159 op/s 2024-09-12T02:11:45.089 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.45 MB/sec execute 572 sec latency 26833.219 ms 2024-09-12T02:11:46.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.42 MB/sec execute 573 sec latency 27833.294 ms 2024-09-12T02:11:46.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:46 smithi122 ceph-mon[30925]: pgmap v1098: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.4 MiB/s wr, 166 op/s 2024-09-12T02:11:46.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:46 smithi151 ceph-mon[35229]: pgmap v1098: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.4 MiB/s wr, 166 op/s 2024-09-12T02:11:46.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:46 smithi183 ceph-mon[35127]: pgmap v1098: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.4 MiB/s wr, 166 op/s 2024-09-12T02:11:47.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.38 MB/sec execute 574 sec latency 28833.409 ms 2024-09-12T02:11:47.323 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:11:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 -- 172.21.15.122:0/1143208305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8101c30 msgr2=0x7f0eb81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/1143208305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8101c30 0x7f0eb81020b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009ca0 tx=0x7f0ea002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 -- 172.21.15.122:0/1143208305 shutdown_connections 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/1143208305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb81025f0 0x7f0eb8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/1143208305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8101c30 0x7f0eb81020b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/1143208305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0eb8100a40 0x7f0eb8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 -- 172.21.15.122:0/1143208305 >> 172.21.15.122:0/1143208305 conn(0x7f0eb80fc1e0 msgr2=0x7f0eb80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 -- 172.21.15.122:0/1143208305 shutdown_connections 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 -- 172.21.15.122:0/1143208305 wait complete. 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.508+0000 7f0ebefe4640 1 Processor -- start 2024-09-12T02:11:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 -- start start 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 0x7f0eb806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0eb8101c30 0x7f0eb819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb81025f0 0x7f0eb819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810c6a0 con 0x7f0eb8100a40 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810c520 con 0x7f0eb8101c30 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebefe4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810c820 con 0x7f0eb81025f0 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 0x7f0eb806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 0x7f0eb806a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34664/0 (socket says 172.21.15.122:34664) 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 -- 172.21.15.122:0/170745564 learned_addr learned my addr 172.21.15.122:0/170745564 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0eb8101c30 0x7f0eb819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebd581640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb81025f0 0x7f0eb819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 -- 172.21.15.122:0/170745564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb81025f0 msgr2=0x7f0eb819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb81025f0 0x7f0eb819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 -- 172.21.15.122:0/170745564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0eb8101c30 msgr2=0x7f0eb819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0eb8101c30 0x7f0eb819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 -- 172.21.15.122:0/170745564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb819fa40 con 0x7f0eb8100a40 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.509+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 0x7f0eb806a410 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c970 tx=0x7f0eb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.510+0000 7f0eb5ffb640 1 -- 172.21.15.122:0/170745564 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004220 con 0x7f0eb8100a40 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.510+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb819fcd0 con 0x7f0eb8100a40 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.510+0000 7f0eb5ffb640 1 -- 172.21.15.122:0/170745564 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb00043c0 con 0x7f0eb8100a40 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.510+0000 7f0eb5ffb640 1 -- 172.21.15.122:0/170745564 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb001d060 con 0x7f0eb8100a40 2024-09-12T02:11:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.510+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a01c0 con 0x7f0eb8100a40 2024-09-12T02:11:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.512+0000 7f0eb5ffb640 1 -- 172.21.15.122:0/170745564 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0eb00079b0 con 0x7f0eb8100a40 2024-09-12T02:11:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0eb5ffb640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ea40a2cb0 0x7f0ea40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0eb5ffb640 1 -- 172.21.15.122:0/170745564 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0eb00ced30 con 0x7f0eb8100a40 2024-09-12T02:11:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ea40a2cb0 0x7f0ea40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 Processor -- start 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ea40a2cb0 0x7f0ea40a5170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0eb806a180 tx=0x7f0ea0000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 -- start start 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c89da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c8c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e4e2be82f0 con 0x55e4e2c8a370 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4e2b90270 con 0x55e4e2c849a0 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.513+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4e2be42b0 con 0x55e4e2c852e0 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c8c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c8c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34678/0 (socket says 172.21.15.122:34678) 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 -- 172.21.15.122:0/4199288885 learned_addr learned my addr 172.21.15.122:0/4199288885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c89da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 -- 172.21.15.122:0/4199288885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 msgr2=0x55e4e2c89da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c89da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 -- 172.21.15.122:0/4199288885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 msgr2=0x55e4e2c84da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 -- 172.21.15.122:0/4199288885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e2b8f340 con 0x55e4e2c8a370 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c89da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0ebd581640 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c8c830 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800ab80 tx=0x7f0ea800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184670e990247a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0eaeffd640 1 -- 172.21.15.122:0/4199288885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea801d070 con 0x55e4e2c8a370 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0eaeffd640 1 -- 172.21.15.122:0/4199288885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8018b20 con 0x55e4e2c8a370 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.514+0000 7f0eaeffd640 1 -- 172.21.15.122:0/4199288885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8010af0 con 0x55e4e2c8a370 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4199288885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 msgr2=0x55e4e2c8c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c8c830 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800ab80 tx=0x7f0ea800f790 comp rx=0 tx=0).stop 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4199288885 shutdown_connections 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c8c830 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c89da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4199288885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4199288885 >> 172.21.15.122:0/4199288885 conn(0x55e4e2c80010 msgr2=0x55e4e2c80410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4199288885 shutdown_connections 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.515+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4199288885 wait complete. 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 -- start start 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 Processor -- start 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c8ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c8a370 0x55e4e2c96170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e4e2c8d340 con 0x55e4e2c849a0 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4e2b71290 con 0x55e4e2c8a370 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4e2c8d4c0 con 0x55e4e2c852e0 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ec1e55740 10 client.0 _ll_register_callbacks cb 0x55e4e2ba2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ebcd80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c8ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ebcd80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c8ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34690/0 (socket says 172.21.15.122:34690) 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ebd581640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c8a370 0x55e4e2c96170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0ebcd80640 1 -- 172.21.15.122:0/4118928502 learned_addr learned my addr 172.21.15.122:0/4118928502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.516+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebd581640 1 -- 172.21.15.122:0/4118928502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 msgr2=0x55e4e2c8f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebd581640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c8f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebd581640 1 -- 172.21.15.122:0/4118928502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 msgr2=0x55e4e2c8ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebd581640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c8ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebd581640 1 -- 172.21.15.122:0/4118928502 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e28fb210 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c8ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ebd581640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c8a370 0x55e4e2c96170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ea801b040 tx=0x7f0ea8018670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35fc304d279e17ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0e97fff640 1 -- 172.21.15.122:0/4118928502 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea801d050 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea8018b20 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0e97fff640 1 -- 172.21.15.122:0/4118928502 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0ea8005040 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0e97fff640 1 -- 172.21.15.122:0/4118928502 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0ea8005240 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ec1e55740 10 client.27755 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.517+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e4e29c7620 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0e97fff640 1 -- 172.21.15.122:0/4118928502 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8018b20 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0e97fff640 1 -- 172.21.15.122:0/4118928502 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f0ea802d020 con 0x55e4e2c8a370 2024-09-12T02:11:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ec1e55740 10 client.27755 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ec1e55740 10 client.27755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ec1e55740 20 client.27755 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2c9c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ec1e55740 4 client.27755 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e2bf5a20 con 0x55e4e2c9a5d0 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2c9c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0e97fff640 10 client.27755 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.518+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2c9c9c0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd29481d6de3544 server_cookie=a518044d83279d70 in_seq=0 out_seq=0 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0e97fff640 1 -- 172.21.15.122:0/4118928502 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4e2bf5a20 con 0x55e4e2c9a5d0 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0e97fff640 10 client.27755 handle_command_reply: tid=0 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 client.27755 shutdown 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c8a370 msgr2=0x55e4e2c96170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c8a370 0x55e4e2c96170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ea801b040 tx=0x7f0ea8018670 comp rx=0 tx=0).stop 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 shutdown_connections 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2c9c9c0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c8a370 0x55e4e2c96170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0e97fff640 0 client.27755 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/4118928502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c8ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 >> 172.21.15.122:0/4118928502 conn(0x55e4e2c80010 msgr2=0x55e4e2c81820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 shutdown_connections 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.519+0000 7f0ec1e55740 1 -- 172.21.15.122:0/4118928502 wait complete. 2024-09-12T02:11:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.520+0000 7f0ec1e55740 20 client.27755 trim_cache size 0 max 16384 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.537+0000 7f0ec1e55740 1 Processor -- start 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 -- start start 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c852e0 0x55e4e2c96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c8a370 0x55e4e2c81400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81ab900 con 0x55e4e2c849a0 2024-09-12T02:11:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a91d0 con 0x55e4e2c852e0 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a7000 con 0x55e4e2c8a370 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34706/0 (socket says 172.21.15.122:34706) 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 -- 172.21.15.122:0/3307891518 learned_addr learned my addr 172.21.15.122:0/3307891518 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 -- 172.21.15.122:0/3307891518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c8a370 msgr2=0x55e4e2c81400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c8a370 0x55e4e2c81400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 -- 172.21.15.122:0/3307891518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c852e0 msgr2=0x55e4e2c96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c852e0 0x55e4e2c96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.538+0000 7f0ebcd80640 1 -- 172.21.15.122:0/3307891518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e2c8db90 con 0x55e4e2c849a0 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84d80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0097f80 tx=0x7f0eb00cc6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335ee6a7e9775c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0eae7fc640 1 -- 172.21.15.122:0/3307891518 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00b3070 con 0x55e4e2c849a0 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0eae7fc640 1 -- 172.21.15.122:0/3307891518 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0099900 con 0x55e4e2c849a0 2024-09-12T02:11:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0eae7fc640 1 -- 172.21.15.122:0/3307891518 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0084b90 con 0x55e4e2c849a0 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 -- 172.21.15.122:0/3307891518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 msgr2=0x55e4e2c84d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84d80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0097f80 tx=0x7f0eb00cc6c0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 -- 172.21.15.122:0/3307891518 shutdown_connections 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c8a370 0x55e4e2c81400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c852e0 0x55e4e2c96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/3307891518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c849a0 0x55e4e2c84d80 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.539+0000 7f0ec1e55740 1 -- 172.21.15.122:0/3307891518 >> 172.21.15.122:0/3307891518 conn(0x55e4e2c80010 msgr2=0x55e4e2c7f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 -- 172.21.15.122:0/3307891518 shutdown_connections 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 -- 172.21.15.122:0/3307891518 wait complete. 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 -- start start 2024-09-12T02:11:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 Processor -- start 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c93ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c93ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a7690 con 0x55e4e2c8a370 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a8b00 con 0x55e4e2c849a0 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a7d60 con 0x55e4e2c852e0 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.540+0000 7f0ec1e55740 10 client.0 _ll_register_callbacks cb 0x7f0ea801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c95000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34718/0 (socket says 172.21.15.122:34718) 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 -- 172.21.15.122:0/2003492752 learned_addr learned my addr 172.21.15.122:0/2003492752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c93ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c93ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 -- 172.21.15.122:0/2003492752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 msgr2=0x55e4e2c93ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c93ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 -- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 msgr2=0x55e4e2c93ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c93ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 -- 172.21.15.122:0/2003492752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e2c95610 con 0x55e4e2c8a370 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0eb7fff640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c93ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ebd581640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c95000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8004800 tx=0x7f0ea800a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9409614897809bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0e97fff640 1 -- 172.21.15.122:0/2003492752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea801d040 con 0x55e4e2c8a370 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0e97fff640 1 -- 172.21.15.122:0/2003492752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0ea8004d00 con 0x55e4e2c8a370 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.541+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0099900 con 0x55e4e2c8a370 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0e97fff640 1 -- 172.21.15.122:0/2003492752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0ea800a9d0 con 0x55e4e2c8a370 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 10 client.18846 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e4e2d05300 con 0x55e4e2c8a370 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0e97fff640 1 -- 172.21.15.122:0/2003492752 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0099900 con 0x55e4e2c8a370 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0e97fff640 1 -- 172.21.15.122:0/2003492752 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f0ea8024720 con 0x55e4e2c8a370 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 10 client.18846 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 10 client.18846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 20 client.18846 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2d07d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 4 client.18846 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4e2befd70 con 0x55e4e2c9a5d0 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.542+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2d07d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.543+0000 7f0ebcd80640 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2d07d40 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c5a0bae46191f1 server_cookie=f743d5922fe9839e in_seq=0 out_seq=0 2024-09-12T02:11:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.543+0000 7f0e97fff640 10 client.18846 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:47.597 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.614+0000 7f0e97fff640 1 -- 172.21.15.122:0/2003492752 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4e2befd70 con 0x55e4e2c9a5d0 2024-09-12T02:11:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.614+0000 7f0e97fff640 10 client.18846 handle_command_reply: tid=0 2024-09-12T02:11:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.614+0000 7f0ec1e55740 1 client.18846 shutdown 2024-09-12T02:11:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 msgr2=0x55e4e2c95000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c95000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8004800 tx=0x7f0ea800a7b0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 shutdown_connections 2024-09-12T02:11:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e4e2c9a5d0 0x55e4e2d07d40 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e4e2c8a370 0x55e4e2c95000 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4e2c852e0 0x55e4e2c93ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 --2- 172.21.15.122:0/2003492752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4e2c849a0 0x55e4e2c93ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0e97fff640 0 client.18846 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 >> 172.21.15.122:0/2003492752 conn(0x55e4e2c80010 msgr2=0x55e4e2c7f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 shutdown_connections 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.615+0000 7f0ec1e55740 1 -- 172.21.15.122:0/2003492752 wait complete. 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ec1e55740 20 client.18846 trim_cache size 0 max 16384 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:11:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:11:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ea40a2cb0 msgr2=0x7f0ea40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ea40a2cb0 0x7f0ea40a5170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0eb806a180 tx=0x7f0ea0000f40 comp rx=0 tx=0).stop 2024-09-12T02:11:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 msgr2=0x7f0eb806a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 0x7f0eb806a410 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c970 tx=0x7f0eb000ce30 comp rx=0 tx=0).stop 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 shutdown_connections 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0ea40a2cb0 0x7f0ea40a5170 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb81025f0 0x7f0eb819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0eb8101c30 0x7f0eb819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 --2- 172.21.15.122:0/170745564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0eb8100a40 0x7f0eb806a410 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 >> 172.21.15.122:0/170745564 conn(0x7f0eb80fc1e0 msgr2=0x7f0eb8105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 shutdown_connections 2024-09-12T02:11:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:11:47.616+0000 7f0ebefe4640 1 -- 172.21.15.122:0/170745564 wait complete. 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 -- 172.21.15.122:0/1172439179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c101420 msgr2=0x7fce6c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 --2- 172.21.15.122:0/1172439179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c101420 0x7fce6c10d950 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fce64009cd0 tx=0x7fce6402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 -- 172.21.15.122:0/1172439179 shutdown_connections 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 --2- 172.21.15.122:0/1172439179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c101420 0x7fce6c10d950 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 --2- 172.21.15.122:0/1172439179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c100a80 0x7fce6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 --2- 172.21.15.122:0/1172439179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c106a80 0x7fce6c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 -- 172.21.15.122:0/1172439179 >> 172.21.15.122:0/1172439179 conn(0x7fce6c0fc7a0 msgr2=0x7fce6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.057+0000 7fce7289f640 1 -- 172.21.15.122:0/1172439179 shutdown_connections 2024-09-12T02:11:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.058+0000 7fce7289f640 1 -- 172.21.15.122:0/1172439179 wait complete. 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.058+0000 7fce7289f640 1 Processor -- start 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.058+0000 7fce7289f640 1 -- start start 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 0x7fce6c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c101420 0x7fce6c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c106a80 0x7fce6c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c110e80 con 0x7fce6c100a80 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c110d00 con 0x7fce6c106a80 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c111000 con 0x7fce6c101420 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 0x7fce6c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 0x7fce6c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34728/0 (socket says 172.21.15.122:34728) 2024-09-12T02:11:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 -- 172.21.15.122:0/576159994 learned_addr learned my addr 172.21.15.122:0/576159994 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6b7fe640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c101420 0x7fce6c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 -- 172.21.15.122:0/576159994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c101420 msgr2=0x7fce6c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c101420 0x7fce6c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce70e3c640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c106a80 0x7fce6c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 -- 172.21.15.122:0/576159994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c106a80 msgr2=0x7fce6c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c106a80 0x7fce6c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 -- 172.21.15.122:0/576159994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c19f650 con 0x7fce6c100a80 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6b7fe640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c101420 0x7fce6c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce6bfff640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 0x7fce6c19a6a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c890 tx=0x7fce5c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004220 con 0x7fce6c100a80 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c0043c0 con 0x7fce6c100a80 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c19f940 con 0x7fce6c100a80 2024-09-12T02:11:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c0049e0 con 0x7fce6c100a80 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.059+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c1a7220 con 0x7fce6c100a80 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.060+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6c100ee0 con 0x7fce6c100a80 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.062+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fce5c0079b0 con 0x7fce6c100a80 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.062+0000 7fce697fa640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fce540a2b80 0x7fce540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.062+0000 7fce6b7fe640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fce540a2b80 0x7fce540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.062+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fce5c0cf130 con 0x7fce6c100a80 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.063+0000 7fce6b7fe640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fce540a2b80 0x7fce540a5040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce60005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.065+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce5c014030 con 0x7fce6c100a80 2024-09-12T02:11:48.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.35 MB/sec execute 575 sec latency 29833.480 ms 2024-09-12T02:11:48.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.266+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce6c19ba00 con 0x7fce6c100a80 2024-09-12T02:11:48.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.267+0000 7fce697fa640 1 -- 172.21.15.122:0/576159994 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11929 (secure 0 0 0) 0x7fce5c09c090 con 0x7fce6c100a80 2024-09-12T02:11:48.267 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:11:48.267 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"btime":"2024-09-12T02:06:54:525648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:06:54.525634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:11:48.268 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 45 2024-09-12T02:11:48.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fce540a2b80 msgr2=0x7fce540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fce540a2b80 0x7fce540a5040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce60005ea0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 msgr2=0x7fce6c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 0x7fce6c19a6a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c890 tx=0x7fce5c00cd50 comp rx=0 tx=0).stop 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 shutdown_connections 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fce540a2b80 0x7fce540a5040 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c106a80 0x7fce6c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c101420 0x7fce6c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 --2- 172.21.15.122:0/576159994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce6c100a80 0x7fce6c19a6a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 >> 172.21.15.122:0/576159994 conn(0x7fce6c0fc7a0 msgr2=0x7fce6c10c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 shutdown_connections 2024-09-12T02:11:48.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:48.269+0000 7fce7289f640 1 -- 172.21.15.122:0/576159994 wait complete. 2024-09-12T02:11:48.549 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:48 smithi122 ceph-mon[30925]: pgmap v1099: 97 pgs: 97 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.6 MiB/s wr, 163 op/s 2024-09-12T02:11:48.549 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:48 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/576159994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:48.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:48 smithi151 ceph-mon[35229]: pgmap v1099: 97 pgs: 97 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.6 MiB/s wr, 163 op/s 2024-09-12T02:11:48.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:48 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/576159994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:48.920 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id c507febe --release --await' 2024-09-12T02:11:48.920 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id c507febe --release --await 2024-09-12T02:11:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:48 smithi183 ceph-mon[35127]: pgmap v1099: 97 pgs: 97 active+clean; 514 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.6 MiB/s wr, 163 op/s 2024-09-12T02:11:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:48 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/576159994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:11:49.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259340 19.32 MB/sec execute 576 sec latency 30833.533 ms 2024-09-12T02:11:49.212 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:49.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 -- 172.21.15.122:0/822789414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 msgr2=0x7f997410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 --2- 172.21.15.122:0/822789414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997410ea90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f996c009ca0 tx=0x7f996c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 -- 172.21.15.122:0/822789414 shutdown_connections 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 --2- 172.21.15.122:0/822789414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997410ea90 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 --2- 172.21.15.122:0/822789414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974102790 0x7f99741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 --2- 172.21.15.122:0/822789414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9974101de0 0x7f99741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 -- 172.21.15.122:0/822789414 >> 172.21.15.122:0/822789414 conn(0x7f99740fd8c0 msgr2=0x7f99740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 -- 172.21.15.122:0/822789414 shutdown_connections 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.733+0000 7f997a173640 1 -- 172.21.15.122:0/822789414 wait complete. 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 Processor -- start 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 -- start start 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974101de0 0x7f99741983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9974102790 0x7f99741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9974111f90 con 0x7f9974106f10 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9974111e10 con 0x7f9974101de0 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.734+0000 7f997a173640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9974112110 con 0x7f9974102790 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34738/0 (socket says 172.21.15.122:34738) 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 -- 172.21.15.122:0/3716221849 learned_addr learned my addr 172.21.15.122:0/3716221849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f99737fe640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974101de0 0x7f99741983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9972ffd640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9974102790 0x7f99741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 -- 172.21.15.122:0/3716221849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9974102790 msgr2=0x7f99741988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9974102790 0x7f99741988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 -- 172.21.15.122:0/3716221849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974101de0 msgr2=0x7f99741983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974101de0 0x7f99741983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 -- 172.21.15.122:0/3716221849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997419d360 con 0x7f9974106f10 2024-09-12T02:11:49.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9973fff640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997419cc80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f996c031a60 tx=0x7f996c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9970ff9640 1 -- 172.21.15.122:0/3716221849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c003b70 con 0x7f9974106f10 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997419d5f0 con 0x7f9974106f10 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9970ff9640 1 -- 172.21.15.122:0/3716221849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f996c003d10 con 0x7f9974106f10 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f9970ff9640 1 -- 172.21.15.122:0/3716221849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c004770 con 0x7f9974106f10 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.735+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741a4f30 con 0x7f9974106f10 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.737+0000 7f9970ff9640 1 -- 172.21.15.122:0/3716221849 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f996c02f790 con 0x7f9974106f10 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 Processor -- start 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 -- start start 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e419440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e423d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e424240 0x55df3e426630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55df3e37e620 con 0x55df3e419980 2024-09-12T02:11:49.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df3e325e50 con 0x55df3e424240 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f997c54c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df3e37a5e0 con 0x55df3e419060 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f99737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e423d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f99737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e423d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34742/0 (socket says 172.21.15.122:34742) 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f99737fe640 1 -- 172.21.15.122:0/1256898160 learned_addr learned my addr 172.21.15.122:0/1256898160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f9970ff9640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f995c0a2b80 0x7f995c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e424240 0x55df3e426630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f99737fe640 1 -- 172.21.15.122:0/1256898160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 msgr2=0x55df3e419440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f9973fff640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f995c0a2b80 0x7f995c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e419440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.738+0000 7f99737fe640 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e419440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f99737fe640 1 -- 172.21.15.122:0/1256898160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e424240 msgr2=0x55df3e426630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f99737fe640 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e424240 0x55df3e426630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f99737fe640 1 -- 172.21.15.122:0/1256898160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3e316d40 con 0x55df3e419980 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e419440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f9970ff9640 1 -- 172.21.15.122:0/3716221849 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f996c0f02f0 con 0x7f9974106f10 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f99737fe640 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e423d00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f996400ad00 tx=0x7f996400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f71112372758742 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f9959ffb640 1 -- 172.21.15.122:0/1256898160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996401d070 con 0x55df3e419980 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f9973fff640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f995c0a2b80 0x7f995c0a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f996c0b8f70 tx=0x7f996c0878c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f9959ffb640 1 -- 172.21.15.122:0/1256898160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9964018b20 con 0x55df3e419980 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.739+0000 7f9959ffb640 1 -- 172.21.15.122:0/1256898160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964010c20 con 0x55df3e419980 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 -- 172.21.15.122:0/1256898160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 msgr2=0x55df3e423d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e423d00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f996400ad00 tx=0x7f996400f790 comp rx=0 tx=0).stop 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 -- 172.21.15.122:0/1256898160 shutdown_connections 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e424240 0x55df3e426630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e423d00 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 --2- 172.21.15.122:0/1256898160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e419440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 -- 172.21.15.122:0/1256898160 >> 172.21.15.122:0/1256898160 conn(0x55df3e4155f0 msgr2=0x55df3e4159d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 -- 172.21.15.122:0/1256898160 shutdown_connections 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 -- 172.21.15.122:0/1256898160 wait complete. 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 -- start start 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.740+0000 7f997c54c740 1 Processor -- start 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e4211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419980 0x55df3e421710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e421c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55df3e426e10 con 0x55df3e424240 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df3e306f10 con 0x55df3e419980 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df3e426f90 con 0x55df3e419060 2024-09-12T02:11:49.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 10 client.0 _ll_register_callbacks cb 0x55df3e32adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e421c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e421c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34750/0 (socket says 172.21.15.122:34750) 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 -- 172.21.15.122:0/3429727251 learned_addr learned my addr 172.21.15.122:0/3429727251 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e4211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f99737fe640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419980 0x55df3e421710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 -- 172.21.15.122:0/3429727251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 msgr2=0x55df3e4211d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e4211d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 -- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419980 msgr2=0x55df3e421710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419980 0x55df3e421710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 -- 172.21.15.122:0/3429727251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3e08d580 con 0x55df3e424240 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9972ffd640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e421c50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9968018f60 tx=0x7f99680047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3daeb7703c8bb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9952ffd640 1 -- 172.21.15.122:0/3429727251 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968011e20 con 0x55df3e424240 2024-09-12T02:11:49.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f9952ffd640 1 -- 172.21.15.122:0/3429727251 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9968015450 con 0x55df3e424240 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.741+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9964018b20 con 0x55df3e424240 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f9952ffd640 1 -- 172.21.15.122:0/3429727251 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9968015650 con 0x55df3e424240 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 10 client.18864 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55df3e157960 con 0x55df3e424240 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f9952ffd640 1 -- 172.21.15.122:0/3429727251 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9964018b20 con 0x55df3e424240 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f9952ffd640 1 -- 172.21.15.122:0/3429727251 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f9968023920 con 0x55df3e424240 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 10 client.18864 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 10 client.18864 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 20 client.18864 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e433e20 0x55df3e436230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 4 client.18864 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3e38bb90 con 0x55df3e433e20 2024-09-12T02:11:49.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.742+0000 7f99737fe640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e433e20 0x55df3e436230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.743+0000 7f9952ffd640 10 client.18864 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.743+0000 7f99737fe640 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e433e20 0x55df3e436230 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0228463d3e86f2 server_cookie=567fdeea7009ce9c in_seq=0 out_seq=0 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f9952ffd640 1 -- 172.21.15.122:0/3429727251 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df3e38bb90 con 0x55df3e433e20 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f9952ffd640 10 client.18864 handle_command_reply: tid=0 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 client.18864 shutdown 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 msgr2=0x55df3e421c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e421c50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9968018f60 tx=0x7f99680047d0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 shutdown_connections 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e433e20 0x55df3e436230 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e421c50 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419980 0x55df3e421710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f9952ffd640 0 client.18864 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:49.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 --2- 172.21.15.122:0/3429727251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419060 0x55df3e4211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 >> 172.21.15.122:0/3429727251 conn(0x55df3e4155f0 msgr2=0x55df3e41b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 shutdown_connections 2024-09-12T02:11:49.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 1 -- 172.21.15.122:0/3429727251 wait complete. 2024-09-12T02:11:49.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.744+0000 7f997c54c740 20 client.18864 trim_cache size 0 max 16384 2024-09-12T02:11:49.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 Processor -- start 2024-09-12T02:11:49.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 -- start start 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e414a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e41bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e424240 0x55df3e43bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99741afe70 con 0x55df3e419980 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99741ad740 con 0x55df3e419060 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f997c54c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99741ab570 con 0x55df3e424240 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e41bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e41bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34754/0 (socket says 172.21.15.122:34754) 2024-09-12T02:11:49.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 -- 172.21.15.122:0/475199589 learned_addr learned my addr 172.21.15.122:0/475199589 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:49.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f9972ffd640 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e424240 0x55df3e43bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f9972ffd640 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e414a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 -- 172.21.15.122:0/475199589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e424240 msgr2=0x55df3e43bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e424240 0x55df3e43bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 -- 172.21.15.122:0/475199589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 msgr2=0x55df3e414a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e414a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f99737fe640 1 -- 172.21.15.122:0/475199589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3e420180 con 0x55df3e419980 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.761+0000 7f9972ffd640 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e414a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f99737fe640 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e41bac0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9964010770 tx=0x7f9964010700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1208327768634dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f99597fa640 1 -- 172.21.15.122:0/475199589 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964031070 con 0x55df3e419980 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f99597fa640 1 -- 172.21.15.122:0/475199589 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f996400a730 con 0x55df3e419980 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f99597fa640 1 -- 172.21.15.122:0/475199589 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996401d040 con 0x55df3e419980 2024-09-12T02:11:49.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 -- 172.21.15.122:0/475199589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 msgr2=0x55df3e41bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e41bac0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9964010770 tx=0x7f9964010700 comp rx=0 tx=0).stop 2024-09-12T02:11:49.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 -- 172.21.15.122:0/475199589 shutdown_connections 2024-09-12T02:11:49.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e424240 0x55df3e43bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e419980 0x55df3e41bac0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 --2- 172.21.15.122:0/475199589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e414a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.762+0000 7f997c54c740 1 -- 172.21.15.122:0/475199589 >> 172.21.15.122:0/475199589 conn(0x55df3e4155f0 msgr2=0x55df3e422a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 -- 172.21.15.122:0/475199589 shutdown_connections 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 -- 172.21.15.122:0/475199589 wait complete. 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 -- start start 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 Processor -- start 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e42c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419980 0x55df3e4291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e4296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99741abc00 con 0x55df3e424240 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99741ad070 con 0x55df3e419060 2024-09-12T02:11:49.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99741ac2d0 con 0x55df3e419980 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f997c54c740 10 client.0 _ll_register_callbacks cb 0x7f9968018ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f9972ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e4296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.763+0000 7f9972ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e4296f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34762/0 (socket says 172.21.15.122:34762) 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9972ffd640 1 -- 172.21.15.122:0/1542239009 learned_addr learned my addr 172.21.15.122:0/1542239009 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419980 0x55df3e4291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e42c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 -- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 msgr2=0x55df3e42c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e42c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 -- 172.21.15.122:0/1542239009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 msgr2=0x55df3e4296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e4296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 -- 172.21.15.122:0/1542239009 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3e429de0 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e4296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f99737fe640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419980 0x55df3e4291b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55df3e4222c0 tx=0x7f9964010820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6b66b040986160 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9952ffd640 1 -- 172.21.15.122:0/1542239009 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964031070 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f996400a730 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9952ffd640 1 -- 172.21.15.122:0/1542239009 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9964010ac0 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f9952ffd640 1 -- 172.21.15.122:0/1542239009 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f996401d050 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f997c54c740 10 client.27781 fetch_fsmap learned FSMap version 45 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.764+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55df3e49f950 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f9952ffd640 1 -- 172.21.15.122:0/1542239009 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f996400a730 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f9952ffd640 1 -- 172.21.15.122:0/1542239009 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f99640234d0 con 0x55df3e419980 2024-09-12T02:11:49.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f997c54c740 10 client.27781 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f997c54c740 10 client.27781 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f997c54c740 20 client.27781 populate_metadata read hostname 'smithi122' 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f997c54c740 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e49fda0 0x55df3e4a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f997c54c740 4 client.27781 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c507febe", "release": true, "await": true}] 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c507febe", "release": true, "await": true}) -- 0x55df3e3860a0 con 0x55df3e49fda0 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.765+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e49fda0 0x55df3e4a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.766+0000 7f9972ffd640 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e49fda0 0x55df3e4a21c0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5575006993097940 server_cookie=48966346f0475894 in_seq=0 out_seq=0 2024-09-12T02:11:49.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.766+0000 7f9952ffd640 10 client.27781 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:49.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f9952ffd640 1 -- 172.21.15.122:0/1542239009 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55df3e3860a0 con 0x55df3e49fda0 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f9952ffd640 10 client.27781 handle_command_reply: tid=0 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 client.27781 shutdown 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419980 msgr2=0x55df3e4291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419980 0x55df3e4291b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55df3e4222c0 tx=0x7f9964010820 comp rx=0 tx=0).stop 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 shutdown_connections 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55df3e49fda0 0x55df3e4a21c0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df3e424240 0x55df3e4296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3e419980 0x55df3e4291b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f9952ffd640 0 client.27781 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 --2- 172.21.15.122:0/1542239009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df3e419060 0x55df3e42c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 >> 172.21.15.122:0/1542239009 conn(0x55df3e4155f0 msgr2=0x55df3e421190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 shutdown_connections 2024-09-12T02:11:49.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 1 -- 172.21.15.122:0/1542239009 wait complete. 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.825+0000 7f997c54c740 20 client.27781 trim_cache size 0 max 16384 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 45, 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 88, 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": { 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "c507febe": { 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "version": 88, 2024-09-12T02:11:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age_ref": 0.0, 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "timeout": 90.0, 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "expiration": 153.7, 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "members": { 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "file:/": { 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "excluded": false, 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "state": { 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RELEASED", 2024-09-12T02:11:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0 2024-09-12T02:11:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:11:49.829 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f995c0a2b80 msgr2=0x7f995c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f995c0a2b80 0x7f995c0a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f996c0b8f70 tx=0x7f996c0878c0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 msgr2=0x7f997419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997419cc80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f996c031a60 tx=0x7f996c031a90 comp rx=0 tx=0).stop 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 shutdown_connections 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f995c0a2b80 0x7f995c0a5040 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9974106f10 0x7f997419cc80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9974102790 0x7f99741988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 --2- 172.21.15.122:0/3716221849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974101de0 0x7f99741983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:49.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 >> 172.21.15.122:0/3716221849 conn(0x7f99740fd8c0 msgr2=0x7f99740ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:49.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 shutdown_connections 2024-09-12T02:11:49.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:11:49.826+0000 7f997a173640 1 -- 172.21.15.122:0/3716221849 wait complete. 2024-09-12T02:11:50.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 259503 19.30 MB/sec execute 577 sec latency 31537.587 ms 2024-09-12T02:11:50.713 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:50 smithi122 ceph-mon[30925]: pgmap v1100: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 5.2 MiB/s wr, 179 op/s 2024-09-12T02:11:50.745 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c507febe', total seconds elapsed: 37.2 2024-09-12T02:11:50.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:50 smithi151 ceph-mon[35229]: pgmap v1100: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 5.2 MiB/s wr, 179 op/s 2024-09-12T02:11:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:50 smithi183 ceph-mon[35127]: pgmap v1100: 97 pgs: 97 active+clean; 513 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 5.2 MiB/s wr, 179 op/s 2024-09-12T02:11:51.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 260004 19.31 MB/sec execute 578 sec latency 168.486 ms 2024-09-12T02:11:52.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 260591 19.34 MB/sec execute 579 sec latency 57.116 ms 2024-09-12T02:11:52.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:52 smithi122 ceph-mon[30925]: pgmap v1101: 97 pgs: 97 active+clean; 507 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 4.7 MiB/s wr, 168 op/s 2024-09-12T02:11:52.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:52 smithi151 ceph-mon[35229]: pgmap v1101: 97 pgs: 97 active+clean; 507 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 4.7 MiB/s wr, 168 op/s 2024-09-12T02:11:52.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:52 smithi183 ceph-mon[35127]: pgmap v1101: 97 pgs: 97 active+clean; 507 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 4.7 MiB/s wr, 168 op/s 2024-09-12T02:11:53.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 261251 19.38 MB/sec execute 580 sec latency 63.926 ms 2024-09-12T02:11:54.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 261685 19.39 MB/sec execute 581 sec latency 77.042 ms 2024-09-12T02:11:54.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:54 smithi122 ceph-mon[30925]: pgmap v1102: 97 pgs: 97 active+clean; 505 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 202 op/s 2024-09-12T02:11:54.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:54 smithi151 ceph-mon[35229]: pgmap v1102: 97 pgs: 97 active+clean; 505 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 202 op/s 2024-09-12T02:11:54.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:54 smithi183 ceph-mon[35127]: pgmap v1102: 97 pgs: 97 active+clean; 505 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 202 op/s 2024-09-12T02:11:55.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 262207 19.39 MB/sec execute 582 sec latency 63.463 ms 2024-09-12T02:11:56.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 262709 19.41 MB/sec execute 583 sec latency 66.742 ms 2024-09-12T02:11:56.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:56 smithi122 ceph-mon[30925]: pgmap v1103: 97 pgs: 97 active+clean; 510 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 273 op/s 2024-09-12T02:11:56.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:56 smithi151 ceph-mon[35229]: pgmap v1103: 97 pgs: 97 active+clean; 510 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 273 op/s 2024-09-12T02:11:56.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:56 smithi183 ceph-mon[35127]: pgmap v1103: 97 pgs: 97 active+clean; 510 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 273 op/s 2024-09-12T02:11:57.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 263267 19.43 MB/sec execute 584 sec latency 67.400 ms 2024-09-12T02:11:58.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 263957 19.44 MB/sec execute 585 sec latency 53.974 ms 2024-09-12T02:11:58.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:11:58 smithi122 ceph-mon[30925]: pgmap v1104: 97 pgs: 97 active+clean; 509 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-09-12T02:11:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:11:58 smithi151 ceph-mon[35229]: pgmap v1104: 97 pgs: 97 active+clean; 509 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-09-12T02:11:58.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:11:58 smithi183 ceph-mon[35127]: pgmap v1104: 97 pgs: 97 active+clean; 509 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.4 MiB/s wr, 270 op/s 2024-09-12T02:11:59.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 264536 19.47 MB/sec execute 586 sec latency 61.999 ms 2024-09-12T02:12:00.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 265138 19.49 MB/sec execute 587 sec latency 80.611 ms 2024-09-12T02:12:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 -- 172.21.15.122:0/3499443041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 msgr2=0x7f2448107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 --2- 172.21.15.122:0/3499443041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448107320 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2438009900 tx=0x7f243802a410 comp rx=0 tx=0).stop 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 -- 172.21.15.122:0/3499443041 shutdown_connections 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 --2- 172.21.15.122:0/3499443041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 0x7f2448109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 --2- 172.21.15.122:0/3499443041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448107320 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 --2- 172.21.15.122:0/3499443041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f24481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 -- 172.21.15.122:0/3499443041 >> 172.21.15.122:0/3499443041 conn(0x7f24480fc1e0 msgr2=0x7f24480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 -- 172.21.15.122:0/3499443041 shutdown_connections 2024-09-12T02:12:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.716+0000 7f244fb78640 1 -- 172.21.15.122:0/3499443041 wait complete. 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 Processor -- start 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 -- start start 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f2448197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 0x7f244819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f244810d830 con 0x7f2448104f30 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f244810d6b0 con 0x7f2448107860 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244fb78640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f244810d9b0 con 0x7f2448102600 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.717+0000 7f244d113640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f2448197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f2448197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:50982/0 (socket says 172.21.15.122:50982) 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d113640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57714/0 (socket says 172.21.15.122:57714) 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244e115640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 0x7f244819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 -- 172.21.15.122:0/2499439123 learned_addr learned my addr 172.21.15.122:0/2499439123 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 -- 172.21.15.122:0/2499439123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 msgr2=0x7f244819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 0x7f244819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 -- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 msgr2=0x7f2448198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 -- 172.21.15.122:0/2499439123 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244819fa30 con 0x7f2448102600 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244e115640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 0x7f244819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244d914640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f2448197e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f244400ea60 tx=0x7f244400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f243effd640 1 -- 172.21.15.122:0/2499439123 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244400cde0 con 0x7f2448102600 2024-09-12T02:12:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244819fd20 con 0x7f2448102600 2024-09-12T02:12:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f243effd640 1 -- 172.21.15.122:0/2499439123 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24440035f0 con 0x7f2448102600 2024-09-12T02:12:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f243effd640 1 -- 172.21.15.122:0/2499439123 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444010670 con 0x7f2448102600 2024-09-12T02:12:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.718+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24481a0190 con 0x7f2448102600 2024-09-12T02:12:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.721+0000 7f243effd640 1 -- 172.21.15.122:0/2499439123 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f24440037a0 con 0x7f2448102600 2024-09-12T02:12:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.721+0000 7f24529f1740 1 Processor -- start 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 -- start start 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43aa6f0 0x55bef43af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 0x55bef43b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bef43092f0 con 0x55bef43aa6f0 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bef42b1270 con 0x55bef43af780 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f24529f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bef43052b0 con 0x55bef43a9db0 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f243effd640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24340a2c30 0x7f24340a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 0x55bef43b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 0x55bef43b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40470/0 (socket says 172.21.15.122:40470) 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 -- 172.21.15.122:0/1053949283 learned_addr learned my addr 172.21.15.122:0/1053949283 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d914640 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43aa6f0 0x55bef43af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244e115640 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244e115640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24340a2c30 0x7f24340a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 -- 172.21.15.122:0/1053949283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 msgr2=0x55bef43aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 -- 172.21.15.122:0/1053949283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43aa6f0 msgr2=0x55bef43af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43aa6f0 0x55bef43af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f243effd640 1 -- 172.21.15.122:0/2499439123 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f24440cf2f0 con 0x7f2448102600 2024-09-12T02:12:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.722+0000 7f244d113640 1 -- 172.21.15.122:0/1053949283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef42b0340 con 0x55bef43af780 2024-09-12T02:12:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.723+0000 7f244d113640 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 0x55bef43b1c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2438006010 tx=0x7f2438004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a31f4264261789 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.723+0000 7f244e115640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24340a2c30 0x7f24340a50f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f24400049d0 tx=0x7f2440004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.723+0000 7f2433fff640 1 -- 172.21.15.122:0/1053949283 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438035040 con 0x55bef43af780 2024-09-12T02:12:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.723+0000 7f2433fff640 1 -- 172.21.15.122:0/1053949283 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438004430 con 0x55bef43af780 2024-09-12T02:12:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.723+0000 7f2433fff640 1 -- 172.21.15.122:0/1053949283 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380316b0 con 0x55bef43af780 2024-09-12T02:12:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 -- 172.21.15.122:0/1053949283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 msgr2=0x55bef43b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 0x55bef43b1c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2438006010 tx=0x7f2438004040 comp rx=0 tx=0).stop 2024-09-12T02:12:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 -- 172.21.15.122:0/1053949283 shutdown_connections 2024-09-12T02:12:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43af780 0x55bef43b1c40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43aa6f0 0x55bef43af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 --2- 172.21.15.122:0/1053949283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 -- 172.21.15.122:0/1053949283 >> 172.21.15.122:0/1053949283 conn(0x55bef43a1010 msgr2=0x55bef43a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 -- 172.21.15.122:0/1053949283 shutdown_connections 2024-09-12T02:12:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.724+0000 7f24529f1740 1 -- 172.21.15.122:0/1053949283 wait complete. 2024-09-12T02:12:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 -- start start 2024-09-12T02:12:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 Processor -- start 2024-09-12T02:12:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43bb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bef43b26f0 con 0x55bef43af780 2024-09-12T02:12:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bef4292290 con 0x55bef43aa6f0 2024-09-12T02:12:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bef43b2870 con 0x55bef43a9db0 2024-09-12T02:12:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f244d113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f244d113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40472/0 (socket says 172.21.15.122:40472) 2024-09-12T02:12:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f244d113640 1 -- 172.21.15.122:0/3727108353 learned_addr learned my addr 172.21.15.122:0/3727108353 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f244d914640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f244e115640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.725+0000 7f24529f1740 10 client.0 _ll_register_callbacks cb 0x55bef42c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d113640 1 -- 172.21.15.122:0/3727108353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 msgr2=0x55bef43b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d113640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d113640 1 -- 172.21.15.122:0/3727108353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 msgr2=0x55bef43bb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d113640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43bb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d113640 1 -- 172.21.15.122:0/3727108353 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef401c210 con 0x55bef43aa6f0 2024-09-12T02:12:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d914640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f244d113640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b4450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f243802a930 tx=0x7f24380049a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f053d55e5ee2bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f24317fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-12T02:12:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f24317fa640 1 -- 172.21.15.122:0/3727108353 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438052060 con 0x55bef43aa6f0 2024-09-12T02:12:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f24317fa640 1 -- 172.21.15.122:0/3727108353 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2438004230 con 0x55bef43aa6f0 2024-09-12T02:12:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.726+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2438004430 con 0x55bef43aa6f0 2024-09-12T02:12:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24317fa640 1 -- 172.21.15.122:0/3727108353 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2438035070 con 0x55bef43aa6f0 2024-09-12T02:12:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 10 client.27782 fetch_fsmap learned FSMap version 45 2024-09-12T02:12:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bef40e8620 con 0x55bef43aa6f0 2024-09-12T02:12:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24317fa640 1 -- 172.21.15.122:0/3727108353 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438004430 con 0x55bef43aa6f0 2024-09-12T02:12:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24317fa640 1 -- 172.21.15.122:0/3727108353 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f2438057070 con 0x55bef43aa6f0 2024-09-12T02:12:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 10 client.27782 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:12:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 10 client.27782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 20 client.27782 populate_metadata read hostname 'smithi122' 2024-09-12T02:12:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef43bf770 0x55bef43c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 4 client.27782 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef4316a20 con 0x55bef43bf770 2024-09-12T02:12:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.727+0000 7f244d914640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef43bf770 0x55bef43c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.728+0000 7f24317fa640 10 client.27782 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.728+0000 7f244d914640 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef43bf770 0x55bef43c1b60 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e30fb76cad6f605 server_cookie=2237285e1d9a6b91 in_seq=0 out_seq=0 2024-09-12T02:12:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24317fa640 1 -- 172.21.15.122:0/3727108353 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bef4316a20 con 0x55bef43bf770 2024-09-12T02:12:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24317fa640 10 client.27782 handle_command_reply: tid=0 2024-09-12T02:12:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 client.27782 shutdown 2024-09-12T02:12:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 msgr2=0x55bef43b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b4450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f243802a930 tx=0x7f24380049a0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 shutdown_connections 2024-09-12T02:12:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef43bf770 0x55bef43c1b60 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43bb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b4450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 --2- 172.21.15.122:0/3727108353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 >> 172.21.15.122:0/3727108353 conn(0x55bef43a1010 msgr2=0x55bef43a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24317fa640 0 client.27782 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 shutdown_connections 2024-09-12T02:12:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 1 -- 172.21.15.122:0/3727108353 wait complete. 2024-09-12T02:12:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.729+0000 7f24529f1740 20 client.27782 trim_cache size 0 max 16384 2024-09-12T02:12:00.745 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:00 smithi122 ceph-mon[30925]: pgmap v1105: 97 pgs: 97 active+clean; 520 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 410 op/s 2024-09-12T02:12:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 Processor -- start 2024-09-12T02:12:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 -- start start 2024-09-12T02:12:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43a9db0 0x55bef43a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43bb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 0x55bef43a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24481a9670 con 0x55bef43a9db0 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24481a6f40 con 0x55bef43aa6f0 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f24529f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24481a4d70 con 0x55bef43af780 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f244d914640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43a9db0 0x55bef43a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.752+0000 7f244e115640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 0x55bef43a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 0x55bef43a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51006/0 (socket says 172.21.15.122:51006) 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 -- 172.21.15.122:0/3415110616 learned_addr learned my addr 172.21.15.122:0/3415110616 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244d113640 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 -- 172.21.15.122:0/3415110616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 msgr2=0x55bef43bb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43bb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 -- 172.21.15.122:0/3415110616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43a9db0 msgr2=0x55bef43a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43a9db0 0x55bef43a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 -- 172.21.15.122:0/3415110616 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef439c2d0 con 0x55bef43af780 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244d914640 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43a9db0 0x55bef43a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f244e115640 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 0x55bef43a1930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2440013800 tx=0x7f2440009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc0b4e196471139 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f24337fe640 1 -- 172.21.15.122:0/3415110616 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440004b20 con 0x55bef43af780 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f24337fe640 1 -- 172.21.15.122:0/3415110616 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440004cc0 con 0x55bef43af780 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.753+0000 7f24337fe640 1 -- 172.21.15.122:0/3415110616 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440010690 con 0x55bef43af780 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 -- 172.21.15.122:0/3415110616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 msgr2=0x55bef43a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 0x55bef43a1930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2440013800 tx=0x7f2440009f90 comp rx=0 tx=0).stop 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 -- 172.21.15.122:0/3415110616 shutdown_connections 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43af780 0x55bef43a1930 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 --2- 172.21.15.122:0/3415110616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43a9db0 0x55bef43a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 -- 172.21.15.122:0/3415110616 >> 172.21.15.122:0/3415110616 conn(0x55bef43a1010 msgr2=0x55bef43b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 -- 172.21.15.122:0/3415110616 shutdown_connections 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.754+0000 7f24529f1740 1 -- 172.21.15.122:0/3415110616 wait complete. 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 -- start start 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 Processor -- start 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43b8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24481a5400 con 0x55bef43af780 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24481a6870 con 0x55bef43aa6f0 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24481a5ad0 con 0x55bef43a9db0 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f24529f1740 10 client.0 _ll_register_callbacks cb 0x7f24440976d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f244d914640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f244d113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f244e115640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f244d914640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43bb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.122:51018/0 (socket says 172.21.15.122:51018) 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f244d113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.122:40498/0 (socket says 172.21.15.122:40498) 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.755+0000 7f244d914640 1 -- 172.21.15.122:0/2244380571 learned_addr learned my addr 172.21.15.122:0/2244380571 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244d914640 1 -- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 msgr2=0x55bef43b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244d914640 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244d914640 1 -- 172.21.15.122:0/2244380571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 msgr2=0x55bef43b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244d914640 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43b8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244d914640 1 -- 172.21.15.122:0/2244380571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef4428410 con 0x55bef43a9db0 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244e115640 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f244d914640 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43bb3e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f244401f000 tx=0x7f244400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89bd579cb492c191 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24317fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24317fa640 1 -- 172.21.15.122:0/2244380571 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24440b3070 con 0x55bef43a9db0 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24317fa640 1 -- 172.21.15.122:0/2244380571 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2444003b30 con 0x55bef43a9db0 2024-09-12T02:12:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2440004cc0 con 0x55bef43a9db0 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24317fa640 1 -- 172.21.15.122:0/2244380571 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f244409ff00 con 0x55bef43a9db0 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24529f1740 10 client.27799 fetch_fsmap learned FSMap version 45 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.756+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bef44297f0 con 0x55bef43a9db0 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24317fa640 1 -- 172.21.15.122:0/2244380571 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440004cc0 con 0x55bef43a9db0 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24317fa640 1 -- 172.21.15.122:0/2244380571 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f244409afa0 con 0x55bef43a9db0 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24529f1740 10 client.27799 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24529f1740 10 client.27799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24529f1740 20 client.27799 populate_metadata read hostname 'smithi122' 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24529f1740 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef442a0a0 0x55bef442c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24529f1740 4 client.27799 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bef4310d70 con 0x55bef442a0a0 2024-09-12T02:12:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f244d113640 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef442a0a0 0x55bef442c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f24317fa640 10 client.27799 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.757+0000 7f244d113640 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef442a0a0 0x55bef442c4c0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d5fad76338297d server_cookie=e5a483d71108ded7 in_seq=0 out_seq=0 2024-09-12T02:12:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:00 smithi151 ceph-mon[35229]: pgmap v1105: 97 pgs: 97 active+clean; 520 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 410 op/s 2024-09-12T02:12:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:00 smithi183 ceph-mon[35127]: pgmap v1105: 97 pgs: 97 active+clean; 520 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 410 op/s 2024-09-12T02:12:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24317fa640 1 -- 172.21.15.122:0/2244380571 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bef4310d70 con 0x55bef442a0a0 2024-09-12T02:12:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24317fa640 10 client.27799 handle_command_reply: tid=0 2024-09-12T02:12:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 client.27799 shutdown 2024-09-12T02:12:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 msgr2=0x55bef43bb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43bb3e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f244401f000 tx=0x7f244400ef90 comp rx=0 tx=0).stop 2024-09-12T02:12:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 shutdown_connections 2024-09-12T02:12:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55bef442a0a0 0x55bef442c4c0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bef43af780 0x55bef43b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef43aa6f0 0x55bef43b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 --2- 172.21.15.122:0/2244380571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bef43a9db0 0x55bef43bb3e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24317fa640 0 client.27799 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.973+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 >> 172.21.15.122:0/2244380571 conn(0x55bef43a1010 msgr2=0x55bef439f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 shutdown_connections 2024-09-12T02:12:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f24529f1740 1 -- 172.21.15.122:0/2244380571 wait complete. 2024-09-12T02:12:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f24529f1740 20 client.27799 trim_cache size 0 max 16384 2024-09-12T02:12:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:12:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:12:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:12:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:12:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24340a2c30 msgr2=0x7f24340a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f244fb78640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24340a2c30 0x7f24340a50f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f24400049d0 tx=0x7f2440004ab0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 msgr2=0x7f2448197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.974+0000 7f244fb78640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f2448197e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f244400ea60 tx=0x7f244400ef20 comp rx=0 tx=0).stop 2024-09-12T02:12:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 shutdown_connections 2024-09-12T02:12:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f24340a2c30 0x7f24340a50f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2448107860 0x7f244819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2448104f30 0x7f2448198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 --2- 172.21.15.122:0/2499439123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2448102600 0x7f2448197e10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 >> 172.21.15.122:0/2499439123 conn(0x7f24480fc1e0 msgr2=0x7f2448102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 shutdown_connections 2024-09-12T02:12:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:00.975+0000 7f244fb78640 1 -- 172.21.15.122:0/2499439123 wait complete. 2024-09-12T02:12:01.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 265653 19.53 MB/sec execute 588 sec latency 141.544 ms 2024-09-12T02:12:02.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 266182 19.53 MB/sec execute 589 sec latency 129.376 ms 2024-09-12T02:12:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:02 smithi151 ceph-mon[35229]: pgmap v1106: 97 pgs: 97 active+clean; 521 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 392 op/s 2024-09-12T02:12:02.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:02 smithi183 ceph-mon[35127]: pgmap v1106: 97 pgs: 97 active+clean; 521 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 392 op/s 2024-09-12T02:12:02.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:02 smithi122 ceph-mon[30925]: pgmap v1106: 97 pgs: 97 active+clean; 521 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 392 op/s 2024-09-12T02:12:03.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 266671 19.54 MB/sec execute 590 sec latency 78.279 ms 2024-09-12T02:12:04.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 267131 19.55 MB/sec execute 591 sec latency 76.931 ms 2024-09-12T02:12:04.754 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:04 smithi183 ceph-mon[35127]: pgmap v1107: 97 pgs: 97 active+clean; 522 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 463 op/s 2024-09-12T02:12:04.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:04 smithi122 ceph-mon[30925]: pgmap v1107: 97 pgs: 97 active+clean; 522 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 463 op/s 2024-09-12T02:12:04.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:04 smithi151 ceph-mon[35229]: pgmap v1107: 97 pgs: 97 active+clean; 522 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 463 op/s 2024-09-12T02:12:05.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 267658 19.56 MB/sec execute 592 sec latency 159.646 ms 2024-09-12T02:12:06.090 INFO:tasks.workunit.client.0.smithi122.stdout: 10 268351 19.59 MB/sec execute 593 sec latency 191.173 ms 2024-09-12T02:12:06.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:06 smithi122 ceph-mon[30925]: pgmap v1108: 97 pgs: 97 active+clean; 528 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 511 op/s 2024-09-12T02:12:06.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:06 smithi151 ceph-mon[35229]: pgmap v1108: 97 pgs: 97 active+clean; 528 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 511 op/s 2024-09-12T02:12:06.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:06 smithi183 ceph-mon[35127]: pgmap v1108: 97 pgs: 97 active+clean; 528 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 511 op/s 2024-09-12T02:12:07.091 INFO:tasks.workunit.client.0.smithi122.stdout: 10 268946 19.61 MB/sec execute 594 sec latency 62.164 ms 2024-09-12T02:12:08.091 INFO:tasks.workunit.client.0.smithi122.stdout: 10 269662 19.65 MB/sec execute 595 sec latency 70.316 ms 2024-09-12T02:12:08.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:08 smithi151 ceph-mon[35229]: pgmap v1109: 97 pgs: 97 active+clean; 529 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.0 MiB/s wr, 451 op/s 2024-09-12T02:12:08.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:08 smithi183 ceph-mon[35127]: pgmap v1109: 97 pgs: 97 active+clean; 529 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.0 MiB/s wr, 451 op/s 2024-09-12T02:12:08.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:08 smithi122 ceph-mon[30925]: pgmap v1109: 97 pgs: 97 active+clean; 529 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.0 MiB/s wr, 451 op/s 2024-09-12T02:12:09.092 INFO:tasks.workunit.client.0.smithi122.stdout: 10 270220 19.67 MB/sec execute 596 sec latency 74.220 ms 2024-09-12T02:12:10.091 INFO:tasks.workunit.client.0.smithi122.stdout: 10 270794 19.68 MB/sec execute 597 sec latency 68.872 ms 2024-09-12T02:12:10.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:10 smithi151 ceph-mon[35229]: pgmap v1110: 97 pgs: 97 active+clean; 538 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 539 op/s 2024-09-12T02:12:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:10 smithi183 ceph-mon[35127]: pgmap v1110: 97 pgs: 97 active+clean; 538 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 539 op/s 2024-09-12T02:12:10.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:10 smithi122 ceph-mon[30925]: pgmap v1110: 97 pgs: 97 active+clean; 538 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 539 op/s 2024-09-12T02:12:11.091 INFO:tasks.workunit.client.0.smithi122.stdout: 10 271294 19.70 MB/sec execute 598 sec latency 92.243 ms 2024-09-12T02:12:12.091 INFO:tasks.workunit.client.0.smithi122.stdout: 10 271808 19.71 MB/sec execute 599 sec latency 96.074 ms 2024-09-12T02:12:12.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:12 smithi151 ceph-mon[35229]: pgmap v1111: 97 pgs: 97 active+clean; 541 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-09-12T02:12:12.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:12 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:12 smithi183 ceph-mon[35127]: pgmap v1111: 97 pgs: 97 active+clean; 541 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-09-12T02:12:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:12 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:12 smithi122 ceph-mon[30925]: pgmap v1111: 97 pgs: 97 active+clean; 541 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-09-12T02:12:12.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:12 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:13.091 INFO:tasks.workunit.client.0.smithi122.stdout: 10 cleanup 600 sec 2024-09-12T02:12:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:13.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:13.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:13 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:12:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:13 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:12:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:13.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:13 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 -- 172.21.15.122:0/717727492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb620107900 msgr2=0x7fb620109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 --2- 172.21.15.122:0/717727492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb620107900 0x7fb620109cf0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb618009ca0 tx=0x7fb61802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 -- 172.21.15.122:0/717727492 shutdown_connections 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 --2- 172.21.15.122:0/717727492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb620107900 0x7fb620109cf0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 --2- 172.21.15.122:0/717727492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620104fd0 0x7fb6201073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 --2- 172.21.15.122:0/717727492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6201026a0 0x7fb620104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 -- 172.21.15.122:0/717727492 >> 172.21.15.122:0/717727492 conn(0x7fb6200fc1e0 msgr2=0x7fb6200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 -- 172.21.15.122:0/717727492 shutdown_connections 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.073+0000 7fb625e30640 1 -- 172.21.15.122:0/717727492 wait complete. 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 Processor -- start 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 -- start start 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 0x7fb620197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620104fd0 0x7fb620198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620107900 0x7fb62019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010d7f0 con 0x7fb6201026a0 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010d670 con 0x7fb620107900 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb625e30640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010d970 con 0x7fb620104fd0 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 0x7fb620197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb61effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620104fd0 0x7fb620198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 0x7fb620197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40010/0 (socket says 172.21.15.122:40010) 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.074+0000 7fb61f7fe640 1 -- 172.21.15.122:0/659838408 learned_addr learned my addr 172.21.15.122:0/659838408 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61ffff640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620107900 0x7fb62019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61f7fe640 1 -- 172.21.15.122:0/659838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620104fd0 msgr2=0x7fb620198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620104fd0 0x7fb620198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61f7fe640 1 -- 172.21.15.122:0/659838408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620107900 msgr2=0x7fb62019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620107900 0x7fb62019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61f7fe640 1 -- 172.21.15.122:0/659838408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019fa10 con 0x7fb6201026a0 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61ffff640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620107900 0x7fb62019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61effd640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620104fd0 0x7fb620198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 0x7fb620197df0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb61000d950 tx=0x7fb61000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61cff9640 1 -- 172.21.15.122:0/659838408 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610017070 con 0x7fb6201026a0 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61cff9640 1 -- 172.21.15.122:0/659838408 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610009c30 con 0x7fb6201026a0 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019fd00 con 0x7fb6201026a0 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb61cff9640 1 -- 172.21.15.122:0/659838408 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61000f6b0 con 0x7fb6201026a0 2024-09-12T02:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.075+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201a01e0 con 0x7fb6201026a0 2024-09-12T02:12:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.077+0000 7fb61cff9640 1 -- 172.21.15.122:0/659838408 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb6100109d0 con 0x7fb6201026a0 2024-09-12T02:12:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 Processor -- start 2024-09-12T02:12:14.087 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 -- start start 2024-09-12T02:12:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da3451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 0x5643da34a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da34a780 0x5643da34cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5643da2a42f0 con 0x5643da3456f0 2024-09-12T02:12:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643da24c270 con 0x5643da34a780 2024-09-12T02:12:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb628bc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643da2a02b0 con 0x5643da344db0 2024-09-12T02:12:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 0x5643da34a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61cff9640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb6080a2cb0 0x7fb6080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 0x5643da34a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40024/0 (socket says 172.21.15.122:40024) 2024-09-12T02:12:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61f7fe640 1 -- 172.21.15.122:0/2991222622 learned_addr learned my addr 172.21.15.122:0/2991222622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da3451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb6080a2cb0 0x7fb6080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61effd640 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da34a780 0x5643da34cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61cff9640 1 -- 172.21.15.122:0/659838408 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb6100cebd0 con 0x7fb6201026a0 2024-09-12T02:12:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 -- 172.21.15.122:0/2991222622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da34a780 msgr2=0x5643da34cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da34a780 0x5643da34cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 -- 172.21.15.122:0/2991222622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 msgr2=0x5643da34a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 0x5643da34a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61ffff640 1 -- 172.21.15.122:0/2991222622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643da24b340 con 0x5643da344db0 2024-09-12T02:12:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.078+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 0x5643da34a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.079+0000 7fb61ffff640 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da3451b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb618005e30 tx=0x7fb618031870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e018783d4cb0ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.079+0000 7fb61ffff640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb6080a2cb0 0x7fb6080a5170 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb618009bf0 tx=0x7fb618041000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.079+0000 7fb605ffb640 1 -- 172.21.15.122:0/2991222622 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61804e070 con 0x5643da344db0 2024-09-12T02:12:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.079+0000 7fb605ffb640 1 -- 172.21.15.122:0/2991222622 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb618036040 con 0x5643da344db0 2024-09-12T02:12:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.079+0000 7fb605ffb640 1 -- 172.21.15.122:0/2991222622 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61802fd30 con 0x5643da344db0 2024-09-12T02:12:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- 172.21.15.122:0/2991222622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 msgr2=0x5643da3451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da3451b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb618005e30 tx=0x7fb618031870 comp rx=0 tx=0).stop 2024-09-12T02:12:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- 172.21.15.122:0/2991222622 shutdown_connections 2024-09-12T02:12:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da34a780 0x5643da34cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da3456f0 0x5643da34a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- 172.21.15.122:0/2991222622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da3451b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- 172.21.15.122:0/2991222622 >> 172.21.15.122:0/2991222622 conn(0x5643da33c010 msgr2=0x5643da33c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- 172.21.15.122:0/2991222622 shutdown_connections 2024-09-12T02:12:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- 172.21.15.122:0/2991222622 wait complete. 2024-09-12T02:12:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- start start 2024-09-12T02:12:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 Processor -- start 2024-09-12T02:12:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da34f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da34f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 0x5643da356520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5643da34d6f0 con 0x5643da34a780 2024-09-12T02:12:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643da22d290 con 0x5643da3456f0 2024-09-12T02:12:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643da34d870 con 0x5643da344db0 2024-09-12T02:12:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.080+0000 7fb628bc8740 10 client.0 _ll_register_callbacks cb 0x5643da25e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 0x5643da356520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 0x5643da356520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40030/0 (socket says 172.21.15.122:40030) 2024-09-12T02:12:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 -- 172.21.15.122:0/4154879626 learned_addr learned my addr 172.21.15.122:0/4154879626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da34f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61effd640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da34f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 -- 172.21.15.122:0/4154879626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 msgr2=0x5643da34f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da34f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 -- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 msgr2=0x5643da34f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da34f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 -- 172.21.15.122:0/4154879626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d9fb7210 con 0x5643da34a780 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb61ffff640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 0x5643da356520 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb618009600 tx=0x7fb61802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61553bdec047d5e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb5feffd640 1 -- 172.21.15.122:0/4154879626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618058020 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb5feffd640 1 -- 172.21.15.122:0/4154879626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb61802fb30 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb618036040 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb5feffd640 1 -- 172.21.15.122:0/4154879626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb61804e040 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb628bc8740 10 client.18900 fetch_fsmap learned FSMap version 45 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.081+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5643da083620 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb5feffd640 1 -- 172.21.15.122:0/4154879626 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb618036040 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb5feffd640 1 -- 172.21.15.122:0/4154879626 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fb618048070 con 0x5643da34a780 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb628bc8740 10 client.18900 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb628bc8740 10 client.18900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb628bc8740 20 client.18900 populate_metadata read hostname 'smithi122' 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb628bc8740 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da35a820 0x5643da35cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb628bc8740 4 client.18900 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643da2b1a20 con 0x5643da35a820 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.082+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da35a820 0x5643da35cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da35a820 0x5643da35cc30 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46257c517577418 server_cookie=5130561bb40a5053 in_seq=0 out_seq=0 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb5feffd640 10 client.18900 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb5feffd640 1 -- 172.21.15.122:0/4154879626 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643da2b1a20 con 0x5643da35a820 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb5feffd640 10 client.18900 handle_command_reply: tid=0 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 client.18900 shutdown 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 msgr2=0x5643da356520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 0x5643da356520 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb618009600 tx=0x7fb61802f840 comp rx=0 tx=0).stop 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 shutdown_connections 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da35a820 0x5643da35cc30 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da34a780 0x5643da356520 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da34f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb5feffd640 0 client.18900 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 --2- 172.21.15.122:0/4154879626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da34f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 >> 172.21.15.122:0/4154879626 conn(0x5643da33c010 msgr2=0x5643da33d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 shutdown_connections 2024-09-12T02:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.083+0000 7fb628bc8740 1 -- 172.21.15.122:0/4154879626 wait complete. 2024-09-12T02:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.084+0000 7fb628bc8740 20 client.18900 trim_cache size 0 max 16384 2024-09-12T02:12:14.100 INFO:tasks.workunit.client.0.smithi122.stdout: 10 cleanup 601 sec 2024-09-12T02:12:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 Processor -- start 2024-09-12T02:12:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 -- start start 2024-09-12T02:12:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 0x5643da33d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da33db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da3564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a96a0 con 0x5643da33c010 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a6f70 con 0x5643da3456f0 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb628bc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a4da0 con 0x5643da344db0 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 0x5643da33d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 0x5643da33d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40040/0 (socket says 172.21.15.122:40040) 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 -- 172.21.15.122:0/1569638783 learned_addr learned my addr 172.21.15.122:0/1569638783 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61effd640 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da33db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 -- 172.21.15.122:0/1569638783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 msgr2=0x5643da33db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da33db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 -- 172.21.15.122:0/1569638783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 msgr2=0x5643da3564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61ffff640 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da3564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da3564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 -- 172.21.15.122:0/1569638783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643da3464c0 con 0x5643da33c010 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61ffff640 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da3564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.098+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 0x5643da33d5d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb610097e00 tx=0x7fb6100cc5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15298098dacdce96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb6057fa640 1 -- 172.21.15.122:0/1569638783 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61009f7e0 con 0x5643da33c010 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb6057fa640 1 -- 172.21.15.122:0/1569638783 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6100d3040 con 0x5643da33c010 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb6057fa640 1 -- 172.21.15.122:0/1569638783 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6100ccb90 con 0x5643da33c010 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 -- 172.21.15.122:0/1569638783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 msgr2=0x5643da33d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 0x5643da33d5d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb610097e00 tx=0x7fb6100cc5a0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 -- 172.21.15.122:0/1569638783 shutdown_connections 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da3456f0 0x5643da3564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da344db0 0x5643da33db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 --2- 172.21.15.122:0/1569638783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da33c010 0x5643da33d5d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 -- 172.21.15.122:0/1569638783 >> 172.21.15.122:0/1569638783 conn(0x5643da34a310 msgr2=0x5643da33cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 -- 172.21.15.122:0/1569638783 shutdown_connections 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.099+0000 7fb628bc8740 1 -- 172.21.15.122:0/1569638783 wait complete. 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 -- start start 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 Processor -- start 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da33c010 0x5643da355db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 0x5643da3562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da3456f0 0x5643da3543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a5430 con 0x5643da344db0 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a68a0 con 0x5643da33c010 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a5b00 con 0x5643da3456f0 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb628bc8740 10 client.0 _ll_register_callbacks cb 0x7fb61804c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb61effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 0x5643da3562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb61effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 0x5643da3562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40054/0 (socket says 172.21.15.122:40054) 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb61effd640 1 -- 172.21.15.122:0/3822650462 learned_addr learned my addr 172.21.15.122:0/3822650462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da33c010 0x5643da355db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.100+0000 7fb61ffff640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da3456f0 0x5643da3543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61effd640 1 -- 172.21.15.122:0/3822650462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da3456f0 msgr2=0x5643da3543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61effd640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da3456f0 0x5643da3543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61effd640 1 -- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da33c010 msgr2=0x5643da355db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61effd640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da33c010 0x5643da355db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61effd640 1 -- 172.21.15.122:0/3822650462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643da354b30 con 0x5643da344db0 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da33c010 0x5643da355db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61ffff640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da3456f0 0x5643da3543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb61effd640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 0x5643da3562f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb620199300 tx=0x7fb614012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c320a57c1c47133 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb5feffd640 1 -- 172.21.15.122:0/3822650462 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614010b70 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb5feffd640 1 -- 172.21.15.122:0/3822650462 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb614010d10 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6100d3040 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb5feffd640 1 -- 172.21.15.122:0/3822650462 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb614004020 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb628bc8740 10 client.18906 fetch_fsmap learned FSMap version 45 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5643da3c5fd0 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb5feffd640 1 -- 172.21.15.122:0/3822650462 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6100d3040 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.101+0000 7fb5feffd640 1 -- 172.21.15.122:0/3822650462 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7fb614026030 con 0x5643da344db0 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb628bc8740 10 client.18906 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb628bc8740 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T02:12:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb628bc8740 20 client.18906 populate_metadata read hostname 'smithi122' 2024-09-12T02:12:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb628bc8740 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da34a780 0x5643da3c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb628bc8740 4 client.18906 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:12:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643da2abd70 con 0x5643da34a780 2024-09-12T02:12:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da34a780 0x5643da3c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb5feffd640 10 client.18906 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.102+0000 7fb61f7fe640 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da34a780 0x5643da3c8a40 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722a6528320672e9 server_cookie=28441254790f3687 in_seq=0 out_seq=0 2024-09-12T02:12:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.375+0000 7fb5feffd640 1 -- 172.21.15.122:0/3822650462 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643da2abd70 con 0x5643da34a780 2024-09-12T02:12:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.375+0000 7fb5feffd640 10 client.18906 handle_command_reply: tid=0 2024-09-12T02:12:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.375+0000 7fb628bc8740 1 client.18906 shutdown 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.375+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 msgr2=0x5643da3562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 0x5643da3562f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb620199300 tx=0x7fb614012ca0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 shutdown_connections 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x5643da34a780 0x5643da3c8a40 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643da3456f0 0x5643da3543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643da344db0 0x5643da3562f0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 --2- 172.21.15.122:0/3822650462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643da33c010 0x5643da355db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb5feffd640 0 client.18906 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 >> 172.21.15.122:0/3822650462 conn(0x5643da34a310 msgr2=0x5643da33d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 shutdown_connections 2024-09-12T02:12:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 1 -- 172.21.15.122:0/3822650462 wait complete. 2024-09-12T02:12:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.376+0000 7fb628bc8740 20 client.18906 trim_cache size 0 max 16384 2024-09-12T02:12:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:{ 2024-09-12T02:12:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "message": "", 2024-09-12T02:12:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout: "return_code": 0 2024-09-12T02:12:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stdout:} 2024-09-12T02:12:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb6080a2cb0 msgr2=0x7fb6080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb6080a2cb0 0x7fb6080a5170 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb618009bf0 tx=0x7fb618041000 comp rx=0 tx=0).stop 2024-09-12T02:12:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 msgr2=0x7fb620197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 0x7fb620197df0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb61000d950 tx=0x7fb61000de10 comp rx=0 tx=0).stop 2024-09-12T02:12:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 shutdown_connections 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb6080a2cb0 0x7fb6080a5170 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620107900 0x7fb62019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620104fd0 0x7fb620198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 --2- 172.21.15.122:0/659838408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6201026a0 0x7fb620197df0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 >> 172.21.15.122:0/659838408 conn(0x7fb6200fc1e0 msgr2=0x7fb620102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 shutdown_connections 2024-09-12T02:12:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi122.front.sepia.ceph.com.smithi122.stderr:2024-09-12T02:12:14.377+0000 7fb625e30640 1 -- 172.21.15.122:0/659838408 wait complete. 2024-09-12T02:12:14.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:14 smithi151 ceph-mon[35229]: pgmap v1112: 97 pgs: 97 active+clean; 539 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-09-12T02:12:14.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:14 smithi183 ceph-mon[35127]: pgmap v1112: 97 pgs: 97 active+clean; 539 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-09-12T02:12:14.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:14 smithi122 ceph-mon[30925]: pgmap v1112: 97 pgs: 97 active+clean; 539 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.3 MiB/s wr, 470 op/s 2024-09-12T02:12:15.091 INFO:tasks.workunit.client.0.smithi122.stdout: 9 cleanup 602 sec 2024-09-12T02:12:16.091 INFO:tasks.workunit.client.0.smithi122.stdout: 9 cleanup 603 sec 2024-09-12T02:12:16.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:16 smithi122 ceph-mon[30925]: pgmap v1113: 97 pgs: 97 active+clean; 529 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 459 op/s 2024-09-12T02:12:16.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:16 smithi151 ceph-mon[35229]: pgmap v1113: 97 pgs: 97 active+clean; 529 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 459 op/s 2024-09-12T02:12:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:16 smithi183 ceph-mon[35127]: pgmap v1113: 97 pgs: 97 active+clean; 529 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.3 MiB/s wr, 459 op/s 2024-09-12T02:12:17.091 INFO:tasks.workunit.client.0.smithi122.stdout: 8 cleanup 604 sec 2024-09-12T02:12:17.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:17 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:17 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: 0 cleanup 605 sec 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Operation Count AvgLat MaxLat 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: ---------------------------------------- 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: NTCreateX 379083 7.111 31537.584 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Close 278310 0.002 0.175 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Rename 16069 32.232 29242.890 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Unlink 76659 12.430 31297.812 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Deltree 2 171.025 260.158 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Mkdir 1 0.033 0.033 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Qpathinfo 343659 3.386 1079.831 2024-09-12T02:12:17.986 INFO:tasks.workunit.client.0.smithi122.stdout: Qfileinfo 59974 0.003 99.801 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: Qfsinfo 63035 0.138 13.119 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: Sfileinfo 30899 5.314 31116.933 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: Find 132799 0.434 647.513 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: WriteX 187637 0.979 31131.742 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: ReadX 594559 0.023 189.577 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: LockX 1236 5.620 71.391 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: UnlockX 1236 4.924 68.966 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: Flush 26587 8.409 281.651 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-09-12T02:12:17.987 INFO:tasks.workunit.client.0.smithi122.stdout:Throughput 19.7097 MB/sec 10 clients 10 procs max_latency=31537.587 ms 2024-09-12T02:12:17.988 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T02:12:17.988 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T02:12:17.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:17 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:18 smithi151 ceph-mon[35229]: pgmap v1114: 97 pgs: 97 active+clean; 530 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-09-12T02:12:18.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:18 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:18.910 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:18 smithi122 ceph-mon[30925]: pgmap v1114: 97 pgs: 97 active+clean; 530 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-09-12T02:12:18.910 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:18 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:18.911 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-12T02:12:18.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T02:12:18.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:18 smithi183 ceph-mon[35127]: pgmap v1114: 97 pgs: 97 active+clean; 530 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-09-12T02:12:18.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:18 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:19.464 DEBUG:teuthology.parallel:result is None 2024-09-12T02:12:19.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T02:12:19.501 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T02:12:19.501 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T02:12:19.516 INFO:tasks.quiescer:joining Quiescers 2024-09-12T02:12:19.516 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T02:12:19.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:19.786 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 -- 172.21.15.122:0/3613751830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc100e20 msgr2=0x7fb9fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 --2- 172.21.15.122:0/3613751830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc100e20 0x7fb9fc101200 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009900 tx=0x7fb9ec02a480 comp rx=0 tx=0).stop 2024-09-12T02:12:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 -- 172.21.15.122:0/3613751830 shutdown_connections 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 --2- 172.21.15.122:0/3613751830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 --2- 172.21.15.122:0/3613751830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc102570 0x7fb9fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 --2- 172.21.15.122:0/3613751830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc100e20 0x7fb9fc101200 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 -- 172.21.15.122:0/3613751830 >> 172.21.15.122:0/3613751830 conn(0x7fb9fc0fc7d0 msgr2=0x7fb9fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 -- 172.21.15.122:0/3613751830 shutdown_connections 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.319+0000 7fba027e7640 1 -- 172.21.15.122:0/3613751830 wait complete. 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 Processor -- start 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 -- start start 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc100e20 0x7fb9fc103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9fc102570 0x7fb9fc103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc110f80 con 0x7fb9fc102ef0 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc110e00 con 0x7fb9fc102570 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba027e7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc111100 con 0x7fb9fc100e20 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc1041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40198/0 (socket says 172.21.15.122:40198) 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 -- 172.21.15.122:0/3330325563 learned_addr learned my addr 172.21.15.122:0/3330325563 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fb9fb7fe640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9fc102570 0x7fb9fc103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fb9fbfff640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc100e20 0x7fb9fc103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 -- 172.21.15.122:0/3330325563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc100e20 msgr2=0x7fb9fc103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc100e20 0x7fb9fc103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 -- 172.21.15.122:0/3330325563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9fc102570 msgr2=0x7fb9fc103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9fc102570 0x7fb9fc103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fba00d84640 1 -- 172.21.15.122:0/3330325563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc1a69e0 con 0x7fb9fc102ef0 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.320+0000 7fb9fbfff640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc100e20 0x7fb9fc103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.321+0000 7fba00d84640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc1041d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400eaa0 tx=0x7fb9f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.321+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400cdb0 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.321+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4005020 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.321+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc1a6cd0 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.321+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40116b0 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.321+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc1a71c0 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.322+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9fc101200 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.323+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb9f4005560 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.324+0000 7fb9f97fa640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9e80a2b80 0x7fb9e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.324+0000 7fb9fbfff640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9e80a2b80 0x7fb9e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.324+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb9f40cf5e0 con 0x7fb9fc102ef0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.324+0000 7fb9fbfff640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9e80a2b80 0x7fb9e80a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000c00 tx=0x7fb9ec004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.327+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9f4016030 con 0x7fb9fc102ef0 2024-09-12T02:12:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.528+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9fc104c80 con 0x7fb9fc102ef0 2024-09-12T02:12:20.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.529+0000 7fb9f97fa640 1 -- 172.21.15.122:0/3330325563 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11931 (secure 0 0 0) 0x7fb9f409c540 con 0x7fb9fc102ef0 2024-09-12T02:12:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:12:20.529 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":47,"btime":"2024-09-12T02:12:17:582760+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:12:16.941936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24428,24436,24452]},"id":1}]} 2024-09-12T02:12:20.530 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 47 2024-09-12T02:12:20.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9e80a2b80 msgr2=0x7fb9e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:20.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9e80a2b80 0x7fb9e80a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000c00 tx=0x7fb9ec004100 comp rx=0 tx=0).stop 2024-09-12T02:12:20.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 msgr2=0x7fb9fc1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:20.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc1041d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400eaa0 tx=0x7fb9f400ef60 comp rx=0 tx=0).stop 2024-09-12T02:12:20.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 shutdown_connections 2024-09-12T02:12:20.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb9e80a2b80 0x7fb9e80a5040 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9fc102ef0 0x7fb9fc1041d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9fc102570 0x7fb9fc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 --2- 172.21.15.122:0/3330325563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9fc100e20 0x7fb9fc103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:20.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 >> 172.21.15.122:0/3330325563 conn(0x7fb9fc0fc7d0 msgr2=0x7fb9fc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:20.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 shutdown_connections 2024-09-12T02:12:20.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:20.531+0000 7fba027e7640 1 -- 172.21.15.122:0/3330325563 wait complete. 2024-09-12T02:12:20.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:20 smithi122 ceph-mon[30925]: pgmap v1115: 97 pgs: 97 active+clean; 477 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.6 MiB/s wr, 593 op/s 2024-09-12T02:12:20.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:20 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3330325563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:20.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:20 smithi151 ceph-mon[35229]: pgmap v1115: 97 pgs: 97 active+clean; 477 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.6 MiB/s wr, 593 op/s 2024-09-12T02:12:20.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:20 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3330325563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:20 smithi183 ceph-mon[35127]: pgmap v1115: 97 pgs: 97 active+clean; 477 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.6 MiB/s wr, 593 op/s 2024-09-12T02:12:20.994 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:20 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3330325563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:21.296 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --cancel --all' 2024-09-12T02:12:21.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --cancel --all 2024-09-12T02:12:21.553 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:21.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:21 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:21 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:21.964 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:21 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.048+0000 7f5f72549640 1 -- 172.21.15.122:0/2779733314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c1062b0 msgr2=0x7f5f6c106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.048+0000 7f5f72549640 1 --2- 172.21.15.122:0/2779733314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c106690 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009930 tx=0x7f5f5c02a420 comp rx=0 tx=0).stop 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 -- 172.21.15.122:0/2779733314 shutdown_connections 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 --2- 172.21.15.122:0/2779733314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f6c100c00 0x7f5f6c10d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 --2- 172.21.15.122:0/2779733314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1002a0 0x7f5f6c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 --2- 172.21.15.122:0/2779733314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c106690 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 -- 172.21.15.122:0/2779733314 >> 172.21.15.122:0/2779733314 conn(0x7f5f6c0fc5c0 msgr2=0x7f5f6c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 -- 172.21.15.122:0/2779733314 shutdown_connections 2024-09-12T02:12:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 -- 172.21.15.122:0/2779733314 wait complete. 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 Processor -- start 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 -- start start 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f6c1002a0 0x7f5f6c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 0x7f5f6c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.049+0000 7f5f72549640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f72549640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c110eb0 con 0x7f5f6c100c00 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f72549640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c110d30 con 0x7f5f6c1062b0 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f72549640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c111030 con 0x7f5f6c1002a0 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 0x7f5f6c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 0x7f5f6c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40228/0 (socket says 172.21.15.122:40228) 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1782469031 learned_addr learned my addr 172.21.15.122:0/1782469031 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f6c1002a0 0x7f5f6c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1782469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f6c1002a0 msgr2=0x7f5f6c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f6c1002a0 0x7f5f6c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1782469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1062b0 msgr2=0x7f5f6c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1782469031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c1a4850 con 0x7f5f6c100c00 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 0x7f5f6c1997d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c920 tx=0x7f5f6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f697fa640 1 -- 172.21.15.122:0/1782469031 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60007c70 con 0x7f5f6c100c00 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f697fa640 1 -- 172.21.15.122:0/1782469031 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f60007e10 con 0x7f5f6c100c00 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c1a4a20 con 0x7f5f6c100c00 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f697fa640 1 -- 172.21.15.122:0/1782469031 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60018890 con 0x7f5f6c100c00 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.050+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c1a4f30 con 0x7f5f6c100c00 2024-09-12T02:12:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.052+0000 7f5f697fa640 1 -- 172.21.15.122:0/1782469031 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5f60018a70 con 0x7f5f6c100c00 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 Processor -- start 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 -- start start 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329cc8e0 0x55e5329cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 0x55e5329de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e532931200 con 0x55e5329dbff0 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5328d7dc0 con 0x55e5329cd200 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f74922740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e53292d1c0 con 0x55e5329cc8e0 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f697fa640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f540a2b80 0x7f5f540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 0x55e5329de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 0x55e5329de3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40242/0 (socket says 172.21.15.122:40242) 2024-09-12T02:12:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1352073701 learned_addr learned my addr 172.21.15.122:0/1352073701 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f540a2b80 0x7f5f540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329cc8e0 0x55e5329cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.053+0000 7f5f697fa640 1 -- 172.21.15.122:0/1782469031 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5f600cf240 con 0x7f5f6c100c00 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1352073701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329cc8e0 msgr2=0x55e5329cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329cc8e0 0x55e5329cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1352073701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 msgr2=0x55e5329dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/1352073701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5326a2620 con 0x55e5329dbff0 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329cc8e0 0x55e5329cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 0x55e5329de3e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5f600ccd40 tx=0x7f5f60020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96df68226ee027c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f540a2b80 0x7f5f540a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400afd0 tx=0x7f5f6400a3d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f527fc640 1 -- 172.21.15.122:0/1352073701 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600bbe60 con 0x55e5329dbff0 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f527fc640 1 -- 172.21.15.122:0/1352073701 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6000de20 con 0x55e5329dbff0 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.054+0000 7f5f527fc640 1 -- 172.21.15.122:0/1352073701 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60098110 con 0x55e5329dbff0 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 -- 172.21.15.122:0/1352073701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 msgr2=0x55e5329de3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 0x55e5329de3e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5f600ccd40 tx=0x7f5f60020000 comp rx=0 tx=0).stop 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 -- 172.21.15.122:0/1352073701 shutdown_connections 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329dbff0 0x55e5329de3e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 --2- 172.21.15.122:0/1352073701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329cc8e0 0x55e5329cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 -- 172.21.15.122:0/1352073701 >> 172.21.15.122:0/1352073701 conn(0x55e5329c8f90 msgr2=0x55e5329c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 -- 172.21.15.122:0/1352073701 shutdown_connections 2024-09-12T02:12:22.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 -- 172.21.15.122:0/1352073701 wait complete. 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 -- start start 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.055+0000 7f5f74922740 1 Processor -- start 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5329debc0 con 0x55e5329cd200 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5328cbc50 con 0x55e5329cc8e0 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5329ded40 con 0x55e5329dbff0 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 10 client.0 _ll_register_callbacks cb 0x55e5328ddba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329d8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40258/0 (socket says 172.21.15.122:40258) 2024-09-12T02:12:22.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/351924203 learned_addr learned my addr 172.21.15.122:0/351924203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/351924203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 msgr2=0x55e5329d91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 msgr2=0x55e5329db980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329db980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/351924203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5329d99e0 con 0x55e5329cd200 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329d8c30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c8f0 tx=0x7f5f6009bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d726accafb9d40dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/351924203 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600b3070 con 0x55e5329cd200 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/351924203 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5f6000fb30 con 0x55e5329cd200 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6000de20 con 0x55e5329cd200 2024-09-12T02:12:22.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/351924203 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5f6009bd00 con 0x55e5329cd200 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 10 client.18924 fetch_fsmap learned FSMap version 49 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.056+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e5329ea7c0 con 0x55e5329cd200 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/351924203 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6000de20 con 0x55e5329cd200 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/351924203 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f5f6009aef0 con 0x55e5329cd200 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f74922740 10 client.18924 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f74922740 10 client.18924 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f74922740 20 client.18924 populate_metadata read hostname 'smithi122' 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f74922740 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5329ebb10 0x55e5329edf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f74922740 4 client.18924 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e53293e770 con 0x55e5329ebb10 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5329ebb10 0x55e5329edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5329ebb10 0x55e5329edf00 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e3253477b35d5a server_cookie=279051eac9a3592a in_seq=0 out_seq=0 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.057+0000 7f5f4b7fe640 10 client.18924 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/351924203 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e53293e770 con 0x55e5329ebb10 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f4b7fe640 10 client.18924 handle_command_reply: tid=0 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 client.18924 shutdown 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 msgr2=0x55e5329d8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329d8c30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c8f0 tx=0x7f5f6009bb90 comp rx=0 tx=0).stop 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 shutdown_connections 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e5329ebb10 0x55e5329edf00 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329d8c30 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f4b7fe640 0 client.18924 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 --2- 172.21.15.122:0/351924203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329db980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.058+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 >> 172.21.15.122:0/351924203 conn(0x55e5329c8f90 msgr2=0x55e5329cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.059+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 shutdown_connections 2024-09-12T02:12:22.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.059+0000 7f5f74922740 1 -- 172.21.15.122:0/351924203 wait complete. 2024-09-12T02:12:22.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.059+0000 7f5f74922740 20 client.18924 trim_cache size 0 max 16384 2024-09-12T02:12:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 Processor -- start 2024-09-12T02:12:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 -- start start 2024-09-12T02:12:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 0x55e5329d7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1add10 con 0x55e5329cc8e0 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1ab580 con 0x55e5329cd200 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f74922740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1a93b0 con 0x55e5329dbff0 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 0x55e5329d7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 0x55e5329d7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40266/0 (socket says 172.21.15.122:40266) 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 -- 172.21.15.122:0/38694797 learned_addr learned my addr 172.21.15.122:0/38694797 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 -- 172.21.15.122:0/38694797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 msgr2=0x55e5329f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 -- 172.21.15.122:0/38694797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 msgr2=0x55e5329f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329f0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.076+0000 7f5f6bfff640 1 -- 172.21.15.122:0/38694797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5329dec10 con 0x55e5329cc8e0 2024-09-12T02:12:22.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.077+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 0x55e5329d7c10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0096c0 tx=0x7f5f5c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d57e77c56f544e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.077+0000 7f5f51ffb640 1 -- 172.21.15.122:0/38694797 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c048070 con 0x55e5329cc8e0 2024-09-12T02:12:22.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.077+0000 7f5f51ffb640 1 -- 172.21.15.122:0/38694797 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f5c0035f0 con 0x55e5329cc8e0 2024-09-12T02:12:22.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.077+0000 7f5f51ffb640 1 -- 172.21.15.122:0/38694797 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c035040 con 0x55e5329cc8e0 2024-09-12T02:12:22.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- 172.21.15.122:0/38694797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 msgr2=0x55e5329d7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 0x55e5329d7c10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0096c0 tx=0x7f5f5c005a20 comp rx=0 tx=0).stop 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- 172.21.15.122:0/38694797 shutdown_connections 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cd200 0x55e5329f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- 172.21.15.122:0/38694797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cc8e0 0x55e5329d7c10 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- 172.21.15.122:0/38694797 >> 172.21.15.122:0/38694797 conn(0x55e5329c8f90 msgr2=0x55e5329d7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- 172.21.15.122:0/38694797 shutdown_connections 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- 172.21.15.122:0/38694797 wait complete. 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- start start 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 Processor -- start 2024-09-12T02:12:22.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329ef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329ef750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1a9a40 con 0x55e5329cd200 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1aaeb0 con 0x55e5329cc8e0 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.078+0000 7f5f74922740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1aa110 con 0x55e5329dbff0 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f74922740 10 client.0 _ll_register_callbacks cb 0x7f5f60018ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329ef210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40274/0 (socket says 172.21.15.122:40274) 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/92869579 learned_addr learned my addr 172.21.15.122:0/92869579 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f70ae6640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/92869579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 msgr2=0x55e5329ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 msgr2=0x55e5329eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 -- 172.21.15.122:0/92869579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5329c8dc0 con 0x55e5329cd200 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f6b7fe640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329ef210 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5f60098470 tx=0x7f5f6009ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15c1dd587e8370b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/92869579 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f600b3050 con 0x55e5329cd200 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/92869579 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5f6009ab00 con 0x55e5329cd200 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f5c0035f0 con 0x55e5329cd200 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/92869579 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5f60071e60 con 0x55e5329cd200 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f74922740 10 client.18930 fetch_fsmap learned FSMap version 49 2024-09-12T02:12:22.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.079+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e532a56510 con 0x55e5329cd200 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/92869579 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f5c0035f0 con 0x55e5329cd200 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/92869579 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f5f600a4280 con 0x55e5329cd200 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f74922740 10 client.18930 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f74922740 10 client.18930 resolve_mds: validated gid 24452 aka daemon mds.b 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f74922740 20 client.18930 populate_metadata read hostname 'smithi122' 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f74922740 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e532a56ae0 0x55e532a58f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f74922740 4 client.18930 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 --> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55e532938c80 con 0x55e532a56ae0 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e532a56ae0 0x55e532a58f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f4b7fe640 10 client.18930 ms_handle_connect on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.080+0000 7f5f6bfff640 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e532a56ae0 0x55e532a58f00 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ff0497bb1228b0 server_cookie=fd9d2dcabde1f3f6 in_seq=0 out_seq=0 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f4b7fe640 1 -- 172.21.15.122:0/92869579 <== mds.0 v2:172.21.15.151:6838/1184770501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55e532938c80 con 0x55e532a56ae0 2024-09-12T02:12:22.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f4b7fe640 10 client.18930 handle_command_reply: tid=0 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 client.18930 shutdown 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 msgr2=0x55e5329ef210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329ef210 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5f60098470 tx=0x7f5f6009ff80 comp rx=0 tx=0).stop 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 shutdown_connections 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:6838/1184770501,v1:172.21.15.151:6839/1184770501] conn(0x55e532a56ae0 0x55e532a58f00 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e5329dbff0 0x55e5329ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5329cd200 0x55e5329ef210 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f4b7fe640 0 client.18930 ms_handle_reset on v2:172.21.15.151:6838/1184770501 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 --2- 172.21.15.122:0/92869579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5329cc8e0 0x55e5329eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 >> 172.21.15.122:0/92869579 conn(0x55e5329c8f90 msgr2=0x55e5329d7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 shutdown_connections 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 1 -- 172.21.15.122:0/92869579 wait complete. 2024-09-12T02:12:22.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.081+0000 7f5f74922740 20 client.18930 trim_cache size 0 max 16384 2024-09-12T02:12:22.089 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:12:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 49, 2024-09-12T02:12:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "leader": 24452, 2024-09-12T02:12:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "set_version": 88, 2024-09-12T02:12:22.089 INFO:teuthology.orchestra.run.smithi122.stdout: "sets": {} 2024-09-12T02:12:22.089 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f540a2b80 msgr2=0x7f5f540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f540a2b80 0x7f5f540a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400afd0 tx=0x7f5f6400a3d0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 msgr2=0x7f5f6c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 0x7f5f6c1997d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c920 tx=0x7f5f6000cde0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 shutdown_connections 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f5f540a2b80 0x7f5f540a5040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f6c1062b0 0x7f5f6c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f6c100c00 0x7f5f6c1997d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 --2- 172.21.15.122:0/1782469031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f6c1002a0 0x7f5f6c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 >> 172.21.15.122:0/1782469031 conn(0x7f5f6c0fc5c0 msgr2=0x7f5f6c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 shutdown_connections 2024-09-12T02:12:22.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.082+0000 7f5f72549640 1 -- 172.21.15.122:0/1782469031 wait complete. 2024-09-12T02:12:22.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:22 smithi122 ceph-mon[30925]: pgmap v1116: 97 pgs: 97 active+clean; 416 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 598 op/s 2024-09-12T02:12:22.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:22 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:22.765 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T02:12:22.765 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T02:12:22.778 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T02:12:22.786 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T02:12:22.799 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T02:12:22.799 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-09-12T02:12:22.801 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T02:12:22.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T02:12:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:22 smithi151 ceph-mon[35229]: pgmap v1116: 97 pgs: 97 active+clean; 416 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 598 op/s 2024-09-12T02:12:22.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:22 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.928+0000 7f0c72fbd640 1 -- 172.21.15.122:0/2115007388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 msgr2=0x7f0c6c104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.928+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/2115007388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c104a40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009900 tx=0x7f0c6802a410 comp rx=0 tx=0).stop 2024-09-12T02:12:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- 172.21.15.122:0/2115007388 shutdown_connections 2024-09-12T02:12:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/2115007388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c6c1078b0 0x7f0c6c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/2115007388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c6c104f80 0x7f0c6c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/2115007388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c104a40 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- 172.21.15.122:0/2115007388 >> 172.21.15.122:0/2115007388 conn(0x7f0c6c0fc190 msgr2=0x7f0c6c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- 172.21.15.122:0/2115007388 shutdown_connections 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- 172.21.15.122:0/2115007388 wait complete. 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 Processor -- start 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- start start 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c6c104f80 0x7f0c6c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c6c1078b0 0x7f0c6c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10d830 con 0x7f0c6c102650 2024-09-12T02:12:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10d6b0 con 0x7f0c6c1078b0 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c72fbd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10d9b0 con 0x7f0c6c104f80 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c70d59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c70d59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40292/0 (socket says 172.21.15.122:40292) 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c70d59640 1 -- 172.21.15.122:0/1214140875 learned_addr learned my addr 172.21.15.122:0/1214140875 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.929+0000 7f0c63fff640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c6c104f80 0x7f0c6c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c7155a640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c6c1078b0 0x7f0c6c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c70d59640 1 -- 172.21.15.122:0/1214140875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c6c104f80 msgr2=0x7f0c6c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c70d59640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c6c104f80 0x7f0c6c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c70d59640 1 -- 172.21.15.122:0/1214140875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c6c1078b0 msgr2=0x7f0c6c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c70d59640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c6c1078b0 0x7f0c6c19f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c70d59640 1 -- 172.21.15.122:0/1214140875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c19fb30 con 0x7f0c6c102650 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c70d59640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c197f10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009230 tx=0x7f0c68004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6803f030 con 0x7f0c6c102650 2024-09-12T02:12:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c68003ce0 con 0x7f0c6c102650 2024-09-12T02:12:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68035070 con 0x7f0c6c102650 2024-09-12T02:12:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c19fdc0 con 0x7f0c6c102650 2024-09-12T02:12:22.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.930+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1a0270 con 0x7f0c6c102650 2024-09-12T02:12:22.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.931+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c0fdc80 con 0x7f0c6c102650 2024-09-12T02:12:22.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.932+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0c68042030 con 0x7f0c6c102650 2024-09-12T02:12:22.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.933+0000 7f0c61ffb640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c540a2b90 0x7f0c540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:22.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.933+0000 7f0c63fff640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c540a2b90 0x7f0c540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:22.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.933+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0c680efa30 con 0x7f0c6c102650 2024-09-12T02:12:22.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.934+0000 7f0c63fff640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c540a2b90 0x7f0c540a5050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c199420 tx=0x7f0c5c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:22.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:22.937+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c680bc990 con 0x7f0c6c102650 2024-09-12T02:12:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:22 smithi183 ceph-mon[35127]: pgmap v1116: 97 pgs: 97 active+clean; 416 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 598 op/s 2024-09-12T02:12:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:22 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.107+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c6c062e90 con 0x7f0c6c102650 2024-09-12T02:12:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.108+0000 7f0c61ffb640 1 -- 172.21.15.122:0/1214140875 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22783 (secure 0 0 0) 0x7f0c680bc290 con 0x7f0c6c102650 2024-09-12T02:12:23.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:12:23.110 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":88,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T02:06:54.470390+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c540a2b90 msgr2=0x7f0c540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c540a2b90 0x7f0c540a5050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c199420 tx=0x7f0c5c009110 comp rx=0 tx=0).stop 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 msgr2=0x7f0c6c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c197f10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009230 tx=0x7f0c68004360 comp rx=0 tx=0).stop 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 shutdown_connections 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f0c540a2b90 0x7f0c540a5050 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c6c1078b0 0x7f0c6c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c6c104f80 0x7f0c6c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 --2- 172.21.15.122:0/1214140875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c6c102650 0x7f0c6c197f10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.110+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 >> 172.21.15.122:0/1214140875 conn(0x7f0c6c0fc190 msgr2=0x7f0c6c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.111+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 shutdown_connections 2024-09-12T02:12:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.111+0000 7f0c72fbd640 1 -- 172.21.15.122:0/1214140875 wait complete. 2024-09-12T02:12:23.124 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T01:39:58.671405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T01:44:32.683657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T01:44:34.505051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:12:23.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 -- 172.21.15.122:0/339292433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b8100400 msgr2=0x7fb0b8100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/339292433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b8100400 0x7fb0b8100800 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009900 tx=0x7fb0b402a480 comp rx=0 tx=0).stop 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 -- 172.21.15.122:0/339292433 shutdown_connections 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/339292433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b81016e0 0x7fb0b8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/339292433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0b8100d40 0x7fb0b81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/339292433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b8100400 0x7fb0b8100800 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 -- 172.21.15.122:0/339292433 >> 172.21.15.122:0/339292433 conn(0x7fb0b80fc190 msgr2=0x7fb0b80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 -- 172.21.15.122:0/339292433 shutdown_connections 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.226+0000 7fb0c03b9640 1 -- 172.21.15.122:0/339292433 wait complete. 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 Processor -- start 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 -- start start 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b8100400 0x7fb0b819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0b8100d40 0x7fb0b819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 0x7fb0b81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810f990 con 0x7fb0b81016e0 2024-09-12T02:12:23.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810f810 con 0x7fb0b8100400 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0c03b9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810fb10 con 0x7fb0b8100d40 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 0x7fb0b81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 0x7fb0b81a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40312/0 (socket says 172.21.15.122:40312) 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 -- 172.21.15.122:0/3482509365 learned_addr learned my addr 172.21.15.122:0/3482509365 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0bd954640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0b8100d40 0x7fb0b819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be155640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b8100400 0x7fb0b819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 -- 172.21.15.122:0/3482509365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0b8100d40 msgr2=0x7fb0b819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0b8100d40 0x7fb0b819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 -- 172.21.15.122:0/3482509365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b8100400 msgr2=0x7fb0b819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b8100400 0x7fb0b819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.227+0000 7fb0be956640 1 -- 172.21.15.122:0/3482509365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b81a1cc0 con 0x7fb0b81016e0 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.228+0000 7fb0be956640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 0x7fb0b81a1650 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009fd0 tx=0x7fb0b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.228+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0b81016e0 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.228+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b0003e50 con 0x7fb0b81016e0 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.228+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b81a1fb0 con 0x7fb0b81016e0 2024-09-12T02:12:23.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.228+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00106c0 con 0x7fb0b81016e0 2024-09-12T02:12:23.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.228+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a2420 con 0x7fb0b81016e0 2024-09-12T02:12:23.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.229+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b810b310 con 0x7fb0b81016e0 2024-09-12T02:12:23.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.230+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb0b0003970 con 0x7fb0b81016e0 2024-09-12T02:12:23.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.230+0000 7fb0af7fe640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0a40a2b90 0x7fb0a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.231+0000 7fb0be155640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0a40a2b90 0x7fb0a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.231+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb0b00cede0 con 0x7fb0b81016e0 2024-09-12T02:12:23.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.231+0000 7fb0be155640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0a40a2b90 0x7fb0a40a5050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c00 tx=0x7fb0b40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:23.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.235+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0b009bd40 con 0x7fb0b81016e0 2024-09-12T02:12:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.402+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0b8062e90 con 0x7fb0b81016e0 2024-09-12T02:12:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.403+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3482509365 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb0b0097020 con 0x7fb0b81016e0 2024-09-12T02:12:23.404 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0a40a2b90 msgr2=0x7fb0a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0a40a2b90 0x7fb0a40a5050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c00 tx=0x7fb0b40315d0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 msgr2=0x7fb0b81a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 0x7fb0b81a1650 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009fd0 tx=0x7fb0b000b260 comp rx=0 tx=0).stop 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 shutdown_connections 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb0a40a2b90 0x7fb0a40a5050 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b81016e0 0x7fb0b81a1650 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0b8100d40 0x7fb0b819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 --2- 172.21.15.122:0/3482509365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b8100400 0x7fb0b819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 >> 172.21.15.122:0/3482509365 conn(0x7fb0b80fc190 msgr2=0x7fb0b80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 shutdown_connections 2024-09-12T02:12:23.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.405+0000 7fb0c03b9640 1 -- 172.21.15.122:0/3482509365 wait complete. 2024-09-12T02:12:23.418 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T02:12:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 -- 172.21.15.122:0/1179000295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b81078b0 msgr2=0x7f84b8109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 --2- 172.21.15.122:0/1179000295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b81078b0 0x7f84b8109ca0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009ca0 tx=0x7f84b002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 -- 172.21.15.122:0/1179000295 shutdown_connections 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 --2- 172.21.15.122:0/1179000295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b81078b0 0x7f84b8109ca0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 --2- 172.21.15.122:0/1179000295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b8104f80 0x7f84b8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 --2- 172.21.15.122:0/1179000295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b8102650 0x7f84b8104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 -- 172.21.15.122:0/1179000295 >> 172.21.15.122:0/1179000295 conn(0x7f84b80fc1b0 msgr2=0x7f84b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 -- 172.21.15.122:0/1179000295 shutdown_connections 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.518+0000 7f84bc911640 1 -- 172.21.15.122:0/1179000295 wait complete. 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 Processor -- start 2024-09-12T02:12:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 -- start start 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 0x7f84b8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b8104f80 0x7f84b8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b81078b0 0x7f84b819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810d7f0 con 0x7f84b8102650 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810d670 con 0x7f84b8104f80 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84bc911640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810d970 con 0x7f84b81078b0 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84b659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 0x7f84b8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84b659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 0x7f84b8197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40336/0 (socket says 172.21.15.122:40336) 2024-09-12T02:12:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84b659c640 1 -- 172.21.15.122:0/462698980 learned_addr learned my addr 172.21.15.122:0/462698980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.519+0000 7f84b5d9b640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b8104f80 0x7f84b8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b6d9d640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b81078b0 0x7f84b819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b659c640 1 -- 172.21.15.122:0/462698980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b81078b0 msgr2=0x7f84b819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b659c640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b81078b0 0x7f84b819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b659c640 1 -- 172.21.15.122:0/462698980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b8104f80 msgr2=0x7f84b8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b659c640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b8104f80 0x7f84b8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b659c640 1 -- 172.21.15.122:0/462698980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819fb30 con 0x7f84b8102650 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b6d9d640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b81078b0 0x7f84b819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84b659c640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 0x7f84b8197f10 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f84a800e9f0 tx=0x7f84a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a800cd80 con 0x7f84b8102650 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b819fe20 con 0x7f84b8102650 2024-09-12T02:12:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a8005500 con 0x7f84b8102650 2024-09-12T02:12:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8010690 con 0x7f84b8102650 2024-09-12T02:12:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.520+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b81a0280 con 0x7f84b8102650 2024-09-12T02:12:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.521+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b80fdca0 con 0x7f84b8102650 2024-09-12T02:12:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.522+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f84a80046c0 con 0x7f84b8102650 2024-09-12T02:12:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.523+0000 7f84a77fe640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.523+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f84a80cf510 con 0x7f84b8102650 2024-09-12T02:12:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.523+0000 7f84b5d9b640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.523+0000 7f84b5d9b640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84a00a2b90 0x7f84a00a5050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84b8199420 tx=0x7f84ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.527+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84a809c470 con 0x7f84b8102650 2024-09-12T02:12:23.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.693+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f84b8062e90 con 0x7f84b8102650 2024-09-12T02:12:23.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.693+0000 7f84a77fe640 1 -- 172.21.15.122:0/462698980 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f84a809bd70 con 0x7f84b8102650 2024-09-12T02:12:23.695 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84a00a2b90 msgr2=0x7f84a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84a00a2b90 0x7f84a00a5050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84b8199420 tx=0x7f84ac00b390 comp rx=0 tx=0).stop 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 msgr2=0x7f84b8197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 0x7f84b8197f10 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f84a800e9f0 tx=0x7f84a800edf0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 shutdown_connections 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84b81078b0 0x7f84b819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84b8104f80 0x7f84b8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 --2- 172.21.15.122:0/462698980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b8102650 0x7f84b8197f10 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.695+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 >> 172.21.15.122:0/462698980 conn(0x7f84b80fc1b0 msgr2=0x7f84b8102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.696+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 shutdown_connections 2024-09-12T02:12:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.696+0000 7f84bc911640 1 -- 172.21.15.122:0/462698980 wait complete. 2024-09-12T02:12:23.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:23 smithi122 ceph-mon[30925]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:23.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:23 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1214140875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:12:23.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:23 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3482509365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:12:23.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T02:12:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 -- 172.21.15.122:0/422424434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 msgr2=0x7f4940108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 --2- 172.21.15.122:0/422424434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f4940108e70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4938009ca0 tx=0x7f493802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 -- 172.21.15.122:0/422424434 shutdown_connections 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 --2- 172.21.15.122:0/422424434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f4940108e70 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 --2- 172.21.15.122:0/422424434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4940101c20 0x7f49401020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 --2- 172.21.15.122:0/422424434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4940100a30 0x7f4940100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 -- 172.21.15.122:0/422424434 >> 172.21.15.122:0/422424434 conn(0x7f49400fc1b0 msgr2=0x7f49400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 -- 172.21.15.122:0/422424434 shutdown_connections 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 -- 172.21.15.122:0/422424434 wait complete. 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 Processor -- start 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.809+0000 7f494788d640 1 -- start start 2024-09-12T02:12:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4940100a30 0x7f4940197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4940101c20 0x7f4940198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f494019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f494010d800 con 0x7f49401025e0 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f494010d680 con 0x7f4940100a30 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f494010d980 con 0x7f4940101c20 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f494019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f494019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40348/0 (socket says 172.21.15.122:40348) 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 -- 172.21.15.122:0/708899193 learned_addr learned my addr 172.21.15.122:0/708899193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945629640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4940100a30 0x7f4940197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4944e28640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4940101c20 0x7f4940198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 -- 172.21.15.122:0/708899193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4940101c20 msgr2=0x7f4940198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4940101c20 0x7f4940198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 -- 172.21.15.122:0/708899193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4940100a30 msgr2=0x7f4940197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4940100a30 0x7f4940197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 -- 172.21.15.122:0/708899193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494019f9d0 con 0x7f49401025e0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f4945e2a640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f494019f330 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f4938031ee0 tx=0x7f4938031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938003c30 con 0x7f49401025e0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4938003dd0 con 0x7f49401025e0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494019fc60 con 0x7f49401025e0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938004720 con 0x7f49401025e0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.810+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a01b0 con 0x7f49401025e0 2024-09-12T02:12:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.811+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4940107cf0 con 0x7f49401025e0 2024-09-12T02:12:23.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.813+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f493802f710 con 0x7f49401025e0 2024-09-12T02:12:23.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.814+0000 7f49327fc640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f492c0a2b90 0x7f492c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:23.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.814+0000 7f4945629640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f492c0a2b90 0x7f492c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:23.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.814+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49380f0630 con 0x7f49401025e0 2024-09-12T02:12:23.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.814+0000 7f4945629640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f492c0a2b90 0x7f492c0a5050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4940199290 tx=0x7f493c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:23.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.817+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49380bd620 con 0x7f49401025e0 2024-09-12T02:12:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:23 smithi151 ceph-mon[35229]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:23.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:23 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1214140875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:12:23.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:23 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3482509365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:12:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:23 smithi183 ceph-mon[35127]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T02:12:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:23 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1214140875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:12:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:23 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3482509365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:12:23.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.983+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f494002da50 con 0x7f49401025e0 2024-09-12T02:12:23.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.983+0000 7f49327fc640 1 -- 172.21.15.122:0/708899193 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f49380bcf20 con 0x7f49401025e0 2024-09-12T02:12:23.985 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f492c0a2b90 msgr2=0x7f492c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f492c0a2b90 0x7f492c0a5050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4940199290 tx=0x7f493c005f90 comp rx=0 tx=0).stop 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 msgr2=0x7f494019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f494019f330 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f4938031ee0 tx=0x7f4938031f10 comp rx=0 tx=0).stop 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 shutdown_connections 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f492c0a2b90 0x7f492c0a5050 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49401025e0 0x7f494019f330 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4940101c20 0x7f4940198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 --2- 172.21.15.122:0/708899193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4940100a30 0x7f4940197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:23.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 >> 172.21.15.122:0/708899193 conn(0x7f49400fc1b0 msgr2=0x7f4940104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:23.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 shutdown_connections 2024-09-12T02:12:23.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:23.986+0000 7f494788d640 1 -- 172.21.15.122:0/708899193 wait complete. 2024-09-12T02:12:23.998 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T02:12:24.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 -- 172.21.15.122:0/3204260807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c101b30 msgr2=0x7f983c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 --2- 172.21.15.122:0/3204260807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c101b30 0x7f983c101f30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9830009900 tx=0x7f983002a410 comp rx=0 tx=0).stop 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 -- 172.21.15.122:0/3204260807 shutdown_connections 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 --2- 172.21.15.122:0/3204260807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f983c1036e0 0x7f983c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 --2- 172.21.15.122:0/3204260807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c102d20 0x7f983c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 --2- 172.21.15.122:0/3204260807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c101b30 0x7f983c101f30 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 -- 172.21.15.122:0/3204260807 >> 172.21.15.122:0/3204260807 conn(0x7f983c0fd2b0 msgr2=0x7f983c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 -- 172.21.15.122:0/3204260807 shutdown_connections 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.096+0000 7f9840f2c640 1 -- 172.21.15.122:0/3204260807 wait complete. 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 Processor -- start 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 -- start start 2024-09-12T02:12:24.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c101b30 0x7f983c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 0x7f983c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f983c1036e0 0x7f983c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10d7b0 con 0x7f983c102d20 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10d630 con 0x7f983c101b30 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9840f2c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10d930 con 0x7f983c1036e0 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 0x7f983c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 0x7f983c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40362/0 (socket says 172.21.15.122:40362) 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 -- 172.21.15.122:0/340362494 learned_addr learned my addr 172.21.15.122:0/340362494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f983a59c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c101b30 0x7f983c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f983ad9d640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f983c1036e0 0x7f983c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 -- 172.21.15.122:0/340362494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f983c1036e0 msgr2=0x7f983c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:24.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f983c1036e0 0x7f983c19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 -- 172.21.15.122:0/340362494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c101b30 msgr2=0x7f983c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c101b30 0x7f983c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 -- 172.21.15.122:0/340362494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c19fa80 con 0x7f983c102d20 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.097+0000 7f9839d9b640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 0x7f983c1983a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f982c00d950 tx=0x7f982c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.098+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c017070 con 0x7f983c102d20 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.098+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f982c009c30 con 0x7f983c102d20 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.098+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983c19fd70 con 0x7f983c102d20 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.098+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c00f6b0 con 0x7f983c102d20 2024-09-12T02:12:24.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.098+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983c1a0260 con 0x7f983c102d20 2024-09-12T02:12:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.099+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983c108fc0 con 0x7f983c102d20 2024-09-12T02:12:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.100+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f982c0109d0 con 0x7f983c102d20 2024-09-12T02:12:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.101+0000 7f982b7fe640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98240a2b90 0x7f98240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.101+0000 7f983a59c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98240a2b90 0x7f98240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:24.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.101+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f982c0ced40 con 0x7f983c102d20 2024-09-12T02:12:24.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.101+0000 7f983a59c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98240a2b90 0x7f98240a5050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9830006010 tx=0x7f9830000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.104+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f982c009de0 con 0x7f983c102d20 2024-09-12T02:12:24.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.315+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f983c02da10 con 0x7f983c102d20 2024-09-12T02:12:24.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.322+0000 7f982b7fe640 1 -- 172.21.15.122:0/340362494 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740378 (secure 0 0 0) 0x7f982c09bc60 con 0x7f983c102d20 2024-09-12T02:12:24.338 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:12:24.338 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:12:24.338 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 20, 2024-09-12T02:12:24.338 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "active_gid": 14150, 2024-09-12T02:12:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "active_name": "x", 2024-09-12T02:12:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "active_addrs": { 2024-09-12T02:12:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-09-12T02:12:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.339 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-09-12T02:12:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:6800", 2024-09-12T02:12:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3245830997 2024-09-12T02:12:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-09-12T02:12:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:6801", 2024-09-12T02:12:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3245830997 2024-09-12T02:12:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.341 INFO:teuthology.orchestra.run.smithi122.stdout: "active_addr": "172.21.15.122:6801/3245830997", 2024-09-12T02:12:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "active_change": "2024-09-12T01:37:44.367455+0000", 2024-09-12T02:12:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T02:12:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-12T02:12:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "standbys": [ 2024-09-12T02:12:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.342 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 24109, 2024-09-12T02:12:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "z", 2024-09-12T02:12:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr_features": 4540701547738038271, 2024-09-12T02:12:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: "available_modules": [ 2024-09-12T02:12:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.343 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "alerts", 2024-09-12T02:12:24.344 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.344 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.344 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.344 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:24.344 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:24.344 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.345 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.345 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.345 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.345 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.345 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.346 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.347 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.347 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.347 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.347 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.347 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.347 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.348 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.348 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.348 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.348 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.348 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.348 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.349 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.349 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.349 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.349 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.349 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.349 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.350 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.351 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.353 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_destination": { 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_destination", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_from_name": { 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_from_name", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "Ceph", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email From: name", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_host": { 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_host", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.356 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP server", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_password": { 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_password", 2024-09-12T02:12:24.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Password to authenticate with", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_port": { 2024-09-12T02:12:24.358 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_port", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "465", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP port", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.359 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_sender": { 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_sender", 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.360 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_ssl": { 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_ssl", 2024-09-12T02:12:24.361 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.362 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_user": { 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_user", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "User to authenticate as", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.363 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "balancer", 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.364 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "active": { 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "active", 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.365 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_time": { 2024-09-12T02:12:24.366 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_time", 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0000", 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.367 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_weekday": { 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_weekday", 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.368 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.369 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.369 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.369 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-09-12T02:12:24.369 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-09-12T02:12:24.370 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.370 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:12:24.370 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:12:24.370 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.370 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_max_iterations": { 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "25", 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "250", 2024-09-12T02:12:24.371 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_metrics": { 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_metrics", 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:12:24.372 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_step": { 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_step", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.5", 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0.001", 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "0.999", 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.374 INFO:teuthology.orchestra.run.smithi122.stdout: "end_time": { 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_time", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2359", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "end_weekday": { 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_weekday", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.376 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.377 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.378 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.380 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.382 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: "min_score": { 2024-09-12T02:12:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_score", 2024-09-12T02:12:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.384 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.384 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.384 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.384 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.384 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:12:24.384 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "upmap", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.385 INFO:teuthology.orchestra.run.smithi122.stdout: "crush-compat", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "none", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "read", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap-read" 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Balancer mode", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_ids": { 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_ids", 2024-09-12T02:12:24.386 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:24.387 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.388 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_deviation": { 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_deviation", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_optimizations": { 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.390 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm", 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_down_multiplier": { 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_down_multiplier", 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.391 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.392 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_refresh_rate": { 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_refresh_rate", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "20", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.393 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_starting_port": { 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_starting_port", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4721", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:12:24.394 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_ptrace": { 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_ptrace", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:12:24.395 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_interval": { 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_interval", 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.396 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.7", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm_log_destination": { 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm_log_destination", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.398 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "file", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "file,syslog", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "syslog" 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "cgroups_split": { 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cgroups_split", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.400 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "config_checks_enabled": { 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_checks_enabled", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.401 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "config_dashboard": { 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_dashboard", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:12:24.402 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_alertmanager": { 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_alertmanager", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_base": { 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_base", 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.404 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_elasticsearch": { 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.405 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "elasticsearch container image", 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_grafana": { 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_grafana", 2024-09-12T02:12:24.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_haproxy": { 2024-09-12T02:12:24.407 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_haproxy", 2024-09-12T02:12:24.408 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.408 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.408 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.408 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:12:24.408 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "HAproxy container image", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_agent": { 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.409 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_collector": { 2024-09-12T02:12:24.410 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:12:24.411 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_query": { 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:12:24.412 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger query container image", 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_keepalived": { 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_keepalived", 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.414 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Keepalived container image", 2024-09-12T02:12:24.415 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.415 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.415 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.415 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_loki": { 2024-09-12T02:12:24.415 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_loki", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Loki container image", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.416 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_node_exporter": { 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_node_exporter", 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.417 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.418 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_nvmeof": { 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_nvmeof", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Nvme-of container image", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_prometheus": { 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_prometheus", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.420 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_promtail": { 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_promtail", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Promtail container image", 2024-09-12T02:12:24.421 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_samba": { 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_samba", 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.422 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_snmp_gateway": { 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.423 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.424 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: "container_init": { 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_init", 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "daemon_cache_timeout": { 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.427 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.428 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:12:24.428 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.428 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.428 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.428 INFO:teuthology.orchestra.run.smithi122.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "900", 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.429 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "default_registry": { 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_registry", 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.430 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io", 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "device_cache_timeout": { 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_cache_timeout", 2024-09-12T02:12:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1800", 2024-09-12T02:12:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.433 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:12:24.433 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.433 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.433 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.434 INFO:teuthology.orchestra.run.smithi122.stdout: "device_enhanced_scan": { 2024-09-12T02:12:24.434 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_enhanced_scan", 2024-09-12T02:12:24.434 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.434 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.434 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.434 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.435 INFO:teuthology.orchestra.run.smithi122.stdout: "facts_cache_timeout": { 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "facts_cache_timeout", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.436 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "grafana_dashboards_path": { 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:12:24.437 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "host_check_interval": { 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "host_check_interval", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.438 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "hw_monitoring": { 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hw_monitoring", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "inventory_list_all": { 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "inventory_list_all", 2024-09-12T02:12:24.440 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.441 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.441 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.441 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.441 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.441 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.441 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.442 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.443 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.443 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.443 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.443 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.443 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.445 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.445 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.445 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.445 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.445 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.446 INFO:teuthology.orchestra.run.smithi122.stdout: "log_refresh_metadata": { 2024-09-12T02:12:24.446 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_refresh_metadata", 2024-09-12T02:12:24.446 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.446 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.447 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.447 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.447 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.447 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.447 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.448 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:12:24.448 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.448 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.448 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.448 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.449 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.449 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.449 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.449 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.449 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.450 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.450 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.450 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.450 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:12:24.450 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.450 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.451 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.451 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.451 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.451 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.452 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.452 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.452 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.452 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.452 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.452 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.453 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.453 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.453 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.453 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.453 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.454 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.454 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.454 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.454 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.454 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.454 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.455 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.455 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.455 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.455 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.456 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.456 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.456 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.456 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.456 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.457 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.457 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.457 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.457 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:12:24.458 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:12:24.458 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.458 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.458 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.458 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.458 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.459 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.459 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.459 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:12:24.459 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.459 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.460 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.460 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:12:24.460 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:12:24.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.460 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.461 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.461 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "*", 2024-09-12T02:12:24.461 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.461 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.461 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.462 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:12:24.462 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.462 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.462 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.463 INFO:teuthology.orchestra.run.smithi122.stdout: "max_count_per_host": { 2024-09-12T02:12:24.463 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_count_per_host", 2024-09-12T02:12:24.463 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.463 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.463 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.463 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.464 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.464 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.464 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.464 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:12:24.464 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: "max_osd_draining_count": { 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_osd_draining_count", 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.465 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "migration_current": { 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "migration_current", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "internal - do not modify", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.467 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "root", 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.468 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm-package", 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "root" 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.469 INFO:teuthology.orchestra.run.smithi122.stdout: "oob_default_addr": { 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "oob_default_addr", 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "169.254.1.1", 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.470 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "prometheus_alerts_path": { 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.471 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_insecure": { 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_insecure", 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.472 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.473 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.476 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.476 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.476 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.476 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_password": { 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_password", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.477 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_url": { 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_url", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.478 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_username": { 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_username", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.479 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "secure_monitoring_stack": { 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.480 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "service_discovery_port": { 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "service_discovery_port", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.481 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8765", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "ssh_config_file": { 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssh_config_file", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.482 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "use_agent": { 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_agent", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "use_repo_digest": { 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_repo_digest", 2024-09-12T02:12:24.484 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_failed_host_check": { 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:12:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_daemons": { 2024-09-12T02:12:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.487 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_hosts": { 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crash", 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.489 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.490 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.491 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.492 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.493 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.494 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "retain_interval": { 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retain_interval", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "31536000", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:12:24.495 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_recent_interval": { 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_recent_interval", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1209600", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.496 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dashboard", 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:12:24.497 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.498 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.501 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.502 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.503 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.504 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.505 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.506 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.507 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.509 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:12:24.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:12:24.511 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.512 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.513 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_URL": { 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.518 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.519 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.521 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.522 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.523 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.524 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.526 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.527 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:12:24.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:12:24.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:12:24.530 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.532 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.534 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "45", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.535 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.536 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.537 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.538 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.539 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.540 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.541 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.542 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "cross_origin_url": { 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cross_origin_url", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.543 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "crt_file": { 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crt_file", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.544 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "debug": { 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "debug", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "jwt_token_ttl": { 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "jwt_token_ttl", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.546 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "28800", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-09-12T02:12:24.547 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.549 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.552 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.553 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "motd": { 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "motd", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "The message of the day", 2024-09-12T02:12:24.554 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.555 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.556 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8080", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.557 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.558 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl_server_port": { 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl_server_port", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8443", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.560 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "redirect", 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect" 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-09-12T02:12:24.561 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.562 INFO:teuthology.orchestra.run.smithi122.stdout: "url_prefix": { 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url_prefix", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_monitoring": { 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_monitoring", 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.564 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "monitor device health metrics", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.566 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.567 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.568 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.570 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "mark_out_threshold": { 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mark_out_threshold", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2419200", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.571 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_name": { 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_name", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "device_health_metrics", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:12:24.572 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "retention_period": { 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retention_period", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15552000", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:12:24.573 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_frequency": { 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_frequency", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.574 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "self_heal": { 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "self_heal", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.575 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.576 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_threshold": { 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_threshold", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7257600", 2024-09-12T02:12:24.577 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "diskprediction_local", 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.578 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.579 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.580 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.581 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.582 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.583 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "predict_interval": { 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predict_interval", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.584 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "predictor_model": { 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predictor_model", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "prophetstor", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.585 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.586 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "influx", 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": false, 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "batch_size": { 2024-09-12T02:12:24.587 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "batch_size", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5000", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.588 INFO:teuthology.orchestra.run.smithi122.stdout: "database": { 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "database", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "ceph", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.589 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": { 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hostname", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.590 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "30", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "5", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.591 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.592 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.594 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.595 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.596 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "password": { 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "password", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.597 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "port": { 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "port", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8086", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.598 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server port", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "false", 2024-09-12T02:12:24.599 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "threads": { 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threads", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.600 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "32", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "username": { 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "username", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.601 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "verify_ssl": { 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "verify_ssl", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.602 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "true", 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "insights", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.604 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.605 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.606 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.607 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.608 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "iostat", 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.609 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.610 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.611 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.613 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.614 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "k8sevents", 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_event_retention_days": { 2024-09-12T02:12:24.615 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "config_check_secs": { 2024-09-12T02:12:24.616 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_check_secs", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "10", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.618 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.620 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "localpool", 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "failure_domain": { 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "failure_domain", 2024-09-12T02:12:24.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "host", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.624 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.625 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.626 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.627 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.628 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.629 INFO:teuthology.orchestra.run.smithi122.stdout: "min_size": { 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_size", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.630 INFO:teuthology.orchestra.run.smithi122.stdout: "num_rep": { 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "num_rep", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.631 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_num": { 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_num", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "128", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "prefix": { 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prefix", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:12:24.633 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "subtree": { 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subtree", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "rack", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:12:24.634 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mds_autoscaler", 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.635 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.636 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.638 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.639 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirroring", 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.641 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.642 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.643 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.644 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.645 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.646 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "nfs", 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.647 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.648 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.649 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.650 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "fail_fs": { 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "fail_fs", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.653 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.654 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.656 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.657 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.658 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator": { 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.659 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "rook", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "test_orchestrator" 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Orchestrator backend", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_perf_query", 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.662 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.663 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.664 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.666 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_support", 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.667 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.668 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.669 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.670 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.671 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.672 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_autoscaler", 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.673 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.674 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.675 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.676 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.678 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "threshold": { 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threshold", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1.0", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "scaling threshold", 2024-09-12T02:12:24.680 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "progress", 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_pg_recovery_event": { 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:12:24.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.682 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.682 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.684 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.684 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.684 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.684 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.685 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.688 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.689 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.690 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "max_completed_events": { 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_completed_events", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.691 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "50", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.692 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "cache": { 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cache", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.694 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "exclude_perf_counters": { 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "exclude_perf_counters", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.695 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.696 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.697 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.698 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.699 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.700 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools": { 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools", 2024-09-12T02:12:24.701 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:12:24.702 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "300", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_interval": { 2024-09-12T02:12:24.703 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_interval", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15.0", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.704 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "9283", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "stale_cache_strategy": { 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stale_cache_strategy", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "log", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.707 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "default", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.708 INFO:teuthology.orchestra.run.smithi122.stdout: "default", 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "error" 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.709 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-09-12T02:12:24.710 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.711 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.713 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.714 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.715 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_snap_create": { 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.716 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "trash_purge_schedule": { 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "trash_purge_schedule", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.718 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.719 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "restful", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_auth": { 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_auth", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.720 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.722 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.723 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.724 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.725 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.726 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:24.727 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.729 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rgw", 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.730 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.731 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.732 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.733 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.734 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rook", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.736 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.737 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.739 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "storage_class": { 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "storage_class", 2024-09-12T02:12:24.741 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "local", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "selftest", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.743 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.744 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.745 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.747 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "roption1": { 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption1", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.748 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "roption2": { 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption2", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "xyz", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.749 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption1": { 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption1", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.750 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption2": { 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption2", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.751 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption3": { 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption3", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.752 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption4": { 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption4", 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.753 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption5": { 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption5", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.754 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption6": { 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption6", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.755 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption7": { 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption7", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.756 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "42", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "testkey": { 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testkey", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.757 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "testlkey": { 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testlkey", 2024-09-12T02:12:24.758 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "testnewline": { 2024-09-12T02:12:24.759 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testnewline", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_m_granularity": { 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_m_granularity", 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.761 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "dump_on_update": { 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dump_on_update", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.762 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.763 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.764 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.765 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.766 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.767 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.768 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stats", 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.769 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.770 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.771 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.772 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.773 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.774 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "status", 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.775 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.776 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.777 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.778 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.780 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telegraf", 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "address": { 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "address", 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:24.782 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.783 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.784 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.785 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.786 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.788 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telemetry", 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_basic": { 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_basic", 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.789 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_crash": { 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_crash", 2024-09-12T02:12:24.790 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_device": { 2024-09-12T02:12:24.791 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_device", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_ident": { 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_ident", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_perf": { 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_perf", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "contact": { 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "contact", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": { 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "description", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.796 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "device_url": { 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_url", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.797 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.798 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "24", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "8", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.799 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "last_opt_revision": { 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "last_opt_revision", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.800 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard": { 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard", 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.801 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard_description": { 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard_description", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.802 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.803 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.804 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.805 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.806 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "organization": { 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "organization", 2024-09-12T02:12:24.808 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "proxy": { 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "proxy", 2024-09-12T02:12:24.809 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "url": { 2024-09-12T02:12:24.810 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "test_orchestrator", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.812 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.815 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.817 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.818 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.819 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.820 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.821 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.822 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_clones": { 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_clones", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4", 2024-09-12T02:12:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "periodic_async_work": { 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "periodic_async_work", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.824 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Periodically check for async work", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_delay": { 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.825 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.826 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "discovery_interval": { 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "discovery_interval", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "100", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.828 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "identifier": { 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "identifier", 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.829 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.830 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.831 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.832 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.833 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.834 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.835 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_host": { 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_host", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_port": { 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_port", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10051", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.838 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_sender": { 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_sender", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.839 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 24113, 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "y", 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr_features": 4540701547738038271, 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: "available_modules": [ 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "alerts", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.841 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.842 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.843 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.845 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.846 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_destination": { 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_destination", 2024-09-12T02:12:24.847 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_from_name": { 2024-09-12T02:12:24.848 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_from_name", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "Ceph", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email From: name", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_host": { 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_host", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP server", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.850 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_password": { 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_password", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Password to authenticate with", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.851 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_port": { 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_port", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "465", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP port", 2024-09-12T02:12:24.852 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_sender": { 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_sender", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:12:24.853 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_ssl": { 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_ssl", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.854 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_user": { 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_user", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.855 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "User to authenticate as", 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "balancer", 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "active": { 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "active", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.857 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_time": { 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_time", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0000", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.858 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_weekday": { 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_weekday", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.859 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_max_iterations": { 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "25", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "250", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_metrics": { 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_metrics", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:12:24.861 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_step": { 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_step", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.5", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0.001", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "0.999", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:12:24.862 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "end_time": { 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_time", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2359", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.863 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "end_weekday": { 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_weekday", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-09-12T02:12:24.864 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.867 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.868 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.869 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "min_score": { 2024-09-12T02:12:24.870 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_score", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-09-12T02:12:24.871 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "upmap", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "crush-compat", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "none", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "read", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap", 2024-09-12T02:12:24.872 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap-read" 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Balancer mode", 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_ids": { 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_ids", 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.873 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.874 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_deviation": { 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_deviation", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.875 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_optimizations": { 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:24.876 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_down_multiplier": { 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_down_multiplier", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.878 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_refresh_rate": { 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_refresh_rate", 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "20", 2024-09-12T02:12:24.879 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_starting_port": { 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_starting_port", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.880 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4721", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_ptrace": { 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_ptrace", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.881 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_interval": { 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_interval", 2024-09-12T02:12:24.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:12:24.883 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.7", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm_log_destination": { 2024-09-12T02:12:24.885 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm_log_destination", 2024-09-12T02:12:24.885 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.885 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.885 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.885 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.885 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.888 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.888 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.888 INFO:teuthology.orchestra.run.smithi122.stdout: "file", 2024-09-12T02:12:24.888 INFO:teuthology.orchestra.run.smithi122.stdout: "file,syslog", 2024-09-12T02:12:24.888 INFO:teuthology.orchestra.run.smithi122.stdout: "syslog" 2024-09-12T02:12:24.888 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "cgroups_split": { 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cgroups_split", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.889 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "config_checks_enabled": { 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_checks_enabled", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.890 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "config_dashboard": { 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_dashboard", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.891 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_alertmanager": { 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_alertmanager", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.892 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_base": { 2024-09-12T02:12:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_base", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_elasticsearch": { 2024-09-12T02:12:24.894 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "elasticsearch container image", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.895 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_grafana": { 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_grafana", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_haproxy": { 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_haproxy", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "HAproxy container image", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.897 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_agent": { 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:12:24.898 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_collector": { 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.899 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_query": { 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.900 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger query container image", 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_keepalived": { 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_keepalived", 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.901 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Keepalived container image", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_loki": { 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_loki", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.902 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Loki container image", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_node_exporter": { 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_node_exporter", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.903 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_nvmeof": { 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_nvmeof", 2024-09-12T02:12:24.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Nvme-of container image", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_prometheus": { 2024-09-12T02:12:24.905 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_prometheus", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_promtail": { 2024-09-12T02:12:24.906 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_promtail", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Promtail container image", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.907 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_samba": { 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_samba", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_snmp_gateway": { 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.909 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "container_init": { 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_init", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.910 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "daemon_cache_timeout": { 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.911 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "900", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:12:24.912 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "default_registry": { 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_registry", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:12:24.913 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "device_cache_timeout": { 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_cache_timeout", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1800", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:12:24.914 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "device_enhanced_scan": { 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_enhanced_scan", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.915 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "facts_cache_timeout": { 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "facts_cache_timeout", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.916 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "grafana_dashboards_path": { 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.917 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "host_check_interval": { 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "host_check_interval", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.918 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "hw_monitoring": { 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hw_monitoring", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.919 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "inventory_list_all": { 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "inventory_list_all", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.920 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.921 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.922 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "log_refresh_metadata": { 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_refresh_metadata", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:12:24.923 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.924 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.925 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.926 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.927 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:12:24.928 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "*", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.929 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "max_count_per_host": { 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_count_per_host", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "max_osd_draining_count": { 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_osd_draining_count", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "migration_current": { 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "migration_current", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.932 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "internal - do not modify", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "root", 2024-09-12T02:12:24.933 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm-package", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "root" 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "oob_default_addr": { 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "oob_default_addr", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.934 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "169.254.1.1", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "prometheus_alerts_path": { 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:12:24.935 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_insecure": { 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_insecure", 2024-09-12T02:12:24.936 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_password": { 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_password", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.938 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_url": { 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_url", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.939 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_username": { 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_username", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.940 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "secure_monitoring_stack": { 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:12:24.941 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "service_discovery_port": { 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "service_discovery_port", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8765", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:12:24.942 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "ssh_config_file": { 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssh_config_file", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.943 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "use_agent": { 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_agent", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "use_repo_digest": { 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_repo_digest", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.945 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_failed_host_check": { 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.946 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_daemons": { 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.947 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_hosts": { 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.948 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crash", 2024-09-12T02:12:24.949 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.950 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:24.951 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:24.952 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:24.953 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.954 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "retain_interval": { 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retain_interval", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.955 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "31536000", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_recent_interval": { 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_recent_interval", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.956 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1209600", 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dashboard", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.958 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.959 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.960 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.961 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.962 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.963 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.964 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.965 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.966 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:12:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:12:24.968 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:12:24.969 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:12:24.970 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.971 INFO:teuthology.orchestra.run.smithi122.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.973 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.974 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_URL": { 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.975 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.976 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.977 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.978 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.979 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.980 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.981 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.984 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.985 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.986 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.987 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "45", 2024-09-12T02:12:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:12:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:12:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:12:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:12:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "cross_origin_url": { 2024-09-12T02:12:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cross_origin_url", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "crt_file": { 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crt_file", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "debug": { 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "debug", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "jwt_token_ttl": { 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "jwt_token_ttl", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "28800", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "motd": { 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "motd", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "The message of the day", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8080", 2024-09-12T02:12:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl_server_port": { 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl_server_port", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8443", 2024-09-12T02:12:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "redirect", 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect" 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-09-12T02:12:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "url_prefix": { 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url_prefix", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_monitoring": { 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_monitoring", 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.024 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "monitor device health metrics", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "mark_out_threshold": { 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mark_out_threshold", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2419200", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_name": { 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_name", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "device_health_metrics", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "retention_period": { 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retention_period", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15552000", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_frequency": { 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_frequency", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "self_heal": { 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "self_heal", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:12:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_threshold": { 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_threshold", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7257600", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "diskprediction_local", 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "predict_interval": { 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predict_interval", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "predictor_model": { 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predictor_model", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "prophetstor", 2024-09-12T02:12:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "influx", 2024-09-12T02:12:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": false, 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "batch_size": { 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "batch_size", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5000", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:12:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "database": { 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "database", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "ceph", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": { 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hostname", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "30", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "5", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "password": { 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "password", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "port": { 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "port", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8086", 2024-09-12T02:12:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server port", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "false", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "threads": { 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threads", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "32", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "username": { 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "username", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "verify_ssl": { 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "verify_ssl", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "true", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "insights", 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "iostat", 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.071 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.072 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.074 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "k8sevents", 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_event_retention_days": { 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.075 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "config_check_secs": { 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_check_secs", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.076 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "10", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.078 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.079 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.080 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.082 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "localpool", 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "failure_domain": { 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "failure_domain", 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "host", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.084 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.085 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.086 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.087 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.088 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.089 INFO:teuthology.orchestra.run.smithi122.stdout: "min_size": { 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_size", 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.090 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.092 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.092 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:12:25.092 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:12:25.092 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.092 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "num_rep": { 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "num_rep", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.093 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_num": { 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_num", 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "128", 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "prefix": { 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prefix", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "subtree": { 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subtree", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "rack", 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mds_autoscaler", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.098 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.099 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.101 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.102 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.103 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirroring", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.104 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.105 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.107 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.108 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.109 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "nfs", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.110 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.112 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.113 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.114 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.115 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "fail_fs": { 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "fail_fs", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.116 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.117 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.118 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.119 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.120 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.121 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator": { 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.122 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "rook", 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "test_orchestrator" 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Orchestrator backend", 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_perf_query", 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.123 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.124 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.125 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.126 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.127 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.128 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_support", 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.129 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.130 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.131 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.133 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.134 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_autoscaler", 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.135 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.136 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.137 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.138 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.139 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.140 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.141 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "threshold": { 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threshold", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1.0", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.142 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "scaling threshold", 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "progress", 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.143 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_pg_recovery_event": { 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.144 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.145 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.146 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.148 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.149 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.150 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "max_completed_events": { 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_completed_events", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "50", 2024-09-12T02:12:25.151 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.152 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus", 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.153 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "cache": { 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cache", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.154 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "exclude_perf_counters": { 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "exclude_perf_counters", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:12:25.155 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.156 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.157 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.158 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.159 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.160 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools": { 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.162 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "300", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_interval": { 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_interval", 2024-09-12T02:12:25.163 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15.0", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:25.164 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:25.165 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "9283", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.166 INFO:teuthology.orchestra.run.smithi122.stdout: "stale_cache_strategy": { 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stale_cache_strategy", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "log", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.167 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "default", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "default", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "error" 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.168 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.169 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.170 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_snap_create": { 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "trash_purge_schedule": { 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "trash_purge_schedule", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "restful", 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_auth": { 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_auth", 2024-09-12T02:12:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-09-12T02:12:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rgw", 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rook", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "storage_class": { 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "storage_class", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "local", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "selftest", 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "roption1": { 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption1", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "roption2": { 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption2", 2024-09-12T02:12:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "xyz", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption1": { 2024-09-12T02:12:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption1", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption2": { 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption2", 2024-09-12T02:12:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption3": { 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption3", 2024-09-12T02:12:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption4": { 2024-09-12T02:12:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption4", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption5": { 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption5", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption6": { 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption6", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption7": { 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption7", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "42", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "testkey": { 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testkey", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "testlkey": { 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testlkey", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "testnewline": { 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testnewline", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_m_granularity": { 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_m_granularity", 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "dump_on_update": { 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dump_on_update", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stats", 2024-09-12T02:12:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "status", 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telegraf", 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "address": { 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "address", 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telemetry", 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_basic": { 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_basic", 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_crash": { 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_crash", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_device": { 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_device", 2024-09-12T02:12:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_ident": { 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_ident", 2024-09-12T02:12:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_perf": { 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_perf", 2024-09-12T02:12:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "contact": { 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "contact", 2024-09-12T02:12:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "description": { 2024-09-12T02:12:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "description", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "device_url": { 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_url", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "24", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "8", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "last_opt_revision": { 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "last_opt_revision", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard": { 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard_description": { 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard_description", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "organization": { 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "organization", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "proxy": { 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "proxy", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "url": { 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "test_orchestrator", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_clones": { 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_clones", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4", 2024-09-12T02:12:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "periodic_async_work": { 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "periodic_async_work", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Periodically check for async work", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_delay": { 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "discovery_interval": { 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "discovery_interval", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "100", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "identifier": { 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "identifier", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_host": { 2024-09-12T02:12:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_host", 2024-09-12T02:12:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_port": { 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_port", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10051", 2024-09-12T02:12:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_sender": { 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_sender", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "dashboard", 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "available_modules": [ 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "alerts", 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:12:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_destination": { 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_destination", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_from_name": { 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_from_name", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "Ceph", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email From: name", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_host": { 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_host", 2024-09-12T02:12:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP server", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_password": { 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_password", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Password to authenticate with", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_port": { 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_port", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "465", 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP port", 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_sender": { 2024-09-12T02:12:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_sender", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_ssl": { 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_ssl", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_user": { 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_user", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "User to authenticate as", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "balancer", 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "active": { 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "active", 2024-09-12T02:12:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_time": { 2024-09-12T02:12:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_time", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0000", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_weekday": { 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_weekday", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_max_iterations": { 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "25", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "250", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_metrics": { 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_metrics", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:12:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_step": { 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_step", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.5", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0.001", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "0.999", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:12:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "end_time": { 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_time", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2359", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:12:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "end_weekday": { 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_weekday", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-09-12T02:12:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "min_score": { 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_score", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "upmap", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "crush-compat", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "none", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "read", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap-read" 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Balancer mode", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_ids": { 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_ids", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:12:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_deviation": { 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_deviation", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_optimizations": { 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm", 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_down_multiplier": { 2024-09-12T02:12:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_down_multiplier", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_refresh_rate": { 2024-09-12T02:12:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_refresh_rate", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "20", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_starting_port": { 2024-09-12T02:12:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_starting_port", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4721", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_ptrace": { 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_ptrace", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_interval": { 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_interval", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:12:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.7", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm_log_destination": { 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm_log_destination", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "file", 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "file,syslog", 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "syslog" 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "cgroups_split": { 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cgroups_split", 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "config_checks_enabled": { 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_checks_enabled", 2024-09-12T02:12:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "config_dashboard": { 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_dashboard", 2024-09-12T02:12:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_alertmanager": { 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_alertmanager", 2024-09-12T02:12:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_base": { 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_base", 2024-09-12T02:12:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_elasticsearch": { 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "elasticsearch container image", 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_grafana": { 2024-09-12T02:12:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_grafana", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_haproxy": { 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_haproxy", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "HAproxy container image", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_agent": { 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_collector": { 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_query": { 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger query container image", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_keepalived": { 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_keepalived", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Keepalived container image", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_loki": { 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_loki", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Loki container image", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_node_exporter": { 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_node_exporter", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_nvmeof": { 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_nvmeof", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Nvme-of container image", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_prometheus": { 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_prometheus", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_promtail": { 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_promtail", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Promtail container image", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_samba": { 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_samba", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:12:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_snmp_gateway": { 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "container_init": { 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_init", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "daemon_cache_timeout": { 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "900", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "default_registry": { 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_registry", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io", 2024-09-12T02:12:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "device_cache_timeout": { 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_cache_timeout", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1800", 2024-09-12T02:12:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "device_enhanced_scan": { 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_enhanced_scan", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "facts_cache_timeout": { 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "facts_cache_timeout", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "grafana_dashboards_path": { 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "host_check_interval": { 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "host_check_interval", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "hw_monitoring": { 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hw_monitoring", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "inventory_list_all": { 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "inventory_list_all", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "log_refresh_metadata": { 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_refresh_metadata", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.379 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.380 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.381 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.382 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.383 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "*", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:12:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "max_count_per_host": { 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_count_per_host", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "max_osd_draining_count": { 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_osd_draining_count", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "migration_current": { 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "migration_current", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "internal - do not modify", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "root", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm-package", 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "root" 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "oob_default_addr": { 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "oob_default_addr", 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.389 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "169.254.1.1", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "prometheus_alerts_path": { 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.390 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_insecure": { 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_insecure", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.391 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_password": { 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_password", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_url": { 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_url", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.393 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_username": { 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_username", 2024-09-12T02:12:25.394 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "secure_monitoring_stack": { 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:12:25.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "service_discovery_port": { 2024-09-12T02:12:25.396 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "service_discovery_port", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8765", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.397 INFO:teuthology.orchestra.run.smithi122.stdout: "ssh_config_file": { 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssh_config_file", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "use_agent": { 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_agent", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "use_repo_digest": { 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_repo_digest", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.400 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_failed_host_check": { 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:12:25.401 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_daemons": { 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:12:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_hosts": { 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.403 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crash", 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.404 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.405 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.406 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.407 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.408 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "retain_interval": { 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retain_interval", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "31536000", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_recent_interval": { 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_recent_interval", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1209600", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dashboard", 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.412 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:12:25.413 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.414 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.417 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.418 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.420 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.421 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.422 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.423 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.424 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.425 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.426 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.427 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.428 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_URL": { 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.429 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:25.430 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.431 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.432 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.433 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.434 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.435 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:12:25.436 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:12:25.437 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.438 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:12:25.439 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.440 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.443 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.444 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.445 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.446 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.447 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.448 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "45", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.449 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.450 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.451 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.452 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:12:25.453 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.454 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.455 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.457 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "cross_origin_url": { 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cross_origin_url", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.458 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "crt_file": { 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crt_file", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.459 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "debug": { 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "debug", 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.460 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "jwt_token_ttl": { 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "jwt_token_ttl", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "28800", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.461 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.462 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.463 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.464 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.465 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.466 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.467 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "motd": { 2024-09-12T02:12:25.468 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "motd", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "The message of the day", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.471 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8080", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.472 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.473 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl_server_port": { 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl_server_port", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8443", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.474 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "redirect", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.475 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect" 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-09-12T02:12:25.476 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "url_prefix": { 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url_prefix", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.478 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_monitoring": { 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_monitoring", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "monitor device health metrics", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.480 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.482 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.483 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.484 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "mark_out_threshold": { 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mark_out_threshold", 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.485 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2419200", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_name": { 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_name", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.486 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "device_health_metrics", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "retention_period": { 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retention_period", 2024-09-12T02:12:25.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15552000", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_frequency": { 2024-09-12T02:12:25.488 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_frequency", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "self_heal": { 2024-09-12T02:12:25.489 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "self_heal", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.490 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.491 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_threshold": { 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_threshold", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7257600", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.492 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "diskprediction_local", 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.493 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.496 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.497 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.501 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.501 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "predict_interval": { 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predict_interval", 2024-09-12T02:12:25.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.503 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "predictor_model": { 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predictor_model", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "prophetstor", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.504 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.505 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "influx", 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": false, 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.506 INFO:teuthology.orchestra.run.smithi122.stdout: "batch_size": { 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "batch_size", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5000", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.507 INFO:teuthology.orchestra.run.smithi122.stdout: "database": { 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "database", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "ceph", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.508 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": { 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hostname", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "30", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "5", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:12:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.515 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "password": { 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "password", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.516 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "port": { 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "port", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.517 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8086", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server port", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "false", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "threads": { 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threads", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.519 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "32", 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.520 INFO:teuthology.orchestra.run.smithi122.stdout: "username": { 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "username", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.521 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "verify_ssl": { 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "verify_ssl", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "true", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.522 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "insights", 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.524 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.525 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.526 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.527 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "iostat", 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.529 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.530 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.531 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.532 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "k8sevents", 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_event_retention_days": { 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7", 2024-09-12T02:12:25.535 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "config_check_secs": { 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_check_secs", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.536 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "10", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.537 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.539 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.540 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.541 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "localpool", 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "failure_domain": { 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "failure_domain", 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "host", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.544 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.546 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.547 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "min_size": { 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_size", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "num_rep": { 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "num_rep", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.551 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_num": { 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_num", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "128", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.552 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "prefix": { 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prefix", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:12:25.553 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "subtree": { 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subtree", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "rack", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.554 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mds_autoscaler", 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.555 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.556 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.557 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.558 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.560 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirroring", 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.561 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.563 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.564 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.565 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "nfs", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.567 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.568 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.569 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.570 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.571 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.572 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "fail_fs": { 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "fail_fs", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.573 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.574 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.576 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.577 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.578 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator": { 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "rook", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "test_orchestrator" 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Orchestrator backend", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_perf_query", 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.581 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.582 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.583 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.584 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.585 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_support", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.587 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.588 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.589 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.590 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.591 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_autoscaler", 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.592 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.593 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.594 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.596 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.597 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.598 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "threshold": { 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threshold", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1.0", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.599 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "scaling threshold", 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "progress", 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.600 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_pg_recovery_event": { 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.601 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.603 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.604 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.606 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.607 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "max_completed_events": { 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_completed_events", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "50", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:12:25.608 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.609 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus", 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "cache": { 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cache", 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.610 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "exclude_perf_counters": { 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "exclude_perf_counters", 2024-09-12T02:12:25.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.613 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.614 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.615 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.616 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.617 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools": { 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "300", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.619 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_interval": { 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_interval", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15.0", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.620 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.621 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "9283", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.622 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "stale_cache_strategy": { 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stale_cache_strategy", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "log", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.623 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "default", 2024-09-12T02:12:25.624 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "default", 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "error" 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-09-12T02:12:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.627 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.628 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.629 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.630 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.631 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_snap_create": { 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:12:25.632 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.633 INFO:teuthology.orchestra.run.smithi122.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.634 INFO:teuthology.orchestra.run.smithi122.stdout: "trash_purge_schedule": { 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "trash_purge_schedule", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "restful", 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_auth": { 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_auth", 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.636 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.637 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.639 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.640 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.641 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.642 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-09-12T02:12:25.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-09-12T02:12:25.644 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rgw", 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.646 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.647 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.648 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.649 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.650 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rook", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.652 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.653 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.654 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.655 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.656 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "storage_class": { 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "storage_class", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.657 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "local", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "selftest", 2024-09-12T02:12:25.658 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.660 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "roption1": { 2024-09-12T02:12:25.663 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption1", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "roption2": { 2024-09-12T02:12:25.664 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption2", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "xyz", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption1": { 2024-09-12T02:12:25.665 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption1", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption2": { 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption2", 2024-09-12T02:12:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption3": { 2024-09-12T02:12:25.667 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption3", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption4": { 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption4", 2024-09-12T02:12:25.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption5": { 2024-09-12T02:12:25.669 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption5", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption6": { 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption6", 2024-09-12T02:12:25.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption7": { 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption7", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.671 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "42", 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "testkey": { 2024-09-12T02:12:25.672 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testkey", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.673 INFO:teuthology.orchestra.run.smithi122.stdout: "testlkey": { 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testlkey", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.674 INFO:teuthology.orchestra.run.smithi122.stdout: "testnewline": { 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testnewline", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_m_granularity": { 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_m_granularity", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.676 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "dump_on_update": { 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dump_on_update", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.677 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.678 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.680 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.681 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.682 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stats", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.684 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.685 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.687 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.688 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "status", 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.689 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.690 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.692 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.693 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.694 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telegraf", 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "address": { 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "address", 2024-09-12T02:12:25.695 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.696 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.697 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.698 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.699 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.700 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.701 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.702 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telemetry", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_basic": { 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_basic", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.703 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_crash": { 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_crash", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.704 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_device": { 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_device", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.705 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_ident": { 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_ident", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.706 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_perf": { 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_perf", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.707 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "contact": { 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "contact", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.708 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "description": { 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "description", 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.709 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "device_url": { 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_url", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.710 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.712 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "24", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "8", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "last_opt_revision": { 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "last_opt_revision", 2024-09-12T02:12:25.713 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard": { 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard", 2024-09-12T02:12:25.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard_description": { 2024-09-12T02:12:25.715 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard_description", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "organization": { 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "organization", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "proxy": { 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "proxy", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "url": { 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:12:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "test_orchestrator", 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_clones": { 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_clones", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:12:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "periodic_async_work": { 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "periodic_async_work", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Periodically check for async work", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_delay": { 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-09-12T02:12:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix", 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-09-12T02:12:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "discovery_interval": { 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "discovery_interval", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "100", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "identifier": { 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "identifier", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-09-12T02:12:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-09-12T02:12:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-09-12T02:12:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_host": { 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_host", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_port": { 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_port", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10051", 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_sender": { 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_sender", 2024-09-12T02:12:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "services": { 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "dashboard": "https://172.21.15.122:8443/" 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "always_on_modules": { 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "octopus": [ 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-09-12T02:12:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "pacific": [ 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-09-12T02:12:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "quincy": [ 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-09-12T02:12:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "reef": [ 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-09-12T02:12:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "squid": [ 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "last_failure_osd_epoch": 3, 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "active_clients": [ 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3110672081 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "libcephsqlite", 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3549926238 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 2349071867 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 730445688 2024-09-12T02:12:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-12T02:12:25.762 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:12:25.763 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:24 smithi151 ceph-mon[35229]: pgmap v1117: 97 pgs: 97 active+clean; 325 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-09-12T02:12:25.763 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:24 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/462698980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:12:25.763 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:24 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/708899193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:12:25.763 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:24 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/340362494' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:12:25.764 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:24 smithi183 ceph-mon[35127]: pgmap v1117: 97 pgs: 97 active+clean; 325 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-09-12T02:12:25.764 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:24 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/462698980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:12:25.764 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:24 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/708899193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:12:25.764 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:24 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/340362494' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98240a2b90 msgr2=0x7f98240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98240a2b90 0x7f98240a5050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9830006010 tx=0x7f9830000bc0 comp rx=0 tx=0).stop 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 msgr2=0x7f983c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 0x7f983c1983a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f982c00d950 tx=0x7f982c00de10 comp rx=0 tx=0).stop 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 shutdown_connections 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f98240a2b90 0x7f98240a5050 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f983c1036e0 0x7f983c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f983c102d20 0x7f983c1983a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 --2- 172.21.15.122:0/340362494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f983c101b30 0x7f983c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 >> 172.21.15.122:0/340362494 conn(0x7f983c0fd2b0 msgr2=0x7f983c106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 shutdown_connections 2024-09-12T02:12:25.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:24.327+0000 7f9840f2c640 1 -- 172.21.15.122:0/340362494 wait complete. 2024-09-12T02:12:25.825 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T02:12:25.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T02:12:25.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:24 smithi122 ceph-mon[30925]: pgmap v1117: 97 pgs: 97 active+clean; 325 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-09-12T02:12:25.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:24 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/462698980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:12:25.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:24 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/708899193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:12:25.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:24 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/340362494' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:12:26.091 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 451, 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 446, 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 820320, 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 276112, 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:26.568 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.054092100, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.026750359, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.031522452, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 445, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.019411264, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000043620 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:26.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 820320, 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 276112 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 6, 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 343, 2024-09-12T02:12:26.570 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3344, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 217904, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.024323530, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.018210742, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.003267230, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 342, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.018862564, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000055153 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:26.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 217904 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-09-12T02:12:26.572 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000281941, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000203778, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:26.573 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:26.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:26.575 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:26.576 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148744, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 30, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:26.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:26.578 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 453, 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 756567, 2024-09-12T02:12:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 453, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 453, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 756567, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:26.580 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:12:26.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:26 smithi122 ceph-mon[30925]: pgmap v1118: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-09-12T02:12:26.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:26 smithi151 ceph-mon[35229]: pgmap v1118: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-09-12T02:12:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:26 smithi183 ceph-mon[35127]: pgmap v1118: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 599 op/s 2024-09-12T02:12:27.243 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T02:12:27.243 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T02:12:27.243 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T02:12:27.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T02:12:27.486 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:27.957 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:12:27.957 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 4, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 4, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 448, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.003026350, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000035787, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.002184690, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:27.958 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 3, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.003967614, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001322538 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:12:27.959 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:12:27.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.960 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:27.960 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 457, 2024-09-12T02:12:27.960 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 452, 2024-09-12T02:12:27.960 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 821952, 2024-09-12T02:12:27.960 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 279936, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.053677909, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.026317408, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.032733747, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:27.961 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 451, 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.019661601, 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000043595 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.962 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 821952, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 279936 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 345, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 218160, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.022985328, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.018437472, 2024-09-12T02:12:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001356086, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 343, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.015761766, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000045952 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:12:27.965 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 218160 2024-09-12T02:12:27.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.965 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-12T02:12:27.965 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-09-12T02:12:27.965 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-12T02:12:27.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.966 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-12T02:12:27.966 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:27.966 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:27.966 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:27.966 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:27.966 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:27.967 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:27.968 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:27.969 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148721, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 30, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-09-12T02:12:27.970 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:27.971 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-12T02:12:27.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:27.972 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:27.972 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-12T02:12:27.972 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-12T02:12:27.972 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-12T02:12:27.972 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 459, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 757347, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 459, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 459, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 757347, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:27.973 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:27.974 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:27.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:27.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:27.974 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:12:28.656 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T02:12:28.656 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T02:12:28.656 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T02:12:28.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T02:12:28.899 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:28 smithi183 ceph-mon[35127]: pgmap v1119: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 512 op/s 2024-09-12T02:12:28.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:28 smithi122 ceph-mon[30925]: pgmap v1119: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 512 op/s 2024-09-12T02:12:29.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:28 smithi151 ceph-mon[35229]: pgmap v1119: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 512 op/s 2024-09-12T02:12:29.415 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:12:29.415 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:29.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 453, 2024-09-12T02:12:29.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 448, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 820864, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 277392, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.055656467, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.027691622, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.031999762, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 447, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.025067803, 2024-09-12T02:12:29.416 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000056080 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 820864, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 277392 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 6, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 344, 2024-09-12T02:12:29.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3344, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 218080, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.025367527, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.018588237, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.003808757, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 343, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.019704627, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000057447 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.418 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 218080 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-09-12T02:12:29.419 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000424014, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000345241, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:29.420 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:29.421 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:29.422 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:29.423 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148721, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 30, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-12T02:12:29.424 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:29.425 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 455, 2024-09-12T02:12:29.426 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 756827, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 455, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 455, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 756827, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:29.427 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:29.428 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:29.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:29.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:29.428 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:12:30.110 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T02:12:30.110 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T02:12:30.110 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T02:12:30.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T02:12:30.354 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:30.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:30 smithi122 ceph-mon[30925]: pgmap v1120: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 9.0 MiB/s wr, 438 op/s 2024-09-12T02:12:30.836 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-12T02:12:30.836 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:30.836 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 458, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 453, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 822224, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 280592, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.055522217, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.026768412, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.033391250, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 452, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.019510393, 2024-09-12T02:12:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000043164 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 822224, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 280592 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 6, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 348, 2024-09-12T02:12:30.838 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3344, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 218784, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.025276457, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.018938961, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.003399923, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 347, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.022571873, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000065048 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.839 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 218784 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-09-12T02:12:30.840 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000290042, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000206979, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:30.841 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:30.842 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:30.843 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:30.844 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148704, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 29, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-12T02:12:30.845 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:30.846 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 460, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 757477, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 460, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-12T02:12:30.847 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 460, 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 757477, 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-12T02:12:30.848 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-12T02:12:30.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:30 smithi183 ceph-mon[35127]: pgmap v1120: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 9.0 MiB/s wr, 438 op/s 2024-09-12T02:12:31.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:30 smithi151 ceph-mon[35229]: pgmap v1120: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 9.0 MiB/s wr, 438 op/s 2024-09-12T02:12:31.511 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T02:12:31.511 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T02:12:31.512 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T02:12:31.512 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T02:12:31.839 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:12:32.635 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-12T02:12:32.636 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:32.636 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 311596, 2024-09-12T02:12:32.636 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 225917, 2024-09-12T02:12:32.636 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 82823558, 2024-09-12T02:12:32.637 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 170179204, 2024-09-12T02:12:32.637 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 329, 2024-09-12T02:12:32.637 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:32.637 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 17.422275685, 2024-09-12T02:12:32.637 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 7.614651547, 2024-09-12T02:12:32.638 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 10.814335623, 2024-09-12T02:12:32.638 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 1.330120690, 2024-09-12T02:12:32.638 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:32.638 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 225916, 2024-09-12T02:12:32.638 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 18.433402312, 2024-09-12T02:12:32.639 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000081594 2024-09-12T02:12:32.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.639 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:32.639 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 472008, 2024-09-12T02:12:32.639 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.064529443, 2024-09-12T02:12:32.639 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000136 2024-09-12T02:12:32.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.640 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1011488, 2024-09-12T02:12:32.640 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 291408 2024-09-12T02:12:32.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.641 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:32.641 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 361434, 2024-09-12T02:12:32.641 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 521218, 2024-09-12T02:12:32.642 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 114523859, 2024-09-12T02:12:32.642 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 512796499, 2024-09-12T02:12:32.642 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 50, 2024-09-12T02:12:32.642 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 6, 2024-09-12T02:12:32.642 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 22.635916543, 2024-09-12T02:12:32.643 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 9.015354476, 2024-09-12T02:12:32.643 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 13.738099668, 2024-09-12T02:12:32.643 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 2.345786221, 2024-09-12T02:12:32.643 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:32.643 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 521213, 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 53.761483035, 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000103146 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 753271, 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.140094432, 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000185 2024-09-12T02:12:32.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 56512, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 813568 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 49024, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 61441, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 11905672, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 105453523, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 10, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 4.272634008, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 1.823603325, 2024-09-12T02:12:32.645 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 1.294968980, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 1.059086845, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 61443, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 27.716346713, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000451090 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 48505, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.009287188, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000191 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:12:32.646 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 92954, 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 104.275449869, 2024-09-12T02:12:32.647 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001121796 2024-09-12T02:12:32.648 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.648 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-12T02:12:32.648 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:32.648 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 50681, 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 5.808497544, 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000114608 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 234413, 2024-09-12T02:12:32.649 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 189617, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 189617, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 828.639522041, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.004370069 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 44796, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 5, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 2, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 2795, 2024-09-12T02:12:32.650 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 19, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 25, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 1221, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 382, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 171, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 668, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 675, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 444, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 3, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 3, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 4, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 357623, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 168915, 2024-09-12T02:12:32.651 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 44795, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 21, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 2960, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 8, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 308, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 7, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 108, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 49, 2024-09-12T02:12:32.652 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 49, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 33, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 4, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 29, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 16, 2024-09-12T02:12:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 4, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 15326, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 131717, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 64938, 2024-09-12T02:12:32.654 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 25013, 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 11232, 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 127752 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 13, 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 21, 2024-09-12T02:12:32.655 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 14, 2024-09-12T02:12:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 28, 2024-09-12T02:12:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 28, 2024-09-12T02:12:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-12T02:12:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-12T02:12:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:12:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 31458, 2024-09-12T02:12:32.657 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 31458, 2024-09-12T02:12:32.657 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 22, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 1842, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 4, 2024-09-12T02:12:32.660 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 8, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 44, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 158365, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 157030, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 157030, 2024-09-12T02:12:32.661 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 1335, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 3316, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 294, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 292, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 292, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 1, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 5, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 307293958, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 309065584, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-09-12T02:12:32.662 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-12T02:12:32.663 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 47627, 2024-09-12T02:12:32.663 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 271.117083450, 2024-09-12T02:12:32.663 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.005692508 2024-09-12T02:12:32.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.663 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-09-12T02:12:32.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 32, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 31636, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 31604, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 44, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 261, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 217, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 1346, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 68476, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 67130, 2024-09-12T02:12:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 3, 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 35834, 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 35831, 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 84204, 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 231712 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 311029, 2024-09-12T02:12:32.665 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 49, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 234413, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 4512, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 12407, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 31440, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 249.017317634, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.007920398 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.666 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 337, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.729227356, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002163879 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 68569, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 78.232236848, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001140927 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-09-12T02:12:32.668 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000247159, 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000123579 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.669 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.670 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 20, 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.019162668, 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000958133 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-12T02:12:32.671 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000222974, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000222974 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 50395, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 204.190676951, 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.004051804 2024-09-12T02:12:32.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 6477, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 79.464375512, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.012268700 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.673 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 23, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2.981794320, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.129643231 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.674 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 63, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 78.464066116, 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 1.245461366 2024-09-12T02:12:32.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-12T02:12:32.676 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 836, 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.660689917, 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000790298 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.677 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.678 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-12T02:12:32.678 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 8, 2024-09-12T02:12:32.678 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.018608514, 2024-09-12T02:12:32.678 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002326064 2024-09-12T02:12:32.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-12T02:12:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 31446, 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 134.860896072, 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.004288650 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.680 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 3, 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-09-12T02:12:32.681 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 19789, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 19789, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 1572, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:32.682 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:32.683 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:32.683 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:32.683 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:32.683 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:32.683 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:32.684 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:32.684 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:32.684 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:32.684 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:32.685 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:32.686 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 295524, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 104, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:32.687 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7592, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 156, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 1146774, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13981, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-12T02:12:32.688 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 101781, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 330938576, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 37, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 101781, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 101781, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2153.270365849, 2024-09-12T02:12:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.021155916 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 101781, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 140270 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 101781, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 945, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 100836, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 2571, 2024-09-12T02:12:32.690 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 15638, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 885, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 48508, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 875, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 19, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 32262, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 60, 2024-09-12T02:12:32.691 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 31276, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-12T02:12:32.692 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-12T02:12:32.693 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 88, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 7, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 12, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 12, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 6353, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 14, 2024-09-12T02:12:32.694 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1809 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 1876 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T02:12:32.695 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 10, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 31458, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 31458, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:32.696 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 722050, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 144227824, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 722050, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 722050, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 144227824, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.697 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 101781, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 334115746, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 101781, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 334115746, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.699 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 101781, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 101781, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 101781, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 101781, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.700 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 31458, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 3177258, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 31458, 2024-09-12T02:12:32.701 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 881, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 3177258, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x563e2c78e960": { 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:32.702 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 158365, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 304871280, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 158365, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 47627, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 304871280, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:32.703 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.704 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:32.704 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-12T02:12:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:32 smithi183 ceph-mon[35127]: pgmap v1121: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 4.3 MiB/s wr, 246 op/s 2024-09-12T02:12:32.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:32 smithi122 ceph-mon[30925]: pgmap v1121: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 4.3 MiB/s wr, 246 op/s 2024-09-12T02:12:33.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:32 smithi151 ceph-mon[35229]: pgmap v1121: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 4.3 MiB/s wr, 246 op/s 2024-09-12T02:12:34.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:34 smithi183 ceph-mon[35127]: pgmap v1122: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-12T02:12:34.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:34 smithi122 ceph-mon[30925]: pgmap v1122: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-12T02:12:35.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:34 smithi151 ceph-mon[35229]: pgmap v1122: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-09-12T02:12:35.507 INFO:tasks.check_counter:Daemon mds.b mds.exported=8 2024-09-12T02:12:35.508 INFO:tasks.check_counter:Daemon mds.b mds.imported=7 2024-09-12T02:12:35.508 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T02:12:35.508 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T02:12:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 3482, 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 3484, 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 30380601, 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1107283, 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-09-12T02:12:36.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.276603772, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.088314706, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.084172172, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.082003254, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3482, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.113759661, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000032670 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1197, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1539, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 25468752, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1068042, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:36.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.114210365, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.041191062, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.035813296, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.028208066, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1537, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.047975280, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000031213 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 688288 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1642, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1609, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 24150539, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 646544, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.150120901, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.049242081, 2024-09-12T02:12:36.497 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.076133707, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.028163619, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1609, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.065031686, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000040417 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 950800, 2024-09-12T02:12:36.498 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 282752 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 10035, 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.202644538, 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000020193 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-12T02:12:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 12043, 2024-09-12T02:12:36.501 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 12043, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 12043, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:12:36.502 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-09-12T02:12:36.503 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:12:36.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-12T02:12:36.505 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:12:36.506 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 63248, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-12T02:12:36.507 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 77, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 7, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 81630926, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 81629788, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.508 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 63248 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 13, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 12066, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 12053, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 11, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 32, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 21, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 12043, 2024-09-12T02:12:36.509 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 12145, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 102, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 64640, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215328 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-09-12T02:12:36.510 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-12T02:12:36.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-12T02:12:36.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-12T02:12:36.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 1, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 0, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2798, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.523 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:36.524 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:36.525 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 233760, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 82, 2024-09-12T02:12:36.526 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7592, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 156, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:36.527 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8521864, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 48300, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 5820, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-09-12T02:12:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 5820, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 5820, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 7.586898376, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001303590 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 5820, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 5821 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 5820, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 5820, 2024-09-12T02:12:36.529 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 3136, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 2682, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-12T02:12:36.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-12T02:12:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-12T02:12:36.532 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 88, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 5, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 10, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 11, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 1, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.533 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-12T02:12:36.534 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 6317, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 79403076, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 6317, 2024-09-12T02:12:36.535 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 6317, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 79403076, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:36.536 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 5820, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 77437444, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 5820, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 77437444, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 5820, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 5820, 2024-09-12T02:12:36.538 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 5820, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 5820, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55c15611eb40": { 2024-09-12T02:12:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:36.541 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:36.542 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:36.542 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.542 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:36.542 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-12T02:12:36.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:36 smithi151 ceph-mon[35229]: pgmap v1123: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-12T02:12:36.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:36 smithi183 ceph-mon[35127]: pgmap v1123: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-12T02:12:36.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:36 smithi122 ceph-mon[30925]: pgmap v1123: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-12T02:12:37.457 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T02:12:37.457 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T02:12:37.457 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T02:12:37.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T02:12:37.711 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 975826, 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1448977, 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 309096049, 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1561823818, 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 15, 2024-09-12T02:12:38.283 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 9, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 54.793342229, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 23.848993412, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 35.457133667, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 3.371461857, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1448974, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 60.303433835, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000041618 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2120788, 2024-09-12T02:12:38.284 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.379623133, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000179 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 964544, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 330928 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 270516, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 356412, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 58955509, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 416584425, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 22.122274276, 2024-09-12T02:12:38.285 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 10.183838000, 2024-09-12T02:12:38.286 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 9.780715764, 2024-09-12T02:12:38.286 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 3.205668802, 2024-09-12T02:12:38.286 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:38.286 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 356413, 2024-09-12T02:12:38.286 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 13.336412301, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000037418 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 472013, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.083269019, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000176 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 690240 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 275109, 2024-09-12T02:12:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 333704, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 65526582, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 471225812, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 18, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 6, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 20.564225021, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 8.150946939, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 8.379740166, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 4.461018169, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 333702, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 14.843705739, 2024-09-12T02:12:38.288 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000044481 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 396148, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.075270180, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000190 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-12T02:12:38.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 243561, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 368.045073789, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001511100 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 152104, 2024-09-12T02:12:38.290 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 42.228361170, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000277628 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 634791, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 618148, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 618148, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 5781.341575630, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.009352681 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.291 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 16643, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 672, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 55, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 65, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 7638, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 4679, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 2788, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 171, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 948, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 142, 2024-09-12T02:12:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 3, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 952131, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 561765, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 16642, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 5, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 7407, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 11, 2024-09-12T02:12:38.293 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 120, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 12, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 422, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 78, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 77, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 6026335, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 78, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:12:38.294 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 11, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 11, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 67, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 86, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 54998, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 384143, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 212998, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 82276, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 38336, 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 373856 2024-09-12T02:12:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 17, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 9, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 6, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 11, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 55, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 14, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 97707, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 97693, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-12T02:12:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 22, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 7112, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 11, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 120, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-12T02:12:38.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 492557, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 385602, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 380991, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 111566, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 4611, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 584, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 456, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 450, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 134, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 10, 2024-09-12T02:12:38.298 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 6, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 808378504, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 1042952018, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 147196, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1349.296631687, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.009166666 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-12T02:12:38.299 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 33, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 97838, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 97805, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 75, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 641, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 566, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 7638, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 202932, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 195294, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 108221, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 108221, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 124756, 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223520 2024-09-12T02:12:38.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 806020, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 39364, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 634791, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 13513, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 78763, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 97637, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1537.827274288, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.015750456 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 28281, 2024-09-12T02:12:38.301 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 599.861739353, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.021210768 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.302 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 219480, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 948.020691758, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.004319394 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-12T02:12:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 99, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.551294544, 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.005568631 2024-09-12T02:12:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 9, 2024-09-12T02:12:38.306 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.005531627, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000614625 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 152526, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1335.641703374, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.008756813 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 20113, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 284.443582351, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.014142275 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.307 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 101, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.652866905, 2024-09-12T02:12:38.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.006464028 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 304, 2024-09-12T02:12:38.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 85.202383747, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.280270999 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1992, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 16.013115597, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.008038712 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.310 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 97606, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 973.121392086, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.009969893 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 3, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 41555, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 41555, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 1488, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-12T02:12:38.313 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:38.314 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:38.315 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 289510, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 125, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 264, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 3, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-12T02:12:38.316 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7592, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 156, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 5981950, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 84636, 2024-09-12T02:12:38.317 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 254215, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 1065278117, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 29, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 254215, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 254215, 2024-09-12T02:12:38.318 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1658.342870941, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.006523387 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 254215, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 258334 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 254215, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 2499, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 251716, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-12T02:12:38.319 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 76, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 577, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 2421, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 149617, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 1130, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 54, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 98557, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 78, 2024-09-12T02:12:38.320 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 1154, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-12T02:12:38.321 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-12T02:12:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 88, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 6, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 12, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 12, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 3430, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1240 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:12:38.323 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 7205 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 10, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 97693, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 97693, 2024-09-12T02:12:38.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1521447, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 296617543, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1521447, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1521447, 2024-09-12T02:12:38.325 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 296617543, 2024-09-12T02:12:38.326 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:38.326 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.326 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.326 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.326 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 254215, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1075145110, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 254215, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1075145110, 2024-09-12T02:12:38.327 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:38.328 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 254215, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 254215, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 254215, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 254215, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:38.329 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 97693, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 9866993, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 97693, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2421, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 9866993, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.330 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x56004e552960": { 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 492557, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1038757714, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 492557, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 147196, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1038757714, 2024-09-12T02:12:38.331 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:38.332 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:38.332 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:38.332 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:38.332 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.332 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:38.332 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-12T02:12:38.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:38 smithi183 ceph-mon[35127]: pgmap v1124: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T02:12:38.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:38 smithi122 ceph-mon[30925]: pgmap v1124: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T02:12:39.066 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:38 smithi151 ceph-mon[35229]: pgmap v1124: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T02:12:39.095 INFO:tasks.check_counter:Daemon mds.h mds.exported=11 2024-09-12T02:12:39.095 INFO:tasks.check_counter:Daemon mds.h mds.imported=12 2024-09-12T02:12:39.095 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T02:12:39.095 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T02:12:39.343 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:12:39.913 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-12T02:12:39.913 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:39.913 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 425, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 425, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 12721653, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 134345, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.048844476, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.013564929, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.019620113, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.011236409, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 424, 2024-09-12T02:12:39.914 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.030988617, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000073086 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:39.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 532, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 526, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 49216635, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 193991, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.073930536, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.012813371, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.051647480, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.010340682, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 525, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.028896200, 2024-09-12T02:12:39.916 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000055040 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 202368, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 58784 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 483, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 555, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 58844946, 2024-09-12T02:12:39.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 367069, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.081730987, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.011600126, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.054140179, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.012264350, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 553, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.021939710, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000039673 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 213792 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2219, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.158317634, 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000071346 2024-09-12T02:12:39.919 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-12T02:12:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 13194, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 8726, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 4468, 2024-09-12T02:12:39.921 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 727, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 2, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-12T02:12:39.922 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:12:39.923 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.924 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-12T02:12:39.925 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.926 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 63507, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 25, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-12T02:12:39.927 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 307293958, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 309065584, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 309061283, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 63507 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 17, 2024-09-12T02:12:39.928 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 13158, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 13141, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 26, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 117, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 91, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 13194, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 13459, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 265, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 61448, 2024-09-12T02:12:39.929 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215328 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.930 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-12T02:12:39.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-12T02:12:39.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.936 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-12T02:12:39.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.940 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.941 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 1, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 0, 2024-09-12T02:12:39.942 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2806, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:39.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:39.944 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 221559, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 64, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-12T02:12:39.946 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 7592, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 156, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 9361229, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 52969, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:39.947 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 1328, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 26, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 4, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 1328, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1328, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 3.423955115, 2024-09-12T02:12:39.948 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002578279 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1328, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1329 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 1328, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 1328, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 701, 2024-09-12T02:12:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 624, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 2, 2024-09-12T02:12:39.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-12T02:12:39.951 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-12T02:12:39.952 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 9, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 9, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:12:39.953 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-12T02:12:39.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1436, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 120645495, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1436, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1436, 2024-09-12T02:12:39.955 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 120645495, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:39.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 1328, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 120205442, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1328, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 120205442, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-12T02:12:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 1328, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1328, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1328, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1328, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:39.959 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x5570b7cb46e0": { 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:39.960 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-12T02:12:39.961 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-12T02:12:40.631 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-12T02:12:40.631 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-12T02:12:40.631 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T02:12:40.632 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T02:12:40.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:40 smithi151 ceph-mon[35229]: pgmap v1125: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:12:40.895 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:12:40.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:40 smithi183 ceph-mon[35127]: pgmap v1125: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:12:40.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:40 smithi122 ceph-mon[30925]: pgmap v1125: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 42473, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 30058, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 11120520, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 36819408, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 2.141514217, 2024-09-12T02:12:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.792988443, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 1.305500529, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.283337450, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 30056, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.623284410, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000020737 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 48586, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.006497984, 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000133 2024-09-12T02:12:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 137102, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 189534, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 42384475, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 137193136, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 19, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 7.672891474, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 3.181577312, 2024-09-12T02:12:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 5.117643417, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.293503012, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 189532, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.529766905, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000018623 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 289871, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.043084739, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000148 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:12:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 689968 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 240214, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 181590, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 66653245, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 66487014, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 10.994568332, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 4.254525599, 2024-09-12T02:12:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 8.500750706, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.331168623, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 181590, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.028281502, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000016676 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 396766, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.049873116, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000125 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 968416, 2024-09-12T02:12:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 289264 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 34731, 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 18.604647853, 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000535678 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21565, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.974978788, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000045211 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 96737, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 66668, 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-12T02:12:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 66668, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 413.974423360, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006209492 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 30069, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-09-12T02:12:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 16, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 1, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 5, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 10, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 14, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 145, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-09-12T02:12:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 190841, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 77061, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 30069, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 25, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 2873, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 5, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 128, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 5, 2024-09-12T02:12:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 26, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 10, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 77, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 6026335, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 10, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:12:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 10, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 2, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 900, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 43519, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 19379, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 4721, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 2491, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 66438 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 59, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 40, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 52, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 9, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 12033, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 12033, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-12T02:12:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 22, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 831, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 5, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-12T02:12:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 63248, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 63248, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 77, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 77, 2024-09-12T02:12:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 7, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 81630926, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16941, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 52.996434390, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003128294 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-12T02:12:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 26, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 12098, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 12072, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 32, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 77, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 45, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 77, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 15815, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 15738, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 12398, 2024-09-12T02:12:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 12398, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 68280, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223520 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 164124, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 5, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 96737, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 2140, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 27081, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 12012, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 185.848810835, 2024-09-12T02:12:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.015471928 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 158, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.107233001, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000678689 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-12T02:12:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16149, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 12.879007291, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000797511 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-12T02:12:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-12T02:12:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21747, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 11.398814400, 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000524155 2024-09-12T02:12:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2873, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 55.403211067, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.019284097 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-12T02:12:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1696, 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.500701370, 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000295224 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-12T02:12:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 12033, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 147.836645396, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.012285934 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 4125, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 4125, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1490, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 287906, 2024-09-12T02:12:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 107, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7592, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 156, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 209924, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2296, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 32381, 2024-09-12T02:12:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 79886888, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 32, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 32381, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 32381, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 122.247816788, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003775294 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 32381, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 33840 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 32381, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 887, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 31494, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 877, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 17818, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 546, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-12T02:12:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 12063, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 10, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-12T02:12:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-12T02:12:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 88, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-12T02:12:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 7, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 1986, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 510 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:12:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 863 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T02:12:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 10, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 12033, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 12033, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 419785, 2024-09-12T02:12:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 82347564, 2024-09-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 419785, 2024-09-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 419785, 2024-09-12T02:12:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 82347564, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 32381, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 81102221, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 32381, 2024-09-12T02:12:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 81102221, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T02:12:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 32381, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 32381, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 32381, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 32381, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 12033, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1215333, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 12033, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 877, 2024-09-12T02:12:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1215333, 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5618e5f6c960": { 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 63248, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 77436622, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 63248, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 16941, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 77436622, 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:41.746 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T02:12:42.769 INFO:tasks.check_counter:Daemon mds.c mds.exported=5 2024-09-12T02:12:42.769 INFO:tasks.check_counter:Daemon mds.c mds.imported=5 2024-09-12T02:12:42.769 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T02:12:42.769 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T02:12:42.899 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:42 smithi183 ceph-mon[35127]: pgmap v1126: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T02:12:42.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:42 smithi122 ceph-mon[30925]: pgmap v1126: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T02:12:43.027 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:12:43.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:42 smithi151 ceph-mon[35229]: pgmap v1126: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T02:12:43.562 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T02:12:43.562 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:43.562 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 101, 2024-09-12T02:12:43.562 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 100, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 169200, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 60688, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.012437633, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.004668730, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.008288931, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 99, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.006385814, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000064503 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 169200, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 60688 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 76, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 160, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 91920, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2024-09-12T02:12:43.564 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.004724584, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.002869465, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001149136, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 74, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004865040, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000065743 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:43.565 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 91920 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 448, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.002757516, 2024-09-12T02:12:43.566 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000040046, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002718865, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004736146, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001578715 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:12:43.567 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:43.568 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:43.569 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:43.570 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 150885, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 29, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-09-12T02:12:43.571 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:43.572 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 102, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 154970, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 102, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 102, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 154970, 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:43.573 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:43.574 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:43.574 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:43.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:43.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:43.574 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T02:12:44.261 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-09-12T02:12:44.261 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-09-12T02:12:44.261 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T02:12:44.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T02:12:44.510 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:12:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:44 smithi183 ceph-mon[35127]: pgmap v1127: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:12:44.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:44 smithi122 ceph-mon[30925]: pgmap v1127: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 457, 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 452, 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 821952, 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 279952, 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:45.100 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.055296417, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.023124885, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.037860677, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 451, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.022714630, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000050365 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:45.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 821952, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 279952 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 346, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 320, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 218336, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.018719621, 2024-09-12T02:12:45.102 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.014025268, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001532276, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 344, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.017566434, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000051065 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:45.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 218336 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 448, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-12T02:12:45.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.002413663, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000037303, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002350161, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004160649, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001386883 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:45.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:45.106 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:45.107 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:45.108 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 149775, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 31, 2024-09-12T02:12:45.109 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T02:12:45.111 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 459, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 757347, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 459, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 459, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 757347, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:45.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:45.113 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:45.113 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:45.113 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T02:12:45.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:44 smithi151 ceph-mon[35229]: pgmap v1127: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T02:12:45.804 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.i 2024-09-12T02:12:45.805 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.i 2024-09-12T02:12:45.805 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T02:12:45.805 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T02:12:46.049 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1821, 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1790, 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 289265717, 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 705906, 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-12T02:12:46.608 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.296985331, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.053591241, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.214959445, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.028190879, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1789, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.055700280, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000031134 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.609 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 949136, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 285504 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1725, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2074, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 319214732, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1240637, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-12T02:12:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.282646842, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.051747100, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.180150928, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.037351630, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2072, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.063334761, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000030566 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.611 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 692400 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3826, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3826, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 433130177, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1215902, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-12T02:12:46.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.510036448, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.089686773, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.297169163, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.096685377, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3825, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.114298851, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000029882 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.613 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T02:12:46.614 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11152, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.382210657, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000034272 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.615 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-12T02:12:46.616 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 98014, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 52892, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 45122, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 22454, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-12T02:12:46.617 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-12T02:12:46.618 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:12:46.619 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:12:46.620 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-12T02:12:46.621 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-12T02:12:46.622 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 492557, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-12T02:12:46.623 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 134, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 42, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 808378504, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 1042952018, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 1042951419, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 492557 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 27, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 97836, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 97809, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 66, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 636, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 570, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 98014, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 99280, 2024-09-12T02:12:46.625 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1266, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 139896, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215328 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-12T02:12:46.626 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.627 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.628 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.629 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-12T02:12:46.630 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.631 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.631 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.631 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-12T02:12:46.632 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.634 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.635 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.636 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.637 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-12T02:12:46.639 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2819, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T02:12:46.640 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:12:46.641 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:12:46.642 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 240733, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 94, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T02:12:46.643 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7592, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 156, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 69153561, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 393705, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T02:12:46.644 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 6868, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 6868, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6868, 2024-09-12T02:12:46.645 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 11.398889603, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001659710 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6868, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 6869 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 6868, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 6868, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-12T02:12:46.646 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 3869, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 2997, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-12T02:12:46.647 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-12T02:12:46.648 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-12T02:12:46.649 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 88, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 5, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 11, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 11, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-12T02:12:46.650 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-12T02:12:46.651 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 7368, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1040918960, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 7368, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:46.652 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 7368, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1040918960, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T02:12:46.653 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 6868, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 1038769458, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 6868, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1038769458, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 6868, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 6868, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 6868, 2024-09-12T02:12:46.655 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 6868, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T02:12:46.656 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55f835ba6b40": { 2024-09-12T02:12:46.657 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T02:12:46.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T02:12:46.659 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.659 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T02:12:46.659 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T02:12:46.926 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:46 smithi183 ceph-mon[35127]: pgmap v1128: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:12:46.985 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:46 smithi151 ceph-mon[35229]: pgmap v1128: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:12:46.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:46 smithi122 ceph-mon[30925]: pgmap v1128: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T02:12:47.457 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T02:12:47.457 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T02:12:47.458 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T02:12:47.468 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T02:12:47.468 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T02:12:47.468 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:12:47.489 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T02:12:47.489 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:12:47.548 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T02:12:47.857 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1723763713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0106f10 msgr2=0x7fa8f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1723763713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0106f10 0x7fa8f010ea90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009ca0 tx=0x7fa8e802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1723763713 shutdown_connections 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1723763713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0106f10 0x7fa8f010ea90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1723763713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f0102790 0x7fa8f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1723763713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f0101de0 0x7fa8f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1723763713 >> 172.21.15.122:0/1723763713 conn(0x7fa8f00fd8c0 msgr2=0x7fa8f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1723763713 shutdown_connections 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.303+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1723763713 wait complete. 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 Processor -- start 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 -- start start 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 0x7fa8f0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f0102790 0x7fa8f0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f0106f10 0x7fa8f019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0111fb0 con 0x7fa8f0101de0 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0111e30 con 0x7fa8f0106f10 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8f5d65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0112130 con 0x7fa8f0102790 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8ef7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 0x7fa8f0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8ef7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 0x7fa8f0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51588/0 (socket says 172.21.15.122:51588) 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8ef7fe640 1 -- 172.21.15.122:0/1036495489 learned_addr learned my addr 172.21.15.122:0/1036495489 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.304+0000 7fa8effff640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f0106f10 0x7fa8f019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8eeffd640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f0102790 0x7fa8f0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ef7fe640 1 -- 172.21.15.122:0/1036495489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f0102790 msgr2=0x7fa8f0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ef7fe640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f0102790 0x7fa8f0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ef7fe640 1 -- 172.21.15.122:0/1036495489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f0106f10 msgr2=0x7fa8f019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ef7fe640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f0106f10 0x7fa8f019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ef7fe640 1 -- 172.21.15.122:0/1036495489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f019d520 con 0x7fa8f0101de0 2024-09-12T02:12:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ef7fe640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 0x7fa8f0198570 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d770 tx=0x7fa8e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e0005500 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f019d810 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004e00 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.305+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f01a50f0 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.306+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f01021c0 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.307+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa8e0004fa0 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.308+0000 7fa8ecff9640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa8d80a2b80 0x7fa8d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.308+0000 7fa8eeffd640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa8d80a2b80 0x7fa8d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.308+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa8e00ceac0 con 0x7fa8f0101de0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.308+0000 7fa8eeffd640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa8d80a2b80 0x7fa8d80a5040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa8f01087b0 tx=0x7fa8e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.312+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8e009ba20 con 0x7fa8f0101de0 2024-09-12T02:12:48.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.488+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8f010a150 con 0x7fa8f0101de0 2024-09-12T02:12:48.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.489+0000 7fa8ecff9640 1 -- 172.21.15.122:0/1036495489 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+601 (secure 0 0 0) 0x7fa8e009b320 con 0x7fa8f0101de0 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/3688521366 2024-09-13T01:37:44.367349+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3407930740 2024-09-13T01:37:44.367349+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/3688521366 2024-09-13T01:37:44.367349+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1117597805 2024-09-13T01:37:44.367349+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/2717375533 2024-09-13T01:37:44.367349+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3107764322 2024-09-13T01:37:21.796085+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1950823209 2024-09-13T01:37:21.796085+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1950823209 2024-09-13T01:37:21.796085+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1058528799 2024-09-13T01:37:21.796085+0000 2024-09-12T02:12:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/472737560 2024-09-13T01:37:21.796085+0000 2024-09-12T02:12:48.491 INFO:teuthology.orchestra.run.smithi122.stderr:listed 10 entries 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa8d80a2b80 msgr2=0x7fa8d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa8d80a2b80 0x7fa8d80a5040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa8f01087b0 tx=0x7fa8e4008040 comp rx=0 tx=0).stop 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 msgr2=0x7fa8f0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 0x7fa8f0198570 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d770 tx=0x7fa8e000dc30 comp rx=0 tx=0).stop 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 shutdown_connections 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fa8d80a2b80 0x7fa8d80a5040 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f0106f10 0x7fa8f019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8f0102790 0x7fa8f0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 --2- 172.21.15.122:0/1036495489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f0101de0 0x7fa8f0198570 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:48.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 >> 172.21.15.122:0/1036495489 conn(0x7fa8f00fd8c0 msgr2=0x7fa8f00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:48.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 shutdown_connections 2024-09-12T02:12:48.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:48.491+0000 7fa8f5d65640 1 -- 172.21.15.122:0/1036495489 wait complete. 2024-09-12T02:12:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:48 smithi183 ceph-mon[35127]: pgmap v1129: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:12:48.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:48 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1036495489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:12:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:48 smithi122 ceph-mon[30925]: pgmap v1129: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:12:48.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:48 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1036495489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:12:49.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:48 smithi151 ceph-mon[35229]: pgmap v1129: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T02:12:49.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:48 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1036495489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:12:49.284 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T02:12:49.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T02:12:49.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T02:12:49.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.396 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T02:12:49.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.442 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-12T02:12:49.489 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T02:12:49.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:12:49.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-12T02:12:49.665 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T02:12:49.665 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-12T02:12:49.666 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:12:49.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055985,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497396k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18567 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248696k,nr_inodes=812174,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/f12d4f69ede93deb2359f71f78181cb27e92e655a624465c53c9e2d3f7072023/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/f12d4f69ede93deb2359f71f78181cb27e92e655a624465c53c9e2d3f7072023/diff,workdir=/var/lib/containers/storage/overlay/f12d4f69ede93deb2359f71f78181cb27e92e655a624465c53c9e2d3f7072023/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/59fa9bb2e369d1a0a1801834a1a719c82831094030362535fd0fbc050fbc65e0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/59fa9bb2e369d1a0a1801834a1a719c82831094030362535fd0fbc050fbc65e0/diff,workdir=/var/lib/containers/storage/overlay/59fa9bb2e369d1a0a1801834a1a719c82831094030362535fd0fbc050fbc65e0/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248696k,nr_inodes=812174,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/233d61dc792a090f4b01d66557a570a10442a46f7f0ecb6473e41f721072ab30/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/233d61dc792a090f4b01d66557a570a10442a46f7f0ecb6473e41f721072ab30/diff,workdir=/var/lib/containers/storage/overlay/233d61dc792a090f4b01d66557a570a10442a46f7f0ecb6473e41f721072ab30/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7085335c141458e827dcc4de1dd8e1e94452d9dffbcb16374bb221fa72848b1b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/7085335c141458e827dcc4de1dd8e1e94452d9dffbcb16374bb221fa72848b1b/diff,workdir=/var/lib/containers/storage/overlay/7085335c141458e827dcc4de1dd8e1e94452d9dffbcb16374bb221fa72848b1b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a2d4b5b8931cc35c54a3808ddb6f2e028a11b3762b00f8fc27a03845d31c7719/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/a2d4b5b8931cc35c54a3808ddb6f2e028a11b3762b00f8fc27a03845d31c7719/diff,workdir=/var/lib/containers/storage/overlay/a2d4b5b8931cc35c54a3808ddb6f2e028a11b3762b00f8fc27a03845d31c7719/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8416efa4df254a40a525b41cd7e02966a24aa66e9afd0e320e8715991b4f0865/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/8416efa4df254a40a525b41cd7e02966a24aa66e9afd0e320e8715991b4f0865/diff,workdir=/var/lib/containers/storage/overlay/8416efa4df254a40a525b41cd7e02966a24aa66e9afd0e320e8715991b4f0865/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3a09016cec67ecdbc4f3ddf2442124e6cbbe604b07e0b99d62d1d2c392df88ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/3a09016cec67ecdbc4f3ddf2442124e6cbbe604b07e0b99d62d1d2c392df88ca/diff,workdir=/var/lib/containers/storage/overlay/3a09016cec67ecdbc4f3ddf2442124e6cbbe604b07e0b99d62d1d2c392df88ca/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b83f4be483bc83423ba02f6f634f63fb777a334a62c1f0fea7f3377299267b99/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/b83f4be483bc83423ba02f6f634f63fb777a334a62c1f0fea7f3377299267b99/diff,workdir=/var/lib/containers/storage/overlay/b83f4be483bc83423ba02f6f634f63fb777a334a62c1f0fea7f3377299267b99/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5a9e9bd2141ec94ca7dfeff11d8649b3253a2c131a2a2c06929ba84b420ee95f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/5a9e9bd2141ec94ca7dfeff11d8649b3253a2c131a2a2c06929ba84b420ee95f/diff,workdir=/var/lib/containers/storage/overlay/5a9e9bd2141ec94ca7dfeff11d8649b3253a2c131a2a2c06929ba84b420ee95f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4fa8db45f4f1cfb694d52841c7d23a0ee052015a79019180569b267178cef73e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AIWBHWRRVYMW7HLCLRPAYA5WGO:/var/lib/containers/storage/overlay/l/7S6RV27CHCDJQCEADWCGPEWEEE,upperdir=/var/lib/containers/storage/overlay/4fa8db45f4f1cfb694d52841c7d23a0ee052015a79019180569b267178cef73e/diff,workdir=/var/lib/containers/storage/overlay/4fa8db45f4f1cfb694d52841c7d23a0ee052015a79019180569b267178cef73e/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497396k,nr_inodes=819200,mode=755,inode64 0 0 2024-09-12T02:12:49.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.715 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-12T02:12:49.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:12:49.761 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-12T02:12:49.823 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-09-12T02:12:49.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:12:49.825 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T02:12:49.835 INFO:tasks.cephadm:Teardown begin 2024-09-12T02:12:49.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T02:12:49.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:50.086 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:50.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.558+0000 7f89a3f7e640 1 -- 172.21.15.122:0/2902213367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 msgr2=0x7f899c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.558+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/2902213367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c106e60 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8998009930 tx=0x7f899802a420 comp rx=0 tx=0).stop 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.558+0000 7f89a3f7e640 1 -- 172.21.15.122:0/2902213367 shutdown_connections 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/2902213367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f899c101420 0x7f899c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/2902213367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c100a80 0x7f899c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/2902213367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c106e60 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- 172.21.15.122:0/2902213367 >> 172.21.15.122:0/2902213367 conn(0x7f899c0fc7a0 msgr2=0x7f899c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- 172.21.15.122:0/2902213367 shutdown_connections 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- 172.21.15.122:0/2902213367 wait complete. 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 Processor -- start 2024-09-12T02:12:50.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- start start 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f899c100a80 0x7f899c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c101420 0x7f899c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f899c110e40 con 0x7f899c106a80 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f899c110cc0 con 0x7f899c100a80 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.559+0000 7f89a3f7e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f899c110fc0 con 0x7f899c101420 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46958/0 (socket says 172.21.15.122:46958) 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 -- 172.21.15.122:0/3639370586 learned_addr learned my addr 172.21.15.122:0/3639370586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a1d1a640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f899c100a80 0x7f899c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a1519640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c101420 0x7f899c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 -- 172.21.15.122:0/3639370586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c101420 msgr2=0x7f899c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c101420 0x7f899c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 -- 172.21.15.122:0/3639370586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f899c100a80 msgr2=0x7f899c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f899c100a80 0x7f899c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 -- 172.21.15.122:0/3639370586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c19d370 con 0x7f899c106a80 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a251b640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c19cc90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f899400cad0 tx=0x7f899400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899400ad60 con 0x7f899c106a80 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8994005020 con 0x7f899c106a80 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c19d660 con 0x7f899c106a80 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899400f6b0 con 0x7f899c106a80 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.560+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a4f40 con 0x7f899c106a80 2024-09-12T02:12:50.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.561+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c100ee0 con 0x7f899c106a80 2024-09-12T02:12:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.562+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8994005560 con 0x7f899c106a80 2024-09-12T02:12:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.563+0000 7f8992ffd640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89880a2b80 0x7f89880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.563+0000 7f89a1d1a640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89880a2b80 0x7f89880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.563+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f89940cef80 con 0x7f899c106a80 2024-09-12T02:12:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.564+0000 7f89a1d1a640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89880a2b80 0x7f89880a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8998009900 tx=0x7f89980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.568+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8994014030 con 0x7f899c106a80 2024-09-12T02:12:50.739 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:50 smithi122 ceph-mon[30925]: pgmap v1130: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:12:50.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.771+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f899c02d7c0 con 0x7f899c106a80 2024-09-12T02:12:50.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.772+0000 7f8992ffd640 1 -- 172.21.15.122:0/3639370586 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11936 (secure 0 0 0) 0x7f899409bee0 con 0x7f899c106a80 2024-09-12T02:12:50.772 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:12:50.773 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":50,"btime":"2024-09-12T02:12:22:603776+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:12:21.942151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T02:12:50.773 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 50 2024-09-12T02:12:50.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89880a2b80 msgr2=0x7f89880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89880a2b80 0x7f89880a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8998009900 tx=0x7f89980058d0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 msgr2=0x7f899c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c19cc90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f899400cad0 tx=0x7f899400ced0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 shutdown_connections 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f89880a2b80 0x7f89880a5040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c106a80 0x7f899c19cc90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f899c101420 0x7f899c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 --2- 172.21.15.122:0/3639370586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f899c100a80 0x7f899c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 >> 172.21.15.122:0/3639370586 conn(0x7f899c0fc7a0 msgr2=0x7f899c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 shutdown_connections 2024-09-12T02:12:50.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:50.774+0000 7f89a3f7e640 1 -- 172.21.15.122:0/3639370586 wait complete. 2024-09-12T02:12:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:50 smithi183 ceph-mon[35127]: pgmap v1130: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:12:51.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:50 smithi151 ceph-mon[35229]: pgmap v1130: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T02:12:51.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:12:51.669 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:51.911 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:51 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/3639370586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:51.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:51 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/3639370586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:52.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:51 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/3639370586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 -- 172.21.15.122:0/3130299508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 msgr2=0x7f1f2010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 --2- 172.21.15.122:0/3130299508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f2010d7a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009cd0 tx=0x7f1f1802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 -- 172.21.15.122:0/3130299508 shutdown_connections 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 --2- 172.21.15.122:0/3130299508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f2010d7a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 --2- 172.21.15.122:0/3130299508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f201008d0 0x7f1f20100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 --2- 172.21.15.122:0/3130299508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201068d0 0x7f1f20106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 -- 172.21.15.122:0/3130299508 >> 172.21.15.122:0/3130299508 conn(0x7f1f200fc610 msgr2=0x7f1f200fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:52.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 -- 172.21.15.122:0/3130299508 shutdown_connections 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 -- 172.21.15.122:0/3130299508 wait complete. 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 Processor -- start 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.130+0000 7f1f255a2640 1 -- start start 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f255a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201008d0 0x7f1f20199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f255a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f20199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f255a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f201068d0 0x7f1f201a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:52.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f255a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20110cd0 con 0x7f1f20101270 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f255a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20110b50 con 0x7f1f201068d0 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f255a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20110e50 con 0x7f1f201008d0 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f20199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f20199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46970/0 (socket says 172.21.15.122:46970) 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 -- 172.21.15.122:0/4289889458 learned_addr learned my addr 172.21.15.122:0/4289889458 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f24da1640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f201068d0 0x7f1f201a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1ffff640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201008d0 0x7f1f20199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 -- 172.21.15.122:0/4289889458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201008d0 msgr2=0x7f1f20199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:52.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201008d0 0x7f1f20199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 -- 172.21.15.122:0/4289889458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f201068d0 msgr2=0x7f1f201a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f201068d0 0x7f1f201a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 -- 172.21.15.122:0/4289889458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f201a45e0 con 0x7f1f20101270 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1ffff640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201008d0 0x7f1f20199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.131+0000 7f1f1f7fe640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f20199560 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800c890 tx=0x7f1f0800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.132+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004220 con 0x7f1f20101270 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.132+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f201a47b0 con 0x7f1f20101270 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.132+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f080043c0 con 0x7f1f20101270 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.132+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004a00 con 0x7f1f20101270 2024-09-12T02:12:52.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.132+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f201a4cf0 con 0x7f1f20101270 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.133+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f20100d30 con 0x7f1f20101270 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.134+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1f080079b0 con 0x7f1f20101270 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.134+0000 7f1f1d7fa640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1f0c0a2b80 0x7f1f0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.135+0000 7f1f1ffff640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1f0c0a2b80 0x7f1f0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.135+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1f080ced40 con 0x7f1f20101270 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.135+0000 7f1f1ffff640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1f0c0a2b80 0x7f1f0c0a5040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1f10005e10 tx=0x7f1f10005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:52.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.139+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f08014030 con 0x7f1f20101270 2024-09-12T02:12:52.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.345+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f201a4fb0 con 0x7f1f20101270 2024-09-12T02:12:52.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.346+0000 7f1f1d7fa640 1 -- 172.21.15.122:0/4289889458 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11936 (secure 0 0 0) 0x7f1f0809bca0 con 0x7f1f20101270 2024-09-12T02:12:52.347 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:12:52.347 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":50,"btime":"2024-09-12T02:12:22:603776+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3513109229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3513109229},{"type":"v1","addr":"172.21.15.122:6835","nonce":3513109229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/6361172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":6361172},{"type":"v1","addr":"172.21.15.122:6837","nonce":6361172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3259334666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3259334666},{"type":"v1","addr":"172.21.15.183:6839","nonce":3259334666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2056437657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2056437657},{"type":"v1","addr":"172.21.15.122:6839","nonce":2056437657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/1896979097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":1896979097},{"type":"v1","addr":"172.21.15.122:6841","nonce":1896979097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":18258,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1657619890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1657619890},{"type":"v1","addr":"172.21.15.183:6835","nonce":1657619890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T01:44:36.280693+0000","modified":"2024-09-12T02:12:21.942151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24436,"mds_2":24428},"failed":[],"damaged":[],"stopped":[],"info":{"gid_18282":{"gid":18282,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/503509846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":503509846},{"type":"v1","addr":"172.21.15.151:6833","nonce":503509846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/3975453954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3975453954},{"type":"v1","addr":"172.21.15.183:6833","nonce":3975453954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2236567181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2236567181},{"type":"v1","addr":"172.21.15.151:6835","nonce":2236567181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":62,"addr":"172.21.15.151:6837/3984907767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3984907767},{"type":"v1","addr":"172.21.15.151:6837","nonce":3984907767}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/3258783936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3258783936},{"type":"v1","addr":"172.21.15.183:6837","nonce":3258783936}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.151:6839/1184770501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1184770501},{"type":"v1","addr":"172.21.15.151:6839","nonce":1184770501}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24436,24428]},"id":1}]} 2024-09-12T02:12:52.347 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 50 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1f0c0a2b80 msgr2=0x7f1f0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1f0c0a2b80 0x7f1f0c0a5040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1f10005e10 tx=0x7f1f10005da0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 msgr2=0x7f1f20199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f20199560 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800c890 tx=0x7f1f0800cd50 comp rx=0 tx=0).stop 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 shutdown_connections 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f1f0c0a2b80 0x7f1f0c0a5040 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f201068d0 0x7f1f201a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f20101270 0x7f1f20199560 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 --2- 172.21.15.122:0/4289889458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f201008d0 0x7f1f20199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:52.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 >> 172.21.15.122:0/4289889458 conn(0x7f1f200fc610 msgr2=0x7f1f200fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:52.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 shutdown_connections 2024-09-12T02:12:52.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:52.348+0000 7f1f255a2640 1 -- 172.21.15.122:0/4289889458 wait complete. 2024-09-12T02:12:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:52 smithi183 ceph-mon[35127]: pgmap v1131: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T02:12:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:52 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/4289889458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:52.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:52 smithi122 ceph-mon[30925]: pgmap v1131: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T02:12:52.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:52 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/4289889458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:53.032 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:12:53.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:52 smithi151 ceph-mon[35229]: pgmap v1131: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T02:12:53.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:52 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/4289889458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:12:53.277 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 -- 172.21.15.122:0/2447528798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61010c170 msgr2=0x7fb61010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 --2- 172.21.15.122:0/2447528798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61010c170 0x7fb61010e610 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb608009ca0 tx=0x7fb60802a950 comp rx=0 tx=0).stop 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 -- 172.21.15.122:0/2447528798 shutdown_connections 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 --2- 172.21.15.122:0/2447528798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61010c170 0x7fb61010e610 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 --2- 172.21.15.122:0/2447528798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610103660 0x7fb61010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 --2- 172.21.15.122:0/2447528798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb610102d40 0x7fb610103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 -- 172.21.15.122:0/2447528798 >> 172.21.15.122:0/2447528798 conn(0x7fb6100fc7a0 msgr2=0x7fb6100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 -- 172.21.15.122:0/2447528798 shutdown_connections 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 -- 172.21.15.122:0/2447528798 wait complete. 2024-09-12T02:12:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 Processor -- start 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.754+0000 7fb6163d7640 1 -- start start 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb6163d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb610102d40 0x7fb610198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb6163d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 0x7fb6101988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb6163d7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb61010c170 0x7fb61019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb6163d7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb610111f70 con 0x7fb610103660 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb6163d7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb610111df0 con 0x7fb61010c170 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb6163d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101120f0 con 0x7fb610102d40 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 0x7fb6101988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 0x7fb6101988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46998/0 (socket says 172.21.15.122:46998) 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 -- 172.21.15.122:0/941369159 learned_addr learned my addr 172.21.15.122:0/941369159 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60ffff640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb610102d40 0x7fb610198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb614974640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb61010c170 0x7fb61019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 -- 172.21.15.122:0/941369159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb610102d40 msgr2=0x7fb610198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb610102d40 0x7fb610198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 -- 172.21.15.122:0/941369159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb61010c170 msgr2=0x7fb61019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb61010c170 0x7fb61019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 -- 172.21.15.122:0/941369159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61019d320 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.755+0000 7fb60f7fe640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 0x7fb6101988b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c970 tx=0x7fb5f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.756+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8007bb0 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.756+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61019d610 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.756+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f8007d50 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.756+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8005260 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.756+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6101a4ef0 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.757+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb610103120 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.758+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb5f8005400 con 0x7fb610103660 2024-09-12T02:12:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.758+0000 7fb60d7fa640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb5fc0a2b80 0x7fb5fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.759+0000 7fb60ffff640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb5fc0a2b80 0x7fb5fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.759+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb5f80cefe0 con 0x7fb610103660 2024-09-12T02:12:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.759+0000 7fb60ffff640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb5fc0a2b80 0x7fb5fc0a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb600005fd0 tx=0x7fb60000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.763+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5f8014030 con 0x7fb610103660 2024-09-12T02:12:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.932+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6101a5210 con 0x7fb610103660 2024-09-12T02:12:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.933+0000 7fb60d7fa640 1 -- 172.21.15.122:0/941369159 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22783 (secure 0 0 0) 0x7fb5f809bfe0 con 0x7fb610103660 2024-09-12T02:12:53.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-12T02:12:53.934 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":88,"fsid":"5731f69a-70a7-11ef-bcea-c7b262605968","created":"2024-09-12T01:36:52.283857+0000","modified":"2024-09-12T02:06:54.470390+0000","last_up_change":"2024-09-12T01:43:44.547061+0000","last_in_change":"2024-09-12T01:43:21.733053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T01:39:58.671405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T01:44:32.683657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T01:44:34.505051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f05d0c-f7d1-4a68-8a67-6daf86c09856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6803","nonce":1924566162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6805","nonce":1924566162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6809","nonce":1924566162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1924566162},{"type":"v1","addr":"172.21.15.122:6807","nonce":1924566162}]},"public_addr":"172.21.15.122:6803/1924566162","cluster_addr":"172.21.15.122:6805/1924566162","heartbeat_back_addr":"172.21.15.122:6809/1924566162","heartbeat_front_addr":"172.21.15.122:6807/1924566162","state":["exists","up"]},{"osd":1,"uuid":"474588fe-b8aa-4c63-a179-9fb1a91609c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6801","nonce":148046823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6803","nonce":148046823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6807","nonce":148046823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":148046823},{"type":"v1","addr":"172.21.15.151:6805","nonce":148046823}]},"public_addr":"172.21.15.151:6801/148046823","cluster_addr":"172.21.15.151:6803/148046823","heartbeat_back_addr":"172.21.15.151:6807/148046823","heartbeat_front_addr":"172.21.15.151:6805/148046823","state":["exists","up"]},{"osd":2,"uuid":"f1bc9ea7-5717-4f77-a9a1-dee0231402da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6801","nonce":3556820790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6803","nonce":3556820790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6807","nonce":3556820790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3556820790},{"type":"v1","addr":"172.21.15.183:6805","nonce":3556820790}]},"public_addr":"172.21.15.183:6801/3556820790","cluster_addr":"172.21.15.183:6803/3556820790","heartbeat_back_addr":"172.21.15.183:6807/3556820790","heartbeat_front_addr":"172.21.15.183:6805/3556820790","state":["exists","up"]},{"osd":3,"uuid":"a02adc8a-32be-42b2-9b14-d6f23ac36887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6811","nonce":533436685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6813","nonce":533436685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6817","nonce":533436685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":533436685},{"type":"v1","addr":"172.21.15.122:6815","nonce":533436685}]},"public_addr":"172.21.15.122:6811/533436685","cluster_addr":"172.21.15.122:6813/533436685","heartbeat_back_addr":"172.21.15.122:6817/533436685","heartbeat_front_addr":"172.21.15.122:6815/533436685","state":["exists","up"]},{"osd":4,"uuid":"79ee3483-5327-4ea0-ab19-e2614d5aec71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6809","nonce":190394298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6811","nonce":190394298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6815","nonce":190394298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":190394298},{"type":"v1","addr":"172.21.15.151:6813","nonce":190394298}]},"public_addr":"172.21.15.151:6809/190394298","cluster_addr":"172.21.15.151:6811/190394298","heartbeat_back_addr":"172.21.15.151:6815/190394298","heartbeat_front_addr":"172.21.15.151:6813/190394298","state":["exists","up"]},{"osd":5,"uuid":"f02377d1-5979-48eb-945c-65d7675d8283","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6809","nonce":685154842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6811","nonce":685154842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6815","nonce":685154842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":685154842},{"type":"v1","addr":"172.21.15.183:6813","nonce":685154842}]},"public_addr":"172.21.15.183:6809/685154842","cluster_addr":"172.21.15.183:6811/685154842","heartbeat_back_addr":"172.21.15.183:6815/685154842","heartbeat_front_addr":"172.21.15.183:6813/685154842","state":["exists","up"]},{"osd":6,"uuid":"3c4782f3-b6a1-45f7-83b4-e109d523dffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6819","nonce":913950299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6821","nonce":913950299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6825","nonce":913950299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":913950299},{"type":"v1","addr":"172.21.15.122:6823","nonce":913950299}]},"public_addr":"172.21.15.122:6819/913950299","cluster_addr":"172.21.15.122:6821/913950299","heartbeat_back_addr":"172.21.15.122:6825/913950299","heartbeat_front_addr":"172.21.15.122:6823/913950299","state":["exists","up"]},{"osd":7,"uuid":"142f0be6-d6a8-49d4-a4a1-462d5aece8d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6817","nonce":1163877446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6819","nonce":1163877446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6823","nonce":1163877446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1163877446},{"type":"v1","addr":"172.21.15.151:6821","nonce":1163877446}]},"public_addr":"172.21.15.151:6817/1163877446","cluster_addr":"172.21.15.151:6819/1163877446","heartbeat_back_addr":"172.21.15.151:6823/1163877446","heartbeat_front_addr":"172.21.15.151:6821/1163877446","state":["exists","up"]},{"osd":8,"uuid":"0da6dee8-5cbe-43a4-8be6-b4c58619d9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6817","nonce":830128598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6819","nonce":830128598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6823","nonce":830128598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":830128598},{"type":"v1","addr":"172.21.15.183:6821","nonce":830128598}]},"public_addr":"172.21.15.183:6817/830128598","cluster_addr":"172.21.15.183:6819/830128598","heartbeat_back_addr":"172.21.15.183:6823/830128598","heartbeat_front_addr":"172.21.15.183:6821/830128598","state":["exists","up"]},{"osd":9,"uuid":"dc2fd685-4e31-49ab-b2ed-8244461230a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6827","nonce":163584006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6829","nonce":163584006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6833","nonce":163584006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":163584006},{"type":"v1","addr":"172.21.15.122:6831","nonce":163584006}]},"public_addr":"172.21.15.122:6827/163584006","cluster_addr":"172.21.15.122:6829/163584006","heartbeat_back_addr":"172.21.15.122:6833/163584006","heartbeat_front_addr":"172.21.15.122:6831/163584006","state":["exists","up"]},{"osd":10,"uuid":"d577aa12-49ce-464c-b1b7-18e5b0765883","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6825","nonce":1544038944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6827","nonce":1544038944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6831","nonce":1544038944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1544038944},{"type":"v1","addr":"172.21.15.151:6829","nonce":1544038944}]},"public_addr":"172.21.15.151:6825/1544038944","cluster_addr":"172.21.15.151:6827/1544038944","heartbeat_back_addr":"172.21.15.151:6831/1544038944","heartbeat_front_addr":"172.21.15.151:6829/1544038944","state":["exists","up"]},{"osd":11,"uuid":"07eb6a15-ca7e-4169-8e16-6f47e7046f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6825","nonce":2785958349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6827","nonce":2785958349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6831","nonce":2785958349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2785958349},{"type":"v1","addr":"172.21.15.183:6829","nonce":2785958349}]},"public_addr":"172.21.15.183:6825/2785958349","cluster_addr":"172.21.15.183:6827/2785958349","heartbeat_back_addr":"172.21.15.183:6831/2785958349","heartbeat_front_addr":"172.21.15.183:6829/2785958349","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:03.529989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:29.720158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:39:54.719005+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:19.592867+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:40:45.179232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:11.189617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:41:36.296291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:00.921934+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:27.132187+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:42:52.382325+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:18.019431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T01:43:43.126188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6800/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3407930740":"2024-09-13T01:37:44.367349+0000","172.21.15.122:6801/3688521366":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/1117597805":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/2717375533":"2024-09-13T01:37:44.367349+0000","172.21.15.122:0/3107764322":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6801/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:6800/1950823209":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/1058528799":"2024-09-13T01:37:21.796085+0000","172.21.15.122:0/472737560":"2024-09-13T01:37:21.796085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb5fc0a2b80 msgr2=0x7fb5fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb5fc0a2b80 0x7fb5fc0a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb600005fd0 tx=0x7fb60000b390 comp rx=0 tx=0).stop 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 msgr2=0x7fb6101988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 0x7fb6101988b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c970 tx=0x7fb5f800ce30 comp rx=0 tx=0).stop 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 shutdown_connections 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fb5fc0a2b80 0x7fb5fc0a5040 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb61010c170 0x7fb61019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb610103660 0x7fb6101988b0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 --2- 172.21.15.122:0/941369159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb610102d40 0x7fb610198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 >> 172.21.15.122:0/941369159 conn(0x7fb6100fc7a0 msgr2=0x7fb6100fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 shutdown_connections 2024-09-12T02:12:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:53.935+0000 7fb6163d7640 1 -- 172.21.15.122:0/941369159 wait complete. 2024-09-12T02:12:54.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T02:12:54.859 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:54.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:54 smithi183 ceph-mon[35127]: pgmap v1132: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T02:12:54.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:54 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/941369159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:12:54.980 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:54 smithi122 ceph-mon[30925]: pgmap v1132: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T02:12:54.980 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:54 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/941369159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:12:55.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:54 smithi151 ceph-mon[35229]: pgmap v1132: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T02:12:55.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:54 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/941369159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:12:55.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 -- 172.21.15.122:0/3379323514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888102790 msgr2=0x7f18881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:55.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 --2- 172.21.15.122:0/3379323514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888102790 0x7f18881067e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 -- 172.21.15.122:0/3379323514 shutdown_connections 2024-09-12T02:12:55.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 --2- 172.21.15.122:0/3379323514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1888106f10 0x7f188810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 --2- 172.21.15.122:0/3379323514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888102790 0x7f18881067e0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 --2- 172.21.15.122:0/3379323514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1888101de0 0x7f18881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 -- 172.21.15.122:0/3379323514 >> 172.21.15.122:0/3379323514 conn(0x7f18880fd8c0 msgr2=0x7f18880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 -- 172.21.15.122:0/3379323514 shutdown_connections 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.368+0000 7f188c86b640 1 -- 172.21.15.122:0/3379323514 wait complete. 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.369+0000 7f188c86b640 1 Processor -- start 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.369+0000 7f188c86b640 1 -- start start 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.369+0000 7f188c86b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 0x7f1888198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.369+0000 7f188c86b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1888102790 0x7f1888198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.369+0000 7f188c86b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1888106f10 0x7f188819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:55.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188c86b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1888111f70 con 0x7f1888101de0 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188c86b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1888111df0 con 0x7f1888102790 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188c86b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18881120f0 con 0x7f1888106f10 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 0x7f1888198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 0x7f1888198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47012/0 (socket says 172.21.15.122:47012) 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 -- 172.21.15.122:0/2760726240 learned_addr learned my addr 172.21.15.122:0/2760726240 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f1886d9d640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1888106f10 0x7f188819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f187ffff640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1888102790 0x7f1888198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 -- 172.21.15.122:0/2760726240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1888106f10 msgr2=0x7f188819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1888106f10 0x7f188819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 -- 172.21.15.122:0/2760726240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1888102790 msgr2=0x7f1888198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:55.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1888102790 0x7f1888198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 -- 172.21.15.122:0/2760726240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188819d2f0 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188659c640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 0x7f1888198340 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f187800c920 tx=0x7f187800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878007c70 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1878007e10 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188819d5e0 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878018890 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.370+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18881a4ec0 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.371+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18881021c0 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.372+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1878018a70 con 0x7f1888101de0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.373+0000 7f187f7fe640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18740a2b80 0x7f18740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.373+0000 7f187ffff640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18740a2b80 0x7f18740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:55.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.373+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f18780cf240 con 0x7f1888101de0 2024-09-12T02:12:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.374+0000 7f187ffff640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18740a2b80 0x7f18740a5040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f18880fe130 tx=0x7f1870031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:55.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.378+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1878014030 con 0x7f1888101de0 2024-09-12T02:12:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.578+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1888109bc0 con 0x7f1888101de0 2024-09-12T02:12:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.582+0000 7f187f7fe640 1 -- 172.21.15.122:0/2760726240 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7f187800de20 con 0x7f1888101de0 2024-09-12T02:12:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18740a2b80 msgr2=0x7f18740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18740a2b80 0x7f18740a5040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f18880fe130 tx=0x7f1870031490 comp rx=0 tx=0).stop 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 msgr2=0x7f1888198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 0x7f1888198340 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f187800c920 tx=0x7f187800cde0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 shutdown_connections 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7f18740a2b80 0x7f18740a5040 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1888106f10 0x7f188819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1888102790 0x7f1888198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 --2- 172.21.15.122:0/2760726240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1888101de0 0x7f1888198340 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:55.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.585+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 >> 172.21.15.122:0/2760726240 conn(0x7f18880fd8c0 msgr2=0x7f18880feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:55.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.586+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 shutdown_connections 2024-09-12T02:12:55.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:55.586+0000 7f188c86b640 1 -- 172.21.15.122:0/2760726240 wait complete. 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c[64735]: ignoring --setuser ceph since I am not root 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c[64735]: ignoring --setgroup ceph since I am not root 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-mds[64739]: -- 172.21.15.183:0/3129875546 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649e075f860 con 0x5649e073c800 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c[64735]: starting mds.c at 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l[64133]: ignoring --setuser ceph since I am not root 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l[64133]: ignoring --setgroup ceph since I am not root 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-mds[64137]: -- 172.21.15.183:0/1587482224 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c0955d860 con 0x556c0953b400 2024-09-12T02:12:55.953 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:12:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l[64133]: starting mds.l at 2024-09-12T02:12:55.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:55 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T02:12:55.579+0000 7f42ad3d2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:12:56.017 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: ignoring --setuser ceph since I am not root 2024-09-12T02:12:56.017 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: ignoring --setgroup ceph since I am not root 2024-09-12T02:12:56.017 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-mds[64109]: -- 172.21.15.151:0/3226734285 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ece2b7860 con 0x559ece294800 2024-09-12T02:12:56.017 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: starting mds.k at 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: ignoring --setuser ceph since I am not root 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: ignoring --setgroup ceph since I am not root 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-mds[65031]: -- 172.21.15.151:0/707257123 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557c59b5c680 con 0x557c59b31400 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-mds[65031]: -- 172.21.15.151:0/707257123 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c59b53860 con 0x557c59b31400 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: starting mds.b at 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e[64404]: ignoring --setuser ceph since I am not root 2024-09-12T02:12:56.018 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e[64404]: ignoring --setgroup ceph since I am not root 2024-09-12T02:12:56.019 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-mds[64408]: -- 172.21.15.151:0/2342575297 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610ef513860 con 0x5610ef4f0800 2024-09-12T02:12:56.019 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e[64404]: starting mds.e at 2024-09-12T02:12:56.019 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: ignoring --setuser ceph since I am not root 2024-09-12T02:12:56.019 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: ignoring --setgroup ceph since I am not root 2024-09-12T02:12:56.019 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-mds[64709]: -- 172.21.15.151:0/2722925316 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606bb4e7860 con 0x5606bb4c4800 2024-09-12T02:12:56.019 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:12:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: starting mds.h at 2024-09-12T02:12:56.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T02:12:56.541 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: pgmap v1133: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2760726240' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2760726240' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T02:12:56.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:56 smithi151 ceph-mon[35229]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T02:12:56.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: pgmap v1133: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T02:12:56.661 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2760726240' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:12:56.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:12:56.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:12:56.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-12T02:12:56.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2760726240' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:12:56.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T02:12:56.662 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:56 smithi122 ceph-mon[30925]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: pgmap v1133: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2760726240' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2760726240' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T02:12:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:56 smithi183 ceph-mon[35127]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T02:12:57.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 -- 172.21.15.122:0/3861355624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928100a80 msgr2=0x7fc928100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:57.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 --2- 172.21.15.122:0/3861355624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928100a80 0x7fc928100ee0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc910009930 tx=0x7fc91002a420 comp rx=0 tx=0).stop 2024-09-12T02:12:57.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 -- 172.21.15.122:0/3861355624 shutdown_connections 2024-09-12T02:12:57.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 --2- 172.21.15.122:0/3861355624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928101420 0x7fc92810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 --2- 172.21.15.122:0/3861355624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928100a80 0x7fc928100ee0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 --2- 172.21.15.122:0/3861355624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928106a80 0x7fc928106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 -- 172.21.15.122:0/3861355624 >> 172.21.15.122:0/3861355624 conn(0x7fc9280fc7a0 msgr2=0x7fc9280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 -- 172.21.15.122:0/3861355624 shutdown_connections 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.010+0000 7fc92ca97640 1 -- 172.21.15.122:0/3861355624 wait complete. 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 Processor -- start 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 -- start start 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928100a80 0x7fc928198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 0x7fc9281989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928106a80 0x7fc92819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc928111fa0 con 0x7fc928101420 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc928111e20 con 0x7fc928100a80 2024-09-12T02:12:57.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92ca97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc928112120 con 0x7fc928106a80 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 0x7fc9281989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 0x7fc9281989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47032/0 (socket says 172.21.15.122:47032) 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc92659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928100a80 0x7fc928198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.011+0000 7fc91dd9b640 1 -- 172.21.15.122:0/2775490390 learned_addr learned my addr 172.21.15.122:0/2775490390 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc926d9d640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928106a80 0x7fc92819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91dd9b640 1 -- 172.21.15.122:0/2775490390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928106a80 msgr2=0x7fc92819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91dd9b640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928106a80 0x7fc92819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91dd9b640 1 -- 172.21.15.122:0/2775490390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928100a80 msgr2=0x7fc928198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91dd9b640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928100a80 0x7fc928198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91dd9b640 1 -- 172.21.15.122:0/2775490390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92819d430 con 0x7fc928101420 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91dd9b640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 0x7fc9281989c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc910009230 tx=0x7fc910004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91003f030 con 0x7fc928101420 2024-09-12T02:12:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc910003ad0 con 0x7fc928101420 2024-09-12T02:12:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92819d6c0 con 0x7fc928101420 2024-09-12T02:12:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910035070 con 0x7fc928101420 2024-09-12T02:12:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.012+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281a5000 con 0x7fc928101420 2024-09-12T02:12:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.013+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc928100ee0 con 0x7fc928101420 2024-09-12T02:12:57.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.015+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc910042030 con 0x7fc928101420 2024-09-12T02:12:57.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.015+0000 7fc91ffff640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9140a2c00 0x7fc9140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:57.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.016+0000 7fc92659c640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9140a2c00 0x7fc9140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:57.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.016+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc9100efce0 con 0x7fc928101420 2024-09-12T02:12:57.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.016+0000 7fc92659c640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9140a2c00 0x7fc9140a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc91800bb60 tx=0x7fc91800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:57.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.019+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc910049030 con 0x7fc928101420 2024-09-12T02:12:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.219+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc928108a60 con 0x7fc928101420 2024-09-12T02:12:57.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.592+0000 7fc91ffff640 1 -- 172.21.15.122:0/2775490390 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7fc9100bc4e0 con 0x7fc928101420 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9140a2c00 msgr2=0x7fc9140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9140a2c00 0x7fc9140a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc91800bb60 tx=0x7fc91800b390 comp rx=0 tx=0).stop 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 msgr2=0x7fc9281989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 0x7fc9281989c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc910009230 tx=0x7fc910004270 comp rx=0 tx=0).stop 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 shutdown_connections 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fc9140a2c00 0x7fc9140a50c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928106a80 0x7fc92819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928101420 0x7fc9281989c0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 --2- 172.21.15.122:0/2775490390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928100a80 0x7fc928198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.594+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 >> 172.21.15.122:0/2775490390 conn(0x7fc9280fc7a0 msgr2=0x7fc9280fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.595+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 shutdown_connections 2024-09-12T02:12:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:57.595+0000 7fc92ca97640 1 -- 172.21.15.122:0/2775490390 wait complete. 2024-09-12T02:12:57.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6832/3838006467,v1:172.21.15.183:6833/3838006467] up:boot 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.183:6836/3267479523,v1:172.21.15.183:6837/3267479523] up:boot 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6832/3001181675,v1:172.21.15.151:6834/3001181675] up:boot 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6833/1025071564,v1:172.21.15.151:6835/1025071564] up:boot 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6836/3219361360,v1:172.21.15.151:6838/3219361360] up:boot 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: mds.? [v2:172.21.15.151:6837/3932940450,v1:172.21.15.151:6839/3932940450] up:boot 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:12:57.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:12:57.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:12:57.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:12:57.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:57 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2775490390' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6832/3838006467,v1:172.21.15.183:6833/3838006467] up:boot 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.183:6836/3267479523,v1:172.21.15.183:6837/3267479523] up:boot 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6832/3001181675,v1:172.21.15.151:6834/3001181675] up:boot 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6833/1025071564,v1:172.21.15.151:6835/1025071564] up:boot 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6836/3219361360,v1:172.21.15.151:6838/3219361360] up:boot 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: mds.? [v2:172.21.15.151:6837/3932940450,v1:172.21.15.151:6839/3932940450] up:boot 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:12:57.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:12:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:12:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:12:57.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:57 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2775490390' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6832/3838006467,v1:172.21.15.183:6833/3838006467] up:boot 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.183:6836/3267479523,v1:172.21.15.183:6837/3267479523] up:boot 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6832/3001181675,v1:172.21.15.151:6834/3001181675] up:boot 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6833/1025071564,v1:172.21.15.151:6835/1025071564] up:boot 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6836/3219361360,v1:172.21.15.151:6838/3219361360] up:boot 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: mds.? [v2:172.21.15.151:6837/3932940450,v1:172.21.15.151:6839/3932940450] up:boot 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:12:57.960 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:12:57.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:12:57.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:12:57.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='mgr.14150 172.21.15.122:0/2562521913' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:12:57.961 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:57 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2775490390' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:12:58.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T02:12:58.573 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:12:58.850 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:58 smithi122 ceph-mon[30925]: pgmap v1135: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-12T02:12:58.851 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:58 smithi122 ceph-mon[30925]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:12:58.851 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:58 smithi122 ceph-mon[30925]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:12:58.851 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:58 smithi122 ceph-mon[30925]: Cluster is now healthy 2024-09-12T02:12:58.851 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:58 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/2775490390' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:12:58.851 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:58 smithi122 ceph-mon[30925]: fsmap 12 up:standby 2024-09-12T02:12:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:58 smithi151 ceph-mon[35229]: pgmap v1135: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-12T02:12:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:58 smithi151 ceph-mon[35229]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:12:58.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:58 smithi151 ceph-mon[35229]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:12:58.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:58 smithi151 ceph-mon[35229]: Cluster is now healthy 2024-09-12T02:12:58.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:58 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/2775490390' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:12:58.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:58 smithi151 ceph-mon[35229]: fsmap 12 up:standby 2024-09-12T02:12:58.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:58 smithi183 ceph-mon[35127]: pgmap v1135: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-12T02:12:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:58 smithi183 ceph-mon[35127]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:12:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:58 smithi183 ceph-mon[35127]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:12:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:58 smithi183 ceph-mon[35127]: Cluster is now healthy 2024-09-12T02:12:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:58 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/2775490390' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:12:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:58 smithi183 ceph-mon[35127]: fsmap 12 up:standby 2024-09-12T02:12:59.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1301367880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8106f10 msgr2=0x7fd5e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1301367880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8106f10 0x7fd5e810ea90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009ca0 tx=0x7fd5e002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1301367880 shutdown_connections 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1301367880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8106f10 0x7fd5e810ea90 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1301367880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e8102790 0x7fd5e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1301367880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 0x7fd5e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1301367880 >> 172.21.15.122:0/1301367880 conn(0x7fd5e80fd8c0 msgr2=0x7fd5e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1301367880 shutdown_connections 2024-09-12T02:12:59.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.058+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1301367880 wait complete. 2024-09-12T02:12:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 Processor -- start 2024-09-12T02:12:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 -- start start 2024-09-12T02:12:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 0x7fd5e8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 0x7fd5e8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e8106f10 0x7fd5e819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8111f70 con 0x7fd5e8102790 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8111df0 con 0x7fd5e8101de0 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5eeab9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e81120f0 con 0x7fd5e8106f10 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 0x7fd5e8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 0x7fd5e8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47044/0 (socket says 172.21.15.122:47044) 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 -- 172.21.15.122:0/1515294781 learned_addr learned my addr 172.21.15.122:0/1515294781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5ec855640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 0x7fd5e8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5ed056640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e8106f10 0x7fd5e819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 -- 172.21.15.122:0/1515294781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e8106f10 msgr2=0x7fd5e819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e8106f10 0x7fd5e819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 -- 172.21.15.122:0/1515294781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 msgr2=0x7fd5e8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 0x7fd5e8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.059+0000 7fd5e7fff640 1 -- 172.21.15.122:0/1515294781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e819d2e0 con 0x7fd5e8102790 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5ec855640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 0x7fd5e8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5e7fff640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 0x7fd5e8198870 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400d950 tx=0x7fd5d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4017070 con 0x7fd5e8102790 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e819d5d0 con 0x7fd5e8102790 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d4009c30 con 0x7fd5e8102790 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400f6b0 con 0x7fd5e8102790 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.060+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a4eb0 con 0x7fd5e8102790 2024-09-12T02:12:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.061+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e81021c0 con 0x7fd5e8102790 2024-09-12T02:12:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.062+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd5d40109d0 con 0x7fd5e8102790 2024-09-12T02:12:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.063+0000 7fd5e5ffb640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5d80a2b80 0x7fd5d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.063+0000 7fd5ec855640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5d80a2b80 0x7fd5d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.063+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd5d40cee00 con 0x7fd5e8102790 2024-09-12T02:12:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.063+0000 7fd5ec855640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5d80a2b80 0x7fd5d80a5040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00abe0 tx=0x7fd5dc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:59.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.067+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5d4009de0 con 0x7fd5e8102790 2024-09-12T02:12:59.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.233+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd5e810a150 con 0x7fd5e8102790 2024-09-12T02:12:59.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.602+0000 7fd5e5ffb640 1 -- 172.21.15.122:0/1515294781 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7fd5d409bc50 con 0x7fd5e8102790 2024-09-12T02:12:59.603 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-12T02:12:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.604+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5d80a2b80 msgr2=0x7fd5d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.604+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5d80a2b80 0x7fd5d80a5040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00abe0 tx=0x7fd5dc00a5c0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.604+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 msgr2=0x7fd5e8198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.604+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 0x7fd5e8198870 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400d950 tx=0x7fd5d400de10 comp rx=0 tx=0).stop 2024-09-12T02:12:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 shutdown_connections 2024-09-12T02:12:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fd5d80a2b80 0x7fd5d80a5040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e8106f10 0x7fd5e819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5e8102790 0x7fd5e8198870 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 --2- 172.21.15.122:0/1515294781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e8101de0 0x7fd5e8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:59.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 >> 172.21.15.122:0/1515294781 conn(0x7fd5e80fd8c0 msgr2=0x7fd5e80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:59.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 shutdown_connections 2024-09-12T02:12:59.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:12:59.605+0000 7fd5eeab9640 1 -- 172.21.15.122:0/1515294781 wait complete. 2024-09-12T02:12:59.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:12:59 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1515294781' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:12:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:12:59 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1515294781' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:12:59.978 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:12:59 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1515294781' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:13:00.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T02:13:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:00 smithi151 ceph-mon[35229]: pgmap v1136: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-12T02:13:00.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:00 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1515294781' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:13:00.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:00 smithi151 ceph-mon[35229]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T02:13:00.925 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:00 smithi122 ceph-mon[30925]: pgmap v1136: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-12T02:13:00.925 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:00 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1515294781' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:13:00.926 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:00 smithi122 ceph-mon[30925]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T02:13:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:00 smithi183 ceph-mon[35127]: pgmap v1136: 97 pgs: 97 active+clean; 309 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-12T02:13:00.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:00 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1515294781' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:13:00.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:00 smithi183 ceph-mon[35127]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T02:13:01.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 -- 172.21.15.122:0/3436867801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0102790 msgr2=0x7fcbd01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:01.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/3436867801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0102790 0x7fcbd01067e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009900 tx=0x7fcbc002a410 comp rx=0 tx=0).stop 2024-09-12T02:13:01.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 -- 172.21.15.122:0/3436867801 shutdown_connections 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/3436867801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0106f10 0x7fcbd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/3436867801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0102790 0x7fcbd01067e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/3436867801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbd0101de0 0x7fcbd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 -- 172.21.15.122:0/3436867801 >> 172.21.15.122:0/3436867801 conn(0x7fcbd00fd8c0 msgr2=0x7fcbd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 -- 172.21.15.122:0/3436867801 shutdown_connections 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.007+0000 7fcbd7a12640 1 -- 172.21.15.122:0/3436867801 wait complete. 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 Processor -- start 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 -- start start 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbd0101de0 0x7fcbd00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0102790 0x7fcbd0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 0x7fcbd0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0111f70 con 0x7fcbd0106f10 2024-09-12T02:13:01.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0111df0 con 0x7fcbd0101de0 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd7a12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01120f0 con 0x7fcbd0102790 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd5faf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 0x7fcbd0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd5faf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 0x7fcbd0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60762/0 (socket says 172.21.15.122:60762) 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd5faf640 1 -- 172.21.15.122:0/1824339716 learned_addr learned my addr 172.21.15.122:0/1824339716 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd57ae640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbd0101de0 0x7fcbd00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.008+0000 7fcbd4fad640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0102790 0x7fcbd0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd5faf640 1 -- 172.21.15.122:0/1824339716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0102790 msgr2=0x7fcbd0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd5faf640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0102790 0x7fcbd0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd5faf640 1 -- 172.21.15.122:0/1824339716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbd0101de0 msgr2=0x7fcbd00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd5faf640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbd0101de0 0x7fcbd00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd5faf640 1 -- 172.21.15.122:0/1824339716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd01a6870 con 0x7fcbd0106f10 2024-09-12T02:13:01.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd4fad640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0102790 0x7fcbd0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd5faf640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 0x7fcbd0074f70 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800cce0 tx=0x7fcbc800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc800f450 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc800fad0 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8018c10 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd01a6b60 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.009+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd01a70a0 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.010+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd01021c0 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.011+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcbc800f5f0 con 0x7fcbd0106f10 2024-09-12T02:13:01.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.012+0000 7fcbc67fc640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbbc0a2c00 0x7fcbbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:01.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.012+0000 7fcbd57ae640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbbc0a2c00 0x7fcbbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:01.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.012+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8002+0+0 (secure 0 0 0) 0x7fcbc80cf4c0 con 0x7fcbd0106f10 2024-09-12T02:13:01.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.012+0000 7fcbd57ae640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbbc0a2c00 0x7fcbbc0a50c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005d00 tx=0x7fcbcc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:01.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.016+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbc8014030 con 0x7fcbd0106f10 2024-09-12T02:13:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.181+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbd0109bc0 con 0x7fcbd0106f10 2024-09-12T02:13:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.611+0000 7fcbc67fc640 1 -- 172.21.15.122:0/1824339716 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7fcbc809c550 con 0x7fcbd0106f10 2024-09-12T02:13:01.611 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-12T02:13:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbbc0a2c00 msgr2=0x7fcbbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbbc0a2c00 0x7fcbbc0a50c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005d00 tx=0x7fcbcc00b410 comp rx=0 tx=0).stop 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 msgr2=0x7fcbd0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 0x7fcbd0074f70 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800cce0 tx=0x7fcbc800a620 comp rx=0 tx=0).stop 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 shutdown_connections 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:6800/3245830997,v1:172.21.15.122:6801/3245830997] conn(0x7fcbbc0a2c00 0x7fcbbc0a50c0 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005d00 tx=0x7fcbcc00b410 comp rx=0 tx=0).stop 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbd0106f10 0x7fcbd0074f70 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0102790 0x7fcbd0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 --2- 172.21.15.122:0/1824339716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbd0101de0 0x7fcbd00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:01.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.614+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 >> 172.21.15.122:0/1824339716 conn(0x7fcbd00fd8c0 msgr2=0x7fcbd00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.617+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 shutdown_connections 2024-09-12T02:13:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-12T02:13:01.617+0000 7fcbd7a12640 1 -- 172.21.15.122:0/1824339716 wait complete. 2024-09-12T02:13:01.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:01 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1824339716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:13:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:01 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1824339716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:13:01.989 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:01 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1824339716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:13:02.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:13:02.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:13:02.415 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:13:02.445 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T02:13:02.445 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:13:02.461 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:13:02.477 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:13:02.500 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T02:13:02.501 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T02:13:02.501 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:13:02.612 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 systemd[1]: Stopping Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:02 smithi183 ceph-mon[35127]: pgmap v1138: 65 pgs: 65 active+clean; 577 KiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:13:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:02 smithi183 ceph-mon[35127]: from='client.? 172.21.15.122:0/1824339716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:13:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:02 smithi183 ceph-mon[35127]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T02:13:02.865 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:02 smithi151 ceph-mon[35229]: pgmap v1138: 65 pgs: 65 active+clean; 577 KiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:13:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:02 smithi151 ceph-mon[35229]: from='client.? 172.21.15.122:0/1824339716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:13:02.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:02 smithi151 ceph-mon[35229]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T02:13:02.880 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 ceph-mon[30925]: pgmap v1138: 65 pgs: 65 active+clean; 577 KiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:13:02.881 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 ceph-mon[30925]: from='client.? 172.21.15.122:0/1824339716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:13:02.881 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 ceph-mon[30925]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T02:13:02.881 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T02:13:02.809+0000 7f42b2bdd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:02.881 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a[30921]: 2024-09-12T02:13:02.809+0000 7f42b2bdd640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T02:13:03.193 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:02 smithi122 podman[111855]: 2024-09-12 02:13:02.881359594 +0000 UTC m=+0.191881448 container died 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:03.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:03 smithi122 podman[111855]: 2024-09-12 02:13:03.293582505 +0000 UTC m=+0.604104357 container cleanup 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T02:13:03.489 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:03 smithi122 bash[111855]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:13:03.813 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:03 smithi122 podman[111917]: 2024-09-12 02:13:03.529161458 +0000 UTC m=+0.646313071 container remove 2745149ee0394004a84f14ad1b735668155b24a1ed84bff70e1cb3e51490b2ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:13:04.400 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:04 smithi122 podman[112006]: 2024-09-12 02:13:04.149879781 +0000 UTC m=+0.489603438 container died 27e7bd00a285b569df7a8f910d0ef98b942b35bd8c6c8f7f9db52eb1f4a79eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=awesome_bouman, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T02:13:04.400 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:04 smithi122 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:13:04.400 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:04 smithi122 systemd[1]: Stopped Ceph mon.a for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T02:13:04.400 INFO:journalctl@ceph.mon.a.smithi122.stdout:Sep 12 02:13:04 smithi122 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service: Consumed 42.750s CPU time. 2024-09-12T02:13:04.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T02:13:04.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:04.477 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T02:13:04.478 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T02:13:04.478 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.b 2024-09-12T02:13:04.858 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:04 smithi151 systemd[1]: Stopping Ceph mon.b for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:05.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-b[35225]: 2024-09-12T02:13:04.856+0000 7fba154bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:05.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-b[35225]: 2024-09-12T02:13:04.856+0000 7fba154bd640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T02:13:05.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:04 smithi151 podman[73059]: 2024-09-12 02:13:04.91907912 +0000 UTC m=+0.188867041 container died 088859bd539afd174ae540f68806c6924b3b620e82a6e132dc0bc0c109569e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:13:05.610 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:05 smithi151 podman[73059]: 2024-09-12 02:13:05.333801712 +0000 UTC m=+0.603589633 container cleanup 088859bd539afd174ae540f68806c6924b3b620e82a6e132dc0bc0c109569e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T02:13:05.610 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:05 smithi151 bash[73059]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-b 2024-09-12T02:13:05.822 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T02:13:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:05 smithi151 podman[73071]: 2024-09-12 02:13:05.611027299 +0000 UTC m=+0.693420439 container remove 088859bd539afd174ae540f68806c6924b3b620e82a6e132dc0bc0c109569e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T02:13:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:05 smithi151 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-12T02:13:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:05 smithi151 systemd[1]: Stopped Ceph mon.b for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T02:13:05.866 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 12 02:13:05 smithi151 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.b.service: Consumed 17.153s CPU time. 2024-09-12T02:13:05.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:05.871 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T02:13:05.871 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T02:13:05.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.c 2024-09-12T02:13:06.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:05 smithi183 systemd[1]: Stopping Ceph mon.c for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:06.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c[35123]: 2024-09-12T02:13:06.216+0000 7f8e04b6f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:06.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c[35123]: 2024-09-12T02:13:06.216+0000 7f8e04b6f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T02:13:06.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:06 smithi183 podman[73428]: 2024-09-12 02:13:06.339238252 +0000 UTC m=+0.249859645 container died 9a051b6e9f2993edd11110bcdc955a8c5505bd91d5a4d2974da56d9ec81dbc45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T02:13:07.019 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:06 smithi183 podman[73428]: 2024-09-12 02:13:06.768129107 +0000 UTC m=+0.678750501 container cleanup 9a051b6e9f2993edd11110bcdc955a8c5505bd91d5a4d2974da56d9ec81dbc45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T02:13:07.221 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:13:06 smithi183 bash[73428]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mon-c 2024-09-12T02:13:07.223 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T02:13:07.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:07.274 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T02:13:07.274 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T02:13:07.274 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:13:07.685 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:07 smithi122 systemd[1]: Stopping Ceph mgr.x for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:07.989 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x[31152]: 2024-09-12T02:13:07.684+0000 7fdae0d99640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:07.989 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:07 smithi122 podman[114821]: 2024-09-12 02:13:07.715368811 +0000 UTC m=+0.210233962 container died bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T02:13:08.370 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:08 smithi122 podman[114821]: 2024-09-12 02:13:08.135120322 +0000 UTC m=+0.629985468 container cleanup bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:08.370 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:08 smithi122 bash[114821]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x 2024-09-12T02:13:08.618 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T02:13:08.640 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:08 smithi122 podman[114832]: 2024-09-12 02:13:08.370787033 +0000 UTC m=+0.651067388 container remove bdeedc2eecd22736c695250ef5724cd0a39bd9b435052978df6bb637530ea558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T02:13:08.640 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:08 smithi122 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-12T02:13:08.640 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:08 smithi122 systemd[1]: Stopped Ceph mgr.x for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T02:13:08.640 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Sep 12 02:13:08 smithi122 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.x.service: Consumed 50.041s CPU time. 2024-09-12T02:13:08.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:08.656 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T02:13:08.656 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T02:13:08.656 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.y 2024-09-12T02:13:09.002 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 02:13:08 smithi151 systemd[1]: Stopping Ceph mgr.y for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:09.309 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 02:13:09 smithi151 podman[73184]: 2024-09-12 02:13:09.002665254 +0000 UTC m=+0.150861210 container died c7ca2074172f2c2858539a301a7735217d63fa814fb16fc5c90de8649e6299da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T02:13:09.615 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 02:13:09 smithi151 podman[73184]: 2024-09-12 02:13:09.41677681 +0000 UTC m=+0.564972759 container cleanup c7ca2074172f2c2858539a301a7735217d63fa814fb16fc5c90de8649e6299da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T02:13:09.616 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Sep 12 02:13:09 smithi151 bash[73184]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-y 2024-09-12T02:13:09.843 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T02:13:09.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:09.879 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T02:13:09.879 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T02:13:09.879 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.z 2024-09-12T02:13:10.203 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:13:09 smithi183 systemd[1]: Stopping Ceph mgr.z for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:10.507 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:13:10 smithi183 podman[73562]: 2024-09-12 02:13:10.219521186 +0000 UTC m=+0.148096443 container died 673b42d8bd2ecb33eeb78c4f6a5a0d2a17a1dc6f78b0e774e7b1afd41bf97c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T02:13:10.842 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:13:10 smithi183 podman[73562]: 2024-09-12 02:13:10.598868276 +0000 UTC m=+0.527443520 container cleanup 673b42d8bd2ecb33eeb78c4f6a5a0d2a17a1dc6f78b0e774e7b1afd41bf97c3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:13:10.842 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:13:10 smithi183 bash[73562]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mgr-z 2024-09-12T02:13:10.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T02:13:11.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:11.006 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T02:13:11.006 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T02:13:11.006 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.0 2024-09-12T02:13:11.432 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:11 smithi122 systemd[1]: Stopping Ceph osd.0 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:11.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:11 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0[45064]: 2024-09-12T02:13:11.431+0000 7f3b94d66640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:11.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:11 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0[45064]: 2024-09-12T02:13:11.432+0000 7f3b94d66640 -1 osd.0 91 *** Got signal Terminated *** 2024-09-12T02:13:11.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:11 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0[45064]: 2024-09-12T02:13:11.432+0000 7f3b94d66640 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:13:17.342 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:17 smithi122 podman[114943]: 2024-09-12 02:13:17.065864669 +0000 UTC m=+5.856182673 container died 0e96fe80206423e18ef2580214b1e180b180bd2c1000733caae3a39b6bc4c20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T02:13:17.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:17 smithi122 podman[114943]: 2024-09-12 02:13:17.54137941 +0000 UTC m=+6.331697451 container cleanup 0e96fe80206423e18ef2580214b1e180b180bd2c1000733caae3a39b6bc4c20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T02:13:17.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:17 smithi122 bash[114943]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0 2024-09-12T02:13:18.166 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:17 smithi122 podman[114958]: 2024-09-12 02:13:17.818158974 +0000 UTC m=+0.747428917 container remove 0e96fe80206423e18ef2580214b1e180b180bd2c1000733caae3a39b6bc4c20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:18.438 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:18 smithi122 podman[115049]: 2024-09-12 02:13:18.069246952 +0000 UTC m=+0.015580822 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:13:18.438 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:18 smithi122 podman[115049]: 2024-09-12 02:13:18.190773722 +0000 UTC m=+0.137107625 container create 3b759dcb98af01f946a6de22ab94cb2c2d0a6098374131f83cc568444125eebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T02:13:18.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:18 smithi122 podman[115049]: 2024-09-12 02:13:18.438774267 +0000 UTC m=+0.385108127 container init 3b759dcb98af01f946a6de22ab94cb2c2d0a6098374131f83cc568444125eebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T02:13:18.739 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:18 smithi122 podman[115049]: 2024-09-12 02:13:18.442599686 +0000 UTC m=+0.388933550 container start 3b759dcb98af01f946a6de22ab94cb2c2d0a6098374131f83cc568444125eebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T02:13:18.740 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:18 smithi122 podman[115049]: 2024-09-12 02:13:18.530564884 +0000 UTC m=+0.476898762 container attach 3b759dcb98af01f946a6de22ab94cb2c2d0a6098374131f83cc568444125eebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T02:13:18.740 INFO:journalctl@ceph.osd.0.smithi122.stdout:Sep 12 02:13:18 smithi122 podman[115049]: 2024-09-12 02:13:18.627024881 +0000 UTC m=+0.573358744 container died 3b759dcb98af01f946a6de22ab94cb2c2d0a6098374131f83cc568444125eebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T02:13:19.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T02:13:19.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:19.454 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T02:13:19.455 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T02:13:19.455 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.1 2024-09-12T02:13:19.781 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:19 smithi151 systemd[1]: Stopping Ceph osd.1 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:20.115 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1[39583]: 2024-09-12T02:13:19.779+0000 7f3263cdd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:20.115 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1[39583]: 2024-09-12T02:13:19.779+0000 7f3263cdd640 -1 osd.1 91 *** Got signal Terminated *** 2024-09-12T02:13:20.116 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1[39583]: 2024-09-12T02:13:19.779+0000 7f3263cdd640 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:13:25.558 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:25 smithi151 podman[73311]: 2024-09-12 02:13:25.241953378 +0000 UTC m=+5.584417058 container died fcb5a81c9cc5356dedc24090d85458948c42bd321defe0d5f8fec3a99da8f9b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:13:25.866 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:25 smithi151 podman[73311]: 2024-09-12 02:13:25.640486387 +0000 UTC m=+5.982950063 container cleanup fcb5a81c9cc5356dedc24090d85458948c42bd321defe0d5f8fec3a99da8f9b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T02:13:25.866 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:25 smithi151 bash[73311]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1 2024-09-12T02:13:26.310 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:25 smithi151 podman[73325]: 2024-09-12 02:13:25.967592403 +0000 UTC m=+0.721310596 container remove fcb5a81c9cc5356dedc24090d85458948c42bd321defe0d5f8fec3a99da8f9b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:26.597 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:26 smithi151 podman[73413]: 2024-09-12 02:13:26.214037581 +0000 UTC m=+0.016241995 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:13:26.597 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:26 smithi151 podman[73413]: 2024-09-12 02:13:26.344403436 +0000 UTC m=+0.146607839 container create 558fc40d5057fefa1f05217209ba816ea89aef08fd640a075616f39ec4020751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T02:13:26.866 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:26 smithi151 podman[73413]: 2024-09-12 02:13:26.597801557 +0000 UTC m=+0.400005961 container init 558fc40d5057fefa1f05217209ba816ea89aef08fd640a075616f39ec4020751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T02:13:26.866 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:26 smithi151 podman[73413]: 2024-09-12 02:13:26.601603307 +0000 UTC m=+0.403807710 container start 558fc40d5057fefa1f05217209ba816ea89aef08fd640a075616f39ec4020751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T02:13:26.866 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:26 smithi151 podman[73413]: 2024-09-12 02:13:26.72317369 +0000 UTC m=+0.525378112 container attach 558fc40d5057fefa1f05217209ba816ea89aef08fd640a075616f39ec4020751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:13:27.243 INFO:journalctl@ceph.osd.1.smithi151.stdout:Sep 12 02:13:26 smithi151 podman[73413]: 2024-09-12 02:13:26.905720315 +0000 UTC m=+0.707924731 container died 558fc40d5057fefa1f05217209ba816ea89aef08fd640a075616f39ec4020751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:13:27.593 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T02:13:27.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:27.634 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T02:13:27.634 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T02:13:27.634 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.2 2024-09-12T02:13:27.952 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:27 smithi183 systemd[1]: Stopping Ceph osd.2 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:28.452 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2[40084]: 2024-09-12T02:13:27.952+0000 7f8444840640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:28.452 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2[40084]: 2024-09-12T02:13:27.952+0000 7f8444840640 -1 osd.2 91 *** Got signal Terminated *** 2024-09-12T02:13:28.453 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2[40084]: 2024-09-12T02:13:27.952+0000 7f8444840640 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:13:33.702 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:33 smithi183 podman[73688]: 2024-09-12 02:13:33.420577221 +0000 UTC m=+5.586486856 container died fba22aa1be470c92f377ce3ee5017b744b3dc2c7d807202d85b9006a8a53def9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T02:13:34.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:33 smithi183 podman[73688]: 2024-09-12 02:13:33.736356066 +0000 UTC m=+5.902265700 container cleanup fba22aa1be470c92f377ce3ee5017b744b3dc2c7d807202d85b9006a8a53def9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T02:13:34.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:33 smithi183 bash[73688]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2 2024-09-12T02:13:34.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:33 smithi183 podman[73704]: 2024-09-12 02:13:33.979892055 +0000 UTC m=+0.553215188 container remove fba22aa1be470c92f377ce3ee5017b744b3dc2c7d807202d85b9006a8a53def9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T02:13:34.599 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:34 smithi183 podman[73793]: 2024-09-12 02:13:34.198970681 +0000 UTC m=+0.016018192 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:13:34.599 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:34 smithi183 podman[73793]: 2024-09-12 02:13:34.339301025 +0000 UTC m=+0.156348514 container create 380db0d3908d6cafd3b4824f85c0762007e9f13607c0312dfdfd48144da05c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T02:13:34.953 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:34 smithi183 podman[73793]: 2024-09-12 02:13:34.600283562 +0000 UTC m=+0.417331050 container init 380db0d3908d6cafd3b4824f85c0762007e9f13607c0312dfdfd48144da05c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T02:13:34.953 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:34 smithi183 podman[73793]: 2024-09-12 02:13:34.603572332 +0000 UTC m=+0.420619823 container start 380db0d3908d6cafd3b4824f85c0762007e9f13607c0312dfdfd48144da05c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:13:34.953 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:34 smithi183 podman[73793]: 2024-09-12 02:13:34.708846291 +0000 UTC m=+0.525893799 container attach 380db0d3908d6cafd3b4824f85c0762007e9f13607c0312dfdfd48144da05c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T02:13:34.953 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:13:34 smithi183 podman[73793]: 2024-09-12 02:13:34.804261609 +0000 UTC m=+0.621309114 container died 380db0d3908d6cafd3b4824f85c0762007e9f13607c0312dfdfd48144da05c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T02:13:35.540 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T02:13:35.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:35.585 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T02:13:35.585 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T02:13:35.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.3 2024-09-12T02:13:35.956 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:35 smithi122 systemd[1]: Stopping Ceph osd.3 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:36.227 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:35 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3[51916]: 2024-09-12T02:13:35.987+0000 7efccd9da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:36.227 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:35 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3[51916]: 2024-09-12T02:13:35.987+0000 7efccd9da640 -1 osd.3 91 *** Got signal Terminated *** 2024-09-12T02:13:36.227 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:35 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3[51916]: 2024-09-12T02:13:35.987+0000 7efccd9da640 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:13:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:37.831+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:38.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:37 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:37.867+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:38.317 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:38 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:38.010+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:38.793 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:38.488+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:39.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:38.792+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:39.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:38 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:38.905+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:39.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:38 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:38.971+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:39.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:39.319+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:39.761 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:39.495+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:39.760+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:40.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:39 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:39.950+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:40.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:40 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:40.002+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:40.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:40.354+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:40.937 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:40.518+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:40.937 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:40.767+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:41.202 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:40.936+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:41.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:40 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:40.937+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:41.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:40 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:40.988+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:41.614 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:41 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:41.327+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:41.775 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:41.522+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:41.905 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:41 smithi122 podman[115151]: 2024-09-12 02:13:41.615091183 +0000 UTC m=+5.837807550 container died 07d05a42b41c2ba302be08cebbfae04d723f73d7de9f1872c5082c4cb1768914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:13:42.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:41.969+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:41.775+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:42.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:41 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:41.905+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:42.239 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 podman[115151]: 2024-09-12 02:13:42.055053641 +0000 UTC m=+6.277770018 container cleanup 07d05a42b41c2ba302be08cebbfae04d723f73d7de9f1872c5082c4cb1768914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:13:42.239 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 bash[115151]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3 2024-09-12T02:13:42.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:41 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:41.973+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:42.711 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:42.301+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:42.712 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 podman[115180]: 2024-09-12 02:13:42.273421456 +0000 UTC m=+0.656752243 container remove 07d05a42b41c2ba302be08cebbfae04d723f73d7de9f1872c5082c4cb1768914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T02:13:42.952 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:42.561+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:42.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:42.727+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:42.989 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:42 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:42.912+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:42.990 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 podman[115267]: 2024-09-12 02:13:42.614898149 +0000 UTC m=+0.016028161 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:13:42.990 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 podman[115267]: 2024-09-12 02:13:42.732808504 +0000 UTC m=+0.133938504 container create 90be1e9387e3826b8dacfcc0496768e09e7c2e076648d4fd32ea43947dc8b145 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T02:13:43.291 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 podman[115267]: 2024-09-12 02:13:42.993834699 +0000 UTC m=+0.394964697 container init 90be1e9387e3826b8dacfcc0496768e09e7c2e076648d4fd32ea43947dc8b145 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T02:13:43.291 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:42 smithi122 podman[115267]: 2024-09-12 02:13:42.997217526 +0000 UTC m=+0.398347525 container start 90be1e9387e3826b8dacfcc0496768e09e7c2e076648d4fd32ea43947dc8b145 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T02:13:43.291 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:43 smithi122 podman[115267]: 2024-09-12 02:13:43.085726534 +0000 UTC m=+0.486856549 container attach 90be1e9387e3826b8dacfcc0496768e09e7c2e076648d4fd32ea43947dc8b145 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T02:13:43.291 INFO:journalctl@ceph.osd.3.smithi122.stdout:Sep 12 02:13:43 smithi122 podman[115267]: 2024-09-12 02:13:43.185970148 +0000 UTC m=+0.587100159 container died 90be1e9387e3826b8dacfcc0496768e09e7c2e076648d4fd32ea43947dc8b145 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T02:13:43.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:42 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:42.987+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:43.453 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:43.002+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:43.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:43.290+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:43.871 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T02:13:43.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:43.913 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T02:13:43.913 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T02:13:43.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.4 2024-09-12T02:13:43.952 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:43.757+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:43.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:43.576+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:44.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:43 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:43.896+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:44.240 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:43 smithi151 systemd[1]: Stopping Ceph osd.4 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:44.240 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:43 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:43.948+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:44.452 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:44.025+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:44.615 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:44.240+0000 7f4702d82640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:44.616 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:44.240+0000 7f4702d82640 -1 osd.4 91 *** Got signal Terminated *** 2024-09-12T02:13:44.616 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:44.240+0000 7f4702d82640 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:13:44.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:44 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:44.270+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:44.952 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:44.801+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:44.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:44.573+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:45.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:44 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:44.891+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:45.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:44 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:44.984+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:45.453 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:45 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:45.001+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:45.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:45.268+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:45.799 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:45 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:45.536+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:46.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:45 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:45.799+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:46.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:45 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:45.799+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:46.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:45 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:45.956+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:46.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:45 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:45.956+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:46.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:45 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:45.907+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:46.365 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:46 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:46.039+0000 7f46fb390640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.451616+0000 front 2024-09-12T02:13:19.451652+0000 (oldest deadline 2024-09-12T02:13:45.351893+0000) 2024-09-12T02:13:46.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:45 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:45.943+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:46.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:46 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:46.268+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:46.815 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:46 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:46.538+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:46.816 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:46 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:46.754+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:46.816 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:46 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:46.754+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:47.202 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:46 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:46.916+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:47.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:46 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:46.916+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:47.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:46 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:46.949+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:47.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:46 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:46.949+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:47.365 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:47 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:47.003+0000 7f46fb390640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.451616+0000 front 2024-09-12T02:13:19.451652+0000 (oldest deadline 2024-09-12T02:13:45.351893+0000) 2024-09-12T02:13:47.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:46 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:46.965+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:47.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:47 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:47.265+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:47.759 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:47 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:47.498+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:48.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:47 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:47.759+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:48.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:47 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:47.759+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:48.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:47 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:47.881+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:48.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:47 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:47.881+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:48.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:47 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:47.962+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:48.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:47 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:47.962+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:48.365 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:48 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:48.029+0000 7f46fb390640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.451616+0000 front 2024-09-12T02:13:19.451652+0000 (oldest deadline 2024-09-12T02:13:45.351893+0000) 2024-09-12T02:13:48.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:47 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:47.944+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:48.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:47 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:47.944+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:48.730 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:48.453+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:48.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:48 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:48.288+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:49.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:48.730+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:49.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:48.730+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:49.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:48.899+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:49.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:48 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:48.899+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:49.294 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:49.002+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:49.294 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:49.002+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:49.365 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:49 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4[46347]: 2024-09-12T02:13:49.000+0000 7f46fb390640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.451616+0000 front 2024-09-12T02:13:19.451652+0000 (oldest deadline 2024-09-12T02:13:45.351893+0000) 2024-09-12T02:13:49.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:48 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:48.966+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:49.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:48 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:48.966+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:49.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:49 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:49.443+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:49.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:49 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:49.687+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:49.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:49 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:49.687+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:49.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:49.293+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:49.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:49 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:49.293+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:49.975 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:49 smithi151 podman[73526]: 2024-09-12 02:13:49.662013217 +0000 UTC m=+5.543258456 container died 56c5070ccc6fba7545cdc84465213c546c9915183eff7ea5fadae05e464389ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T02:13:50.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:49 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:49.892+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:50.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:49 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:49.892+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:50.334 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:50 smithi151 podman[73526]: 2024-09-12 02:13:50.074443711 +0000 UTC m=+5.955688949 container cleanup 56c5070ccc6fba7545cdc84465213c546c9915183eff7ea5fadae05e464389ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:13:50.334 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:50 smithi151 bash[73526]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4 2024-09-12T02:13:50.335 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:50 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:50.006+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:50.335 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:50 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:50.006+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:50.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:50 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:50.051+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:50.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:50 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:50.051+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:50.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:50 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:50.290+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:50.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:50 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:50.290+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:50.615 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:50 smithi151 podman[73542]: 2024-09-12 02:13:50.334736877 +0000 UTC m=+0.673791361 container remove 56c5070ccc6fba7545cdc84465213c546c9915183eff7ea5fadae05e464389ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:50.872 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:50 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:50.471+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:50.873 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:50 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:50.698+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:50.873 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:50 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:50.698+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:51.045 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:50 smithi151 podman[73630]: 2024-09-12 02:13:50.653753336 +0000 UTC m=+0.016324661 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:13:51.045 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:50 smithi151 podman[73630]: 2024-09-12 02:13:50.794542023 +0000 UTC m=+0.157113382 container create 7c3cfd7d6d05cdaa44478a0ac63c77f2e30d0caa2ef92cc107039694ca49a20a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T02:13:51.202 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:50 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:50.872+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:51.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:50 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:50.872+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:51.354 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:51 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:51.044+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:51.354 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:51 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:51.044+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:51.355 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:51 smithi151 podman[73630]: 2024-09-12 02:13:51.047519082 +0000 UTC m=+0.410090403 container init 7c3cfd7d6d05cdaa44478a0ac63c77f2e30d0caa2ef92cc107039694ca49a20a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:13:51.355 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:51 smithi151 podman[73630]: 2024-09-12 02:13:51.050744208 +0000 UTC m=+0.413315528 container start 7c3cfd7d6d05cdaa44478a0ac63c77f2e30d0caa2ef92cc107039694ca49a20a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:51.355 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:51 smithi151 podman[73630]: 2024-09-12 02:13:51.156188425 +0000 UTC m=+0.518759742 container attach 7c3cfd7d6d05cdaa44478a0ac63c77f2e30d0caa2ef92cc107039694ca49a20a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T02:13:51.355 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 12 02:13:51 smithi151 podman[73630]: 2024-09-12 02:13:51.240089655 +0000 UTC m=+0.602660977 container died 7c3cfd7d6d05cdaa44478a0ac63c77f2e30d0caa2ef92cc107039694ca49a20a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T02:13:51.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:51 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:51.038+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:51.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:51 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:51.038+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:51.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:51 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:51.246+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:51.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:51 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:51.246+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:51.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:51 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:51.354+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:51.883 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:51 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:51.495+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:51.884 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:51 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:51.738+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:51.884 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:51 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:51.738+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:51.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T02:13:52.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:52.032 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T02:13:52.032 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T02:13:52.032 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.5 2024-09-12T02:13:52.202 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:51 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:51.883+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:52.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:51 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:51.883+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:52.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:52 smithi183 systemd[1]: Stopping Ceph osd.5 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:13:52.313 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:52 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:52.065+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:52.313 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:52 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:52.065+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:52.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:52.002+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:52.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:52.002+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:52.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:52.206+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:52.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:52.206+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:52.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:52 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:52.313+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:52.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:52.467+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:52.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:52.356+0000 7f25c0a45640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:13:52.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:52.356+0000 7f25c0a45640 -1 osd.5 91 *** Got signal Terminated *** 2024-09-12T02:13:52.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:52.356+0000 7f25c0a45640 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:13:53.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:52.728+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:53.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:52.728+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:53.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:52.927+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:53.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:52 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:52.927+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:53.282 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:53 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:53.254+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:53.282 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:53 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:53.254+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:53.282 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:53 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:53.014+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:53.282 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:53 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:53.014+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:53.365 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:53 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:53.287+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:53.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:53 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:53.067+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:53.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:53 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:53.067+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:53.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:53 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:53.427+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:54.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:53 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:53.878+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:54.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:53 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:53.878+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:54.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:53 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:53.755+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:54.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:53 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:53.755+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:54.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:54.073+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:54.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:54.073+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:54.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:54.304+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:54.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:54 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:54.304+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:13:54.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:54 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:54.063+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:54.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:54 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:54.063+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:54.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:54 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:54.238+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:54.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:54 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:54.238+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:54.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:54.425+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:54.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:54.425+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:13:55.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:54.857+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:55.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:54.857+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:55.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:54.739+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:55.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:54.739+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:55.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:54 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:54.739+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:13:55.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:55.101+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:55.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:55.101+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:55.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:55.346+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:55.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:55 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:55.346+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:13:55.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:55 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:55.202+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:55.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:55 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:55.202+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:55.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:55 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:55.113+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:55.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:55 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:55.113+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:55.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:55.451+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:55.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:55.451+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:13:56.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:55.813+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:56.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:55.813+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:56.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:55.783+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:56.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:55.783+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:56.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:55 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:55.783+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:13:56.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:56 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:56.213+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:56.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:56 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:56.213+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:56.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:56 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:56.213+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:13:56.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:56 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:56.090+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:56.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:56 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:56.090+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:56.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:56 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:56.090+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:13:56.537 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:56 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:56.364+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:56.537 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:56 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:56.364+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:13:56.537 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:56 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:56.143+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:56.537 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:56 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:56.143+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:56.537 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:56 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:56.143+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:13:56.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:56.410+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:56.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:56.410+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:13:57.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:56.743+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:57.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:56.743+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:57.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:56.743+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:13:57.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:56.798+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.673411+0000 front 2024-09-12T02:13:14.673345+0000 (oldest deadline 2024-09-12T02:13:39.973460+0000) 2024-09-12T02:13:57.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:56.798+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:19.974030+0000 front 2024-09-12T02:13:19.974145+0000 (oldest deadline 2024-09-12T02:13:45.874062+0000) 2024-09-12T02:13:57.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:56 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5[46566]: 2024-09-12T02:13:56.798+0000 7f25b9053640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.875629+0000 front 2024-09-12T02:13:32.875653+0000 (oldest deadline 2024-09-12T02:13:56.375789+0000) 2024-09-12T02:13:57.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:57 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:57.107+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:57.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:57 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:57.107+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:57.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:57 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:57.107+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:13:57.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:57.215+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:57.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:57.215+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:57.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:57.215+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:13:57.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:57.124+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:57.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:57.124+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:57.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:57.124+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:13:57.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:57 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:57.400+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:57.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:57 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:57.400+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:13:57.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:57 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:57.401+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:57.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:57 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:57.401+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:13:58.099 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:57 smithi183 podman[73907]: 2024-09-12 02:13:57.800327638 +0000 UTC m=+5.569772874 container died 53a54b31a17b53b62e0577e509cef320df6e4f497522999e3337444a0e325c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T02:13:58.099 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:57 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:57.772+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:58.100 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:57 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:57.772+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:58.100 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:57 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:57.772+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:13:58.392 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:58 smithi183 podman[73907]: 2024-09-12 02:13:58.207541446 +0000 UTC m=+5.976986680 container cleanup 53a54b31a17b53b62e0577e509cef320df6e4f497522999e3337444a0e325c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T02:13:58.392 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:58 smithi183 bash[73907]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5 2024-09-12T02:13:58.408 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:58 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:58.144+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:58.408 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:58 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:58.144+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:58.408 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:58 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:58.144+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:13:58.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:58 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:58.124+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:58.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:58 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:58.124+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:58.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:58 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:58.124+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:13:58.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:58 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:58.169+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:58.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:58 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:58.169+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:58.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:58 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:58.169+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:13:58.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:58 smithi183 podman[73925]: 2024-09-12 02:13:58.442301723 +0000 UTC m=+0.637965112 container remove 53a54b31a17b53b62e0577e509cef320df6e4f497522999e3337444a0e325c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:58.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:58 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:58.391+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:58.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:58 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:58.391+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:13:58.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:58 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:58.408+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:58.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:58 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:58.408+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:13:59.003 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:58 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:58.760+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:13:59.004 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:58 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:58.760+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:13:59.004 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:58 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:58.760+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:13:59.004 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:58 smithi183 podman[74010]: 2024-09-12 02:13:58.651183287 +0000 UTC m=+0.015489239 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:13:59.004 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:58 smithi183 podman[74010]: 2024-09-12 02:13:58.76840069 +0000 UTC m=+0.132706643 container create 8aa3cf17a3c71480de094e37293eb24df07df573575ed3cf27c4fd4017d7a272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:13:59.004 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:59 smithi183 podman[74010]: 2024-09-12 02:13:59.004266432 +0000 UTC m=+0.368572379 container init 8aa3cf17a3c71480de094e37293eb24df07df573575ed3cf27c4fd4017d7a272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T02:13:59.344 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:59 smithi183 podman[74010]: 2024-09-12 02:13:59.007584699 +0000 UTC m=+0.371890655 container start 8aa3cf17a3c71480de094e37293eb24df07df573575ed3cf27c4fd4017d7a272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:13:59.345 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:59 smithi183 podman[74010]: 2024-09-12 02:13:59.095796072 +0000 UTC m=+0.460102029 container attach 8aa3cf17a3c71480de094e37293eb24df07df573575ed3cf27c4fd4017d7a272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T02:13:59.345 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:13:59 smithi183 podman[74010]: 2024-09-12 02:13:59.200096997 +0000 UTC m=+0.564402945 container died 8aa3cf17a3c71480de094e37293eb24df07df573575ed3cf27c4fd4017d7a272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:13:59.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:59 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:59.099+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:13:59.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:59 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:59.099+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:13:59.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:59 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:13:59.099+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:13:59.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:59 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:59.151+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:13:59.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:59 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:59.151+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:13:59.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:13:59 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:13:59.151+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:13:59.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:59 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:59.393+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:13:59.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:13:59 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:13:59.393+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:13:59.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:59 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:59.170+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:13:59.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:59 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:59.170+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:13:59.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:13:59 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:13:59.170+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:13:59.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:59 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:59.344+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:13:59.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:13:59 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:13:59.344+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:13:59.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T02:13:59.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:13:59.836 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T02:13:59.836 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T02:13:59.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.6 2024-09-12T02:14:00.145 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:13:59 smithi122 systemd[1]: Stopping Ceph osd.6 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:00.145 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:00.105+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:00.146 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:00.105+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:00.146 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:00.105+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:00.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:59 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:59.797+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:00.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:59 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:59.797+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:00.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:13:59 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:13:59.797+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:00.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:00.144+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:14:00.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:00.144+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:14:00.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:00.144+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:14:00.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:00.168+0000 7f352fd72640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:00.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:00.168+0000 7f352fd72640 -1 osd.6 91 *** Got signal Terminated *** 2024-09-12T02:14:00.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:00 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:00.168+0000 7f352fd72640 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:14:00.616 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:00 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:00.444+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:00.616 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:00 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:00.444+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:00.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:00 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:00.212+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:00.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:00 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:00.212+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:00.617 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:00 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:00.212+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:00.705 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:00 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:00.353+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:00.705 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:00 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:00.353+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:01.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:00 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:00.814+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:01.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:00 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:00.814+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:01.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:00 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:00.814+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:01.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:01.162+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:14:01.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:01.162+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:14:01.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:01.162+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:14:01.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:01.107+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:01.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:01.107+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:01.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:01.107+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:01.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:01.450+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:01.616 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:01.450+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:01.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:01.234+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:01.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:01.234+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:01.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:01 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:01.234+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:01.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:01 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:01.329+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:01.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:01 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:01.329+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:02.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:01 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:01.806+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:02.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:01 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:01.806+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:02.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:01 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:01.806+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:02.470 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:02 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:02.187+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:02.470 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:02 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:02.187+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:02.470 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:02 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:02.187+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:02.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:02.191+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:14:02.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:02.191+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:14:02.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:02.191+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:14:02.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:02.136+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:02.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:02.136+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:02.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:02 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:02.136+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:02.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:02 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:02.377+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:02.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:02 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:02.377+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:02.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:02 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:02.470+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:02.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:02 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:02.470+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:03.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:02 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:02.806+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:03.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:02 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:02.806+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:03.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:02 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:02.806+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:03.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:03.176+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:14:03.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:03.176+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:14:03.489 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:03.176+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:14:03.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:03.108+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:03.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:03.108+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:03.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:03 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:03.108+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:03.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:03 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:03.423+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:03.615 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:03 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:03.423+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:03.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:03 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:03.175+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:03.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:03 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:03.175+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:03.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:03 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:03.175+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:03.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:03 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:03.175+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:03.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:03 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:03.357+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:03.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:03 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:03.357+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:04.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:03 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:03.840+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:04.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:03 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:03.840+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:04.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:03 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:03.840+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:04.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:03 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:03.840+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:04.390 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:04.139+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:04.391 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:04.139+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:04.391 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:04.139+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:04.391 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:04.139+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:04.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:04.104+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:04.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:04.104+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:04.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:04.104+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:04.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:04.158+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:14.316071+0000 front 2024-09-12T02:13:14.316013+0000 (oldest deadline 2024-09-12T02:13:37.216159+0000) 2024-09-12T02:14:04.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:04.158+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.218111+0000 front 2024-09-12T02:13:24.217997+0000 (oldest deadline 2024-09-12T02:13:46.517955+0000) 2024-09-12T02:14:04.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:04.158+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.220155+0000 front 2024-09-12T02:13:32.220207+0000 (oldest deadline 2024-09-12T02:13:55.720043+0000) 2024-09-12T02:14:04.490 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:04 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6[58391]: 2024-09-12T02:14:04.158+0000 7f352c388640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:35.720401+0000 front 2024-09-12T02:13:35.720354+0000 (oldest deadline 2024-09-12T02:14:03.221343+0000) 2024-09-12T02:14:04.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:04.371+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:04.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:04.371+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:04.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:04.371+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:04.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:04.390+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:04.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:04.390+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:04.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:04 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:04.390+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:05.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:04.836+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:05.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:04.836+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:05.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:04.836+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:05.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:04 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:04.836+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:05.391 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:05 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:05.056+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:05.391 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:05 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:05.056+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:05.391 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:05 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:05.056+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:05.421 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:05.154+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:05.422 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:05.154+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:05.422 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:05.154+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:05.422 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:05.154+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:05.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:05.372+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:05.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:05.372+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:05.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:05.372+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:05.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:05.421+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:05.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:05.421+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:05.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:05.421+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:05.944 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:05 smithi122 podman[115368]: 2024-09-12 02:14:05.64738194 +0000 UTC m=+5.610202153 container died a7b336e634da17795cf416d48a0eb4f379c8a2edbac88d28b1644ec10176a6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T02:14:06.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:05.865+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:06.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:05.865+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:06.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:05.865+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:06.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:05 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:05.865+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:06.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 podman[115368]: 2024-09-12 02:14:06.143713602 +0000 UTC m=+6.106533806 container cleanup a7b336e634da17795cf416d48a0eb4f379c8a2edbac88d28b1644ec10176a6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T02:14:06.239 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 bash[115368]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6 2024-09-12T02:14:06.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:06 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:06.070+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:06.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:06 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:06.070+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:06.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:06 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:06.070+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:06.409 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:06.145+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:06.409 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:06.145+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:06.409 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:06.145+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:06.409 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:06.145+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:06.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:06.371+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:06.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:06.371+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:06.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:06.371+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:06.739 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 podman[115383]: 2024-09-12 02:14:06.395327427 +0000 UTC m=+0.750327366 container remove a7b336e634da17795cf416d48a0eb4f379c8a2edbac88d28b1644ec10176a6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T02:14:06.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:06.408+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:06.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:06.408+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:06.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:06 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:06.408+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:07.054 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 podman[115473]: 2024-09-12 02:14:06.746048623 +0000 UTC m=+0.107248218 container create 72b0415cea133fd32aab8921b62410dd7a1c5f39f59847408eb0a56ce0256f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:14:07.054 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 podman[115473]: 2024-09-12 02:14:06.654506854 +0000 UTC m=+0.015706453 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:14:07.054 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 podman[115473]: 2024-09-12 02:14:06.957317059 +0000 UTC m=+0.318516656 container init 72b0415cea133fd32aab8921b62410dd7a1c5f39f59847408eb0a56ce0256f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T02:14:07.054 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:06 smithi122 podman[115473]: 2024-09-12 02:14:06.960585803 +0000 UTC m=+0.321785403 container start 72b0415cea133fd32aab8921b62410dd7a1c5f39f59847408eb0a56ce0256f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:14:07.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:06.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:07.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:06.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:07.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:06.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:07.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:06 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:06.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:07.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:07.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:07.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:07.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:07.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:07.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:07.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:07.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:07.460 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:07 smithi122 podman[115473]: 2024-09-12 02:14:07.057130656 +0000 UTC m=+0.418330262 container attach 72b0415cea133fd32aab8921b62410dd7a1c5f39f59847408eb0a56ce0256f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T02:14:07.460 INFO:journalctl@ceph.osd.6.smithi122.stdout:Sep 12 02:14:07 smithi122 podman[115473]: 2024-09-12 02:14:07.148071258 +0000 UTC m=+0.509270852 container died 72b0415cea133fd32aab8921b62410dd7a1c5f39f59847408eb0a56ce0256f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T02:14:07.460 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:07.053+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:07.460 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:07.053+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:07.461 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:07.053+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:07.461 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:07 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:07.053+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:07.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:07.326+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:07.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:07.326+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:07.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:07.326+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:07.794 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T02:14:07.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:07.835 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T02:14:07.835 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T02:14:07.836 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.7 2024-09-12T02:14:07.861 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:07.433+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:07.861 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:07.433+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:07.861 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:07 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:07.433+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:08.115 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:07 smithi151 systemd[1]: Stopping Ceph osd.7 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:08.116 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:08.108+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:08.116 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:08.108+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:08.116 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:08.108+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:08.116 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:08.108+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:08.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:07.805+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:08.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:07.805+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:08.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:07.805+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:08.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:07 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:07.805+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:08.474 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:08.165+0000 7fe6ed26b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:08.477 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:08.165+0000 7fe6ed26b640 -1 osd.7 91 *** Got signal Terminated *** 2024-09-12T02:14:08.477 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:08.165+0000 7fe6ed26b640 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:14:08.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:08.024+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:08.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:08.024+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:08.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:08.024+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:08.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:08 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:08.024+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:08.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:08.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:08.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:08.291+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:08.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:08.291+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:08.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:08.473+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:08.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:08.473+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:08.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:08.473+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:09.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:08.807+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:09.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:08.807+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:09.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:08.807+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:09.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:08 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:08.807+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:09.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:09.100+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:09.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:09.100+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:09.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:09.100+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:09.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:09.100+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:09.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:09.033+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:09.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:09.033+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:09.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:09.033+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:09.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:09 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:09.033+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:09.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:09.295+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:09.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:09.295+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:09.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:09.295+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:09.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:09.434+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:09.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:09.434+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:09.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:09 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:09.434+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:10.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:09.784+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:10.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:09.784+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:10.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:09.784+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:10.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:09 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:09.784+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:10.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:10.052+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:10.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:10.052+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:10.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:10.052+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:10.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:10.052+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:10.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.035+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:10.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.035+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:10.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.035+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:10.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.035+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:10.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:10.299+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:10.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:10.299+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:10.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:10.299+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:10.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:10.433+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:10.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:10.433+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:10.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:10 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:10.433+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:11.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:10.833+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:11.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:10.833+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:11.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:10.833+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:11.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:10 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:10.833+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:11.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.988+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:11.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.988+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:11.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.988+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:11.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:10 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:10.988+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:11.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:11.068+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:11.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:11.068+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:11.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:11.068+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:11.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:11.068+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:11.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:11.281+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:11.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:11.281+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:11.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:11.281+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:11.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:11.451+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:11.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:11.451+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:11.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:11 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:11.451+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:12.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:11.786+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:12.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:11.786+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:12.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:11.786+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:12.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:11 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:11.786+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:12.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:12.105+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:12.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:12.105+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:12.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:12.105+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:12.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:12.105+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:12.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:12 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:12.009+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:12.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:12 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:12.009+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:12.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:12 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:12.009+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:12.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:12 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:12.009+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:12.490 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:12 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:12.009+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:12.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:12.259+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:12.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:12.259+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:12.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:12.259+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:12.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:12.448+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:20.179049+0000 front 2024-09-12T02:13:20.179066+0000 (oldest deadline 2024-09-12T02:13:50.780040+0000) 2024-09-12T02:14:12.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:12.448+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.881257+0000 front 2024-09-12T02:13:31.881215+0000 (oldest deadline 2024-09-12T02:13:54.181377+0000) 2024-09-12T02:14:12.866 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7[52821]: 2024-09-12T02:14:12.448+0000 7fe6e9881640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.882334+0000 front 2024-09-12T02:13:38.882377+0000 (oldest deadline 2024-09-12T02:14:03.582452+0000) 2024-09-12T02:14:13.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:12.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:13.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:12.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:13.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:12.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:13.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:12 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:12.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:13.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:13 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:13.097+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:13.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:13 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:13.097+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:13.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:13 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:13.097+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:13.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:13 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:13.097+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:13.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:13 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:13.097+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:13.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:13 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:13.052+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:13.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:13 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:13.052+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:13.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:13 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:13.052+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:13.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:13 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:13.052+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:13.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:13 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:13.052+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:13.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:13.222+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:13.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:13.222+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:13.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:13.222+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:14.109 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:13 smithi151 podman[73745]: 2024-09-12 02:14:13.751045236 +0000 UTC m=+5.722269228 container died eb3ba75362b5bd8d16f15ccbe2a514c9cf30d24027dcb1d2eefd4171b67d7a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:14:14.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:13.813+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:14.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:13.813+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:14.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:13.813+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:14.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:13.813+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:14.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:13 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:13.813+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:14.365 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:14 smithi151 podman[73745]: 2024-09-12 02:14:14.265734179 +0000 UTC m=+6.236958170 container cleanup eb3ba75362b5bd8d16f15ccbe2a514c9cf30d24027dcb1d2eefd4171b67d7a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T02:14:14.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:14 smithi151 bash[73745]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7 2024-09-12T02:14:14.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:14 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:14.109+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:14.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:14 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:14.109+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:14.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:14 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:14.109+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:14.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:14 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:14.109+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:14.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:14 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:14.109+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:14.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.012+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:14.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.012+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:14.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.012+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:14.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.012+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:14.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.012+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:14.702 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:14.205+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:14.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:14.205+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:14.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:14.205+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:14.865 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:14 smithi151 podman[73761]: 2024-09-12 02:14:14.550954898 +0000 UTC m=+0.796371382 container remove eb3ba75362b5bd8d16f15ccbe2a514c9cf30d24027dcb1d2eefd4171b67d7a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T02:14:15.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:14.808+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:15.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:14.808+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:15.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:14.808+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:15.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:14.808+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:15.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:14 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:14.808+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:15 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:15.141+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:15 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:15.141+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:15 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:15.141+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:15 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:15.141+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:15 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:15.141+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:15 smithi151 podman[73850]: 2024-09-12 02:14:15.035294754 +0000 UTC m=+0.110562860 container create ed95288c20dbcca57ebff2c5af95516913d9ab5986882841a8b92120576e09c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:15 smithi151 podman[73850]: 2024-09-12 02:14:14.940649016 +0000 UTC m=+0.015917129 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:15 smithi151 podman[73850]: 2024-09-12 02:14:15.280210038 +0000 UTC m=+0.355478155 container init ed95288c20dbcca57ebff2c5af95516913d9ab5986882841a8b92120576e09c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T02:14:15.366 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:15 smithi151 podman[73850]: 2024-09-12 02:14:15.283729069 +0000 UTC m=+0.358997180 container start ed95288c20dbcca57ebff2c5af95516913d9ab5986882841a8b92120576e09c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T02:14:15.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.993+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:15.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.993+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:15.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.993+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:15.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.993+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:15.489 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:14 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:14.993+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:15.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:15.243+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:15.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:15.243+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:15.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:15.243+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:15.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:15.243+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:15.783 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:15 smithi151 podman[73850]: 2024-09-12 02:14:15.39188066 +0000 UTC m=+0.467148767 container attach ed95288c20dbcca57ebff2c5af95516913d9ab5986882841a8b92120576e09c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:14:15.783 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 12 02:14:15 smithi151 podman[73850]: 2024-09-12 02:14:15.476692614 +0000 UTC m=+0.551960734 container died ed95288c20dbcca57ebff2c5af95516913d9ab5986882841a8b92120576e09c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T02:14:16.160 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T02:14:16.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:15.761+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:16.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:15.761+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:16.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:15.761+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:16.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:15.761+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:16.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:15 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:15.761+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:16.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:15 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:15.962+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:16.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:15 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:15.962+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:16.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:15 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:15.962+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:16.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:15 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:15.962+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:16.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:15 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:15.962+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:16.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:16.244 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T02:14:16.244 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T02:14:16.244 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.8 2024-09-12T02:14:16.426 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:16.158+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:16.426 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:16.158+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:16.426 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:16.158+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:16.426 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:16.158+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:16.426 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:16.158+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:16.560 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 systemd[1]: Stopping Ceph osd.8 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:16.560 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:16.560 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:16.560 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:16.560 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:16.560 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.560+0000 7f0b72a38640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:16.561 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.560+0000 7f0b72a38640 -1 osd.8 91 *** Got signal Terminated *** 2024-09-12T02:14:16.561 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:16.560+0000 7f0b72a38640 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:14:16.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:16.774+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:16.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:16.774+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:16.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:16.774+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:16.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:16.774+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:16.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:16 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:16.774+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:17.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:16.937+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:17.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:16.937+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:17.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:16.937+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:17.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:16.937+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:17.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:16 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:16.937+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:17.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:17 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:17.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:17.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:17 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:17.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:17.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:17 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:17.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:17.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:17 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:17.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:17.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:17 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:17.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:17.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:17.275+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:17.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:17.275+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:17.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:17.275+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:17.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:17.275+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:18.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:17.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:18.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:17.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:18.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:17.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:18.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:17.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:18.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:17 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:17.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:18.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:17.941+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:18.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:17.941+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:18.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:17.941+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:18.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:17.941+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:18.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:17 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:17.941+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:18.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:18.180+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:18.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:18.180+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:18.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:18.180+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:18.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:18.180+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:18.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:18 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:18.180+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:18.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:18.311+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:18.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:18.311+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:18.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:18.311+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:18.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:18.311+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:19.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:18.851+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:19.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:18.851+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:19.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:18.851+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:19.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:18.851+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:19.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:18 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:18.851+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:19.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:18.963+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:19.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:18.963+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:19.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:18.963+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:19.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:18.963+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:19.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:18 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:18.963+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:19.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:19.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:19.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:19.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:19.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:19.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:19.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:19.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:19.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:19 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:19.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:19.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:19.289+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:19.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:19.289+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:19.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:19.289+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:19.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:19.289+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:20.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:19.868+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:20.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:19.868+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:20.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:19.868+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:20.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:19.868+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:20.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:19 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:19.868+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:20.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:19.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:20.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:19.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:20.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:19.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:20.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:19.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:20.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:19.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:20.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:19 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:19.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:20.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:20 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:20.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:20.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:20 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:20.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:20.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:20 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:20.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:20.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:20 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:20.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:20.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:20 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:20.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:20.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:20 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:20.208+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:20.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:20.303+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:20.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:20.303+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:20.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:20.303+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:20.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:20.303+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:20.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:20.303+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:54.370279+0000 front 2024-09-12T02:13:54.370330+0000 (oldest deadline 2024-09-12T02:14:19.670433+0000) 2024-09-12T02:14:21.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:20.834+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:21.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:20.834+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:21.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:20.834+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:21.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:20.834+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:21.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:20.834+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:21.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:20.953+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:21.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:20.953+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:21.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:20.953+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:21.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:20.953+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:21.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:20.953+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:21.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:20 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:20.953+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:21.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:21 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:21.176+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:21.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:21 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:21.176+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:21.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:21 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:21.176+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:21.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:21 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:21.176+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:21.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:21 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:21.176+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:21.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:21 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:21.176+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:21.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:21.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.163842+0000 front 2024-09-12T02:13:12.163864+0000 (oldest deadline 2024-09-12T02:13:37.463828+0000) 2024-09-12T02:14:21.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:21.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.467453+0000 front 2024-09-12T02:13:31.467479+0000 (oldest deadline 2024-09-12T02:13:54.367681+0000) 2024-09-12T02:14:21.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:21.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:38.468859+0000 front 2024-09-12T02:13:38.468817+0000 (oldest deadline 2024-09-12T02:14:04.369032+0000) 2024-09-12T02:14:21.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:21.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.469869+0000 front 2024-09-12T02:13:48.469976+0000 (oldest deadline 2024-09-12T02:14:14.369852+0000) 2024-09-12T02:14:21.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8[53069]: 2024-09-12T02:14:21.290+0000 7f0b6f04e640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:54.370279+0000 front 2024-09-12T02:13:54.370330+0000 (oldest deadline 2024-09-12T02:14:19.670433+0000) 2024-09-12T02:14:22.128 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:22 smithi183 podman[74115]: 2024-09-12 02:14:22.100932097 +0000 UTC m=+5.663359611 container stop 908fcc2c8ea4858bcd4d3c9f3cb41ec95aad5bb1529266fdb509b1d075fbf8bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T02:14:22.128 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:21.863+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:22.128 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:21.863+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:22.129 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:21.863+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:22.129 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:21.863+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:22.129 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:21.863+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:22.129 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:21 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:21.863+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:22.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:21.945+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:22.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:21.945+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:22.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:21.945+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:22.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:21.945+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:22.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:21.945+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:22.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:21 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:21.945+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:22.452 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:22 smithi183 podman[74115]: 2024-09-12 02:14:22.128936315 +0000 UTC m=+5.691363841 container died 908fcc2c8ea4858bcd4d3c9f3cb41ec95aad5bb1529266fdb509b1d075fbf8bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:14:22.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:22 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:22.226+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:22.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:22 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:22.226+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:22.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:22 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:22.226+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:22.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:22 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:22.226+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:22.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:22 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:22.226+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:22.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:22 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:22.226+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:22.753 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:22 smithi183 podman[74115]: 2024-09-12 02:14:22.543680292 +0000 UTC m=+6.106107805 container cleanup 908fcc2c8ea4858bcd4d3c9f3cb41ec95aad5bb1529266fdb509b1d075fbf8bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T02:14:22.753 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:22 smithi183 bash[74115]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8 2024-09-12T02:14:23.159 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:22 smithi183 podman[74130]: 2024-09-12 02:14:22.753484403 +0000 UTC m=+0.646502503 container remove 908fcc2c8ea4858bcd4d3c9f3cb41ec95aad5bb1529266fdb509b1d075fbf8bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T02:14:23.159 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:22 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:22.817+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:23.159 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:22 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:22.817+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:23.159 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:22 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:22.817+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:23.160 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:22 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:22.817+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:23.160 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:22 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:22.817+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:23.160 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:22 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:22.817+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:23.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:22 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:22.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:23.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:22 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:22.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:23.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:22 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:22.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:23.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:22 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:22.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:23.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:22 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:22.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:23.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:22 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:22.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:23.453 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:23 smithi183 podman[74217]: 2024-09-12 02:14:23.062275638 +0000 UTC m=+0.015541321 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:14:23.453 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:23 smithi183 podman[74217]: 2024-09-12 02:14:23.187628488 +0000 UTC m=+0.140894220 container create d798985def4b13f1e099050a9eead71c713daabe18ed3828f278b8d3df8959b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:14:23.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:23 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:23.199+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:23.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:23 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:23.199+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:23.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:23 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:23.199+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:23.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:23 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:23.199+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:23.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:23 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:23.199+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:23.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:23 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:23.199+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:23.801 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:23 smithi183 podman[74217]: 2024-09-12 02:14:23.490597331 +0000 UTC m=+0.443863019 container init d798985def4b13f1e099050a9eead71c713daabe18ed3828f278b8d3df8959b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T02:14:23.801 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:23 smithi183 podman[74217]: 2024-09-12 02:14:23.494337885 +0000 UTC m=+0.447603569 container start d798985def4b13f1e099050a9eead71c713daabe18ed3828f278b8d3df8959b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T02:14:23.801 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:23 smithi183 podman[74217]: 2024-09-12 02:14:23.590425041 +0000 UTC m=+0.543690727 container attach d798985def4b13f1e099050a9eead71c713daabe18ed3828f278b8d3df8959b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T02:14:23.802 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:14:23 smithi183 podman[74217]: 2024-09-12 02:14:23.680088021 +0000 UTC m=+0.633353717 container died d798985def4b13f1e099050a9eead71c713daabe18ed3828f278b8d3df8959b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T02:14:24.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:23 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:23.800+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:24.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:23 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:23.800+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:24.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:23 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:23.801+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:24.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:23 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:23.801+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:24.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:23 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:23.801+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:24.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:23 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:23.801+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:24.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:23.905+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:24.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:23.905+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:24.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:23.905+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:24.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:23.905+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:24.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:23.905+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:24.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:23 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:23.905+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:24.344 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T02:14:24.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:24.385 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T02:14:24.385 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T02:14:24.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.9 2024-09-12T02:14:24.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:24 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:24.172+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:24.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:24 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:24.172+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:24.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:24 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:24.172+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:24.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:24 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:24.172+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:24.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:24 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:24.172+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:24.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:24 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:24.172+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:24.713 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 systemd[1]: Stopping Ceph osd.9 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:24.989 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.713+0000 7f1fc3bf2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:24.989 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.713+0000 7f1fc3bf2640 -1 osd.9 91 *** Got signal Terminated *** 2024-09-12T02:14:24.989 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.713+0000 7f1fc3bf2640 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:14:24.989 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:24.989 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:24.989 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:24.990 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:24.990 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:24.990 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:24 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:24.942+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:25.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:24.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:25.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:24.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:25.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:24.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:25.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:24.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:25.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:24.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:25.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:24.820+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:25.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:25 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:25.217+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:25.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:25 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:25.217+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:25.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:25 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:25.217+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:25.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:25 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:25.217+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:25.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:25 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:25.217+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:25.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:25 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:25.217+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:26.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:25 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:25.862+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:26.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:25 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:25.862+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:26.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:25 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:25.862+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:26.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:25 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:25.862+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:26.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:25 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:25.862+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:26.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:25 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:25.862+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:26.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:25 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:25.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:26.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:25 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:25.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:26.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:25 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:25.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:26.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:25 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:25.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:26.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:25 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:25.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:26.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:25 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:25.908+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:26.517 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:26.517 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:26.517 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:26.517 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:26.517 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:26.518 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:26.518 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:26 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:26.195+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:27.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:26.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:27.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:26.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:27.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:26.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:27.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:26.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:27.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:26.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:27.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:26 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:26.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:27.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:26.944+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:27.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:26.944+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:27.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:26.944+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:27.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:26.944+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:27.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:26.944+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:27.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:26 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:26.944+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:27.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:27.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:27.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:27.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:27.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:27.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:27.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:27 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:27.161+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:28.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:27.886+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:28.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:27.886+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:28.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:27.886+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:28.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:27.886+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:28.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:27.886+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:28.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:27 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:27.886+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:28.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:27 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:27.915+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:28.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:27 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:27.915+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:28.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:27 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:27.915+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:28.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:27 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:27.915+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:28.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:27 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:27.915+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:28.240 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:27 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:27.915+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:28.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:28.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:28.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:28.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:28.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:28.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:28.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:28 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:28.142+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:29.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:28 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:28.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:29.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:28 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:28.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:29.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:28 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:28.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:29.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:28 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:28.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:29.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:28 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:28.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:29.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:28 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:28.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:29.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:28 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:28.943+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:12.800158+0000 front 2024-09-12T02:13:12.800137+0000 (oldest deadline 2024-09-12T02:13:38.700305+0000) 2024-09-12T02:14:29.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:28 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:28.943+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:24.403029+0000 front 2024-09-12T02:13:24.402884+0000 (oldest deadline 2024-09-12T02:13:49.102982+0000) 2024-09-12T02:14:29.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:28 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:28.943+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:29.604017+0000 front 2024-09-12T02:13:29.604200+0000 (oldest deadline 2024-09-12T02:13:56.004689+0000) 2024-09-12T02:14:29.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:28 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:28.943+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.606230+0000 front 2024-09-12T02:13:40.606258+0000 (oldest deadline 2024-09-12T02:14:06.506434+0000) 2024-09-12T02:14:29.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:28 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:28.943+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:46.506942+0000 front 2024-09-12T02:13:46.507074+0000 (oldest deadline 2024-09-12T02:14:11.206944+0000) 2024-09-12T02:14:29.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:28 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9[64892]: 2024-09-12T02:14:28.943+0000 7f1fc0208640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:57.008563+0000 front 2024-09-12T02:13:57.008453+0000 (oldest deadline 2024-09-12T02:14:19.308363+0000) 2024-09-12T02:14:29.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:29.365 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:29.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:29.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:29.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:29.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:29.366 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:29 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:29.095+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:30.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:29.900+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:30.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:29.900+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:30.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:29.900+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:30.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:29.900+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:30.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:29.900+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:30.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:29.900+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:30.430 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:30 smithi122 podman[115574]: 2024-09-12 02:14:30.170277298 +0000 UTC m=+5.583524044 container died 8d551e800f3379a217393c04fac15c5bfd5d6b0985eb0a777e818fcc237a5944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:30.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:30 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:30.121+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:30.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:30 smithi122 podman[115574]: 2024-09-12 02:14:30.58856446 +0000 UTC m=+6.001811205 container cleanup 8d551e800f3379a217393c04fac15c5bfd5d6b0985eb0a777e818fcc237a5944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T02:14:30.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:30 smithi122 bash[115574]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9 2024-09-12T02:14:31.182 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:30 smithi122 podman[115592]: 2024-09-12 02:14:30.781666427 +0000 UTC m=+0.609442625 container remove 8d551e800f3379a217393c04fac15c5bfd5d6b0985eb0a777e818fcc237a5944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T02:14:31.202 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:30 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:30.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:31.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:30 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:30.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:31.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:30 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:30.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:31.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:30 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:30.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:31.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:30 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:30.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:31.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:30 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:30.879+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:31.435 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:31 smithi122 podman[115678]: 2024-09-12 02:14:31.18244413 +0000 UTC m=+0.095410360 container create 05da4aedf9bc62fca6613b7d36a1108c25bc31c1b5bedc33a56d2ad9fc78a9fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T02:14:31.435 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:31 smithi122 podman[115678]: 2024-09-12 02:14:31.10252677 +0000 UTC m=+0.015492965 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:14:31.435 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:31 smithi122 podman[115678]: 2024-09-12 02:14:31.435198708 +0000 UTC m=+0.348164903 container init 05da4aedf9bc62fca6613b7d36a1108c25bc31c1b5bedc33a56d2ad9fc78a9fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T02:14:31.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:31.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:31.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:31.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:31.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:31.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:31.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:31 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:31.133+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:31.739 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:31 smithi122 podman[115678]: 2024-09-12 02:14:31.439321035 +0000 UTC m=+0.352287229 container start 05da4aedf9bc62fca6613b7d36a1108c25bc31c1b5bedc33a56d2ad9fc78a9fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:14:31.740 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:31 smithi122 podman[115678]: 2024-09-12 02:14:31.551758886 +0000 UTC m=+0.464725086 container attach 05da4aedf9bc62fca6613b7d36a1108c25bc31c1b5bedc33a56d2ad9fc78a9fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T02:14:31.740 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:31 smithi122 podman[115678]: 2024-09-12 02:14:31.621200599 +0000 UTC m=+0.534166788 container died 05da4aedf9bc62fca6613b7d36a1108c25bc31c1b5bedc33a56d2ad9fc78a9fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T02:14:32.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:31 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:31.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:32.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:31 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:31.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:32.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:31 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:31.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:32.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:31 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:31.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:32.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:31 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:31.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:32.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:31 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:31.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:32.239 INFO:journalctl@ceph.osd.9.smithi122.stdout:Sep 12 02:14:32 smithi122 podman[115701]: 2024-09-12 02:14:32.222655699 +0000 UTC m=+0.592103718 container remove 05da4aedf9bc62fca6613b7d36a1108c25bc31c1b5bedc33a56d2ad9fc78a9fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:14:32.246 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T02:14:32.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:32.286 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T02:14:32.286 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T02:14:32.286 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.10 2024-09-12T02:14:32.614 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.125+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 systemd[1]: Stopping Ceph osd.10 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.613+0000 7f476a1ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:32.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.613+0000 7f476a1ea640 -1 osd.10 91 *** Got signal Terminated *** 2024-09-12T02:14:32.865 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:32 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:32.613+0000 7f476a1ea640 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:14:33.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:32 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:32.890+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:33.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:32 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:32.890+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:33.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:32 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:32.890+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:33.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:32 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:32.890+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:33.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:32 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:32.890+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:33.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:32 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:32.890+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:33.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:33 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:33.130+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:34.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:33 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:33.883+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:34.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:33 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:33.883+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:34.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:33 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:33.883+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:34.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:33 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:33.883+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:34.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:33 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:33.883+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:34.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:33 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:33.883+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:34.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:34.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:34.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:34.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:34.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:34.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:34.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:34 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:34.148+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:35.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:34.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:35.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:34.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:35.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:34.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:35.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:34.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:35.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:34.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:35.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:34.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:35.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:35.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:35.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:35.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:35.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:35.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:35.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:35 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:35.153+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:36.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:35.906+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:36.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:35.906+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:36.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:35.906+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:36.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:35.906+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:36.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:35.906+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:36.205 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:35 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:35.906+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:36.574 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:36.575 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:36.575 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:36.575 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:36.575 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:36.575 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:36.575 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:36 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:36.183+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:37.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:36 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:36.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:37.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:36 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:36.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:37.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:36 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:36.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:37.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:36 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:36.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:37.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:36 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:36.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:37.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:36 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:36.887+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:37.615 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.589037+0000 front 2024-09-12T02:13:15.588932+0000 (oldest deadline 2024-09-12T02:13:37.288880+0000) 2024-09-12T02:14:37.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:23.190092+0000 front 2024-09-12T02:13:23.190157+0000 (oldest deadline 2024-09-12T02:13:47.290246+0000) 2024-09-12T02:14:37.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:32.492012+0000 front 2024-09-12T02:13:32.491993+0000 (oldest deadline 2024-09-12T02:13:55.992026+0000) 2024-09-12T02:14:37.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:37.093048+0000 front 2024-09-12T02:13:37.093092+0000 (oldest deadline 2024-09-12T02:14:02.993236+0000) 2024-09-12T02:14:37.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:48.894102+0000 front 2024-09-12T02:13:48.894063+0000 (oldest deadline 2024-09-12T02:14:12.994176+0000) 2024-09-12T02:14:37.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:55.295386+0000 front 2024-09-12T02:13:55.295531+0000 (oldest deadline 2024-09-12T02:14:19.395355+0000) 2024-09-12T02:14:37.616 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:37 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10[59370]: 2024-09-12T02:14:37.147+0000 7f47627f8640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-09-12T02:14:04.096489+0000 front 2024-09-12T02:14:04.096471+0000 (oldest deadline 2024-09-12T02:14:25.796341+0000) 2024-09-12T02:14:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:37.847+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:37.847+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:37.847+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:37.847+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:37.847+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:38.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:37 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:37.847+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:38.499 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:38 smithi151 podman[73964]: 2024-09-12 02:14:38.191294527 +0000 UTC m=+5.703369587 container died dc321a7eec3be2966bde194f6e120b2c720471ccb6d968b5a9d69f8b12a0db2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T02:14:38.824 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:38 smithi151 podman[73964]: 2024-09-12 02:14:38.601135972 +0000 UTC m=+6.113211078 container cleanup dc321a7eec3be2966bde194f6e120b2c720471ccb6d968b5a9d69f8b12a0db2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T02:14:38.824 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:38 smithi151 bash[73964]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10 2024-09-12T02:14:39.115 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:38 smithi151 podman[73981]: 2024-09-12 02:14:38.824502366 +0000 UTC m=+0.630201880 container remove dc321a7eec3be2966bde194f6e120b2c720471ccb6d968b5a9d69f8b12a0db2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:39.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:38 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:38.802+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:39.486 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:39 smithi151 podman[74068]: 2024-09-12 02:14:39.127393996 +0000 UTC m=+0.016117853 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:14:39.486 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:39 smithi151 podman[74068]: 2024-09-12 02:14:39.234088354 +0000 UTC m=+0.122812210 container create cc4b443e71dc6ea5648013ddbd2843da8a0be2c3096e9fd44107558f2566a66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T02:14:39.486 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:39 smithi151 podman[74068]: 2024-09-12 02:14:39.486874075 +0000 UTC m=+0.375597934 container init cc4b443e71dc6ea5648013ddbd2843da8a0be2c3096e9fd44107558f2566a66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:14:39.866 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:39 smithi151 podman[74068]: 2024-09-12 02:14:39.49014462 +0000 UTC m=+0.378868476 container start cc4b443e71dc6ea5648013ddbd2843da8a0be2c3096e9fd44107558f2566a66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T02:14:39.866 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:39 smithi151 podman[74068]: 2024-09-12 02:14:39.578714376 +0000 UTC m=+0.467438256 container attach cc4b443e71dc6ea5648013ddbd2843da8a0be2c3096e9fd44107558f2566a66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:14:39.866 INFO:journalctl@ceph.osd.10.smithi151.stdout:Sep 12 02:14:39 smithi151 podman[74068]: 2024-09-12 02:14:39.6840061 +0000 UTC m=+0.572729959 container died cc4b443e71dc6ea5648013ddbd2843da8a0be2c3096e9fd44107558f2566a66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:40.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:39 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:39.852+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:40.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T02:14:40.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:40.372 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T02:14:40.372 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T02:14:40.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.11 2024-09-12T02:14:40.688 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 systemd[1]: Stopping Ceph osd.11 for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.688+0000 7f4be6f96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.688+0000 7f4be6f96640 -1 osd.11 91 *** Got signal Terminated *** 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.688+0000 7f4be6f96640 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:40.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:40 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:40.825+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:42.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:41 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:41.859+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:43.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:43.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:43.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:43.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:43.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:43.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:43.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:43.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:42 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:42.867+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-12T02:14:20.565785+0000 front 2024-09-12T02:14:20.565806+0000 (oldest deadline 2024-09-12T02:14:42.265881+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:44.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:43 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:43.905+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-12T02:14:20.565785+0000 front 2024-09-12T02:14:20.565806+0000 (oldest deadline 2024-09-12T02:14:42.265881+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-09-12T02:13:15.955441+0000 front 2024-09-12T02:13:15.955314+0000 (oldest deadline 2024-09-12T02:13:37.655270+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-09-12T02:13:21.156361+0000 front 2024-09-12T02:13:21.156414+0000 (oldest deadline 2024-09-12T02:13:45.256212+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T02:13:31.657688+0000 front 2024-09-12T02:13:31.657663+0000 (oldest deadline 2024-09-12T02:13:53.957834+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-09-12T02:13:40.359225+0000 front 2024-09-12T02:13:40.359179+0000 (oldest deadline 2024-09-12T02:14:03.259132+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-09-12T02:13:47.860919+0000 front 2024-09-12T02:13:47.860905+0000 (oldest deadline 2024-09-12T02:14:13.661357+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T02:13:53.661872+0000 front 2024-09-12T02:13:53.661939+0000 (oldest deadline 2024-09-12T02:14:21.862589+0000) 2024-09-12T02:14:45.203 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-09-12T02:14:12.964970+0000 front 2024-09-12T02:14:12.964878+0000 (oldest deadline 2024-09-12T02:14:38.264923+0000) 2024-09-12T02:14:45.204 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:44 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11[59767]: 2024-09-12T02:14:44.873+0000 7f4be35ac640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-12T02:14:20.565785+0000 front 2024-09-12T02:14:20.565806+0000 (oldest deadline 2024-09-12T02:14:42.265881+0000) 2024-09-12T02:14:46.603 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:46 smithi183 podman[74331]: 2024-09-12 02:14:46.27604226 +0000 UTC m=+5.709161538 container died 0bb907b9b9bea95b8138f4eea3e6d828cdcfa6109a65e5819287a9d35fdfe0cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T02:14:46.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:46 smithi183 podman[74331]: 2024-09-12 02:14:46.604335363 +0000 UTC m=+6.037454642 container cleanup 0bb907b9b9bea95b8138f4eea3e6d828cdcfa6109a65e5819287a9d35fdfe0cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T02:14:46.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:46 smithi183 bash[74331]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11 2024-09-12T02:14:46.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:46 smithi183 podman[74346]: 2024-09-12 02:14:46.797307869 +0000 UTC m=+0.514899693 container remove 0bb907b9b9bea95b8138f4eea3e6d828cdcfa6109a65e5819287a9d35fdfe0cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T02:14:47.453 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:47 smithi183 podman[74434]: 2024-09-12 02:14:47.198196333 +0000 UTC m=+0.107127688 container create d3c0fcc81093cc5ae8f954373d3a9f64df8d71ff7dd5a338fd665b03860b4983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:14:47.453 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:47 smithi183 podman[74434]: 2024-09-12 02:14:47.106702016 +0000 UTC m=+0.015633341 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:14:47.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:47 smithi183 podman[74434]: 2024-09-12 02:14:47.472640691 +0000 UTC m=+0.381572002 container init d3c0fcc81093cc5ae8f954373d3a9f64df8d71ff7dd5a338fd665b03860b4983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T02:14:47.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:47 smithi183 podman[74434]: 2024-09-12 02:14:47.475963991 +0000 UTC m=+0.384895302 container start d3c0fcc81093cc5ae8f954373d3a9f64df8d71ff7dd5a338fd665b03860b4983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:14:47.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:47 smithi183 podman[74434]: 2024-09-12 02:14:47.567192249 +0000 UTC m=+0.476123562 container attach d3c0fcc81093cc5ae8f954373d3a9f64df8d71ff7dd5a338fd665b03860b4983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:14:47.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 02:14:47 smithi183 podman[74434]: 2024-09-12 02:14:47.665119658 +0000 UTC m=+0.574050986 container died d3c0fcc81093cc5ae8f954373d3a9f64df8d71ff7dd5a338fd665b03860b4983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T02:14:48.267 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T02:14:48.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:48.306 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T02:14:48.306 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T02:14:48.307 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.a 2024-09-12T02:14:48.609 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 02:14:48 smithi122 systemd[1]: Stopping Ceph mds.a for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:48.989 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 02:14:48 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a[71914]: 2024-09-12T02:14:48.608+0000 7f0a56ba3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:48.989 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 02:14:48 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a[71914]: 2024-09-12T02:14:48.608+0000 7f0a56ba3640 -1 mds.a *** got signal Terminated *** 2024-09-12T02:14:51.908 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 02:14:51 smithi122 podman[115792]: 2024-09-12 02:14:51.575346255 +0000 UTC m=+3.075549646 container died b2eb61ef28b76fcdbe18f98e14210bb5555a112192abed87c7bdd39aab9d6f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:14:52.200 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 02:14:51 smithi122 podman[115792]: 2024-09-12 02:14:51.999271639 +0000 UTC m=+3.499475036 container cleanup b2eb61ef28b76fcdbe18f98e14210bb5555a112192abed87c7bdd39aab9d6f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:14:52.200 INFO:journalctl@ceph.mds.a.smithi122.stdout:Sep 12 02:14:51 smithi122 bash[115792]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-a 2024-09-12T02:14:52.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T02:14:52.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:52.460 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T02:14:52.461 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T02:14:52.461 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.d 2024-09-12T02:14:52.739 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 02:14:52 smithi122 systemd[1]: Stopping Ceph mds.d for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:53.239 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 02:14:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-d[70726]: 2024-09-12T02:14:52.795+0000 7fd7bd489640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:53.240 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 02:14:52 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-d[70726]: 2024-09-12T02:14:52.795+0000 7fd7bd489640 -1 mds.d *** got signal Terminated *** 2024-09-12T02:14:56.187 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 02:14:55 smithi122 podman[115912]: 2024-09-12 02:14:55.919567754 +0000 UTC m=+3.237061191 container died 4525fa5dcf64282cc0a14d6e09d6faf42831dbdc9b8eee4f8f0844bba9969d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:14:56.489 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 02:14:56 smithi122 podman[115912]: 2024-09-12 02:14:56.270158563 +0000 UTC m=+3.587651988 container cleanup 4525fa5dcf64282cc0a14d6e09d6faf42831dbdc9b8eee4f8f0844bba9969d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T02:14:56.489 INFO:journalctl@ceph.mds.d.smithi122.stdout:Sep 12 02:14:56 smithi122 bash[115912]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-d 2024-09-12T02:14:56.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T02:14:56.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:14:56.736 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T02:14:56.736 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T02:14:56.736 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.g 2024-09-12T02:14:57.090 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 02:14:56 smithi122 systemd[1]: Stopping Ceph mds.g for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:14:57.489 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 02:14:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g[71516]: 2024-09-12T02:14:57.090+0000 7f2eaac19640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:14:57.489 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 02:14:57 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g[71516]: 2024-09-12T02:14:57.090+0000 7f2eaac19640 -1 mds.g *** got signal Terminated *** 2024-09-12T02:15:00.538 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 02:15:00 smithi122 podman[116031]: 2024-09-12 02:15:00.256361725 +0000 UTC m=+3.296525744 container died 4aa22800bb3467c5ba9ac5070abb9380e0380e2486e9f6cf9705d25c31633b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T02:15:00.858 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 02:15:00 smithi122 podman[116031]: 2024-09-12 02:15:00.640699864 +0000 UTC m=+3.680863884 container cleanup 4aa22800bb3467c5ba9ac5070abb9380e0380e2486e9f6cf9705d25c31633b87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T02:15:00.859 INFO:journalctl@ceph.mds.g.smithi122.stdout:Sep 12 02:15:00 smithi122 bash[116031]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-g 2024-09-12T02:15:01.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T02:15:01.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:01.036 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T02:15:01.037 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T02:15:01.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.j 2024-09-12T02:15:01.489 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:01 smithi122 systemd[1]: Stopping Ceph mds.j for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:01.489 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j[70308]: 2024-09-12T02:15:01.344+0000 7f5827fe2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:01.489 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:01 smithi122 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j[70308]: 2024-09-12T02:15:01.344+0000 7f5827fe2640 -1 mds.j *** got signal Terminated *** 2024-09-12T02:15:04.911 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:04 smithi122 podman[116152]: 2024-09-12 02:15:04.585525441 +0000 UTC m=+3.332558248 container died 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-12T02:15:04.911 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:04 smithi122 podman[116152]: 2024-09-12 02:15:04.911397023 +0000 UTC m=+3.658429784 container cleanup 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T02:15:05.239 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:04 smithi122 bash[116152]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j 2024-09-12T02:15:05.239 INFO:journalctl@ceph.mds.j.smithi122.stdout:Sep 12 02:15:05 smithi122 podman[116164]: 2024-09-12 02:15:05.104043745 +0000 UTC m=+0.511064411 container remove 285a0d556f53c894cacae479d617e395bb9df9b2d0f80269c6851b8ad3bb0912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T02:15:05.334 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T02:15:05.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:05.366 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T02:15:05.367 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T02:15:05.367 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.b 2024-09-12T02:15:05.667 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:05 smithi151 systemd[1]: Stopping Ceph mds.b for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:05.923 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: 2024-09-12T02:15:05.667+0000 7f76352a9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:05.923 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:05 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b[65027]: 2024-09-12T02:15:05.667+0000 7f76352a9640 -1 mds.b *** got signal Terminated *** 2024-09-12T02:15:07.985 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:07 smithi151 podman[74170]: 2024-09-12 02:15:07.673717653 +0000 UTC m=+2.115275501 container died f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T02:15:08.277 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:08 smithi151 podman[74170]: 2024-09-12 02:15:08.084176071 +0000 UTC m=+2.525733918 container cleanup f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T02:15:08.277 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:08 smithi151 bash[74170]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b 2024-09-12T02:15:08.511 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T02:15:08.533 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:08 smithi151 podman[74194]: 2024-09-12 02:15:08.277494966 +0000 UTC m=+0.593991019 container remove f3305815c64ce187a459df829fe2a18cde3fd2e6467b6cfe51a55d58a17d4e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T02:15:08.533 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:08 smithi151 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-12T02:15:08.533 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:08 smithi151 systemd[1]: Stopped Ceph mds.b for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T02:15:08.533 INFO:journalctl@ceph.mds.b.smithi151.stdout:Sep 12 02:15:08 smithi151 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.b.service: Consumed 8min 52.878s CPU time. 2024-09-12T02:15:08.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:08.547 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T02:15:08.547 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T02:15:08.547 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.e 2024-09-12T02:15:08.866 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:15:08 smithi151 systemd[1]: Stopping Ceph mds.e for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:09.365 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:15:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e[64404]: 2024-09-12T02:15:08.884+0000 7f3eaad15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:09.366 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:15:08 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e[64404]: 2024-09-12T02:15:08.884+0000 7f3eaad15640 -1 mds.e *** got signal Terminated *** 2024-09-12T02:15:11.956 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:15:11 smithi151 podman[74305]: 2024-09-12 02:15:11.672865525 +0000 UTC m=+2.907998064 container died ac2e85ffdfb7b5d82758d38867ca97708b75eb2dadcf999019f9966ba122d071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:15:12.324 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:15:12 smithi151 podman[74305]: 2024-09-12 02:15:12.056321106 +0000 UTC m=+3.291453634 container cleanup ac2e85ffdfb7b5d82758d38867ca97708b75eb2dadcf999019f9966ba122d071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T02:15:12.324 INFO:journalctl@ceph.mds.e.smithi151.stdout:Sep 12 02:15:12 smithi151 bash[74305]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-e 2024-09-12T02:15:12.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T02:15:12.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:12.490 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T02:15:12.490 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T02:15:12.490 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.h 2024-09-12T02:15:12.615 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:15:12 smithi151 systemd[1]: Stopping Ceph mds.h for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:13.115 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:15:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T02:15:12.810+0000 7ffa686cd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:13.115 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:15:12 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h[64705]: 2024-09-12T02:15:12.810+0000 7ffa686cd640 -1 mds.h *** got signal Terminated *** 2024-09-12T02:15:15.961 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:15:15 smithi151 podman[74425]: 2024-09-12 02:15:15.66729656 +0000 UTC m=+2.959981658 container died 7c09818ca91006a61d36727367f1e06c838e5dca42a23ec4daaaaedc24afc494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T02:15:16.358 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:15:16 smithi151 podman[74425]: 2024-09-12 02:15:16.044579822 +0000 UTC m=+3.337264920 container cleanup 7c09818ca91006a61d36727367f1e06c838e5dca42a23ec4daaaaedc24afc494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:15:16.358 INFO:journalctl@ceph.mds.h.smithi151.stdout:Sep 12 02:15:16 smithi151 bash[74425]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-h 2024-09-12T02:15:16.531 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T02:15:16.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:16.564 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T02:15:16.564 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T02:15:16.564 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.k 2024-09-12T02:15:16.933 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:15:16 smithi151 systemd[1]: Stopping Ceph mds.k for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:16.933 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:15:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: 2024-09-12T02:15:16.874+0000 7f8a47982640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:16.933 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:15:16 smithi151 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k[64105]: 2024-09-12T02:15:16.874+0000 7f8a47982640 -1 mds.k *** got signal Terminated *** 2024-09-12T02:15:19.949 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:15:19 smithi151 podman[74545]: 2024-09-12 02:15:19.68246182 +0000 UTC m=+2.902255783 container died fbb294fe9528930e406da6ac134b15c924cc4e6c32953217fe60fccf1f82e27e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T02:15:20.365 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:15:20 smithi151 podman[74545]: 2024-09-12 02:15:20.041751592 +0000 UTC m=+3.261545552 container cleanup fbb294fe9528930e406da6ac134b15c924cc4e6c32953217fe60fccf1f82e27e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:15:20.366 INFO:journalctl@ceph.mds.k.smithi151.stdout:Sep 12 02:15:20 smithi151 bash[74545]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-k 2024-09-12T02:15:20.619 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T02:15:20.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:20.651 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T02:15:20.651 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T02:15:20.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.c 2024-09-12T02:15:20.945 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:15:20 smithi183 systemd[1]: Stopping Ceph mds.c for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:21.202 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:15:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c[64735]: 2024-09-12T02:15:20.943+0000 7f04a39f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:21.203 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:15:20 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c[64735]: 2024-09-12T02:15:20.943+0000 7f04a39f0640 -1 mds.c *** got signal Terminated *** 2024-09-12T02:15:23.928 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:15:23 smithi183 podman[74537]: 2024-09-12 02:15:23.653813462 +0000 UTC m=+2.804589611 container died 9902db5727df5c4ee0b641a7fb98ab6aa39b8c2d4927b5fbb085158386dcb3cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T02:15:24.202 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:15:24 smithi183 podman[74537]: 2024-09-12 02:15:24.027298377 +0000 UTC m=+3.178074515 container cleanup 9902db5727df5c4ee0b641a7fb98ab6aa39b8c2d4927b5fbb085158386dcb3cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T02:15:24.203 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 02:15:24 smithi183 bash[74537]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-c 2024-09-12T02:15:24.462 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T02:15:24.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:24.498 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T02:15:24.498 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T02:15:24.498 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.f 2024-09-12T02:15:24.952 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:15:24 smithi183 systemd[1]: Stopping Ceph mds.f for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:24.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:15:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: 2024-09-12T02:15:24.809+0000 7f3ec153c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:24.953 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:15:24 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f[64435]: 2024-09-12T02:15:24.809+0000 7f3ec153c640 -1 mds.f *** got signal Terminated *** 2024-09-12T02:15:29.089 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:15:28 smithi183 podman[74668]: 2024-09-12 02:15:28.81949239 +0000 UTC m=+4.104525834 container died 00a7be5dffbc5a959c26c5b106e74b671a93fa8ab5fe799a25dd8c5021ea578f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T02:15:29.431 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:15:29 smithi183 podman[74668]: 2024-09-12 02:15:29.189387758 +0000 UTC m=+4.474421202 container cleanup 00a7be5dffbc5a959c26c5b106e74b671a93fa8ab5fe799a25dd8c5021ea578f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:15:29.432 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 02:15:29 smithi183 bash[74668]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-f 2024-09-12T02:15:29.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T02:15:29.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:29.600 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T02:15:29.600 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T02:15:29.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.i 2024-09-12T02:15:29.952 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:29 smithi183 systemd[1]: Stopping Ceph mds.i for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:29.953 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i[65036]: 2024-09-12T02:15:29.933+0000 7ff596db9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:29.953 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:29 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i[65036]: 2024-09-12T02:15:29.933+0000 7ff596db9640 -1 mds.i *** got signal Terminated *** 2024-09-12T02:15:33.859 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:33 smithi183 podman[74787]: 2024-09-12 02:15:33.534098107 +0000 UTC m=+3.710111285 container died 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T02:15:33.859 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:33 smithi183 podman[74787]: 2024-09-12 02:15:33.859289422 +0000 UTC m=+4.035302603 container cleanup 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:15:34.185 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:33 smithi183 bash[74787]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i 2024-09-12T02:15:34.430 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T02:15:34.451 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:34 smithi183 podman[74801]: 2024-09-12 02:15:34.18576429 +0000 UTC m=+0.645318581 container remove 5fb738928dc863661aa5ba0169273cb5d54255e85dab4c2492070f8164b710f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T02:15:34.452 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:34 smithi183 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-12T02:15:34.452 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:34 smithi183 systemd[1]: Stopped Ceph mds.i for 5731f69a-70a7-11ef-bcea-c7b262605968. 2024-09-12T02:15:34.452 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 02:15:34 smithi183 systemd[1]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.i.service: Consumed 1.223s CPU time. 2024-09-12T02:15:34.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:34.462 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T02:15:34.462 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T02:15:34.463 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.l 2024-09-12T02:15:34.702 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:15:34 smithi183 systemd[1]: Stopping Ceph mds.l for 5731f69a-70a7-11ef-bcea-c7b262605968... 2024-09-12T02:15:35.202 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:15:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l[64133]: 2024-09-12T02:15:34.804+0000 7f5ab0a15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:15:35.203 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:15:34 smithi183 ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l[64133]: 2024-09-12T02:15:34.804+0000 7f5ab0a15640 -1 mds.l *** got signal Terminated *** 2024-09-12T02:15:39.105 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:15:38 smithi183 podman[74908]: 2024-09-12 02:15:38.822999432 +0000 UTC m=+4.143957828 container died 669df5a4698d1b7081fc968d9b16a74d5403064bbd72d08cabf23cb5637fe173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T02:15:39.452 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:15:39 smithi183 podman[74908]: 2024-09-12 02:15:39.288314023 +0000 UTC m=+4.609272416 container cleanup 669df5a4698d1b7081fc968d9b16a74d5403064bbd72d08cabf23cb5637fe173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T02:15:39.453 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 02:15:39 smithi183 bash[74908]: ceph-5731f69a-70a7-11ef-bcea-c7b262605968-mds-l 2024-09-12T02:15:39.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5731f69a-70a7-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T02:15:39.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:15:39.729 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T02:15:39.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:15:39.929 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T02:15:42.353 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:15:42.554 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T02:15:45.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:15:45.457 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T02:15:47.854 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:15:47.887 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:15:47.920 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:15:47.948 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T02:15:47.949 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi122/crash 2024-09-12T02:15:47.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:15:47.996 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:15:47.996 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:15:47.999 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi151/crash 2024-09-12T02:15:48.000 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:15:48.042 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:15:48.042 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:15:48.045 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi183/crash 2024-09-12T02:15:48.046 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:15:48.078 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:15:48.078 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:15:48.080 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T02:15:48.080 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T02:15:48.124 INFO:teuthology.orchestra.run.smithi122.stdout:2024-09-12T01:48:10.173309+0000 mds.h (mds.2) 2 : cluster [WRN] Scrub error on inode 0x10000000009 (/volumes/qa/sv_1/7eec84f0-36f8-4977-b35c-6cefbfc555ff/client.0/tmp/clients/client0/~dmtmp) see mds.h log and `damage ls` output for details 2024-09-12T02:15:48.124 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-12T02:15:48.124 INFO:tasks.cephadm:Compressing logs... 2024-09-12T02:15:48.124 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:15:48.167 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:15:48.169 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:15:48.193 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T02:15:48.193 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-09-12T02:15:48.193 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T02:15:48.193 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:15:48.193 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:15:48.193 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T02:15:48.194 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:15:48.194 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T02:15:48.194 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T02:15:48.197 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.a.log: 83.3% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:15:48.197 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T02:15:48.198 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:15:48.198 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.5% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:15:48.201 INFO:teuthology.orchestra.run.smithi122.stderr: 94.8% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T02:15:48.202 INFO:teuthology.orchestra.run.smithi122.stderr: 94.7% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T02:15:48.208 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T02:15:48.216 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log: gzip -5 91.1% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:15:48.216 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T02:15:48.216 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T02:15:48.216 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T02:15:48.216 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.j.log: 87.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:15:48.216 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:15:48.218 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-09-12T02:15:48.218 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T02:15:48.218 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.b.log: /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T02:15:48.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T02:15:48.220 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T02:15:48.220 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:15:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.7% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T02:15:48.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T02:15:48.225 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T02:15:48.225 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:15:48.225 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:15:48.225 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T02:15:48.225 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T02:15:48.225 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T02:15:48.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T02:15:48.227 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:15:48.228 INFO:teuthology.orchestra.run.smithi122.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:15:48.382 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 83.6% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:15:48.385 INFO:teuthology.orchestra.run.smithi183.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:15:48.385 INFO:teuthology.orchestra.run.smithi183.stderr: 87.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:15:48.443 INFO:teuthology.orchestra.run.smithi183.stderr: 93.6% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T02:15:48.446 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 83.7% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:15:48.449 INFO:teuthology.orchestra.run.smithi151.stderr: 90.6% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:15:48.466 INFO:teuthology.orchestra.run.smithi151.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:15:48.495 INFO:teuthology.orchestra.run.smithi183.stderr: 90.6% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:15:48.546 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T02:15:48.549 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:15:48.566 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:15:48.616 INFO:teuthology.orchestra.run.smithi122.stderr: 90.7% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T02:15:48.740 INFO:teuthology.orchestra.run.smithi183.stderr: 94.8% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T02:15:48.965 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:15:49.414 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T02:15:51.358 INFO:teuthology.orchestra.run.smithi151.stderr: 95.3% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T02:15:52.348 INFO:teuthology.orchestra.run.smithi122.stderr: 92.4% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T02:15:58.697 INFO:teuthology.orchestra.run.smithi151.stderr: 91.7% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T02:16:05.989 INFO:teuthology.orchestra.run.smithi183.stderr: 95.3% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T02:16:17.523 INFO:teuthology.orchestra.run.smithi122.stderr: 91.3% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T02:16:21.348 INFO:teuthology.orchestra.run.smithi122.stderr: 91.6% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T02:16:24.953 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T02:16:31.635 INFO:teuthology.orchestra.run.smithi122.stderr: 91.4% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T02:16:31.637 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-12T02:16:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m43.456s 2024-09-12T02:16:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:user 2m22.439s 2024-09-12T02:16:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m4.597s 2024-09-12T02:16:45.961 INFO:teuthology.orchestra.run.smithi183.stderr: 91.2% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T02:16:55.687 INFO:teuthology.orchestra.run.smithi183.stderr: 91.1% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T02:16:56.484 INFO:teuthology.orchestra.run.smithi183.stderr: 94.7% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T02:17:11.010 INFO:teuthology.orchestra.run.smithi151.stderr: 95.4% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T02:17:17.816 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T02:17:18.132 INFO:teuthology.orchestra.run.smithi183.stderr: 91.5% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T02:17:18.647 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T02:17:18.648 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-12T02:17:18.648 INFO:teuthology.orchestra.run.smithi183.stderr:real 1m30.459s 2024-09-12T02:17:18.648 INFO:teuthology.orchestra.run.smithi183.stderr:user 4m11.505s 2024-09-12T02:17:18.648 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m8.212s 2024-09-12T02:18:59.005 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T02:21:57.041 INFO:teuthology.orchestra.run.smithi151.stderr: 91.4% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T02:23:30.119 INFO:teuthology.orchestra.run.smithi151.stderr: 91.5% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T02:24:08.160 INFO:teuthology.orchestra.run.smithi151.stderr: 91.2% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T02:24:41.574 INFO:teuthology.orchestra.run.smithi151.stderr: 91.3% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T02:32:43.867 INFO:teuthology.orchestra.run.smithi151.stderr: 93.0% -- replaced with /var/log/ceph/5731f69a-70a7-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T02:32:43.869 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-12T02:32:43.869 INFO:teuthology.orchestra.run.smithi151.stderr:real 16m55.685s 2024-09-12T02:32:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:user 12m22.637s 2024-09-12T02:32:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m31.642s 2024-09-12T02:32:43.870 INFO:tasks.cephadm:Archiving logs... 2024-09-12T02:32:43.871 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi122/log 2024-09-12T02:32:43.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:33:08.053 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi151/log 2024-09-12T02:33:08.054 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:35:06.015 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi183/log 2024-09-12T02:35:06.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:35:43.897 INFO:tasks.cephadm:Removing cluster... 2024-09-12T02:35:43.897 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --force 2024-09-12T02:35:44.100 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T02:35:44.680 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --force 2024-09-12T02:35:44.992 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T02:35:45.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5731f69a-70a7-11ef-bcea-c7b262605968 --force 2024-09-12T02:35:46.001 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 5731f69a-70a7-11ef-bcea-c7b262605968 2024-09-12T02:35:46.553 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T02:35:46.553 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:35:46.573 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:35:46.593 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:35:46.609 INFO:tasks.cephadm:Teardown complete 2024-09-12T02:35:46.609 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T02:35:46.620 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T02:35:46.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:35:46.623 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:35:46.636 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:35:46.764 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T02:35:46.775 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T02:35:46.775 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:35:46.778 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:35:46.781 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:35:46.793 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:35:46.794 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:35:46.795 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:35:46.797 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:35:46.797 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-12T02:35:46.798 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -3291us[-3278us] +/- 64ms 2024-09-12T02:35:46.798 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +1202us[+1202us] +/- 75ms 2024-09-12T02:35:46.798 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +3107us[+3107us] +/- 74ms 2024-09-12T02:35:46.798 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:35:46.807 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:35:46.807 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-12T02:35:46.807 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -3417us[-3430us] +/- 64ms 2024-09-12T02:35:46.807 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 +1080us[+1068us] +/- 75ms 2024-09-12T02:35:46.807 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +2975us[+2975us] +/- 74ms 2024-09-12T02:35:46.807 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:35:46.828 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:35:46.828 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-12T02:35:46.828 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -3448us[-3455us] +/- 64ms 2024-09-12T02:35:46.828 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +1023us[+1017us] +/- 75ms 2024-09-12T02:35:46.828 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +2929us[+2923us] +/- 74ms 2024-09-12T02:35:46.828 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:35:46.829 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T02:35:46.840 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T02:35:46.841 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T02:35:46.852 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:35:46.939 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:35:47.042 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:35:47.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:35:47.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:35:47.178 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:35:47.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:35:47.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:35:47.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:35:47.237 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T02:35:47.251 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T02:35:47.262 INFO:teuthology.task.internal:Duration was 3952.505537 seconds 2024-09-12T02:35:47.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T02:35:47.274 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T02:35:47.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:35:47.278 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:35:47.280 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:35:47.315 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:35:47.328 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:35:47.329 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:35:47.668 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T02:35:47.668 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-12T02:35:47.668 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:35:47.689 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-09-12T02:35:47.690 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:35:47.714 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-12T02:35:47.715 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:35:47.753 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T02:35:47.754 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:35:47.756 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:35:47.759 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:35:47.821 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T02:35:47.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:35:47.823 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:35:47.826 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:35:48.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T02:35:48.844 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T02:35:48.844 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:35:48.870 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:35:48.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:35:48.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T02:35:48.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:35:48.978 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:35:48.981 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:35:48.998 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-12T02:35:49.006 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-12T02:35:49.015 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-09-12T02:35:49.035 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:35:49.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:35:49.067 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:35:49.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:35:49.092 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:35:49.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:35:49.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T02:35:49.123 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T02:35:49.125 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi122 2024-09-12T02:35:49.125 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:35:49.186 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi151 2024-09-12T02:35:49.186 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:35:49.226 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901560/remote/smithi183 2024-09-12T02:35:49.226 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:35:49.266 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T02:35:49.266 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:35:49.269 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:35:49.271 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:35:49.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T02:35:49.336 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T02:35:49.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T02:35:49.352 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T02:35:49.353 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:35:49.356 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:35:49.358 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:35:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: 265972 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:35 /home/ubuntu/cephtest 2024-09-12T02:35:49.372 INFO:teuthology.orchestra.run.smithi151.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:35 /home/ubuntu/cephtest 2024-09-12T02:35:49.374 INFO:teuthology.orchestra.run.smithi183.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:35 /home/ubuntu/cephtest 2024-09-12T02:35:49.375 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T02:35:49.392 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T02:35:49.428 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 3952.5055372714996 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-12T02:35:49.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:35:49.525 INFO:teuthology.run:FAIL